Date: 20-Feb-2008
From: Mark Davies <mark_daviesbyu.edu>
Subject: 360,000,000 word BYU Corpus of American English
E-mail this message to a friend
We are pleased to announce the release of the 360+ million word ''BYUCorpus of American English'' (1990-2007), which is freely available online(http://www.americancorpus.org). New texts will be added at least two timeseach year from this point on (20 million new words each year; 4 millionwords in each of the five genres), and it will thus serve as a uniquelinguistic history of American English since 1990.
CONTENT
The corpus is composed of more than 360 million words in nearly 150,000texts, including 20 million words each year from 1990-2007. For each year(and therefore overall, as well), the corpus is evenly divided between thefive genres of spoken, fiction, popular magazines, newspapers, and academicjournals. The texts come from a variety of sources:
Spoken: (76+ million words) Transcripts of unscripted conversation fromnearly 150 different TV and radio programs (examples: All Things Considered(NPR), Newshour (PBS), Good Morning America (ABC), Today Show (NBC), 60Minutes (CBS), Hannity and Colmes (Fox), Jerry Springer, etc).
Fiction: (70 million words) Short stories and plays from literarymagazines, children’s magazines, popular magazines, first chapters of firstedition books 1990-present, and movie scripts.
Popular Magazines: (78+ million words) Nearly 100 different magazines, witha good mix (overall, and by year) between specific domains (news, health,home and gardening, women, financial, religion, sports, etc). A fewexamples are Time, Men’s Health, Good Housekeeping, Cosmopolitan, ChristianCentury, Fortune, Sports Illustrated, etc.
Newspapers: (73+ million words) Ten newspapers from across the US,including: USA Today, New York Times, Atlanta Journal Constitution, SanFrancisco Chronicle, etc. There is also a good mix between differentsections of the newspapers, such as local news, opinion, sports, financial,etc.
Academic Journals: (73+ million words) Nearly 100 different peer-reviewedjournals. These were selected to cover the entire range of the Library ofCongress classification system (e.g. a certain percentage from B(philosophy, psychology, religion), D (world history), K (education), T(technology), etc.), both overall and by number of words per year
QUERIES
-- The interface is the same as the interface for the 100 million wordBritish National Corpus and 100 million word TIME Magazine corpus (seehttp://corpus.byu.edu/)
-- Queries by word, phrase, alternates, substring, part of speech, lemma,synonyms (see below), and customized lists (see below)
-- The corpus is tagged by CLAWS, the same tagger that was used for the BNCand the TIME corpus
-- Chart listings (totals for all matching forms in each genre or year,1990-present, as well as for sub-genres) and table listings (frequency foreach matching form in each genre or year)
-- Full collocates searching (up to ten words left and right of node word)
-- Comparisons between genres or time periods (e.g. collocates of 'chair'in fiction or academic, nouns with 'break the [N]' in newspapers oracademic, adjectives that occur primarily in sports magazines, or verbsthat are more common 2004-2007 than previously)
-- One-step comparisons of collocates of related words, to study semanticor cultural differences between words (e.g. comparison of collocates of'small' and 'little', or 'men' and 'women', or 'rob' vs 'steal')
-- Include semantic information from a 60,000 entry thesaurus directly aspart of the query syntax (e.g. frequency and distribution of synonyms of'beautiful', synonyms of 'strong' occurring in fiction but not academic,synonyms of 'clean' + noun ('clean the floor', 'washed the dishes')
-- Create your own 'customized' word lists, and then re-use these as partof subsequent queries (e.g. lists related to a particular semantic category(clothes, foods, emotions), or a user-defined part of speech)
NOTE
Due to copyright and licensing issues, the corpus is not available infull-text form. Rather, as with our interface to the BNC and TIME, allaccess will be via the web interface, which allows full frequency anddistributional charts, and limited KWIC displays (up to 100 words per entry)
============================================Mark DaviesProfessor of (Corpus) LinguisticsBrigham Young University(phone) 801-422-9168 / (fax) 801-422-0906Web: davies-linguistics.byu.edu
** Corpus design and use // Linguistic databases **** Historical linguistics // Language variation **** English, Spanish, and Portuguese **============================================
Linguistic Field(s): Computational Linguistics; Lexicography; Text/Corpus Linguistics
|