learn.ifs.name

Automatic Speech Recognition

Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 16

23 cards in “16.1 The ASR task” · whole deck

16.1 The ASR task

  1. What are the four dimensions along which the ASR task varies?
    Vocabulary size; who the speaker is talking to; channel and noise; and accent or speaker-class characteristics.
  2. How does vocabulary size affect ASR difficulty? Examples?
    Small vocabularies are easy and long solved — yes/no (2 words), digit recognition (11 words: zero–nine plus “oh”). Open-ended tasks with 60,000+ words (videos, conversations) are much harder.
  3. What is digit recognition?
    Recognizing sequences of digits: an 11-word vocabulary of zero to nine plus “oh”.
  4. Rank these from easiest to hardest to recognize: conversational speech, read speech, speech to machines.
    Speech to machines and read speech are relatively easy; conversational speech between humans (e.g. a business meeting) is the hardest.
  5. Why is speech to machines or read speech easier to recognize?
    When talking to machines or reading without an audience, people simplify their speech: they talk more slowly and clearly.
  6. How do channel and noise affect ASR?
    Speech recorded in a quiet room with head-mounted microphones is easier than speech from a distant microphone on a noisy street or in a car with the window open.
  7. How do accent and speaker class affect ASR?
    Speech is easier when the speaker uses the dialect the system was trained on. Regional or ethnic dialects, or children's speech, can be hard if training used only standard-dialect or adult speakers.
  8. LibriSpeech: what is it?
    A large open-source read-speech 16 kHz dataset: over 1000 hours of audiobooks from the LibriVox project (volunteers reading copyright-free books), with sentence-level aligned transcripts.
  9. How were LibriSpeech's “clean” and “other” portions defined?
    A recognizer trained on WSJ read speech was run on all audio; per-speaker WER was computed and speakers were split roughly in half: lower-WER speakers = “clean”, higher-WER = “other”.
  10. Switchboard corpus: contents and special advantage?
    Early-1990s prompted telephone conversations between strangers: 2430 conversations averaging 6 min, 240 hours of 8 kHz speech, ~3 million words. Advantage: huge amounts of auxiliary hand labeling (parses, dialogue acts, phonetic and prosodic labels, discourse structure).
  11. CALLHOME corpus?
    Late-1990s corpus of 120 unscripted 30-minute telephone conversations between native English speakers who were usually close friends or family.
  12. CHiME Challenge and the CHiME 6 task?
    A series of difficult shared tasks on robust ASR. CHiME 6: conversational ASR at real dinner parties — 20 parties in real homes, 4 participants each, recorded with distant microphones in kitchen, dining area and living room.
  13. AMI Meeting Corpus?
    100 hours of recorded group meetings (some natural, some organized), with manual transcriptions and some extra hand-labels.
  14. CORAAL?
    Over 150 anonymized sociolinguistic interviews with African American speakers, for studying African American English (AAE); transcripts aligned at the utterance level.
  15. HKUST Mandarin corpus?
    1206 transcribed ten-minute telephone conversations between Mandarin speakers across China, between friends and strangers.
  16. AISHELL-1?
    170 hours of Mandarin read speech of sentences from various domains, read mainly by speakers from northern China.
  17. Common Voice?
    A free crowd-sourced multilingual corpus of transcribed read speech (MP3) for ASR. Volunteers read scripts (often from Wikipedia), and others verify the recordings. At writing: 33,150 hours in 133 languages.
  18. FLEURS?
    A parallel speech dataset built on the MT benchmark FLoRes-101 (3001 English Wikipedia sentences translated into 101 languages). For 2009 sentences in each of 102 languages, 3 native speakers read each, ~12 hours of speech per language.
  19. Approximate state-of-the-art WER on LibriSpeech clean vs other?
    Clean: 1.4%. Other: 2.6%. (English read speech is transcribed very accurately, ~2%.)
  20. Approximate WERs for Switchboard, CALLHOME, and AMI?
    Switchboard 5.8%; CALLHOME 11%; AMI meetings 11%.
  21. Approximate WERs for CORAAL and CHiME6?
    CORAAL (AAE) 16.2%; CHiME6 dinner parties 25.5%.
  22. Approximate WERs for low-resource examples (Common Voice Vietnamese, Swahili; FLEURS Bengali)?
    Vietnamese 39.8%; Swahili 51.2%; Bengali 50%.
  23. What metric is used for Mandarin, and what are the CERs for AISHELL-1 vs HKUST?
    Character error rate (CER). AISHELL-1 read speech 3.9%; HKUST telephone conversations 18.5%.

Source