learn.ifs.name

Phonetics and Speech Feature Extraction

Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 15

64 cards in “15.4 Acoustic phonetics” · whole deck

15.4 Acoustic phonetics

  1. Write the sine wave equation.
    \(y = A \sin(2\pi f t)\) — A = amplitude, f = frequency.
  2. What is frequency, and what is its unit?
    The number of times per second a wave repeats (cycles per second), measured in hertz (Hz).
  3. What is the amplitude of a sine wave?
    Its maximum value on the y-axis.
  4. What is the period T and how is it computed?
    The time for one cycle to complete: \(T = 1/f\). A 10 Hz wave has T = 0.1 s.
  5. What does a speech waveform plot?
    Changes in air pressure over time: compression or rarefaction of air molecules (e.g. at a microphone or eardrum).
  6. What are compression and rarefaction?
    Compression: higher-than-normal air pressure. Rarefaction: lower-than-normal (uncompressed) air pressure.
  7. What are the two steps of analog-to-digital conversion?
    Sampling and quantization.
  8. What is sampling and the sampling rate?
    Sampling: measuring the signal's amplitude at particular times. Sampling rate: number of samples per second.
  9. Why need at least two samples per cycle?
    One for the positive part and one for the negative part; with fewer, the wave's frequency is completely missed.
  10. What is the Nyquist frequency?
    The maximum frequency measurable at a given sampling rate: half the sample rate.
  11. What sampling rate is needed for full accuracy on speech, and why?
    20,000 Hz, since most speech information is below 10,000 Hz.
  12. Why is 8 kHz sampling sufficient for telephone speech?
    The telephone network transmits only frequencies below 4,000 Hz (e.g. Switchboard).
  13. Typical sampling rate for microphone speech?
    16,000 Hz.
  14. What is quantization?
    Representing real-valued amplitudes as integers; the gap between integers is a minimum granularity (quantum size), and values closer than it are represented identically.
  15. Value ranges of 8-bit and 16-bit samples?
    8-bit: −128 to 127. 16-bit: −32768 to 32767.
  16. Typical storage for telephone vs microphone speech?
    Telephone: 8 kHz, 8-bit samples. Microphone: 16 kHz, 16-bit samples.
  17. What are channels in audio file formats?
    Separate signals, e.g. for stereo or two-party conversations; stored in the same file or separate files.
  18. What is µ-law and what is its intuition?
    A log compression format for telephone speech. Human hearing is more sensitive at small intensities, so the log represents small values faithfully at the cost of more error on large values.
  19. Write the µ-law compression equation.
    \(F(x) = \frac{\mathrm{sgn}(x)\log(1+\mu|x|)}{\log(1+\mu)},\ -1 \le x \le 1\), with µ = 255 for 8 bits.
  20. What is linear PCM?
    Linear (unlogged, uncompressed) sample values. PCM = pulse code modulation.
  21. What is the relationship between .wav and RIFF?
    The .wav format is a subset of Microsoft's RIFF, a general format of nested chunks of data and control info. A simple .wav has a 44-byte header (with format chunk) before the data chunk.
  22. Name some standard waveform file formats.
    Microsoft .wav, Apple AIFF (both with headers), and headerless “raw” files.
  23. The [iy] waveform repeats 10 times in 0.03875 s. What is its frequency?
    10 / 0.03875 ≈ 258 Hz.
  24. What causes the periodicity of a voiced waveform?
    Vocal fold vibration: each major amplitude peak corresponds to an opening of the vocal folds.
  25. What is the fundamental frequency (F0)?
    The frequency of vocal fold vibration, i.e. the frequency of the complex wave.
  26. What is a pitch track, and when does it fail?
    A plot of F0 over time. Automatic pitch tracking counts pulses in voiced regions, so it fails where there is no voicing or insufficient sound.
  27. What does the y-axis of a waveform measure, and in what unit?
    Air pressure variation, in Pascals (Pa). Positive = above atmospheric pressure, zero = normal, negative = rarefaction.
  28. Why use RMS amplitude rather than the mean amplitude?
    Positive and negative values would mostly cancel. RMS squares each value before averaging, then takes the square root.
  29. Write the RMS amplitude formula.
    \(\text{RMS} = \sqrt{\frac{1}{N}\sum_{i=1}^{N} x_i^2}\)
  30. Write the formula for the power of a signal.
    \(\text{Power} = \frac{1}{N}\sum_{i=1}^{N} x_i^2\) — related to the square of the amplitude.
  31. What is intensity, and what's its formula?
    Power normalized to the human auditory threshold, in dB: \(\text{Intensity} = 10\log_{10}\frac{1}{NP_0}\sum_{i=1}^{N} x_i^2\), with \(P_0 = 2\times10^{-5}\) Pa.
  32. Where do intensity peaks occur in speech (e.g. Is it a long movie?)?
    At each vowel, with an especially high peak on the word long.
  33. What is pitch?
    The perceptual correlate (mental sensation) of fundamental frequency.
  34. How does pitch relate to frequency across the hearing range?
    Most accurate and roughly linear between 100 and 1000 Hz; above 1000 Hz, logarithmic (high-frequency differences are compressed).
  35. What is a mel?
    A unit of pitch: pairs of sounds perceptually equidistant in pitch are separated by an equal number of mels (Stevens et al. 1937; Stevens & Volkmann 1940).
  36. Write the mel frequency formula.
    \(m = 1127 \ln\left(1 + \frac{f}{700}\right)\)
  37. What is loudness, and why is its relation to power nonlinear?
    The perceptual correlate of power. Nonlinear because (1) the ear has greater resolution at low power, and (2) perceived loudness also depends on frequency.
  38. What are pitch extraction algorithms?
    Algorithms for automatically extracting F0 (a slight abuse of terminology).
  39. How does the autocorrelation method of pitch extraction work?
    Correlates the signal with itself at various offsets; the offset with the highest correlation gives the signal's period.
  40. Name a toolkit with a pitch tracker (and much more) for acoustic analysis.
    Praat (Boersma & Weenink): augmented autocorrelation pitch tracking, spectral and formant analysis, and a scripting language.
  41. How can vowels be spotted in a waveform?
    They are voiced (regular amplitude peaks), relatively long, and relatively loud.
  42. How do stop consonants look in a waveform?
    A period of silence or near-silence (closure) followed by a slight burst of amplitude (release).
  43. How do fricatives like [sh] look in a waveform?
    A noisy, irregular waveform from turbulent air, unlike the regular voicing of vowels.
  44. What is the key insight of Fourier analysis?
    Every complex wave can be represented as a sum of many sine waves of different frequencies.
  45. What is a spectrum?
    A representation of each of a signal's frequency components and their amplitudes (frequency on x-axis, amplitude on y-axis).
  46. What does the spectrum of a 10 Hz + 100 Hz sum of sines look like?
    Two spikes, at 10 Hz and 100 Hz.
  47. In the [ae] of had, what frequencies are visible in the waveform?
    A complex wave ≈234 Hz (10 cycles in 0.0427 s), a smaller wave ≈936 Hz (4× per cycle), and a tiny wave ≈1872 Hz.
  48. What significant frequency components does the DFT spectrum of [ae] in had show?
    Around 930 Hz, 1860 Hz, and 3020 Hz.
  49. What is a decibel (dB)?
    A logarithmic measure of amplitude.
  50. Why is a spectrum useful for phone identification?
    Spectral peaks are characteristic “signatures” of different phones — like spectral lines identifying chemical elements in stars.
  51. What does the cochlea do?
    The inner ear computes a spectrum of the incoming waveform.
  52. What is a spectrogram? Describe its axes.
    A visualization of how a waveform's frequencies change over time. x = time, y = frequency (Hz), darkness = amplitude.
  53. Spectrum vs spectrogram?
    A spectrum shows frequency components at one point in time; a spectrogram shows them over time (a collection of spectra placed end to end).
  54. What is a formant?
    A frequency band particularly amplified by the vocal tract; appears as a dark bar (spectral peak) in a spectrogram.
  55. How do F1 and F2 compare for [ih], [ae], [uh]?
    F1: low for [ih] (~470 Hz), higher for [ae] and [uh] (~800 Hz).
    F2: highest for [ih], middle for [ae], lowest for [uh].
  56. F1 and F2 of [iy] in She just had a baby, vs [ae]?
    [iy]: F1 = 540 Hz (much lower than [ae]), F2 = 2581 Hz (much higher than [ae]).
  57. Which formants determine vowel identity, and what do higher formants reflect?
    F1 and F2 largely determine vowel identity (though they vary by speaker). Higher formants reflect general characteristics of the speaker's vocal tract.
  58. Besides vowels, which phones can formants help identify?
    Nasals [n], [m], [ng] and liquids [l], [r].
  59. Why are formants harder to see in running speech?
    Reduction and coarticulation processes.
  60. What is the source-filter model?
    Explains a sound's acoustics as pulses from the glottis (the source) shaped by the vocal tract (the filter).
  61. What is a harmonic? Example?
    A wave whose frequency is a multiple of the fundamental. A 115 Hz glottal vibration has harmonics at 230, 345, 460 Hz… each generally weaker.
  62. Why does the vocal tract act as a filter?
    Any cavity (like a tube) resonates at frequencies determined by its shape, amplifying some frequencies and damping others.
  63. How do different vowels arise in the source-filter model?
    Moving the tongue and articulators changes the vocal tract's shape, so different harmonics of the same fundamental are amplified. Formants are where the tract amplifies harmonics.
  64. Approximate F1/F2 for [iy] (tea), [ae] (cat), [uw] (moo) in Fig. 15.24?
    [iy]: 268 / 2416 Hz. [ae]: 903 / 1695 Hz. [uw]: 295 / 817 Hz.

Source