learn.ifs.name

Phonetics and Speech Feature Extraction

Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 15

25 cards in “15.5 Log mel spectrum” · whole deck

15.5 Log mel spectrum

  1. What is an acoustic feature vector?
    A vector representing the information in a small time window of the signal; speech processing often first converts the waveform into a sequence of these.
  2. What are the two common starting points for speech recognition systems?
    The raw waveform (processed by convolutional networks) or a higher-level representation such as the log mel spectrum.
  3. Can you mix sampling rates in ASR training and testing?
    No. E.g. if testing on 8 kHz Switchboard, downsample training data to 8 kHz; if any training corpus is telephone speech, downsample all wideband corpora to 8 kHz.
  4. Do higher sampling rates help ASR accuracy?
    Yes, higher sampling rates produce higher ASR accuracy.
  5. What does x[n] denote?
    The sample at time index n in the digitized, quantized waveform.
  6. Stationary vs non-stationary signal?
    Stationary: statistical properties constant within a region. Speech is non-stationary overall, but roughly stationary within a small window.
  7. What is a window in speech processing?
    A function that is non-zero inside a region and zero elsewhere; run across the signal and multiplied by it to produce a windowed waveform.
  8. What is a frame?
    The speech extracted from each window.
  9. What three parameters characterize windowing?
    Window (frame) size in ms, frame stride (shift/offset) between windows, and window shape.
  10. Typical window size and stride?
    25 ms windows with a 10 ms stride.
  11. Windowing equation?
    \(y[n] = w[n]\,s[n]\)
  12. What is the problem with a rectangular window?
    It abruptly cuts off the signal at its boundaries, creating discontinuities that cause problems in Fourier analysis.
  13. What does the Hamming window do?
    Shrinks signal values toward zero at the window boundaries, avoiding discontinuities.
  14. Write the rectangular and Hamming window equations (window of length L).
    Rectangular: \(w[n]=1\) for \(0\le n\le L-1\), else 0.
    Hamming: \(w[n]=0.54-0.46\cos\left(\frac{2\pi n}{L}\right)\) for \(0\le n\le L-1\), else 0.
  15. What is the DFT? What are its input and output?
    Discrete Fourier transform: extracts spectral information for discrete frequency bands from a sampled signal. Input: windowed signal x[n]…x[m]. Output: for each of N bands, a complex number X[k] giving magnitude and phase.
  16. State Euler's formula.
    \(e^{j\theta} = \cos\theta + j\sin\theta\), with j the imaginary unit.
  17. Write the DFT definition.
    \(X[k] = \sum_{n=0}^{N-1} x[n]\, e^{-j\frac{2\pi}{N}kn}\)
  18. What is the FFT and its limitation?
    Fast Fourier transform: a very efficient algorithm for computing the DFT; only works for N that are powers of 2.
  19. Why collect energies on the mel scale instead of equally across frequency?
    Human hearing is less sensitive at high frequencies. Low-frequency info (formants) is crucial for vowels/nasals; high-frequency info (stop bursts, fricative noise) is less crucial. Modeling this improves ASR.
  20. Write the mel formula used for the filter bank.
    \(\mathrm{mel}(f) = 1127\ln\left(1+\frac{f}{700}\right)\)
  21. What is a mel filter bank?
    A bank of triangular filters spaced logarithmically along the mel scale (fine resolution at low, coarse at high frequencies), multiplied by the spectrum to get a mel spectrum (Davis & Mermelstein, 1980).
  22. Why take the log of the mel spectrum values? (2 reasons)
    (1) Human response to signal level is logarithmic: less sensitive to small differences at high amplitudes. (2) Makes features less sensitive to input power variations, e.g. the mouth moving relative to the microphone.
  23. What is a channel in a filterbank output? Typical number?
    Each scalar output of one filter: the log energy of one mel-spaced frequency band. A frame gives a vector of, say, 80 or 128 channels.
  24. How are log mel features commonly normalized before a neural network?
    Scaled to between −1 and 1 with zero mean across the entire pretraining dataset.
  25. Order the log mel spectrum pipeline.
    Sample & quantize → window (e.g. 25 ms Hamming, 10 ms stride) → DFT/FFT → mel filter bank → log → normalize.

Source