Interpretability

10.1 Contextual embeddings

1. How do contextual embeddings differ from static embeddings like word2vec?
   Static embeddings give one vector per word type; contextual embeddings give a different vector each time a word appears in a different context.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 243
2. Where do contextual embeddings come from in a transformer?
   The output of each component (attention heads, feedforward network) added to the residual stream, at every layer and every token, is a contextual embedding. Commonly the final layer’s \(h^L_i\) is used as the meaning of token \(x_i\) in context.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 243
3. How are BERT contextual embeddings for a token often computed?
   By mean- or max-pooling the token’s output vectors from the last four layers, rather than using the final layer alone.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 243
4. Static embeddings represent word types; what do contextual embeddings represent?
   Word instances: a particular occurrence of a word type in a particular context.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, pp. 243–244
5. What does it mean that a word is polysemous? Give examples.
   It has multiple senses: mouse (rodent / cursor device), bank (financial institution / sloping mound).
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 244
6. What is a word sense, and where are senses listed?
   A discrete representation of one aspect of a word’s meaning (bank¹, bank²), listed in thesauruses like WordNet, which exists for many languages.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 244
7. How is the term polysemy sometimes used more narrowly, and what is homonymy?
   Sometimes polysemy means only senses related in a structured way (“structured polysemy”), with homonymy for ambiguity between unrelated senses. The book uses polysemy for any sense ambiguity.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 244
8. What do 2-d projections of many BERT embeddings of the word die show?
   Instances cluster by meaning: separate clusters for die as singular of dice, the verb to die, and the German article die.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, pp. 244–245
9. How does the view of word meaning from embeddings differ from WordNet’s?
   WordNet gives discrete lists of senses; embeddings offer a continuous high-dimensional model of meaning that can be clustered but doesn’t divide into fully discrete senses.
   • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 245
10. What is word sense disambiguation (WSD)?
    Selecting the correct sense of a word in context, given a fixed inventory of possible senses (e.g. from WordNet).
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 245
11. What is the “one sense per discourse” rule?
    A word tends to be used in roughly the same sense throughout a discourse.
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 246
12. Describe the best-performing simple WSD algorithm.
    1-nearest-neighbor with contextual embeddings: for each sense in a sense-labeled corpus (e.g. SemCor), average the contextual embeddings of its tokens into a sense embedding \(v_s\). At test time, embed the target token \(t\) and choose \(\arg\max_s \cos(t, v_s)\).

    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 246
13. How is word similarity in context measured with contextual embeddings?
    By the cosine between the contextual embeddings of the two word instances (after some transformation of the embeddings).
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, pp. 246–247
14. What is anisotropy in contextual embeddings, and why is it a problem?
    All vectors tend to point in the same direction, so even random word pairs have cosines close to 1 (anisotropy = the expected cosine between random pairs). This makes raw cosines uninformative.
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 247
15. What causes anisotropy, and how can it be reduced?
    A few “rogue dimensions” with very large magnitudes and variance dominate the cosine. Standardizing (z-scoring) each dimension over a corpus, \(z = \frac{x - \mu}{\sigma}\), makes embeddings more isotropic.
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 247
16. What problem with cosine similarity of contextual embeddings remains after standardization?
    Cosine tends to underestimate human similarity judgments for very frequent words.
    • Jurafsky & Martin, Speech and Language Processing, 3rd ed. draft, ch. 10, §10.1, p. 247