BERT Rediscovers the Classical NLP Pipeline

BERT represents the steps of the traditional NLP pipeline in an interpretable, localizable way, and the layers responsible for each step appear in the expected order: POS tagging, parsing, NER, semantic roles, then coreference. On individual sentences, though, it can revise lower-level decisions using higher-level information.

Behavioural work designs controlled test sets and analyses errors to infer what abstractions a model uses; structural probing inspects the network’s internal representations directly to find localizable regions associated with particular linguistic decisions. Tenney et al. take the structural route.

A common format for structured prediction tasks: a probing classifier gets one or two spans, \(s_1 = [i_1, j_1)\) and optionally \(s_2 = [i_2, j_2)\), and predicts a label (e.g. constituent or relation type), seeing only the contextual vectors within the spans, so it must rely on the encoder for information about the spans’ roles.

Part-of-speech tags, constituents, dependencies, entities, semantic role labeling, coreference, semantic proto-roles (SPR), and relation classification (SemEval), all scored with micro-averaged F1.

To learn how the network represents language as a result of pretraining: fine-tuning would let the encoder rearrange its internal representations to suit the probing task.

Not finding a pattern doesn’t prove it isn’t there, and finding one doesn’t say how the model uses it, so structural analysis should be combined with behavioural studies.

As in ELMo, the probe pools layers as \(h_{i,\tau} = \gamma_\tau \sum_{\ell=0}^{L} s_\tau^{(\ell)} h_i^{(\ell)}\) with \(s_\tau = \text{softmax}(a_\tau)\), learned jointly with the probe on training data; higher weights suggest a layer holds more task-relevant information. The summary is the center of gravity \(\bar{E}_s[\ell] = \sum_\ell \ell \cdot s_\tau^{(\ell)}\).

Train probes \(P_\tau^{(\ell)}\) that may use layers 0 to \(\ell\) (layer 0 alone is a non-contextual baseline); the differential score \(\Delta_\tau^{(\ell)} = \text{Score}(P_\tau^{(\ell)}) - \text{Score}(P_\tau^{(\ell-1)})\) measures how much layer \(\ell\) adds, measured on evaluation data.

\(\bar{E}_\Delta[\ell] = \frac{\sum_{\ell=1}^{L} \ell \cdot \Delta_\tau^{(\ell)}}{\sum_{\ell=1}^{L} \Delta_\tau^{(\ell)}}\), roughly the layer at which an example gets resolved (ignoring those resolved at layer 0). Mixing weights come from training data and say which layers the probe finds useful; cumulative scores come from evaluation data and say how many layers are needed.

Information about a span may be spread across several layers, and the encoder may discard information in higher layers; cumulative probes see all layers up to \(\ell\), so performance can only grow with access to more information.

Both metrics agree: POS tags earliest, then constituents and dependencies, then semantic roles and coreference. Basic syntax appears earlier in the network, high-level semantics in higher layers, consistent with Peters et al. (2018b).

Syntactic information is localizable: mixing weights concentrate on a few layers. Semantic information (relations, proto-roles) is spread across the network: weights are close to uniform and examples are resolved gradually across nearly all layers.

Many examples can be resolved early from shallow heuristic cues, while hard examples need later layers; the highest mixing weights fall on or just after the last layers that still improve F1 for the task.

The same general ordering holds, with a “stretching effect”: each task’s representations concentrate at the same layers relative to the top of the model.

In “he smoked toronto in the playoffs …”, early layers tag toronto as a city (GPE); once the semantic role is resolved (toronto is what got “smoked”, ARG1), the entity type is revised to ORG, the sports team. The pipeline order holds on average but not for every example.

In “china today blacked out a cnn interview”, today was first tagged as a common noun, date and temporal modifier; higher layers reinterpreted “china today” as a proper noun (a TV network), updating the entity type to ORG and the semantic role to agent (ARG0).

They can represent the syntactic and semantic abstractions traditionally thought necessary for language processing, in roughly pipeline order, and also model complex interactions between levels, using high-level information like predicate–argument relations to disambiguate low-level decisions like part of speech.