Discourse Coherence

A discourse is a coherent structured group of sentences (an article, a conversation, a novel); coherence is the relationship between sentences that makes real discourses different from random assemblages of sentences.

The reader can’t see why the second sentence follows the first, and tries to find a connection (a French spinach shortage?), which shows that comprehension involves establishing coherence. Compare “…She had to attend a conference”, a REASON.

Coherence relations between nearby sentences (e.g. REASON); entity-based coherence (being “about” salient entities rather than swinging between them); topical coherence (nearby sentences on the same topic, shown by lexical cohesion).

Sharing identical or semantically related words in nearby sentences (Halliday and Hasan 1976), e.g. house, chimney, garret, closet, window and a repeated shingled.

Conventional discourse structure of whole texts in a genre: sections like Methodology and Results in academic articles, plotlines in stories, claims and premises in persuasive essays.

Measuring text quality (essay grading), summarization (choosing information by coherence relations), and mental health tasks like measuring symptoms of schizophrenia or other disordered language.

The two spans of a relation: the nucleus is more central to the writer’s purpose and interpretable independently; the satellite is less central and interpretable only with respect to the nucleus. Some symmetric relations hold between two nuclei.

Reason: the satellite gives the reason for an animate agent’s action (“[Jane took a train…] [She had to attend a conference.]”). Elaboration: the satellite adds detail (“[Dorothy was from Kansas.] [She lived in the midst of the great Kansas prairies.]”). Evidence: the satellite adds information meant to convince the reader of the nucleus (“[Kevin must be here.] [His car is parked outside.]”).

Attribution: the satellite gives the source of reported speech (“[SAT Analysts estimated] [NUC that sales declined]”). List: a multinuclear series of nuclei without contrast (“[Billy Bones was the mate;] [Long John, he was quartermaster]”).

An arrow from the satellite to the nucleus, labeled with the relation:

Elementary discourse units (discourse segments): the leaves of an RST tree, spans of a sentence, clause or phrase. Finding their boundaries is an important task; they are analogous to constituents in syntax.

The largest discourse corpus: English Penn Treebank documents with full RST parses using 78 relations in 16 classes. RST treebanks also exist for Spanish, German, Basque, Dutch and Brazilian Portuguese.

Nuclei presumably express more important information than satellites, which can be dropped.

Annotators start from discourse connectives (because, although, when, since, as a result) and mark the connective and its two spans, Arg1 and Arg2, then label the sense.

For adjacent sentences with no explicit connective, annotators choose the connective that could have signaled the relation (e.g. implicit = as a result) and then label its sense.

PDTB annotates only pairwise relations between spans, with no commitment to higher-level structure; RST builds a global tree over the whole discourse.

Chinese marks far fewer relations with explicit connectives than English, so annotators mapped sentence pairs directly to 11 sense tags without starting from a connective.

Automatically determining the coherence relations between sentences: a full tree for RST, only labeled span pairs for PDTB.

Early systems post-processed syntactic parses (EDUs are roughly clauses). Modern systems are neural sequence models: an encoder plus a linear layer and softmax label each token 1 if it starts an EDU, else 0, supervised by gold segmentations.

shift: push the first EDU of the queue onto the stack as a single-node subtree. reduce(l, d): merge the top two subtrees with relation label \(l\) and nuclearity \(d \in \{NN, NS, SN\}\). pop root: remove the final tree.

action	stack after
SH	e1
SH	e1, e2
RD(attr, SN)	e1:2
SH	e1:2, e3
SH	e1:2, e3, e4
RD(elab, NS)	e1:2, e3:4
RD(elab, SN)	e1:4
PR	(done)

The nuclearity says which side is the nucleus: SN = satellite first (e1 is the attribution satellite of e2), NS = nucleus first (e4 elaborates e3).

A hierarchical biLSTM encodes words within each EDU and then the EDU sequence; EDU and subtree representations are averages of encoder outputs. A feedforward decoder scores actions from the top three stack subtrees plus the first queue EDU.

Map each gold tree to a sequence of oracle actions and train with cross-entropy over action probabilities (plus L2 regularization).

On the RST Discourse Treebank test set, with gold EDUs or end-to-end, by RST-Pareval metrics on binarized trees: span F1 (S), nuclearity (N), relation (R) and full (F), micro-averaged over spans.

PDTB parsing: flat relations between spans, not trees. Subtasks (Lin et al.): (1) find discourse connectives, (2) find each connective’s two spans, (3) label the relation, (4) assign a relation to every adjacent sentence pair without an explicit connective.

Encode the two spans with BERT, take the [CLS] hidden state, pass it through a one-layer tanh feedforward network and a softmax over the 11 second-level senses (Nie et al. 2019).

The same words have non-discourse uses: and links clauses in an expansion relation, but is NP conjunction in “blue and green”; once is temporal in “once it enters the lungs” but means ‘formerly’ in “once used to make filters”. It is a kind of word sense disambiguation, today done with biLSTM-CRF BIO tagging.

With sequence models (e.g. BERT inputs to a biLSTM), or simple heuristics: almost all relations lie within one sentence or span two adjacent sentences with one argument in each.

At any point one entity is salient (“centered”), and discourses that CONTINUE to keep the same entity salient are more coherent than those that SHIFT between entities. “John went to his music store… He had frequented the store… He was excited…” is better than alternating “It was a store John had frequented… It was closing just as John arrived.”

The backward-looking center \(C_b(U_n)\) is the salient entity being focused on after \(U_n\). The forward-looking centers \(C_f(U_n)\) are the entities evoked by \(U_n\), ranked by salience and grammatical role (subject > object > others). The preferred center \(C_p\) is the highest-ranked \(C_f\): a prediction of what comes next.

	same \(C_b\)	new \(C_b\)
\(C_b = C_p\)	Continue	Smooth-Shift
\(C_b \neq C_p\)	Retain	Rough-Shift

“Same \(C_b\)”: \(C_b(U_{n+1}) = C_b(U_n)\) or \(C_b(U_n)\) undefined; the rows compare \(C_b(U_{n+1})\) with \(C_p(U_{n+1})\).

Rule 1: if any element of \(C_f(U_n)\) is realized by a pronoun in \(U_{n+1}\), then \(C_b(U_{n+1})\) must be realized as a pronoun too. Rule 2: Continue is preferred to Retain, to Smooth-Shift, to Rough-Shift.

The speaker keeps the current entity as \(C_b\) but places it lower in \(C_f\), intending to shift to a new entity in a future utterance.

\(C_f(U_1)\) = {John, music store, piano}, \(C_p\) = John, \(C_b\) undefined. \(C_f(U_2)\) = {John, piano}, \(C_p\) = John, \(C_b\) = John. Since \(C_b(U_2) = C_p(U_2)\) and \(C_b(U_1)\) is undefined: Continue.

Instead of a top-down theory, it uses machine learning to induce which patterns of entity mentions make a discourse coherent (Barzilay and Lapata 2008).

A 2-D array with a row per sentence and a column per discourse entity (after coreference); cells give the entity’s highest grammatical role there: subject S, object O, neither X, absent –. (1) [The committee]S approved [the budget]O. (2) [It]S includes [funds]O for [schools]X. (3) [The committee]S will publish [it]O.

	committee	budget	funds	schools
1	S	O	–	–
2	–	S	O	X
3	S	O	–	–

Count local entity transitions, sequences in \(\{S, O, X, -\}^n\) read down each column (e.g. [S –], [O S]), and use the probability of each transition type as features of the document. Coherent texts have some dense columns (main entities, often subjects) and many sparse ones.

Predicting human coherence ratings (best when there is an end task, like essay grading), or self-supervision: natural documents vs pseudo-documents with changed sentence order, which are less coherent.

Sentence order discrimination: rank the original above a random permutation. Sentence insertion: remove a sentence and find its original position among all positions (harder, documents differ by one sentence). Sentence order reconstruction: put shuffled sentences back in order (hardest).

Lexical chains (Morris and Hirst): sequences of thesaurus-related words through a discourse (pine, bush, trees, trunk), whose number and density track topic structure. TextTiling (Hearst): cosine between word-count vectors of neighboring spans; high within a subtopic, low across a boundary.

Embed each sentence as the sum of its word LSA embeddings, and average the cosine over adjacent pairs: \(coherence(T) = \frac{1}{n-1}\sum_{i=1}^{n-1}\cos(s_i, s_{i+1})\).

Self-supervision: train to distinguish natural discourses from ones with scrambled sentences, letting representation learning discover the features that matter, instead of hand-labeled RST/PDTB relations.

Positive examples are consecutive pairs \((s_i, s_{i+1})\); negatives pair \(s_i\) with a random sentence \(s'\) from the same document. A margin loss \(\max(0, \eta - f^+ + f^-)\) pushes the positive score above the negative; document coherence is the average pair score.

From sentence embeddings: the concatenation \([s; t]\), the difference \(s - t\), the absolute difference \(|s - t|\), and the elementwise product \(s \odot t\), fed to a one-layer feedforward network that outputs a coherence score.

Train a language model and compare each sentence’s log likelihood with and without the preceding context: the gain measures how much the context makes the sentence predictable.

A plot grammar of Russian folktales: character categories (dramatis personae: Hero, Villain, Donor, Helper) and functions (events like “Donor tests Hero”) that occur in a fixed order; each tale picks a subset of functions.

As latent personae from Wikipedia movie plot summaries, using what characters do (villains strangle), what is done to them (villains are foiled) and how they are described (villains are evil).

Pathos (appeal to the listener’s emotions), ethos (appeal to the speaker’s character), logos (the logical structure). Argumentation mining has mostly studied logos.

Claims (central, controversial, needing support), premises (reasons supporting or attacking a claim or another premise), and argumentative relations like SUPPORT and ATTACK.

(1) is the claim; SUPPORT(2, 1), SUPPORT(3, 1), SUPPORT(4, 3).

They are much less local: a persuasive essay often has one main claim with premises spread throughout the text.

Spans as claim, premise or non-argumentative, and span pairs as SUPPORT, ATTACK or neither; some work detects argumentation schemes (argument from example, from cause to effect, from consequences).

Reciprocity (returning favors), social proof (following others), authority (influence of the powerful) and scarcity (valuing what is scarce), used to predict success of online requests, peer-to-peer lending and influence on social media.

An annotation scheme (Teufel et al.) assigning each sentence of a scientific paper one of 15 rhetorical tags reflecting the paper’s knowledge claim, e.g. AIM, OWN METHOD, OWN RESULTS, USE, GAP/WEAK, SUPPORT, ANTISUPPORT; used to train sentence classifiers.

Segmented Discourse Representation Theory (SDRT), the Linguistic Discourse Model, and graph representations (Wolf and Gibson) motivated by crossed bracketings that can’t form a tree.

Remove explicit connectives and re-predict them to get training data for implicit relations (Marcu and Echihabi); DisSent predicts explicit discourse markers between sentences, producing strong sentence representations.

Current discourse focus vs potential foci. Grosz earlier distinguished global focus (entities relevant to the whole discourse) and immediate focus (most central to an utterance).