From Narrative to Network: Auditing Computational Reconstruction of Social Structure in the Icelandic Family Sagas
SagaGraph project working paper. Version pinned to graph release njala-2026-09-07-46b49883, entity table njala-entities-v6, extraction rel-rules-v2, spaCy en_core_web_sm-3.8.16, gold annotation gold-v1.
Abstract
Can modern NLP reliably turn a medieval prose narrative into a social network, and if not exactly, how wrong is it, and where does that wrongness matter? We build a full extraction-to-analysis pipeline for the Icelandic family sagas — segmentation, gazetteer-based entity resolution, rule-based and coreference-aware relation extraction, and a large-language-model baseline — and evaluate every stage against a manually annotated gold subset of Njáls saga (10 chapters, 306 relations, one annotator). Entity linking against a curated 118-entity canonical table reaches 0.98 F1; pronoun coreference resolves 65% of annotated pronouns correctly; relation extraction tops out at 0.45 F1 (lenient, chapter-level) for a dependency-rule classifier with coreference, ahead of a Qwen3-8B structured-extraction baseline at 0.40 F1 by that measure, though the LLM has higher untyped pair-level recall. We then ask the question the field usually skips: given extraction is this imperfect, how much does it distort the network conclusions researchers might draw? A calibrated error-injection study finds centrality ranking comparatively robust (Spearman ρ ≈ 0.75 for betweenness at the measured relation-miss rate) but community assignment fragile (adjusted Rand index falls to 0.35 at the same rate, and to 0.10 at twice it). Applied across four sagas, the resulting networks show a strikingly consistent shape — density 0.030–0.037, negative degree assortativity throughout, modularity 0.47–0.55 — that we read as evidence of genre-level narrative structure rather than of any one story's idiosyncrasy. Structural centrality correlates strongly with raw narrative prominence (mentions, dialogue share, chapter presence) in every saga, which we argue is a substantially weaker finding than it first appears.
1. Introduction
The claim "the sagas encode a real social network, and we can recover it computationally" is easy to state and easy to illustrate with a pretty force-directed graph. It is much harder to defend, because every step between raw prose and a weighted edge list is a place where an automated pipeline can be confidently wrong. A character's patronymic gets attached to the wrong parent; a pronoun three sentences after a killing gets resolved to the victim instead of the killer; a large language model invents a friendship that isn't in the paragraph it was shown. None of these failures announce themselves in the final visualization. They just quietly move a node's centrality rank, merge two communities that should be distinct, or manufacture a feud that the text never describes.
This project treats that problem as the actual research question, not as a caveat appended after the fact. We do not ask "what does the network of Njáls saga look like" without first asking, and measuring, "how much do we trust the process that produced it." Section 9 (Error Propagation) is not a limitations paragraph; it is a designed experiment with its own baseline, its own perturbation model, and its own numbers, because the honest answer to "does structural centrality matter" depends entirely on how sensitive that centrality is to the kind of mistakes an NLP pipeline actually makes.
We deliberately scoped the project to prove this loop completely on one text — Njáls saga, in George Webbe Dasent's 1861 translation — before touching a second saga. Section 10 (Case Studies) and Section 8 (Network Analysis, cross-saga) report on three further sagas (Laxdæla, Grettis saga, Egils saga) extracted with the same pipeline but a lighter-touch, un-curated entity table; we are explicit throughout about which numbers rest on the curated, gold-checked text (Njáls saga) and which rest on the faster bootstrap (the other three), because collapsing that distinction would misrepresent the confidence the project actually has.
2. Historical and Literary Context
The Íslendingasögur ("sagas of Icelanders") are prose narratives composed in Iceland mostly in the thirteenth and fourteenth centuries, describing events set some centuries earlier, in the Icelandic Commonwealth of roughly 930–1262. They are neither straightforward history nor pure invention: they draw on oral tradition, genealogical record-keeping (a culture that names people by patronymic specifically so that lineage claims — to land, to legal standing, to blood-feud obligation — can be checked), and a strong literary convention of terse, understated narration in which social relationships are the primary carrier of plot. A saga's action is very often legally and kinship-structured violence: killings that trigger blood-price negotiations, alliances contracted through marriage, feuds that propagate along family lines exactly because Icelandic law made kinship the unit of liability and obligation. This is part of why the genre is an unusually good — and unusually demanding — test case for social network extraction: the text is dense with named individuals bound by exactly the relation types (kinship, marriage, alliance, feud) that network science already has vocabulary for, but the naming conventions (patronymics, nicknames, nested clauses of who is whose foster-son) are adversarial to any NLP pipeline built for modern journalistic English.
Njáls saga (Brennu-Njáls saga), our primary text, is the longest and most structurally complex of the family sagas: 158 chapters in Dasent's numbering, covering roughly three generations and culminating in the burning of Njal's household and the subsequent vengeance narrative that gives the saga its alternate title. Its scale and its high density of named, recurring characters (some sharing given names across generations, e.g. three distinct Gunnars) make it simultaneously the hardest test of entity resolution in the corpus and the most rewarding one to get right.
3. Related Work
Computational social network extraction from literary and historical narrative has an established track record — from network reconstructions of Shakespeare's plays and nineteenth-century novels built on co-occurrence, to more recent work using coreference and relation extraction on epic and legal-historical corpora. Two methodological lessons from that literature shaped this project directly. First, co-occurrence networks (an edge whenever two names appear in the same window of text) are cheap and always available, but conflate "these two characters are ever discussed near each other" with "these two characters have a social relationship" — a gap that widens exactly in texts, like the sagas, with dense genealogical scene-setting that mentions many kin in a single sentence without any of them interacting. We therefore treat co-occurrence strictly as a baseline (Sections 6–7), never as the semantic network itself. Second, work that evaluates extracted literary networks against network-science ground truth almost never asks how sensitive the downstream metrics are to the measured extraction error rate; sensitivity analyses in the digital-humanities network literature typically stop at reporting extraction precision/recall. Section 9 of this project is an attempt to close exactly that gap, using the project's own measured per-type recall and precision as the perturbation parameters rather than an arbitrary noise level.
For the sagas specifically, our contribution is not a new close reading (we make none) but an audited, reproducible measurement of what a fairly standard modern toolchain (spaCy dependency parses, a hand-built entity gazetteer, an open-weights LLM behind a chat API) actually recovers, and how far that recovery can be trusted for network-level claims.
4. Corpus
Four public-domain English translations, one per saga, never mixed within a saga:
| Saga | Translator | Edition | Chapters | Words (approx.) |
|---|---|---|---|---|
| Njáls saga | Sir George Webbe Dasent (1861) | Project Gutenberg #17919 | 158 | 125,100 |
| Laxdæla saga | Muriel A. C. Press (1899) | Project Gutenberg #17803 | 78 | 71,800 |
| Grettis saga | translator not stated in the eBook; text matches G. A. Hight's 1914 Everyman translation (attribution unverified in-file) | Project Gutenberg #347 | 93 | 69,800 |
| Egils saga | W. C. Green (1893) | Icelandic Saga Database (sagadb.org) transcription | 92 | 71,300 |
Every source file's SHA-256 is recorded in corpus/manifest.yaml, and the pipeline refuses to build a release if the on-disk file no longer matches (sagagraph.corpus.verify_source). This matters because a silently re-downloaded or edited source text would invalidate every downstream number without any visible symptom. Segmentation is chapter → paragraph → sentence, using spaCy's sentence boundary detector; each sentence receives a permanent id of the form <saga>:c<chapter>:p<paragraph>:s<index> that every mention, relation, and evidence record traces back to. Each translation's chapter-heading format, footnote placement, and title-line convention differ (Dasent centers a following all-caps title line; Press interleaves [Sidenote: ...] markers mid-paragraph; Hight/Grettir carries per-chapter endnote blocks; the sagadb.org HTML uses <h2> headings) and required saga-specific parsing rules, documented in the manifest.
We are explicit that translator choice is a confound we did not control for: Dasent's 1861 prose, Press's 1899 prose, and the sagadb transcription of Green's 1893 prose have different sentence lengths, different anglicization conventions for names (Dasent's "Hauskuld" vs. Press's "Hoskuld" for what is etymologically the same name-type, though the specific characters are different people), and different rates of the archaic constructions ("X, Y's son") the relation extractor's kinship rules were tuned against. Cross-saga comparisons (Sections 8 and 10) should be read as comparing four pipeline outputs on four different English prose styles, not as comparing four sagas in a translation-neutral space.
5. Entity Resolution
This is, honestly, the hardest engineering problem in the project, harder than relation extraction. Saga characters are referred to by bare given name, by patronymic ("Hauskuld Njal's son"), by epithet ("Njal of Bergthorsknoll," "the Whiteness-priest"), and the same bare given name frequently denotes two, three, or more distinct people in the same text (Njáls saga alone has, per our canonical table, multiple characters answering to "Gunnar," "Thorstein," "Hall," and "Kol," among others).
For Njáls saga we built a hand-curated canonical entity table: 118 curated entities (people, not places), each with a stable id, a canonical name, an alias list, patronymic and epithet fields, gender, known parent links, and — critically, for disambiguating a shared given name — a curated active chapter window used as a prior, never as an overriding fact. The mention resolver applies a four-rung ladder for a bare given name shared by several entities: (1) the surface form itself carries a disambiguating patronymic or epithet, resolved with confidence 1.0; (2) a qualified mention of exactly one candidate occurred within the previous 60 sentences (confidence 0.85); (3) exactly one candidate's curated active-chapter window contains the current chapter (confidence 0.7); (4) failing both, the candidate with the most recent qualified mention anywhere in the text so far, or the nearest active-window boundary, flagged low-confidence (0.4–0.5) for the review queue. Every one of these fallback resolutions is visible and correctable in the annotation review queue (/extraction), never silently merged.
On the gold-annotated chapters, this resolver achieves 0.9666 precision / 0.9886 recall (F1 0.9775) for entity linking (surface form → correct canonical id) and 0.9866 / 0.9772 (F1 0.9819) for bare mention detection against the gazetteer. For comparison, spaCy's generic en_core_web_sm PERSON-entity recognizer — the "just use off-the-shelf NER" alternative — detects only 53.5% of gold mentions (F1 0.68) on this translation's names, most of which (Hauskuld, Skarphedinn, Bergthorsknoll-as-a-personal-epithet) are far outside its training distribution. The gazetteer built from the curated table, not the generic NER model, is what makes entity resolution viable here at all; this is itself a finding, not just an implementation note — off-the-shelf NER is not a substitute for a domain gazetteer on this genre.
For the three expansion sagas we did not repeat full manual curation (see Section 1 on scope). Instead sagagraph.bootstrap_entities mines the same discovery statistics (per-name frequency, patronymic and epithet co-occurrence counts) used to seed the Njáls saga table and mechanically emits one entity per sufficiently frequent (given name, qualifier) pair, marking each status: auto-bootstrap and marking any residual, un-covered bare-name frequency as an explicit -unresolved entity flagged for a curator. These three tables have not been gold-checked and their entity-linking accuracy is not independently measured; every number computed from them (Sections 8, 10) should be read as provisional until a curator reviews the bootstrap output, which the annotation review queue is built to support.
Gender, needed for pronoun coreference, is asserted for curated entities and guessed for bootstrap entities from Old Norse name morphology (a fixed list of strong feminine suffixes — -dis, -gerd(r), -hild(a/r), -unn, -veig — plus a weaker default for terminal -a). This is a documented heuristic, not a fact: names it cannot classify (e.g. "Njal," "Egil," "Kjartan" pattern outside the suffix list) simply get no gender and therefore no pronoun coreference, a recall gap rather than a fabricated attribute.
5.1 Cross-saga entity resolution
The family sagas share a common narrative world, and a handful of figures recur across several of them. We built a small, deliberately conservative register (data/entities/cross_saga.yaml) linking saga-scoped entity ids believed to denote the same person — currently four persons: Hoskuld Dala-Koll's son and his half-brother Hrut (Njáls saga ↔ Laxdæla saga, on matching genealogy independently narrated in both texts — both name the same father, the same mother, and the same half-sibling relationship under corresponding names), Snorri the Priest/Godi (linked across all three sagas that mention him, on a distinctive epithet and consistent chronological placement), and Gudmund the Mighty (Njáls saga ↔ Grettis saga). Every link was checked against the texts actually in this corpus, not asserted from outside knowledge of saga genealogy, and each carries a confidence rating (high for the genealogically-corroborated Hoskuld/Hrut link, medium for the epithet-only Snorri and Gudmund links) plus an explicit caveat about what the link does not establish. A cross-saga link never merges the underlying entities or their edges; each saga keeps its own independent network position for "its" copy of the person.
Building this register caught a real entity-resolution error worth reporting as its own small methodological result: verifying the Gudmund link required checking every occurrence of "Gudmund" in Grettis saga, which revealed that the bootstrapped entity table had silently merged "Gudmund the Mighty" (the intended, cross-saga-relevant chieftain) together with an unrelated minor figure, "Gudmund the son of Solmund" of Asbjarnarnes, under one bare-name bootstrap entity. We split the entity, promoted both halves to status: curated, and only then completed the link. This is the second time in this project that pursuing a specific downstream claim (here, cross-saga identity; earlier, a review-queue anomaly during entity curation on Njáls saga) surfaced a merge error a purely internal consistency check would not have — evidence that comparing an entity table against an independent signal, not just against itself, is a genuinely useful curation practice.
6. Relation Extraction
Four independent methods produce edges in the same schema (type, subtype, source, target, sentence id, evidence text, confidence, extracting method, verification status), so they can be compared directly and never silently blended without record of which method contributed which edge:
- Sentence co-occurrence — every pair of distinct resolved entities mentioned in the same sentence becomes an undirected, untyped edge. A pure structural baseline; kept as its own network layer (
copresence), never merged into the semantic layers. - Dependency rules — a hand-written classifier over spaCy dependency parses, matching verb-argument and noun-possessor patterns for each of seven relation types (kinship, marriage, friendship, conversation, conflict, killing, support), including dialogue-internal patterns ("I slew him," says X), causative constructions ("she had Soti slain"), and speaker/addressee attribution for reported speech.
- Dependency rules + coreference — the same classifier run over mentions and resolved pronouns, using a same-clause / same-sentence / previous-sentence rule-based pronoun resolver (gender-matched, quote-aware — pronouns inside direct speech are not resolved against the surrounding narration).
- LLM structured extraction — a locally hosted Qwen3-8B model, prompted per paragraph with the resolved character list and asked to return typed relations as JSON, cached by a hash of (model, prompt version, input) so every run is reproducible and free to repeat.
One recurring implementation issue is worth reporting because it is a general hazard for any dependency-rule extractor applied to this genre: en_core_web_sm frequently mislabels an unfamiliar sentence-initial proper noun as the aux or dep child of the main verb rather than as its nsubj ("Hauskuld had a daughter" parses with "Hauskuld" attached as an auxiliary of "had"), because the tagger has never seen these tokens and falls back on a default reading keyed to sentence position. We added an explicit, documented workaround (treat such a mislabeled pre-verbal token bearing a resolved mention as the subject when the verb otherwise has none) after noticing it suppressed several gold-annotated kinship relations entirely; this single fix raised rules+coreference lenient F1 from 0.434 to 0.446 on the gold set. We flag it here because it is exactly the kind of silent, non-obvious failure mode that this project's overall thesis is about: a generic parser trained on modern text will systematically misparse a specific, learnable class of constructions in an older or more formal register, and that misparsing does not look like an error from the output alone.
Writing unit tests for the rule extractor on hand-written minimal sentences (tests/test_relations.py), rather than only ever running it over full saga chapters, surfaced two further instances of the same class of failure, neither visible in the gold-chapter evaluation numbers because neither happened to recur inside the ten annotated chapters. First, a bare, unmodified "Name was slain." is parsed as a copula-plus-adjectival-complement construction ("slain" attached to "was" as acomp, with no syntactic child carrying the subject) rather than as a passive verb — but the identical sentence with any trailing word ("...was slain there.") parses correctly as a passive with nsubjpass. This is a real, general problem for this genre specifically, because a bare terse death announcement in exactly this shape ("Njal was slain," "Otkell was slain") is Dasent's own stock phrasing throughout the text, not a synthetic edge case. Second, in a causative construction ("she had Soti slain"), the same participle is sometimes tagged as a bare infinitive rather than a past participle, which cascades into the rule-based lemmatizer failing to reduce "slain" back to the verb lemma "slay" at all — so a lemma-based rule check silently never fires even though the surface word is unambiguous. Both are fixed with the same kind of targeted, narrowly-scoped, commented workaround as the subject-attachment fix above (a copula-fallback for the first, a surface-form lemma override for the second), and both are covered by regression tests going forward. Applied across the full corpus, the two fixes recovered one additional gold-adjacent kinship-style edge in Njáls saga and, more substantially, three to four additional death events per saga in the other three sagas' death registries — small in aggregate, but we take the general lesson seriously: a dependency-rule extractor's blind spots are not randomly distributed, and the cheapest way to find more of them was writing tests that isolate one construction at a time rather than reading pipeline output.
7. Evaluation
The gold-standard sample covers ten chapters of Njáls saga (1, 2, 17, 36, 45, 54, 77, 92, 111, 129 — chosen to span the opening genealogical material, an early feud, the Hallgerda–Bergthora servant killings, Gunnar's death, and the Burning), annotated by hand by the project author against a written guideline (data/gold/GUIDELINES.md): every proper-name mention, every third-person-singular pronoun in a documented sample, and every kinship/marriage/friendship/conversation/conflict/killing/support relation the text explicitly or implicitly licenses, each tagged explicit (both arguments named in the sentence) or implicit (at least one argument recovered from context). This yields 306 gold relations, 132 of them explicit.
Entity and coreference results are reported in Section 5 (linking) and above (0.6544 accuracy for pronoun coreference over 272 annotated pronouns; 0.7355 restricted to pronouns the system attempted to resolve at all, i.e. had a gender-known candidate).
Relation extraction, lenient scoring (matching on chapter, type, and unordered character pair — crediting the extractor for finding that two people interact of a given type in a chapter, not the exact sentence):
| Method | Relations found | Precision | Recall | F1 | Pair-level F1 (untyped) |
|---|---|---|---|---|---|
| Co-occurrence (baseline) | 162 | 0.469 | 0.753 | — | 0.578 |
| Dependency rules | 44 | 0.854 | 0.184 | 0.303 | 0.400 |
| Rules + coreference | 85 | 0.829 | 0.305 | 0.446 | 0.558 |
| LLM (Qwen3-8B) | 169 | 0.462 | 0.347 | 0.396 | 0.589 |
Reading this table requires care about what each column claims. The rule-based extractor with coreference is the most typed-precise method by a wide margin (0.829) — when it says "kinship," it is very often right — but recovers under a third of gold relations. The LLM recovers relations the rule set structurally cannot reach (marriage stated as a bare noun phrase, distant paraphrase) and wins on untyped pair-level F1, but at much lower typed precision (0.462), because it both proposes relations the sentence does not actually license (hallucination in the literal sense) and disagrees with the gold typology on genuinely ambiguous cases. Coreference is unambiguously worth having: adding it to the rule-based extractor very nearly doubles typed relation recall (0.184 → 0.305) for a small precision cost.
Per-type F1, lenient (Table, RQ2):
| Type | Rules | Rules+coref | LLM |
|---|---|---|---|
| kinship | 0.476 | 0.731 | 0.370 |
| marriage | 0.000 | 0.000 | 0.333 |
| friendship | 0.500 | 0.444 | 0.222 |
| conversation | 0.344 | 0.457 | 0.463 |
| conflict | 0.235 | 0.421 | 0.468 |
| killing | 0.222 | 0.316 | 0.467 |
| support | 0.069 | 0.129 | 0.171 |
Kinship is the rule-based extractor's clear strength (patronymic parsing and possessive kin-noun patterns are both syntactically regular and extremely common in this genre); killing and conflict are where the LLM's world knowledge and paraphrase tolerance actually pay off, since violent events are described with enormous lexical variety across 150 chapters and no fixed rule set will enumerate them all. Marriage is the hardest relation type for every method — rare in the gold sample (a handful of instances) and frequently stated as a static noun phrase ("his wife X") rather than an event, which our current rule set under-covers. Support is uniformly weak: it is the vaguest of the seven categories and the gold annotator's own judgment calls about what counts as "support" versus incidental narrative color were the least consistent of any type (a limitation of the single-annotator gold set, discussed in Section 12).
Method agreement is low between the LLM and either rule-based variant (Jaccard 0.085–0.090) and high between the two rule-based variants (0.608, as expected — coreference is additive, not contradictory). The rule-based extractor and the LLM are, in a real sense, measuring different things from the same text; a reader should not assume that because both methods "extract relations," their outputs are interchangeable or that agreement between them is a strong validity signal. It is not tested here often enough to be one.
8. Network Construction
Edges are aggregated from method="rules+coref" relation instances (the project's designated primary semantic layer, chosen for its lenient-F1 lead among typed methods) by (source, target, type), retaining every contributing sentence as evidence — the non-negotiable provenance requirement: every edge in the shipped graph traces to specific sentence ids and their text, inspectable by clicking the edge in the interactive viewer. Undirected types (kinship, marriage, friendship) are canonicalized by sorted id pair; directed types (conversation, conflict, killing, support) keep direction. The co-occurrence layer is built and shipped separately and is never merged into the semantic graph's node or edge counts reported anywhere in this paper.
For Njáls saga this yields a semantic graph of 160 nodes, 382 edges, density 0.0300, average clustering 0.235, one connected giant component, average shortest-path length 3.31, diameter 8. Ten Louvain communities, modularity 0.553. A signed-graph structural-balance check (kinship/marriage/friendship/support = positive, conflict/killing/feud = negative; conversation and co-presence excluded as unsigned) over the 49 closed triangles in the graph finds 35 balanced (71.4%) — a majority, consistent with balance theory's prediction for a social network with real enemies, but far from unanimous, and we do not read the unbalanced 30% as extraction noise without direct evidence (see Section 11 case studies).
9. Error Propagation (RQ3)
This is the section we consider the project's actual contribution, more than the network itself. The question: given the measured per-type recall and precision from Section 7, how much do the resulting errors move the metrics anyone would actually use to draw a conclusion?
Method: take the shipped Njáls saga semantic graph as ground truth. For each of four perturbation kinds, generate 40 randomized variants and compare each variant's centralities and Louvain communities against the unperturbed graph (Spearman ρ of centrality values; Jaccard-style top-10 overlap; adjusted Rand index and normalized mutual information for community partitions):
- miss — for each edge of type t, drop it with probability
(1 − measured_recall_t) × multiplier(multiplier 0.5×, 1×, 2× the measured rate); - spurious — add
edges × (1 − precision)/precision × multiplierrandom edges, attached preferentially to already high-degree nodes (matching the observed tendency of both rule and LLM false positives to land on frequently-mentioned characters); - merge — collapse k pairs of entities that share a given name in the curated table (simulating an entity-resolution failure that the disambiguation ladder in Section 5 was specifically built to avoid);
- split — split k high-degree nodes into two, randomly partitioning their edges (simulating a missed alias link).
| Perturbation | Setting | ρ betweenness | ρ PageRank | Top-10 betweenness overlap | Community NMI | Community ARI |
|---|---|---|---|---|---|---|
| miss | ×0.5 measured | 0.880 | 0.941 | 84.8% | 0.728 | 0.570 |
| miss | ×1 measured | 0.751 | 0.827 | 76.0% | 0.620 | 0.350 |
| miss | ×2 measured | 0.545 | 0.595 | 61.8% | 0.580 | 0.098 |
| spurious | ×0.5 measured | 0.961 | 0.987 | 96.5% | 0.870 | 0.793 |
| spurious | ×1 measured | 0.931 | 0.977 | 91.0% | 0.815 | 0.719 |
| spurious | ×2 measured | 0.894 | 0.960 | 88.3% | 0.754 | 0.646 |
| merge (k=2/5/10) | — | 0.99 / 0.98 / 0.96 | 1.00 / 0.99 / 0.97 | 95.5 / 92.5 / 83.8% | 0.91 / 0.87 / 0.80 | 0.86 / 0.80 / 0.69 |
| split (k=2/5/10) | — | 0.98 / 0.97 / 0.94 | 0.99 / 0.98 / 0.97 | 95.0 / 89.3 / 78.0% | 0.91 / 0.87 / 0.79 | 0.83 / 0.77 / 0.66 |
Three findings we consider load-bearing for how anyone should use this project's networks:
- Missed edges hurt far more than spurious ones, at the rates we actually measured. At the measured 1× rate (recall ranging 7–61% by type, see Section 7), dropping edges at that rate degrades betweenness rank correlation to 0.751 and community ARI to 0.350; adding spurious edges at the correspondingly measured false-positive rate barely moves anything (ρ 0.931, ARI 0.719). This makes sense structurally — a missed edge can disconnect a bridge, while an extra random edge mostly reinforces existing dense regions — but it means the honest headline risk of this pipeline is recall failure, not the hallucination failure that gets more attention when people worry about LLM extraction specifically.
- Rank-based centrality is comparatively robust; community assignment is not, at any measured error rate. Even at the mild 0.5× miss rate, community ARI is already down to 0.570 (i.e., a partition only moderately more similar to the truth than chance-adjusted agreement suggests) while betweenness rank correlation stays at 0.880. A claim of the form "these two characters are the most central" survives realistic extraction error; a claim of the form "these characters form a single faction against those characters" (a Louvain community claim) does not, and should be treated as substantially weaker evidence.
- Entity-resolution failures (merge/split) are, at the scale we tested, less damaging than relation-extraction misses of comparable count. Ten merged or split high-degree entities move ARI by roughly 0.30, comparable to what a much smaller number of missed edges does. This is a point in favor of the disproportionate curation effort Section 5 spent on entity resolution: a well-resolved entity table is not just "necessary hygiene," it is measurably cheaper to get slightly wrong than the relation layer is.
10. Case Studies
The Hallgerda–Bergthora servant killings (chapters 35–45). This sequence — a chain of tit-for-tat killings between the households of Gunnar and Njal, carried out through their wives' hired men rather than the principals themselves, and resolved each time by Gunnar and Njal personally paying compensation and explicitly renewing their own friendship — is exactly the kind of structure this project's feud_chains detector was built to surface (retaliation: a later killing whose victim is the earlier killer, or their kin/ally, committed by kin/ally of the earlier victim). It did not fire here, because the killers on each side (household servants: Kol, Svart, Sigmund and Skiolld, Thord Freedmanson) are not linked by our automatically extracted kinship or alliance edges to the household heads whose feud the killings actually serve — the text encodes that connection through employment and loyalty, a relation type this pipeline does not currently model. This is a genuine limitation, not an extraction bug: our feud detector under-counts real feud propagation whenever the connecting relation is patron-servant rather than kin-or-declared-ally, and Njáls saga uses exactly that structure repeatedly. We record this explicitly (surfaced in the /timeline page) rather than loosen the retaliation rule until it happens to catch this case, which would risk manufacturing false positives elsewhere.
Gunnar of Lithend as the highest-PageRank node (0.0609) despite dying at chapter 77 of 158. Betweenness centrality (0.621, also the graph's highest) is a better predictor of "narratively load-bearing" than PageRank here: Gunnar's death is a hinge point connecting the saga's first half (his own feuds) to its second (Njal's household, largely uninvolved with Gunnar directly, and the Burning). A purely cumulative, death-blind network model (as ours is by default — see metrics.json "semantic" summary) will keep crediting a dead character with structural importance for the rest of the text; the cumulative_alive series (which removes a node once its death chapter is reached) is provided specifically so a researcher does not misread "high total centrality" as "important for the whole plot" when it may really mean "important, then dead, then residually connected by edges to characters who are still discussing him."
Structural balance's unbalanced 30%. The three most common unbalanced-triangle configurations (++-: 13, +--: 11, ---: 3) recur around exactly the characters most central to conflicting loyalties — Hallgerda (married into Njal's ally Gunnar's household while remaining hostile to Bergthora) and Mord Valgard's son (kin to Gunnar, yet the eventual architect of the plot against Njal's sons). Balance theory predicts these configurations should be unstable and resolve toward balance; the saga's plot is arguably the process of several of them doing exactly that, violently. We flag this as a genuinely interesting structural-balance finding rather than an extraction artifact, though we note the caveat from Section 9: with community-level ARI this fragile to plausible extraction error, an individual triangle's balance label is a much lower-variance, more trustworthy unit of analysis than a Louvain community assignment would be.
11. Network Analysis
Across all four sagas (Section 8's numbers for Njáls saga; comparable figures on the same metrics for the three bootstrap sagas, computed identically but not gold-checked):
| Saga | Nodes | Edges | Density | Avg. clustering | Degree assortativity | Modularity | Balance ratio |
|---|---|---|---|---|---|---|---|
| Njáls saga | 160 | 382 | 0.0300 | 0.235 | −0.152 | 0.553 | 0.714 |
| Laxdæla saga | 121 | 227 | 0.0313 | 0.211 | −0.159 | 0.495 | 0.938 |
| Grettis saga | 119 | 213 | 0.0303 | 0.154 | −0.165 | 0.466 | 0.750 |
| Egils saga | 95 | 165 | 0.0370 | 0.148 | −0.127 | 0.524 | 0.900 |
Two things recur strongly enough across four different texts, four different translators, and (for three of the four) an un-curated entity table that we treat them as tentative genre-level findings rather than coincidence: density clusters tightly around 0.03–0.037 regardless of saga length or cast size, and degree assortativity is consistently and comparably negative (−0.13 to −0.16) — every saga's social network is hub-and-spoke, built around a handful of highly connected principal characters linked to many peripherally-connected minor ones, never a dense clique of hubs. This is a plausible reflection of how saga narrative actually works (a small cast of protagonists whose feuds and marriages the story follows, surrounded by a much larger cast of named-once kinsmen, neighbors, and witnesses), but we hold it as tentative given that three of the four measurements rest on the un-curated bootstrap entity tables. Structural balance ratio varies far more (0.71–0.93) and we do not see an obvious genre-level pattern in it; it may simply track how much interpersonal hostility each saga's plot actually contains.
Structural centrality correlates strongly with raw narrative-prominence measures (mention count, dialogue-turn count, chapter-presence count, event-involvement count) in every saga — for Njáls saga, mentions correlate with degree at Spearman ρ = 0.787, with PageRank at ρ = 0.793, with betweenness at a still-substantial ρ = 0.588 (n = 158, all p ≪ 0.001); the pattern repeats in the other three sagas (mentions~degree ρ between 0.647 and 0.787). We address in Section 13 why we think this correlation, though real, is a considerably weaker finding about "does the network capture something beyond raw textual salience" than it might look.
12. Limitations
Single annotator, single pass. The gold standard (Section 7) was produced by one person against a written guideline, with no second annotator and no formal inter-annotator agreement measurement. The support and friendship relation types in particular involve judgment calls (does "he was a great friend of X" count if it appears in a single throwaway sentence about a character who never reappears?) that a second annotator might resolve differently; the low F1 scores reported for those types partly reflect real extraction difficulty and partly reflect an inherently under-specified category boundary that this project has not independently validated.
Translation effects are unmeasured, not merely acknowledged. We use one translator per saga and never mix translations of the same saga, but we make no attempt to hold translation style constant across sagas, and the cross-saga comparison in Section 11 is consequently a comparison of (saga × translator × pipeline) triples, not of sagas in isolation. A different translator of Njáls saga could plausibly move the rule-based extractor's recall by using different verb choices for the same underlying events (Dasent's "gave him his death-blow" versus a hypothetical modern translation's "killed him outright" hit different lexicon entries in our KILL/BURN word lists).
Extraction errors, not just literary interpretation, drive some network features. Section 9 exists precisely because we do not want a reader to treat any single centrality value, and especially not any single community assignment, as a fact about the saga rather than as a measurement with a quantified, sometimes large, error bar.
Network reductionism. A weighted graph of typed edges is a genuine simplification of a saga's social world: it discards register (a threat is not equivalent to its execution, though both may register as "conflict" edges of different subtype), discards the asymmetry of narrated versus reported time, and discards almost all of the saga's actual literary texture (verse, formulaic description, authorial commentary) in favor of what can be reduced to a (source, target, type) triple. We consider the evidence-linking feature (every edge traceable to its sentence) a partial mitigation — a reader is never more than one click from the actual prose — but it does not eliminate the underlying reduction.
Arbitrary edge definitions. The line between "conversation" and "conflict" (an angry exchange of words), and between "conflict" and "killing" (a wound description that turns out, several sentences later, to be fatal), required specific, somewhat arbitrary rule thresholds (see sagagraph/relations.py, DEATH_WORDS and the gave-blow heuristics) that a different designer would have drawn differently. We documented every threshold in code rather than leaving it implicit, so at least the arbitrariness is inspectable and reproducible rather than hidden.
Literary versus historical interpretation. Nothing in this project's networks is evidence about the historical Icelandic Commonwealth's actual social structure; it is evidence about the social structure as narrated, filtered through medieval compositional convention, subsequent scribal transmission, and a nineteenth-century English translator's lexical choices. We use "social network" throughout in the narratological sense and flag this explicitly because the genre's own scholarly reception has a real history of historicist over-reading that this project does not intend to repeat.
Centrality overinterpretation. Section 13 addresses this directly: raw prominence and structural centrality are correlated closely enough here that "the network reveals hidden importance beyond what a simple mention count already told you" is, at minimum, not demonstrated by this data.
13. Reviewer 2
We use this section, in the spirit the instructions for this project requested, to state the strongest objections to this work's own framing and answer them directly rather than deferring them to a general limitations paragraph.
"Your centrality-prominence correlation just proves your network is measuring the same thing as a word count, so what did the network analysis add?" This is largely correct, and we do not think the honest answer is to argue it away. A Spearman ρ of 0.79 between mention count and PageRank for a graph constructed from mentions is close to a structural near-tautology (an entity mentioned more often has more chances to be linked to more co-occurring or interacting characters). Where we think the network genuinely adds information beyond a mention count is in the relative positions it reveals when the correlation breaks down — Gunnar's high betweenness despite dying at the story's midpoint (Section 10), or a character with moderate mentions but unusually high betweenness because they structurally bridge two otherwise-separate plot threads. We would recommend any future user of this dataset treat "does centrality correlate with mentions" as a sanity check the pipeline should pass, not as evidence of a substantive finding, and reserve substantive claims for cases where the two measures diverge.
"Your relation extraction F1 tops out at 0.45 — isn't that too weak a foundation for any of the network claims above it?" We built Section 9 specifically because we think this objection, stated as "the extraction is imperfect, therefore the network claims are worthless," is too strong, and stated as "the extraction is imperfect, therefore some network claims are much better supported than others," is exactly right. At the measured error rate, betweenness- and PageRank-based claims about the handful of most central characters are quantifiably robust (ρ > 0.75, top-10 overlap > 75%); community-based claims are quantifiably fragile (ARI 0.35). We are explicit, on every page of the site that surfaces community assignments, that they carry this documented fragility.
"Isn't the LLM baseline just badly prompted — a stronger model or better prompt would close the gap?" Almost certainly yes, to some degree; we used a single prompt version (relation_extraction_v1), a single open-weights 8B model, and did not iterate the prompt against the gold set (which would have invalidated the gold set as a held-out evaluation). We report the LLM's actual measured numbers, not a ceiling estimate, and we think the more durable finding is not "the LLM scored 0.396" in isolation but that the LLM and the rule-based extractor agree on fewer than one edge in ten (Jaccard 0.085–0.090) — that structural disagreement, not either method's absolute score, is the strongest evidence that "extract the social network with an LLM" is not yet a solved, well-posed task with one right answer that different reasonable methods converge on.
"Structural balance and community detection are classic over-interpretable network-science tools — why trust Louvain modularity on a 158-node graph built from an 84%-precision extractor?" We do not, unreservedly; see Section 9 and the fragility numbers above, which we consider the honest answer to this objection rather than a rebuttal of it.
14. Conclusion
A modern NLP pipeline can turn a nineteenth-century English translation of a thirteenth-century Icelandic saga into a social network with genuinely useful fidelity for some questions — entity resolution against a curated gazetteer is very accurate (F1 0.98), and the resulting network's centrality rankings survive realistic extraction error well enough to support claims about who structurally matters most. It cannot yet do this with comparable fidelity for finer-grained relation typing (F1 0.40–0.45 even with coreference) or for community-level structural claims, which the same error rates make substantially less trustworthy than they visually appear in a force-directed layout. Applied across four sagas, the pipeline recovers a consistent, plausible network shape — sparse, hub-and-spoke, moderately modular — that we read as tentative evidence of shared narrative structure across the genre, not as a settled historical or literary finding. The single methodological point we would want a future project in this space to take from ours is not any specific number in the tables above, but the discipline the tables were built to enforce: measure the error, then measure what the error does to the conclusion you actually wanted to draw, before drawing it.