Managing structured critical data with Lua

Unofficial ConTeXt Wiki mirror

Last modified: 2026-08-29

Work in progress.

This orientation page and the related guides are currently being drafted and reviewed. Please feel free to edit, correct, or improve them.

Critical apparatus guides: ← Guide 5 — Typesetting an original text and translation in parallel · Orientation · Glossary · Guide 6 of 6 — Final guide


The earlier guides attach formatted apparatus material directly to the text. That approach is clear and effective for a compact edition, but the internal parts of an entry are no longer independently available after they have been combined into a string.

For example: 

Reason] Judgment B D; om. E

contains several facts:

Field Value
Textual unit reason-01
Position 1:1
Layer textual
Editorial operation reading
Lemma Reason
Reading Judgment
Witnesses ms-b,ms-d

When those facts are stored separately, the same collection can be sorted, filtered, validated, and rendered in several forms.

The progression is:

Stage Editorial problem Result
1 A formatted string cannot be queried reliably Each entry becomes a structured record
2 Records must be entered from ConTeXt A stable registration command is exposed
3 Registration order may not equal textual order Records are copied, sorted, and filtered
4 Identifiers and required fields may be wrong Controlled vocabularies and validation rules are applied
5 Editors and readers need different outputs Several views and modes are generated from one registry
6 Multilingual editions need additional distinctions Language, target, and lemma level extend the model

Division of responsibilities. Lua stores, checks, selects, and reorganizes the editorial records. ConTeXt remains responsible for typography, page layout, notes, tables, and publication output.

1. Decide when a Lua registry is justified

Lua is useful when the project must:

Direct ConTeXt commands remain preferable when the apparatus is short, stable, and intended for one principal output.

Editorial model Suitable when
Direct ConTeXt commands The apparatus is compact and rendered mainly in one form
Lua registry Records must be queried, reordered, checked, or rendered in several forms

Editorial decision. Lua should be introduced because the data need processing—not merely because the edition is technically sophisticated.

2. Design one critical record

The basic model uses nine public fields:

Field Purpose Example
id Stable record identifier reason-reading
unit Stable textual-unit identifier reason-01
position Sorting key in the form unitorder:entryorder 1:2
layer Editorial layer textual
kind Editorial operation reading
lemma Edited text to which the record refers Reason
reading Alternative text or explanatory content Judgment
witnesses Comma-separated witness identifiers ms-b,ms-d
responsibility Editor or authority responsible for a decision Smith

Lua parses: 

1:2

into: 

unitorder  = 1
entryorder = 2

Not every field is required for every operation. An omission has no alternative reading, while a conjecture normally requires a responsible editor.

Data-model principle. Store editorial facts, not final punctuation. The renderer decides later whether an omission appears as om., as a complete sentence, or in another language.

3. Create a project namespace

Avoid loose global variables. Create one project namespace: 

\startluacode

userdata.critical = userdata.critical or { }

local critical = userdata.critical

critical.entries = critical.entries or { }

\stopluacode

The registry is: 

critical.entries

The assignment: 

local critical = userdata.critical

creates only a shorter local reference. It does not create another registry.

4. Register one record from ConTeXt

First parse the position: 

local function parse_position(position)
    local unitorder, entryorder =
        string.match(position or "", "^(%d+):(%d+)$")

    return tonumber(unitorder) or 0,
           tonumber(entryorder) or 0
end

Then define the registration function: 

function critical.register(
    id,
    unit,
    position,
    layer,
    kind,
    lemma,
    reading,
    witnesses,
    responsibility
)
    local unitorder, entryorder =
        parse_position(position)

    critical.entries[#critical.entries + 1] = {
        id             = id,
        unit           = unit,
        unitorder      = unitorder,
        entryorder     = entryorder,
        layer          = layer,
        kind           = kind,
        lemma          = lemma,
        reading        = reading,
        witnesses      = witnesses,
        responsibility = responsibility,
    }
end

Expose it to ConTeXt: 

interfaces.implement {
    name      = "registercriticalentry",
    public    = true,
    protected = true,
    arguments = {
        "string", "string", "string",
        "string", "string", "string",
        "string", "string", "string",
    },
    actions   = critical.register,
}

Use the generated command: 

\registercriticalentry
  {reason-reading}
  {reason-01}
  {1:1}
  {textual}
  {reading}
  {Reason}
  {Judgment}
  {ms-b,ms-d}
  {}

Registration is not rendering. The command adds a record to the Lua registry. Nothing becomes visible until a separate placement command selects and renders records.

5. Render one structured record

Declare the printed sigla separately: 

critical.witness_sigla = {
    ["ms-a"] = "A",
    ["ms-b"] = "B",
    ["ms-c"] = "C",
    ["ms-d"] = "D",
    ["ms-e"] = "E",
}

Split a comma-separated list: 

local function split_identifiers(list)
    local result = { }

    for identifier in string.gmatch(list or "", "[^,%s]+") do
        result[#result + 1] = identifier
    end

    return result
end

Resolve the sigla: 

local function format_witnesses(list)
    local result = { }

    for _, identifier in ipairs(split_identifiers(list)) do
        result[#result + 1] =
            critical.witness_sigla[identifier] or
            ("[" .. identifier .. "]")
    end

    return table.concat(result, " ")
end

An unknown identifier remains visible: 

[ms-x]

Define a renderer: 

local function formatted_entry(entry)
    local witnesses = format_witnesses(entry.witnesses)

    if entry.kind == "omission" then
        return entry.lemma .. "] om. " .. witnesses

    elseif entry.kind == "addition" then
        return entry.lemma .. "] add. " ..
               entry.reading .. " " .. witnesses

    elseif entry.kind == "conjecture" then
        return entry.lemma .. "] conj. " ..
               entry.reading .. " " ..
               entry.responsibility

    else
        return entry.lemma .. "] " ..
               entry.reading .. " " .. witnesses
    end
end

The punctuation now belongs to the renderer rather than to the stored record.

6. Compile the minimal registry MWE

Expected contents: 

Reason] Judgment B D
Reason] om. E
understanding] add. and directs the will C

7. Sort without destroying registration order

Lua's table.sort changes the table it receives. Copy the registry before sorting: 

local function copy_entries(entries)
    local result = { }

    for index, entry in ipairs(entries) do
        result[index] = entry
    end

    return result
end

Sort by textual position: 

local function sort_entries(entries)
    table.sort(entries, function(a, b)
        if a.unitorder == b.unitorder then
            if a.entryorder == b.entryorder then
                return a.id < b.id
            end

            return a.entryorder < b.entryorder
        end

        return a.unitorder < b.unitorder
    end)

    return entries
end

Use: 

local selected = sort_entries(
    copy_entries(critical.entries)
)

Why copy first? Preserving registration order allows another report to show how the data were entered while the reader apparatus uses textual order.

8. Filter records by one criterion

A witness filter asks whether an identifier occurs in the witness list: 

local function has_witness(entry, wanted)
    for _, identifier in ipairs(
        split_identifiers(entry.witnesses)
    ) do
        if identifier == wanted then
            return true
        end
    end

    return false
end

A generic selector reduces repetition: 

local function select_entries(test)
    local result = { }

    for _, entry in ipairs(critical.entries) do
        if test(entry) then
            result[#result + 1] = entry
        end
    end

    return sort_entries(result)
end

Examples: 

local textual = select_entries(
    function(entry)
        return entry.layer == "textual"
    end
)

local omissions = select_entries(
    function(entry)
        return entry.kind == "omission"
    end
)

local witness_b = select_entries(
    function(entry)
        return has_witness(entry, "ms-b")
    end
)
Selection Scholarly question
Witness Where does one witness contribute evidence?
Layer Which records belong to textual criticism, translation, or sources?
Operation Where are the omissions, additions, or conjectures?

9. Compile a sorting and filtering MWE

The following compact MWE registers records out of order and creates three views:

The textual view begins with Reason] Judgment B D, even though that record was registered last.

10. Keep data and presentation separate

A stored omission is: 

{
    kind      = "omission",
    lemma     = "Reason",
    witnesses = "ms-e",
}

It is not stored as: 

Reason] om. E

Another renderer could produce: 

Witness E omits “Reason” in reason-01.

without modifying the record.

This is the principal conceptual gain of structured data.

11. Declare controlled vocabularies

Validation requires an explicit statement of what the project permits. 

critical.known_units = {
    ["reason-01"]     = true,
    ["experience-01"] = true,
}

critical.valid_layers = {
    textual     = true,
    translation = true,
    sources     = true,
}

critical.valid_kinds = {
    reading     = true,
    omission    = true,
    addition    = true,
    conjecture  = true,
    translation = true,
    source      = true,
}

Controlled vocabulary. Values such as textual, translation, and omission should be declared once. Otherwise, spelling variants create different categories without warning.

12. Record validation issues

Create a diagnostic list: 

critical.validation_issues = { }

local report = logs.reporter("critical apparatus")

local function add_issue(severity, code, message)
    critical.validation_issues[
        #critical.validation_issues + 1
    ] = {
        severity = severity,
        code     = code,
        message  = message,
    }

    report("%s: %s", severity, message)
end

The validator can continue after finding a problem, so one proof run can report several issues.

13. Validate identifiers and required fields

Useful checks include:

Check Typical severity
Undefined witness warning
Undefined textual unit error
Unknown layer or operation error
Missing lemma error
Missing content for a reading, addition, conjecture, translation, or source error
Duplicate record identifier error
Duplicate textual position warning or error according to project policy

A simple empty-value test is: 

local function is_empty(value)
    return value == nil or
           value:match("^%s*$") ~= nil
end

An omission is deliberately excluded from the rule requiring a reading.

Validation policy. A duplicate position may be legitimate when several records belong to the same textual location. The project must decide whether that case is informational, a warning, or an error.

14. Compile a diagnostic validation MWE

The following MWE intentionally contains invalid records:

Diagnostic MWE. The invalid records are intentional. This example is designed to demonstrate the reporting mechanism, not to provide data for a real edition.

15. Distinguish formal validation from scholarly judgement

Validation can detect:

It cannot decide whether:

Limit of validation. Lua can verify the formal consistency of recorded scholarship. It cannot perform the scholarship itself.

16. Generate several outputs from one registry

One collection can support:

Output Selection Purpose
Reader apparatus textual layer in textual order publish variants
Witness report records containing one witness inspect one manuscript
Omission catalogue kind == "omission" review absent passages
Conjecture list kind == "conjecture" review editorial interventions
Translation review translation layer check terminology
Source report sources layer collect parallels and allusions
Validation proof all detected issues check formal consistency

ConTeXt modes can select the required output: 

\startmode[proof]

\placevalidationsummary
\placevalidationreport

\stopmode

Compile with: 

context --mode=proof edition.tex

Only the selected presentation changes. The registry remains the same.

17. Organize the project files

Move the Lua implementation into a separate file: 

critical-data.lua

Load it with: 

\registerctxluafile{critical-data}{}

A useful project structure is: 

project/
│
├── environment/
│   └── env-critical-edition.mkxl
│
├── lua/
│   └── critical-data.lua
│
├── data/
│   └── critical-records.mkxl
│
├── components/
│   └── chapter-01.mkxl
│
└── edition.tex
File Responsibility
Environment typography, layout, note styles, public commands
Lua file registration, sorting, filtering, validation, reports
Data file record-registration calls
Component edited text and textual-unit references
Main document selected output mode and document assembly

18. Keep the public registration interface stable

The nine-argument command is a contract: 

\registercriticalentry
  {id}
  {unit}
  {unit order:entry order}
  {layer}
  {operation}
  {lemma}
  {reading or content}
  {witness identifiers}
  {responsibility}

The internal Lua representation may later change, provided that this public interface remains compatible.

Interface stability. The ConTeXt command is the boundary between editorial source and Lua implementation. A stable boundary protects the source from internal refactoring.

19. Advanced extension: multilingual records

A multilingual edition needs additional fields:

Field Purpose
language Language of the annotated text
target Stable identifier of the exact word, phrase, or passage
lemmalevel word, phrase, or passage
lexicallemma Dictionary form used in lexical analysis
bibliography Stable bibliographical key
locator Page, section, column, or cited location

The distinction between unit and target is important:

Field Identifies
unit The larger synchronized passage
target The exact range annotated by one record

Controlled lemma levels can be declared as: 

critical.valid_lemma_levels = {
    word    = true,
    phrase  = true,
    passage = true,
}

Scholarly distinction. An apparatus lemma identifies text in the edited passage. A lexical lemma identifies a dictionary form. Store them in separate fields.

20. Advanced extension: query multilingual data

A Greek textual view may select two properties: 

local greek_textual = select_entries(
    function(entry)
        return entry.language == "gr" and
               entry.layer == "textual"
    end
)

A lexical view may select: 

local lexical = select_entries(
    function(entry)
        return entry.layer == "lexical"
    end
)

A passage-level report may select: 

local passages = select_entries(
    function(entry)
        return entry.lemmalevel == "passage"
    end
)

One record may appear in several reports because each report asks a different question. This is not duplication of source data.

Report Selection Editorial purpose
Greek textual records Greek + textual layer review Greek variants
Latin translation records Latin + translation layer inspect translation evidence
Lexical records lexical layer review lemmas and morphology
Bibliographical records bibliography layer inspect cited works and locators
Passage-level records passage lemma level find annotations attached to complete passages

The reader-facing page and the editorial report remain different outputs. The first is designed for reading; the second is designed for inspection and verification.

21. Recognize the limits of the registry

An embedded Lua registry is appropriate when:

A dedicated structured format becomes preferable when:

At that point, continue with Building critical editions from TEI XML.

22. Test the workflow systematically

Compile separate tests for:

Inspect both:

Testing requirement. Successful compilation proves only that the code ran. The PDF, the log, and the validation report must all be inspected before the editorial data are treated as valid.

23. Command summary

Interface Purpose
\registercriticalentry Register one structured record
\placecriticalentries Print records in registration order
\placetextualview Print the textual layer in textual order
\placewitnessview Print records associated with one witness
\placeomissionview Print omission records
\placesourceview Print the source layer
\placevalidationsummary Print record, error, and warning counts
\placevalidationreport Print the complete diagnostic list

These are project interfaces created through interfaces.implement. They are not predefined ConTeXt commands.

24. What this guide has established

The progression is: 

formatted apparatus string
        │
        ▼
structured record
        │
        ▼
Lua registry
        │
        ▼
sorting and filtering
        │
        ▼
validation
        │
        ▼
reader and editorial outputs

Final result of the series. The six guides have progressed from a simple page-bottom apparatus to a structured workflow involving witnesses, several annotation layers, parallel texts, stable editorial units, Lua processing, and formal validation.

Lua does not replace: 

 required.

25. Continue with TEI XML when necessary

The Lua registry is appropriate when the ConTeXt project remains the principal editorial environment.

TEI XML becomes relevant when the edition requires:

The complementary collection begins at:

Building critical editions from TEI XML

Online resources

Related pages

Critical apparatus guides: ← Guide 5 — Typesetting an original text and translation in parallel · Orientation · Glossary · Guide 6 of 6 — End of guides