8. Provenance and logs

Every stored value says where it came from (chapter 1.9). This chapter says how sources are assigned, how ownership moves, how disagreements are kept, and how the full history lives in each owner's signed log.

8.1 Origin #

origin records how a value was first established, and it never changes.

• A value that arrives with its own source (from another owner's data, chapter 7.8) keeps that source's origin. Otherwise it takes the contract's.
• Rule or contract? A value that can be produced again from the record's own values, with the same answer, is derived by a rule; it can be dropped and rebuilt. A value that can't (AI text, a price from comparables, a drawn SKU) came through a contract, carries that contract's origin, and is irreplaceable, so it is kept.
• A migration never becomes a value's origin or owner (8.6).

8.2 Owner and adopting #

owner is who controls a value now. It changes in one way: adopting (✎, "made your own"). The same verb works at every scale.

• An adopted value doesn't update until it is given back: removing the adoption returns it to following its rule or its presenter, and the next run or update replaces it.
• A presenter's terms may forbid adopting (chapter 7.8). Then a consumer's edit is refused.
• Strength limits adopting a produced value (chapter 6.2): must forbids it, should flags it.

8.3 Link, copy, adopt #

A copy is always self-contained: it holds the values and their sources, so it reads without the original.

8.4 Trust and alternatives #

When two sources disagree about a value (a conflict, chapter 9), the trust order picks the current value. The trust order is per package (chapter 11); by default a person comes first. A facet MAY override it.

The losing values are kept, in the record's alternatives, keyed by address like sources:

"alternatives": {
  "title": [
    { "value": "National Lampoon's Christmas Vacation", "origin": "reported", "via": "L-55ZT" }
  ]
}

Each alternative is a value with its full source. An alternative becomes current only by a person's choice or a change in the trust order, and the value it replaces becomes an alternative in turn. Suggestions (values that land: suggestion, chapter 7.5) are kept the same way, in suggestions.

Sources stay hidden when printing in plain mode (chapter 2.6), and so do alternatives and suggestions.

8.5 The log #

Each owner keeps a log: an append-only list of entries, one per write, however many values the write touched (chapter 3.8 gives the JSON).

Every change is a transaction. An entry is the unit: it applies whole or not at all, and a store that fails partway applies nothing of it. Data entry, intake landing, a grant, a review decision and a change to a definition are each one entry with one cause. The unit is one caller's action on one record, or on the few records one operation touches (a merge, an attach); a run over many records writes one entry per record, so one refused row refuses one row (chapter 7.5). What is computed is never a transaction: projections, rule results and pending instructions are read off the state. Nor is a read, except under an emergency edge (chapter 14.13). An event in the world (a sale, a reading) is not a transaction either: it is a record of a kind with a moment facet, and the transaction is the write that records it (chapter 6.11).

• Hash-linked. Each write has a hash: the SHA-256 of its canonical form (chapter 3.9). An entry's hash is the SHA-256 of its canonical form without sig, with each item of writes replaced by the string sha256: and that write's hash in hex. So the entry's hash covers every write through the write's own hash, and a write can be taken out (8.5.2) without moving it. prev is sha256: and the previous entry's hash. The first entry of a chain has no prev.
• Signed. sig is the owner's Ed25519 signature over the entry's hash (its 32 bytes, not the hex text), written as base64url without padding, made with the key published for the owner's domain (chapter 7.8). A redacted entry carries no owner; it is checked against its chain's owner. When the owner's key is known, an entry without a valid sig breaks the chain there.
• Chains. An owner MAY keep several chains, each linear (one per shard of a large store); every entry then names its chain. A record's writes MUST all go to one chain.
• No consensus. There is no mining and no global ledger. Every record has an owner, and the owner's log is the truth for the owner's data. Anyone holding entries can check the links and signatures.

The log records effects, not causes. An entry says exactly what changed in the stored state, so the state can be rebuilt from the log alone (8.5.3). Why it changed is linked, not re-derived: the intake record, the run, the rule or the recipe.

What an entry holds: log (its ID), prev, chain (when there are several), owner, at, by (exactly one of person, contract with run, rule, migration, instruction or take with the presenter's entry), intake (when the write came through intake), source (the default source of its value writes: origin, and owner and adopted when they apply; via is the entry itself), writes, crubal (the language version, on the first entry of a chain and whenever it changes from the entry before, chapter 11.8), and sig.

Each write is one of these:

Definitions are state like records: the owner's working package is what the define writes build, so a rebuild (8.5.3) recovers the definitions in force at any entry together with the data, and replay (chapter 9.6) needs nothing outside the log. A published version of the package is a snapshot of that state (chapter 11.5).

An entry MAY carry witnessed: true (chapter 14.13); such an entry MUST NOT be redacted.

What can be computed from the log and the package versions it installs is never logged: completeness and the other rule results (chapter 6.9), projections (chapter 12.6), and whether instructions are pending.

8.5.1 Values that crossed a border #

When a value comes from another owner, the consumer's entry says so: by is {"take": "catalogTake", "from": "distributor.example/L-91AC"}. The value keeps its origin, and the presenter's entry can be checked against the presenter's own log.

8.5.2 Redaction #

A write, or a whole hop, can be hidden without breaking the chain.

• A redacted write is replaced, inside its entry, by {"redacted": "sha256:…"}, the hash of the write it replaces. The entry's hash is unchanged (8.5), so prev of the next entry still matches, the signature still verifies, and every write the entry kept is still checked by its own hash.
• A redacted entry keeps only log, prev, chain, sig and "redacted": "sha256:…", the hash of the full entry. Its hash is that value. This is how a whole hop is hidden when an owner shares part of a log.

What may be redacted. Only value, observation and read writes, and only in an entry that isn't witnessed (chapter 14.13). An envelope, edge, binding, review, instruction, run, define or install write is never redacted, so a rebuild (8.5.3) never meets a write to a record it hasn't seen; an entry holding one of those is redacted write by write, never whole.

Erasure is redaction applied to a person: every value write that holds their data, in every entry that holds one, is redacted, whatever else the entry holds. The entry that created their record keeps the creation and loses the values beside it, so the record's shell stays and its values are gone, and the log still proves what happened and when without saying what was written. Erasure is the owner's power (chapter 14.1), and a person's own records (14.1) are erased by them.

8.5.3 Rebuild and replay #

Two different things, never to be confused:

• Rebuild applies the log's entries, in order, to an empty store. It needs no rules, no contracts and no engine of a particular version, and it MUST give exactly the state the store had. A store MUST be able to rebuild; the state it keeps is a cache of the log. A redacted write or entry is skipped, so rebuilding gives the state without what they held, which is what erasure means.
• Replay (chapter 9.6) folds kept intake records again under the current definitions. It is new work, and it writes new entries.

8.6 Migrations and history #

A migration's write gets its own log entry, with by: {"migration": "<recipe>"}. It never changes a value's origin or owner: a person's renamed value is still the person's, with the migration in its history.

Values written before an owner kept a log (legacy data) get one entry for the whole backfill, with by: {"migration": "backfill"}. Their origin is what is known about them, and reported when nothing is.

8.7 Read-through values #

Values read through from an upper level or a shared part carry no source on the lower record. Their source lives where they are stored, and the log entries are the upper record's.

8.8 Record members added by this chapter #

Chapter 3.2's record gains three optional members:

and the source object (chapter 3.7) gains adopted: true.

Edit on GitHub · Markdown