State Changes & State Views

Application logic in EvidentSource is two kinds of WebAssembly component, running on the database servers:

State Changes export decide(db, command, context), which turns a command into events and the append conditions they need.
State Views export evolve(context, state, events), which folds a stream of events into state.

Both are defined by one WebAssembly interface, and each SDK wraps it in its own language.

`decide`: turning commands into events

When a command arrives, it runs a State Change at a revision. decide gets:

db, a handle for reading the database at that revision: its events, its State Views, and speculative transactions that show what a candidate transaction would produce, read exactly as they will be once committed;
command, the request body and its content type;
context about the request: its headers, the call’s now (when the server prepared the call, floored at the commit time of the revision it decides against), and who sent it. Headers that carry a credential, such as Authorization and cookies, are withheld, so identify the caller by who sent it, never by a header.

It returns either a decision, the events to append together with the append conditions that must hold when they commit, or an error: a kind such as invalid-argument, conflict or not-found, a message, and an optional code and details.

decide never writes. What it returns becomes one transaction, committed only if every condition still holds.

Here is create-todo, from the TodoMVC example, in Rust, with its comments trimmed:

#[evidentsource_functions::state_change(name = "create-todo")]
fn decide(
    _db: &Database,
    command: CreateTodoCommand,
    _metadata: &StateChangeMetadata,
) -> Result<(NonEmpty<DomainEvent>, Vec<AppendCondition>), StateChangeError> {
    if command.title.trim().is_empty() {
        return Err(StateChangeError::Validation("Todo title cannot be empty".into()));
    }
    let event = DomainEvent::TodoCreated {
        todo_id: command.todo_id,
        title: command.title,
        effective_timestamp: command.effective_timestamp,
    };
    Ok((NonEmpty::new(event), vec![]))
}

Clocks and randomness in `decide`

The server fixes what a component can observe:

The clocks are fixed. The wall clock reads the call’s now: the time the server prepared the call, or the commit time of the revision it decides against if that is later. A command queued on the command log is prepared when it is decided, not when it arrived. The monotonic clock always reads one nanosecond.
The random source is seeded. It is seeded from everything the call receives, so the same inputs draw the same values. The stream is predictable from the command, so don’t use it to mint secrets.

Reading an effective-time-ordered State View from a db with no effective-time cut fails with failed-precondition, so every cut is one decide names. Give db a cut first, at the time the decision means:

at-now(), for what is in effect now: the call’s now, which context.now and the wall clock also read;
at-effective-timestamp at an effective time the command carries;
at-effective-timestamp at the latest instant the server holds, the end of year +262142 ({seconds: 8210266876799, nanoseconds: 999999000}), to read everything the database knows. The Rust SDK names it at_max_effective_time().

Because now is floored at the revision’s commit time, a cut at it covers every event without a time that the revision recorded, even when the node that prepares the call has a clock behind the writer’s. That holds for history written since commit times became monotonic.

A speculative db reads as the database will once its speculated events commit. Event queries filter, order and limit them as they do committed events, and State Views fold them. Each speculated event is stamped as the commit will stamp it, with three differences: it keeps the id you gave it, it has no causationid, and its recorded time is the transaction time of the revision it speculates on, which is also its effective time if you gave it no time.

decide’s time limit counts only its own running time. Time spent waiting for a read is not charged, but the whole call, waiting included, is limited to 25 seconds. A decide that reaches that limit while waiting for a read is answered unavailable, however long its own code ran.

A decision belongs to the database it read

decide runs before the commit, so the name it read can be deleted and created again in between. The transaction carries the database it decided against, and the writer refuses it with HTTP 409, or gRPC FAILED_PRECONDITION, rather than applying the decision to a different database that happens to share the name. There is nothing to re-send: decide again against the new one.

`evolve`: folding events into state

A State View exports evolve, which takes the view’s current state and a stream of events, and returns the new state.

Only the events its definition selects reach it.
One call per fold. The events arrive as a stream, every event since the fold started, and evolve returns the state once. It must read the stream to its end: a call that returns earlier fails, and the server keeps nothing from it.
Waiting for events is not charged. A call’s budget counts the time its own code runs, and grows with every event it is handed. A much longer limit on the whole call, waiting included, backs it up.
The fold can restart. A late event with an earlier effective time makes an effective-time-ordered view fold again from before it.

So evolve must be deterministic and free of side effects: the same state and events must always give the same result.

The server helps by keeping the world out of a State View. Its wall clock reads the Unix epoch, its monotonic clock always reads one nanosecond, and its random source starts from one fixed seed. Each call starts afresh, so nothing a State View kept in memory carries from one fold to the next. Floating-point and SIMD results don’t depend on the machine. What it can’t prevent is a State View whose state depends on where a fold started, such as one that draws a random value per event: a fold resumed from a snapshot is a second call.

Don’t mint identifiers from the random source in evolve. Every call starts the source from the same seed, so the first event of each fold draws the same value. Derive an identifier from the event instead, such as its id or its revision.

A view’s definition declares its state shape: a blob, ordered set, map or vector. A blob fold returns bare bytes; the definition gives their content type and optional schema (the schema itself: JSON Schema for JSON, say). A collection fold edits native resources and returns a collection handle. External collection reads need a presenter: evident.json version 1 serves JSON pages, and evident.table version 1 emits rows as an Arrow stream.

A definition doesn’t change once published, so a different blob label means publishing a new version. That costs no refold: a new version with a byte-for-byte identical component, the same host folding semantics, event selector, fold order and state shape starts from the old version’s snapshots. Change any of those, or rebuild the component so its bytes differ, and it folds from the start. A host upgrade that changes folding semantics also starts a fresh snapshot series. The one-nanosecond clock therefore refolds views that previously used the zero clock, and invalidates their cached HTTP responses.

Parameters

A view’s selector can hold parameters. A reader binds each one, and each set of bindings is its own instance of the view: one todo, one account. Instances aren’t enumerated ahead of time; one comes into being when someone reads it.

When the fold happens

A view’s definition chooses when it is folded:

On read, the default: the first read that needs it folds it, and later reads start from where that left off.
As events commit: the view is kept current, so a read usually doesn’t wait. A view with parameters is still folded on read.

Fold order

A view’s definition also fixes its order:

Revision order: the order the database recorded the events.
Effective-time order: by effective time, bounded by both a revision and an effective-time cut.

See Bi-temporal reads.

Errors

decide and evolve fail the same way, and a client gets the same answer from either. An error has a kind, a message, an optional code your client can match on, and details. Each kind has one HTTP status and one gRPC code:

Kind	HTTP	gRPC
`invalid-argument`	400	`INVALID_ARGUMENT`
`not-found`	404	`NOT_FOUND`
`conflict`	409	`FAILED_PRECONDITION`
`unauthorized`	403	`PERMISSION_DENIED`
`failed-precondition`	409	`FAILED_PRECONDITION`
`not-acceptable`	406	`FAILED_PRECONDITION`
`resource-exhausted`	429	`RESOURCE_EXHAUSTED`
`deadline-exceeded`	504	`DEADLINE_EXCEEDED`
`unavailable`	503	`UNAVAILABLE`
`internal`	500	`INTERNAL`

Over HTTP, the response body is:

{"error": "Balance is too low", "kind": "failed-precondition", "code": "insufficient-funds", "details": {"available": "40"}}

A State Change that fails with unavailable, resource-exhausted or deadline-exceeded applied nothing. Its HTTP answer also carries "outcome": "not_applied" and the transaction_id to retry under.

Over gRPC, the status carries the message, and its details hold a google.rpc.Status whose details include a com.evidentsource.ComponentError with the kind, code and details. Because gRPC sends these in headers, a message over 512 bytes or a code over 128 bytes is cut short, and details over about 2 KiB are replaced by a single details-truncated entry. An asynchronous State Change’s response carries the kind’s gRPC code and the message.

When the server stops a component, it picks the kind itself:

a component that runs past its time limit is deadline-exceeded;
a fold that spends most of its whole-call limit waiting on the server is unavailable;
a component that runs out of memory is resource-exhausted;
when the server has no room to start a component right now, the call is answered unavailable (HTTP 503, gRPC UNAVAILABLE) and can be retried; a read inside decide that fails this way counts as one the server couldn’t serve;
a component that crashes (a trap) is internal, with a message saying what trapped.

A read that decide asks for can fail too. It gets the error to handle or return: a State View’s own error unchanged, not-found for a missing view, invalid-argument for a malformed request, unavailable when the server can’t serve the read right now, or internal for a server fault. Returning it as it is passes it to your client unchanged. If the server couldn’t serve a read, a failed command is answered as unavailable and retried, whatever decide returned.

A failed fold keeps nothing, so the next read folds again from the last snapshot.

You can expire a view’s snapshots. A stored snapshot that can’t be read fails every read that reaches it, and a wrong fold, once stored, is where every later read starts. ExpireStateViewSnapshots over gRPC, or POST /api/v1/db/{database}/state-views/{name}/expire-snapshots over HTTP, makes the view’s next read fold from its first event, for every version of the view. Nothing is deleted: reads move to a fresh set of snapshots, and the old ones are deleted by the cold-snapshot sweep once nothing has written to them for state_view_snapshot_series_ttl_secs (7 days by default). Other views, and the same view in other databases, keep theirs. It takes the deployer role.

Why these two functions

Auditability. The events decide returns are the permanent record. State Views are computed from them, and can be computed again.
A narrow job. Each function has one input shape and one output shape, which makes it straightforward to test, to review, and to generate.
No coordination to write. A decision needs no locks and no distributed transaction: its append conditions carry its consistency.

Where they run

State Changes and State Views are WebAssembly components, built with the SDK for your language and published to a database. The server runs them sandboxed, next to the data.

See SDKs for the languages.