Kafka prerequisites

EvidentSource’s async command interface needs a Kafka broker. If you do not use async commands, you do not need Kafka at all — the gRPC and REST APIs are complete without it, and the server runs with no broker configured.

Any Kafka-compatible broker works. The suite is exercised against Confluent and AutoMQ; MSK and Confluent Cloud are supported deployment targets.

What the server needs

• Bootstrap servers — --kafka-bootstrap-servers
• Authentication — --kafka-security-protocol, --kafka-sasl-mechanism, and --kafka-sasl-jaas-config as your broker requires. AWS_MSK_IAM is supported and uses the task’s IAM role.
• Transactions. Results are published transactionally, so the broker must support them. On a single-broker development cluster set transaction.state.log.replication.factor=1 and transaction.state.log.min.isr=1, or the first publish blocks.
• Topic creation. The server creates each database’s topics itself, so the credentials it runs under need create permission. Database IDs are minted during creation, so the topic names include that attempt’s new ID.
• Topic deletion stays with operators. The server does not need topic-delete permission and never automatically deletes topics.

Topics

Created per database at database-creation time:

Topic	Cleanup policy	Notes
<database>.<database-id>.async-command	delete	you produce commands here
<database>.<database-id>.async-response	delete	results, published transactionally
<database>.<database-id>.async-response-offsets	compact	internal publisher position

--kafka-topic-replication-factor sets the replication factor the server uses when it creates them (default 1; raise it for production).

The offsets topic has one logical key per command partition. Broker compaction controls how many old position records remain and how much a publisher must scan when it restarts. Keep command and response partition counts matched if you require the partition correspondence described in the reference.

Consuming results

Set isolation.level=read_committed. See Kafka streaming for the full contract.

Required operator task: topic cleanup

Deleting an EvidentSource database does not delete its Kafka topics. Failed creation attempts and partial provisioning also retain any topics they created. Operators must arrange cleanup according to the broker’s retention requirements. Kafka’s cleanup.policy=delete expires records; it does not remove the topic.

A recreated database has a new database ID and uses fresh topics. Retained topics for the old ID do not become the replacement database’s command backlog.

1. Identify the exact database ID and the owning deployment. Provisioning logs record the ID and all three topic names before broker requests; delete and failed-create logs report the topics retained for operator cleanup. Keep these records with your broker inventory, especially when deployments share a broker.

2. Confirm that the ID is no longer current in the owning deployment’s databases/<database>/root.json under its configured storage prefix. An ID in current is protected whether its state is creating or live. A failed create request may have an ambiguous storage outcome: resolve it before cleanup. Confirm that a failed creation attempt has finished or been stopped; a provisioning log alone is not evidence of failure. Absence from another deployment’s storage, or an unreadable lifecycle file, does not establish that a topic is safe to delete.

3. Stop producers targeting the old ID, including retrying clients. Allow required consumers to finish and archive retained commands/results as your policy requires. Old producers may otherwise recreate a deleted topic if broker auto-creation is enabled.

4. Using your broker’s administrative tools and operator credentials, delete only the exact topics for the confirmed retired or failed-attempt ID:

   • <database>.<retired-database-id>.async-command
   • <database>.<retired-database-id>.async-response
   • <database>.<retired-database-id>.async-response-offsets

   A failed attempt may have created only a subset. Do not delete by a database-name prefix: it can also select a replacement database or another deployment’s topics.

5. Verify that the selected topics are absent and record the cleanup in your operational inventory. Repeat for each retired or failed-attempt ID.

There is no automatic topic orphan collector. If a crash left topics whose owning deployment or lifecycle cannot be established, retain them until the broker operator resolves that ownership.