Getting data in · Live scoring
The live scoring engine
What the engine does with a scoring command once it is on the road: set a value, or step a saved state by one event.
In one minute
Most matches are scored as a scorecard: a person types the values, like in the Asian Games console. Only a few sports are scored event by event (ball by ball, point by point). The engine has one mode for each. Apply mode sets a value once and is the default. Events mode steps a saved state by one event.
Today, every event-by-event command replays the whole match. For cricket that grows with the square of the match: one ball costs 0.23 ms at ball 10 and 69.1 ms at ball 600 (measured on a laptop, 6 Oct). The agreed design saves the match state in the existing scoreboard row, next to the log, in the same transaction as every event. A ball then reads one row and runs one step, so ball 2,700 costs what ball 1 costs.
The log stays the truth. Corrections and undo replay from a checkpoint saved every 50 events. A background check replays every live match from its log and repairs the saved state if the two ever differ. A match keeps the exact sport code it started with, checked by a fingerprint on every ball.
Built today: apply mode, the log, corrections, checkpoints (used by athletics only). Still to build: the saved state columns, the step-function contract, the background check, code versions side by side, and results.set_line for typed player lines.
What this part does
The engine turns one accepted command into new rows. The road before it (the key, the lock, the version check, the answer) is the same for every command and is explained in Commands and the workflow engine. This page is about what happens in the middle: how the new score is worked out and saved.
Section 5 answered six questions. If any one is answered wrong, a client shows a wrong score, or scoring slows down as the match goes on.
| # | Question | Why it matters |
|---|---|---|
| 1 | How is the new state worked out after a ball: from the last state, or by replaying the match? | A replay gets slower with every ball. Ball 600 must cost what ball 1 costs. |
| 2 | Where does the current state live, and what is the truth if they disagree? | A cached state that drifts from the log shows a wrong score, and nobody knows. |
| 3 | How does a correction or an undo of an old ball change the state? | The third umpire changes a ball from 20 minutes ago. Everything after it must be right again. |
| 4 | How do we stop the same ball being counted twice, or two balls landing in the wrong order? | One scorer offline, two scorers on one match, a provider sending late. |
| 5 | What happens when a sport's rules change during a match, or the code that holds them is updated? | A deploy during a match must not change the score of balls already bowled. |
| 6 | How is the time of each event kept? | The time is a simple typed value ("23", "45+2"). It must stay with the event, and the order must stay right when it is typed late. |
The two modes
| Apply mode (scorecard and operations) | Events mode (a few sports) | |
|---|---|---|
| What a command says | "Set this value": the final score is 3–1, this side ranks 2nd, this batter made 54 off 38 | "This happened": a ball, a goal, a point |
| How the engine works | Apply once: write the value on its row. No replay, no saved state, no step function | Step: the saved state plus this one event gives the new state |
| Where the truth is | The rows themselves: the match, each side, each player line | The log of events. The state is a copy of it |
| Cost per command | The same at any point: a few rows | The same at any point, once S1 to S3 are built |
| Correction | Set the value again. The old value stays in the log | A correction event, replayed from a checkpoint |
| Used by | Every operational workflow: medal tables, schedules, players, imports, results of any sport. Needs no sport worker processes | Live ball-by-ball or point-by-point scoring, where a client pays for it |
The mode is chosen per competition, with an override per match. Scorecard is the default (S11). A Games uses scorecard everywhere. A premium league turns on events for the matches a client pays for.
Mixing the two. An events match can still take scorecard edits. An operator fixes a total by hand, and that hand edit wins over the value the events produced, under the priority list of Section 2. A match can also move from events to scorecard mid-match: the live scorer drops out at over 30 and the desk types the rest of the card. Nothing already scored is lost (S14).
There is no running clock. An event's match time is a value typed with it ("23", "45+2"), part of its input. The server also stamps when the device sent it and when it arrived. Order is the server's sequence number, never the typed minute (S8, M7).
How it works

Apply mode, step by step
- The command arrives on the one road: its key is checked, the subject's lock is taken.
- The input is checked against the command's input model, then the command's validations run.
- One log row is written: what was set, and by whom. The old value stays readable in the log.
- The command's writers set the values on their rows (the fixture, a side, and, once built, a player line).
- A note goes into the outbox in the same transaction, so feeds and screens hear of the change.
- Commit, then answer "accepted" with the new sequence number.
This is what the Asian Games console ran on, and it is built today in packages/core/src/omnium_core/workflows/engine.py:89-228. Its cost does not grow with the log, because nothing reads the log.
Events mode, step by step (agreed design)

- The ball comes in on the road. One id, the match lock, the version check: the sender says which event it last saw (Section 4, L17).
- Read the current state: one row. The
scoreboardrow holds the state, the number of the last event in it (last_seq), the rule version and the code fingerprint. It is read under the match lock, so nobody else can change it meanwhile. - Check the ball against that state. "Is this bowler allowed another over?" is answered from the state, never by reading the log.
- Apply the ball. The sport's step function takes the state and this one ball and returns the new state plus the lines it changed. Every events sport must have one; it is part of the plugin contract.
- Save, in one transaction: the log row, the new state, the batting and bowling lines this ball changed, the published match document, the note to feeds, and the command row.
- Answer "accepted" after the commit.
Three more paths sit beside the main one:
- A correction or an undo of an old ball loads the newest checkpoint before that ball, replays the events after it with the fix in place, and saves. An undo of the last ball replays at most 49 events.
- A background check replays every live match from its log, from scratch, every 5 minutes and at every match end. If the answer differs from the saved state, the log wins: a "rebuild state" command repairs it, and an alert names the match and the first ball where they part.
- A deploy never changes a running match. A match runs the exact code it started with. Several versions of a sport's code are installed side by side. If the code a match needs is missing, the engine refuses to score it and raises an alert.
Why we save the state

Today each cricket ball folds the whole match again, then runs three line actions over the whole log again. Measured on 6 Oct on a laptop, with today's cricket code, one innings with no wickets, sport code only, no database:
| Ball | Work out the match state | The 4 row actions (batting, bowling, innings, result) | Total per ball |
|---|---|---|---|
| 10 | 0.07 ms | 0.16 ms | 0.23 ms |
| 100 | 2.1 ms | 1.1 ms | 3.1 ms |
| 300 | 15.9 ms | 3.0 ms | 18.9 ms |
| 600 | 63.2 ms | 5.9 ms | 69.1 ms |
Twice the balls costs four times the time. A Test match has about 2,700 balls; by the same curve a late ball would take over a second (an estimate from the design doc, not run). On top of this, each ball reads the whole match log from the database twice: 600 rows took 3 ms at the median and 97 ms at the 99th percentile (measured, design doc; the second number is when the rows are not in memory).
A worked example
Two examples: the common case (a scorecard edit), then the events case (a cricket ball and its undo). Team and player codes are example values.
Example · A volleyball set typed into the scorecard
A Games desk scores volleyball as a scorecard. Set 3 of JPN v IRI ends 25–21. The operator types it in the console.
Agreed design (S12, S13). The console sends two results.set_line commands, one per side. Each sets the side's line for segment set-3. The keys points and sets_won are real: they are what volleyball's line action writes today (flows/shared/setfold.py:132-138).
{
"code": "results.set_line",
"key": "c1d7-set3-jpn",
"input": {
"subject": "JPN",
"role": "",
"segment": "set-3",
"values": { "points": 25, "sets_won": 1 },
"saw": 418
}
}
Step by step:
- The road takes the fixture's lock and checks the key
c1d7-set3-jpnhas not been seen. - The input model passes. The validation checks
pointsandsets_wonagainst volleyball's catalogue of line keys. - The version check looks only at the row this command changes: JPN's
set-3line. Itslast_seqis 418, the same assaw, so it goes on. If another operator had changed that line, the answer would beconflict/stale_rowswith the row as it is now (S13). - One log row is written, with the old values and the new ones.
- The writer upserts one
stat_valuerow by its unique key(fixture_id, subject_id, subject_role, segment, occurred_on), setslast_seqto the new log number, and pinspointsandsets_wonso a later import does not overwrite them. - Outbox note, commit, answer
accepted.
The second command does the same for IRI with points: 21, sets_won: 0. The operator then sets the match score with results.edit_side (sides' set counts) and results.set_result (the scoreline), which exist today.
Today. edit_side and set_result work (flows/results/commands.py:138-182). There is no results.set_line: a set's points or a player's line can only be written by a sport's events code. That is the gap S12 closes.
Example · A cricket ball, then an undo
A T20 match is at last_seq 311. The scorer sends ball 312: STARC to KOHLI, four runs.
{
"code": "cricket.delivery",
"input": {
"striker": "KOHLI", "nonStriker": "GILL", "bowler": "STARC",
"batRuns": 4, "extraRuns": 0
},
"saw": 311
}
Agreed design (events mode).
- Lock the fixture. Check the key. Read the
scoreboardrow:last_seq311,code_shaof the match's cricket code. sawis 311 and equalslast_seq, so nothing was missed. The worker for thatcode_shais found.- One round trip to the sport worker:
check(state, event)passes (STARC has not bowled his quota);step(state, event)returns the new state (KOHLI +4, team +4, legal balls +1), the lines it changed (KOHLI's batting line, STARC's bowling line), and no emits. - One transaction: insert
timeline_itemseq 312; updatescoreboard(state,last_seq312,state_hash); upsert only those lines instat_value; write the published document; outbox note; command row. 312 is not a multiple of 50, so no checkpoint. - Commit. Answer
accepted, seq 312. Cost does not depend on the 311 balls before.
Ten seconds later the scorer taps undo: the four was a leg bye.
- The undo is a new command. A void row (seq 313) points at seq 312; the old row is marked voided, never changed.
- The engine loads the newest checkpoint before 312: seq 300.
- It replays the effective log from 301 to 311 (11 steps; ball 312 is gone) and saves the new state, the lines that differ, the document and a feed note marked as a correction.
- Checkpoints at or after seq 312 are dropped. There are none yet.
The scorer then sends the leg bye as a new ball.
Today. The undo is the built-in command core.void with targetSeq: 312 (scoring/pipeline.py:69-70, 1012-1026). It appends the void row, flips the old row's status and drops checkpoints from 312 (pipeline.py:1113-1162). Then cricket refolds the whole match from ball 1, because its state action is not incremental (flows/cricket/actions.py:46).
Low-level design
This part has two halves. First, the code that runs today, read in order. Then the agreed design: tables, contract, one ball statement by statement, corrections, the check, versions, settings. How a sport's commands, validations and actions are written line by line is on Writing a workflow in code; this page does not repeat it.
Today's events path, in the code Built today
The live desk sends a command to the bridge, which calls pipeline.command (packages/admin/src/omnium_admin/routers/bridge.py:262-297). The pipeline's own header names the rings: lock, resolve, schema, validations, record, route, actions, emit.
1. The lock, then the first full log read. From packages/core/src/omnium_core/scoring/pipeline.py:869 and 896-907:
await session.execute(sa.select(sa.func.pg_advisory_xact_lock(ledger.lock_key(fixture.id))))
...
entrants = await ledger.load_entrants(session, fixture.id)
log = await ledger.effective_log(session, fixture.id) # read 1: every row of the match
context = Context(
fixture=fixture_facts(fixture, entrants.sides, await load_setting(session, fixture)),
log=[e.as_json() for e in log], # every event turned into JSON
)
context.derived = await _current_derived(session, fixture) # the scoreboard as of before this ball
context.masters = await load_masters(session, fixture)
The lock is a Postgres advisory lock on the first 8 bytes of the fixture id (scoring/ledger.py:167-176). It is held until commit. Then the validations get the whole log, so a rule like the bowler quota can loop over it. This is the first of two full reads.
2. The append. ledger.append (ledger.py:220-257) takes the next seq by bumping fixture.last_seq (ledger.py:184-202), inserts the row, flushes, and only then checks the declared players. A bad player code raises after the row is written, which is the Section 4 bug the migration fixes first.
3. The refold, and the second full read. From pipeline.py:685-709:
events = log if log is not None else await ledger.effective_log(session, fixture.id) # read 2
determinism = determinism_for(events)
...
ordered = action_order(program)
folds = tuple(node for node in ordered if node.gives is Gives.STATE)
resume = await _resume_points(session, fixture, folds) # checkpoints, incremental only
if fold_state(compiled, context, determinism, folds, result, resume=resume):
await projections.write_scoreboard(session, fixture.id, context.derived)
await _write_checkpoints(session, fixture, context)
command calls refold without passing the log (pipeline.py:949-958), so line 686 reads it again. determinism_for (pipeline.py:296-306) takes the clock from the latest event, not the wall, so a rebuild next year gives today's answer.
4. Full fold or one step. From pipeline.py:559-563 and 594-608:
value = (
step_state(compiled, fold, context, determinism, (resume or {}).get(fold.id))
if fold.incremental
else run_action(compiled, fold, context, determinism) # handed the whole log
)
def step_state(compiled, node, context, determinism, resume):
fn = _callable_for(compiled, node)
mark, state = resume if resume is not None else (0, None)
for event in context.log:
seq = int(event.get("seq") or 0)
if seq <= mark:
continue # already inside the checkpoint
outcome = fn(state, event, determinism=determinism, limits=SYNC_LIMITS)
state, mark = outcome.value, seq
context.reached[node.id] = (mark, state)
return state
An incremental action is handed (state, event) and stepped from the newest checkpoint. Any other state action is handed the whole log. Note that step_state still loops over the whole in-memory log to skip what the checkpoint covers; the log was still read in full.
Which sports step? Only athletics. Its state action is declared @action(gives=Gives.STATE, incremental=True) (packages/flows/src/omnium_flows/athletics/actions.py:27). Cricket's is @action(gives=Gives.STATE, reads=["log"]) (flows/cricket/actions.py:46), and so is volleyball's (flows/volleyball/actions.py:19). The contract allows incremental only on a state action (packages/contract/src/omnium_contract/flows/components.py:244-245).
5. The scoreboard row. From packages/core/src/omnium_core/scoring/projections.py:76-84:
merged = dict(snapshot)
await session.execute(
pg_insert(Scoreboard)
.values(fixture_id=fixture_id, snapshot=merged, updated_at=sa.func.now())
.on_conflict_do_update(
index_elements=[Scoreboard.fixture_id],
set_={"snapshot": merged, "updated_at": sa.func.now()},
)
)
One row per fixture, replaced whole on every ball. Today it is written but never used as the starting point of the next ball; it is a cache for readers and for validations (pipeline.py:1179-1188).
6. The line actions, again over the whole log. Each cricket line action rebuilds every batter's or bowler's numbers from every ball. From flows/cricket/actions.py:449-458:
@action(gives=Gives.LINES, reads=["log"], keys=("runs", "balls_faced", "fours", "sixes"))
def batting_stats(self, log: tuple[Event, ...], ctx: Ctx[CricketFormat]) -> list[Line]:
per: dict[tuple[str, int], dict[str, Any]] = {}
for ball, innings in _each_ball(log):
batter = str(ball.get("striker"))
...
projections.write_lines (projections.py:145-248) upserts all of them into stat_value and deletes any line not returned this time. That replace-and-prune is why a correction needs no undo code today, and also why every ball rewrites every line.
7. Checkpoints. From pipeline.py:791-796 and scoring/checkpoints.py:70-84:
every = int(get_settings().scoring_checkpoint_every) # 50, settings.py:296
for node_id, (seq, state) in context.reached.items():
if seq and seq % every == 0:
await checkpoints.save(session, fixture.id, node_id, seq, state)
async def drop_from(session, fixture_id, seq):
result = await session.execute(
sa.delete(ScoringCheckpoint).where(
ScoringCheckpoint.fixture_id == fixture_id,
ScoringCheckpoint.thru_seq >= seq,
)
)
A checkpoint is written when an incremental fold reaches a seq that is a multiple of 50. A correction or void drops every checkpoint at or after the changed ball (pipeline.py:1155). Because only athletics is incremental, no other sport writes checkpoints today.
8. Corrections and voids. From pipeline.py:1138-1155:
item = await ledger.append(
session, fixture_id=fixture.id, command=authored or node,
payload={} if void else dict(payload.get("input") or {}),
..., supersedes=original,
event_type=ledger.VOID_EVENT if void else original.event_type,
)
await ledger.supersede(session, original, void=void) # flips status only, never the payload
forgotten = await checkpoints.drop_from(session, fixture.id, target_seq)
A correction is a new timeline_item that points at the old one with supersedes_id. ledger.effective_log (ledger.py:322-376) walks each ball forward to its newest correction and drops voided chains. A correction then fires every action, because it cannot know which numbers it moved.
The fingerprint today. Each component gets a source_sha: the sha256 of that one function's source text (components.py:116-120). The publish step copies it to impl_sha (packages/core/src/omnium_core/flows_publish.py:110). Nothing reads impl_sha at run time; only tests do. A fixture is pinned to a rule version once (scoring/store.py:193-199, column fixture.scoring_program_version_id), but the code that runs is whatever is installed.
The tables Agreed, to build
No new tables. The saved state is the scoreboard row that already exists, and checkpoints already exist. The match lock stays on the match's own row. scoreboard is only written by a command that holds that lock.
-- scoreboard EXISTS today: one row per match (models/fixtures.py:476-500),
-- columns fixture_id, snapshot jsonb, updated_at. It becomes the saved state.
ALTER TABLE scoreboard
ADD COLUMN last_seq bigint, -- the last log row folded into snapshot
ADD COLUMN rules_id uuid, -- scoring_program_version the match is pinned to
ADD COLUMN code_sha text, -- fingerprint of the whole sport package (M6)
ADD COLUMN state_hash bytea; -- sha256 of the canonical snapshot, for the check
-- snapshot (jsonb) stays; rule M2 keeps it under 16 KB.
-- scoring_checkpoint EXISTS today: (fixture_id, trigger, thru_seq) -> state
-- (models/scoring.py:185-213).
ALTER TABLE scoring_checkpoint ADD COLUMN code_sha text;
-- stat_value EXISTS today: one row per (fixture, subject, role, segment, occurred_on)
-- with a jsonb of values and last_seq (models/stats_engine.py:83-130).
-- A player's match line lives here in both modes.
-- The log gains two server times (S8, M7).
ALTER TABLE timeline_item
ADD COLUMN sent_at timestamptz, -- when the device sent it
ADD COLUMN received_at timestamptz NOT NULL DEFAULT now();
-- No clock column: the minute is part of the event's input, as typed.
The state must stay small Agreed, to build
Today's cricket state at ball 600 is 149 KB, and 148 KB of it is a list of every delivery (design doc, 6 Oct). So the state grows with the match, just like the replay. Saving it on every ball was measured on a laptop:
| State saved with each ball | Median | 99th percentile | Written to the database's change log per ball |
|---|---|---|---|
| Small: score, 22 player lines, last 12 balls (2 KB) | 0.67 ms | 1.15 ms | 2.6 KB |
| Today's shape at ball 600 (105 KB of text) | 2.62 ms | 7.55 ms | 47.2 KB |
The big state also costs about 1.5 ms in Python to read and write as JSON (measured, design doc). So the rule (M2): the state holds only what the next ball needs and what screens show now. Every ball's detail stays in the log, which feeds read by match and ball number. The state stays under 16 KB at any point in any match.
The step-function contract Agreed, to build
Every events sport provides four pure functions: no database, no clock, no network. The same events always give the same state. The design doc names the file contract/.../flows/sport.py as new. That file already exists today: it holds the Workflow and Sport classes (packages/contract/src/omnium_contract/flows/sport.py:61, 204). So these types are new, and Section 6 settles where they live.
State = dict[str, Any] # JSON only; canonical form is hashed
@dataclass(frozen=True)
class Event:
seq: int # the match's log number, set by the server
code: str # "cricket.delivery"
input: dict[str, Any] # already checked against the command's pydantic model
source: str # who sent it: device, provider or integration
@dataclass(frozen=True)
class Line: # one player's (or side's) numbers in this match
subject: str # participant code, resolved to an id by the engine
role: str = "" # stat_value.subject_role
segment: str = "" # stat_value.segment
values: dict[str, Any] = field(default_factory=dict) # keys from the sport's catalogue
@dataclass(frozen=True)
class Step:
state: State # the whole new state (small, M2)
lines: tuple[Line, ...] = () # only the lines this event changed (M3)
emits: tuple[str, ...] = () # "wicket", "innings_end" ... for feeds and stats
class EventSport(Protocol):
def initial_state(self, format: dict[str, Any], sides: list[dict[str, Any]]) -> State: ...
def check(self, state: State, event: Event) -> Refusal | None: ...
def step(self, state: State, event: Event) -> Step: ...
def summary(self, state: State) -> dict[str, Any]: ... # the score others compare
What each one does:
initial_statebuilds the empty match from its format and sides.checkreplaces today's validations that read the whole log. It may read only the state and the event. It returnsNoneor aRefusalwith a code and a sentence a scorer can read.stepis the only place a sport's rules change the state. A whole replay isinitial_state, thenstepfor every event. The background check and corrections use exactly the same code as a live ball.summaryis the score that other parts compare (Section 2, T6).
A void is never passed to step. The engine removes the voided event and replays, so a sport never writes undo code, as today. Athletics' incremental action (flows/athletics/actions.py:27-79) already has this shape: state plus one event gives a new state.
One ball, statement by statement Agreed, to build
BEGIN;
SELECT last_seq FROM fixture WHERE id = $match FOR NO KEY UPDATE; -- the match lock (L1)
SELECT outcome, answer FROM command WHERE command_id = $id; -- retry check (L3)
SELECT last_seq, code_sha, snapshot FROM scoreboard WHERE fixture_id = $match;
-- last_seq differs from the event the sender saw -> ROLLBACK, "conflict" + missed events (L17)
-- code_sha differs from the loaded code -> ROLLBACK, "try again" + alert (S6)
-- In the sport's worker process, no SQL:
-- check(state, event) -> None or a Refusal
-- step(state, event) -> Step(state, lines, emits)
INSERT INTO timeline_item (fixture_id, seq, unit, event_type, payload, sent_at, received_at, ...)
VALUES ($match, $seq, 'event', $type, $payload, $sent, now(), ...);
UPDATE scoreboard SET snapshot = $state, last_seq = $seq, code_sha = $sha,
state_hash = $h, updated_at = now()
WHERE fixture_id = $match;
INSERT INTO stat_value (fixture_id, subject_id, subject_role, segment, occurred_on, "values", last_seq, ...)
VALUES (...) -- only the lines this event changed
ON CONFLICT (fixture_id, subject_id, subject_role, segment, occurred_on)
DO UPDATE SET "values" = EXCLUDED."values", last_seq = EXCLUDED.last_seq;
INSERT INTO published_document ... ; -- the match document clients read
INSERT INTO domain_event ... ; INSERT INTO command ... ; -- feed note, command row (Section 4)
UPDATE fixture SET last_seq = $seq WHERE id = $match;
COMMIT; -- then "accepted"
Every 50th event also writes a scoring_checkpoint row. Nothing here reads more than one row per table, so the cost does not depend on how long the match is.
Two trips, not nine (M9). Written as above, a ball makes about 9 trips to the database (counted from the SQL). On AWS one trip in the same zone costs about 0.3 to 1 ms (estimate, to measure in Section 12). So the command is sent in 2 trips: one statement that takes the lock and reads the id, the state and the versions together, and one call to a database function that does every write and the commit. That brings a ball to about 2 to 3 ms on prod (estimate). The target (S9) is under 5 ms of server time at any ball, from ball 1 to ball 2,700.
The engine side, as the design sketches it (core/.../engine/events.py, new):
async def apply(session, subject, node, cmd, fixture_seq) -> Answer:
board = await state.read(session, subject.fixture_id) # one row: snapshot, last_seq, code_sha
if cmd.saw is not None and cmd.saw != board.last_seq:
return answers.conflict_missed(await ledger.after(session, subject.fixture_id, cmd.saw))
pool = workers.pool_for(board.rules_id, board.code_sha) # raises CodeMissing -> "try again" + alert
if node.op is Op.APPEND:
event = Event(seq=fixture_seq + 1, code=cmd.code, input=cmd.input, source=cmd.source)
reply = await pool.check_and_step(board.snapshot, event) # one round trip to the worker
if reply.refusal:
return answers.refused(reply.refusal.code, reply.refusal.sentence, reply.refusal)
await ledger.append(session, subject, event, sent_at=cmd.sent_at)
await state.write(session, subject.fixture_id, reply.step, seq=event.seq) # + checkpoint every 50
await projections.upsert_lines(session, subject.fixture_id, reply.step.lines, seq=event.seq)
else: # CORRECT or VOID
reply = await corrections.replay(session, subject, pool, node, cmd)
await publishing.materialize.fixture(session, subject.fixture, reply.step.state, seq=reply.seq)
await outbox.note(session, subject, reply.seq, ("fixture",), emits=reply.step.emits)
return answers.accepted(reply.seq)
The road around it (key, lock, duplicate check, the answer row) is on Commands and the workflow engine.
The scorecard line command Agreed, to build
A new Results command, results.set_line (S12), with input {subject, role, segment, values: {key: value}, saw}.
- Its validation checks every key against the sport's catalogue, as action keys are checked today (
check_values,projections.py:444). - Its writer (
workflows/writers/lines.py, new) upserts onestat_valuerow by its unique keyuq_stat_value_line(fixture_id, subject_id, subject_role, segment, occurred_on). - It sets
last_seqto the new log number, and pins the keys it set, so an import does not overwrite them. This is the same pin ruleedit_sideuses today (workflows/writers/fixtures.py:6-12). - Its version check covers only the rows it changes (S13). Two operators fixing different batters never block each other. Two fixing the same batter: the second gets
conflict.
The user's note on S12: "sometime we need to manually update scorecard in edge cases". That is why the line command is needed even for events matches (S14).
A correction, step by step Agreed, to build
The third umpire changes ball 140 (a four becomes a six) at ball 600. Checkpoints exist at 50, 100, 150, and so on.
| Step | What the engine does | Rows touched |
|---|---|---|
| 1 | Takes the match lock, checks the id | 1 |
| 2 | Writes the correction as a new log row that points at ball 140 | 1 |
| 3 | Loads the checkpoint at seq 100 | 1 |
| 4 | Reads the effective log from seq 101 to 600 | about 500 |
| 5 | Runs step() 500 times, in the sport's worker | none |
| 6 | Saves the new state, rewrites the lines that differ, deletes checkpoints after seq 140, writes new ones at 150 to 600 | about 25 |
| 7 | Writes the published document and a feed note marked "correction" | 2 |
Cost: reading 600 log rows took 3 ms median locally (measured, design doc); 500 steps at under 0.1 ms each is under 50 ms (estimate). A correction gets a CPU budget of 2 s, not 50 ms (M4). An undo of the last ball is the same with at most 49 steps.
Undo and correction are two different fixes (S7), the same split Sportradar uses. Undo takes back the most recent event. Correction changes any past event and says what changed. Both are new events. Nothing in the log is ever changed.
The background check Agreed, to build
BEGIN ISOLATION LEVEL REPEATABLE READ READ ONLY; -- one consistent picture, takes no lock
SELECT last_seq, code_sha, state_hash FROM scoreboard WHERE fixture_id = $match;
SELECT * FROM timeline_item WHERE fixture_id = $match AND seq <= $last_seq ORDER BY seq;
COMMIT;
It replays the effective log from an empty state with the match's own code, hashes the answer, and compares it with state_hash. It runs every 5 minutes for every live match, at every match end, and after every deploy. A full Test match is about 2,700 steps, under 0.3 s (estimate). It never holds the match lock, so scoring does not wait for it.
On a mismatch it does not write the state itself. It sends a "rebuild state" command on the normal road, so the repair is logged and takes the lock like any other change. It raises an alert with the match and the first ball where the two differ (M5). It runs in the worker service (core/.../engine/verify.py, new), not the command service.
Is hashing fast enough? Measured 6 Oct: sha256 of a 10 KB state takes 0.0044 ms (4.4 µs); turning it into canonical JSON, which is needed anyway, takes 0.05 ms.
Today's nearest thing is pipeline.rebuild (pipeline.py:1196-1216), run by hand from the admin route or the worker command line. It clears and refolds without taking the lock.
Code versions side by side Agreed, to build
- Each published version of the flows plugin is installed into its own folder at deploy:
/opt/omnium/flows/<version>/(pip install --target). - The sport worker processes start per sport and version, with Python's spawn method and only that folder on the path. Two versions never share memory or imports.
- A deploy adds a version. It never replaces one that a live match uses. Old versions are removed after the last match on them is final.
- On every ball the engine compares the match's
code_shawith the worker's. A mismatch refuses the ball ("try again" plus an alert) rather than score it with other rules.
Why a new fingerprint. Today's impl_sha hashes one function's own source text. A change to a helper or a constant that the function uses does not change it. So the new code_sha is the sha256 of the wheel's RECORD file. RECORD is the file inside every Python wheel that lists each file in the package with its own hash, so any change anywhere in the package changes it. It is stored on scoring_program_version at publish and on the scoreboard row at the first event, and checked on every ball.
Sport worker processes Agreed, to build
The sport's code runs in small worker processes inside each command container, not on the web server's thread (Section 4, L10). Only events mode needs them; apply mode never starts one. A worker is a process, not a container: 50 sports never means 50 containers. Workers start only for a sport that has a live events match, and stop 30 minutes after its last one.
Messages are one request and one reply per call, as length-prefixed JSON over the worker's pipe:
{"id": "7f3a", "op": "check_and_step", "sport": "cricket", "state": {}, "events": [{}]}
{"id": "7f3a", "ok": true, "refusal": null, "step": {"state": {}, "lines": [], "emits": ["wicket"]}, "cpu_ms": 0.08}
The op replay takes many events (a correction, the background check) and returns the final step plus the state at every 50th event, for checkpoints.
| Limit | Value | Who measures | What happens |
|---|---|---|---|
| CPU time of one call | 50 ms for a ball, 2 s for a replay | The worker, with time.process_time() around the call | A call that finishes is used, even if slow. It is logged, and an alert fires if a sport stays slow. A correct ball is never thrown away |
| Wall clock for one call | 500 ms for a ball, 5 s for a replay | The server, with a deadline on the pipe | The worker is killed and replaced. The command gets "try again". After 5 such failures on one command: refused plus an alert |
| Worker memory | 512 MB (first proposal) | The operating system limit on the process | Killed and replaced, as above |
A looping rule is stopped by the wall-clock kill: Python cannot interrupt a thread, but a process can always be killed. Pool size: 2 warm workers per container and sport version, up to 4. About 100 MB per worker (estimate, to be measured).
Today the components run in the same process as the web server, with no way to interrupt them. The interim guard is a per-transaction statement_timeout and idle_in_transaction_session_timeout of 10 s (pipeline.py:824-842, settings.py:290).
Settings Agreed, to build
| Setting | Where | Default |
|---|---|---|
scoring.mode: scorecard or events | competition.meta, overridden by fixture.meta | scorecard |
scoring_checkpoint_every | settings.py (exists, line 296) | 50 |
scoring_verify_every_s | settings.py (new) | 300 |
scoring_state_max_bytes | settings.py (new); a bigger state fails CI and alerts on prod | 16384 |
scoring_worker_warm / scoring_worker_max | settings.py (new), per sport version per container | 2 / 4 |
scoring_ball_cpu_ms / scoring_replay_cpu_ms | settings.py (new) | 50 / 2000 |
scoring_ball_wall_ms / scoring_replay_wall_ms | settings.py (new) | 500 / 5000 |
The limits are server settings the worker reads when it starts. They are not sent with each message and not shown in any screen (changed after the user's comment on 6 Oct).
Answer codes Agreed, to build
| Answer | Code | When |
|---|---|---|
| accepted | Saved. Carries the new seq | |
| duplicate | This id was saved before; carries the first answer | |
| conflict | stale_rows | Scorecard: a row changed since the sender saw it. Carries the rows now |
| conflict | missed_events | Events: the match moved on. Carries the events the sender missed |
| refused | bad_input, or the sport's own code (bowler_over_limit ...) | With the Section 4 L18 sentence |
| refused | system_failed | Our bug after 5 tries; carries a reference, alert sent |
| try_again | lock_timeout, worker_timeout, code_missing, db_unavailable | The sender sends again with the same id after a short wait |
Module layout Agreed, to build
Today there are two engines: workflows/engine.py (apply once) and scoring/pipeline.py (fold). They become one package, omnium_core/engine/, with the two modes inside it.
| File | New or today | What it holds |
|---|---|---|
core/.../engine/road.py | new | run(): the one road. Picks the mode and calls one of the two below |
core/.../engine/scorecard.py | new, from workflows/engine.py | Apply once: validations and writers (workflows/writers/* stay) |
core/.../engine/events.py | new, from scoring/pipeline.py | Read the scoreboard row, call the sport worker, write the results |
core/.../engine/corrections.py | new, from pipeline.refold | Replay from a checkpoint after a correction or an undo |
core/.../engine/state.py | new | Read and write the scoreboard row, canonical JSON, the hash, checkpoints |
core/.../engine/workers.py | new | Worker processes per sport and version, their messages and time limits |
core/.../engine/verify.py | new | The background check |
core/.../engine/answers.py | new | Answer, Refusal and the fixed answer codes |
core/.../workflows/writers/lines.py | new | The writer for scorecard player lines (S12) |
core/.../scoring/ledger.py | today | The log. append is changed to check players before it writes |
core/.../scoring/projections.py, checkpoints.py | today | Kept; called from engine/ instead of pipeline.py |
contract/.../flows/components.py | today | Gains a required step for events sports; incremental=True becomes the only kind of state action |
Migration order Agreed, to build
One step at a time, each shippable and tested on its own. Live scoring is not on prod, so nothing live is at risk.
ledger.appendchecks players before it writes (bug fix, Section 4).engine/withroad.pyandscorecard.py; Games and Results move onto it;workflows/engine.pyremoved.results.set_lineandwriters/lines.py. Scorecard mode is then complete.- The step contract,
workers.pyandcode_sha; the fingerprint check turned on. - Football gets a step function and a small state; its golden matches must give the same answers through the new path.
events.py,state.pyandcorrections.py; football runs on them.verify.pyin the worker service.- Cricket gets a step function. This is the larger job: its state today carries every delivery.
- The other sports move over;
scoring/pipeline.pyis removed.
Tests that prove it Agreed, to build
| Test | Proves |
|---|---|
Golden replay: each sport's recorded matches give the same state and lines through step as today's fold | The rewrite changes no score |
| Speed (M8): a full-length synthetic match per sport; the last 100 events cost at most 2 times the first 100; state under 16 KB | Ball 2,700 costs what ball 1 costs |
| Correction: change event k at random in a recorded match; the state after replay equals a fresh replay of the corrected log | Corrections and checkpoints are right |
| Crash: kill the server between the writes and the commit, and after the commit; retry with the same id | Never half saved, never twice |
| Two writers: two connections send to one match at once, many times | The lock and the version check hold |
| Worker kill: a sport with a deliberate endless loop | One command gets "try again"; the server keeps answering others |
| Version: start a match on version 1, install version 2, send a ball | The match stays on version 1; a missing version is refused with an alert |
| Background check: damage a scoreboard row by hand | Found within one interval, rebuilt through a command, alert sent |
When things go wrong
Every case ends with the right score and nothing counted twice. The last column names the decision that covers it.
| What goes wrong | What happens | Decision |
|---|---|---|
| A Test match reaches ball 2,700 | The ball still reads one state row and applies one ball. Its cost does not grow | S1, S2, S3 |
| The third umpire changes ball 140 at ball 600 | Load the checkpoint at 100, replay 101 to 600 with the fix, save, drop the checkpoints after 140. Clients get the corrected score and stats | S4, S7 |
| The scorer taps undo on the last ball | The last ball is voided; the engine replays from the newest checkpoint (at most 49 events) and saves | S4, S7 |
| The scorer is offline for 2 minutes and makes 6 changes | The screen sends them in order, one at a time, each with its own id and the last event it saw. All 6 are saved in order | S2 |
| The server crashes after the commit, before the answer | The ball and the state were saved together. The retry gets "duplicate". The state is never one ball behind the log | S1 |
| A bug in a sport's step function gives a wrong total | The background check replays from the log, sees the state differ at ball 312, rebuilds through a command, and alerts with the match and the ball | S5 |
| A deploy happens during a match with new cricket rules | The match keeps its rule version and code. The new code is installed beside the old one; only new matches use it | S6 |
| The code a running match needs is missing after a deploy | The engine refuses to score that match ("try again" plus an alert) instead of scoring it with other rules. A person decides | S6 |
| A correction changes who batted when (a long chain) | The replay from the checkpoint recomputes every ball after the fix. Nobody writes an "inverse" of a ball | S4 |
| Two scorers on one match | The second scorer's ball says which event it saw; an old view gets "conflict" with the events it missed | S2 |
| The state row is lost or damaged | It is only a copy: the engine rebuilds it from the log (the background check, or by hand) | S1, S5 |
| A football goal typed 20 seconds after it happened | The event carries the minute as typed ("23"), plus when it was sent and when it arrived. Order is the server number; the minute shown is the one typed | S8 |
| Two operators fix the same batter's line in a scorecard | The second gets conflict / stale_rows with the line as it is now. Two different batters never block each other | S13 |
| The live scorer drops out mid-match | The desk switches the match to scorecard and types the rest. Clients see one continuous scorecard | S14 |
| A step function loops forever | The wall-clock deadline kills the worker; that one command gets "try again". Others carry on | M6, L10 |
Decisions
All 23 decisions of Section 5 were agreed on 6 Oct 2026.
Scorecard (most matches)
| # | Decision | In plain words |
|---|---|---|
| S11 | Two modes, scorecard and events, set per competition with an override per match. Scorecard is the default | A Games uses scorecard everywhere; a premium league turns on events for the matches a client pays for |
| S12 | Scorecard commands set values directly, apply once: the match, each side, and each player's line in the match. A new command sets a player line, with keys from the sport's catalogue | Today the desk can type a final score but not "KOHLI 54 (38)". With this, a full scorecard can be typed by hand, with the same checks and log |
| S13 | A scorecard edit checks the version of only the rows it changes, and records the old value in the log | Two operators fixing different batters never block each other; two fixing the same batter get "conflict" |
| S14 | An events match can take scorecard edits as hand edits under the Section 2 priority list, and can switch to scorecard mode mid-match without losing what was scored | The live scorer drops out at over 30; the desk types the rest of the card |
Events (a few sports)
| # | Decision | In plain words |
|---|---|---|
| S1 | Each match has one saved state row, written in the same transaction as every ball. The log stays the truth; the state is a copy | Ball 600 starts from ball 599's answer |
| S2 | Every sport provides a step function: state plus one event gives the new state. Required by the plugin contract. A replay is the step function run over every event | Today only athletics has one. Cricket and football must get one before they go live |
| S3 | Nothing on the live path reads the whole log. A test fails any sport whose cost per ball grows with the match | Keeps every ball at the cost of ball 1 |
| S4 | Checkpoints every 50 events. A correction or an undo replays from the newest checkpoint before the changed ball, and drops the checkpoints after it. Corrections get a larger time budget | A fix at ball 140 during ball 600 replays about 500 steps, not the whole match. An undo replays at most 49 |
| S5 | A background check replays every live match from its log every few minutes and at match end. If the saved state differs, the log wins, the state is rebuilt, and an alert names the match and the first ball that differs | A bug in a step function is found by us within minutes, not by a client |
| S6 | A match runs the rule version and the exact code it started with, checked by the code's fingerprint on every ball. Several versions are installed side by side. Missing code means the match is not scored, and an alert fires | A deploy during a match cannot change its rules |
| S7 | Two ways to fix: undo takes back the last event; a correction changes any past event and says what changed. Both are new events | Clients get the corrected score and know it was a correction |
| S8 | An event's match time is a simple value typed with it ("23", "45+2"). There is no running clock in the engine. The server stamps sent and arrived times. Order is the server's number | A goal typed 20 seconds late still says minute 23 everywhere |
| S9 | Target: one ball takes under 5 ms of server time at any point in a match, from ball 1 to ball 2,700. Load tests use a full Test match | A number we can test in Section 12 |
| S10 | The ball's transaction writes only the log row, the state, the lines this ball changed, the published document, the feed note and the command row. Heavier stats are worked out by the stats queue right after | The scorer's answer never waits for a season table |
Engineering
| # | Decision | In plain words |
|---|---|---|
| M1 | The existing scoreboard row becomes the saved state: it gains last_seq, rules_id, code_sha and state_hash. Written only in a ball's transaction, under the match lock | The current score is one row, never worked out from the whole match |
| M2 | The state stays under 16 KB at any point in any match. It never holds a list of every ball | Today's 150 KB state makes each ball about 4 times slower to save and writes 18 times more data |
| M3 | The step function returns the new state and the lines it changed. Only those lines are written, by key | A ball writes 2 or 3 lines, not 22 |
| M4 | Checkpoints every 50 events, with the code fingerprint. A correction replays in the same transaction, with a 2 s CPU budget, and rewrites the checkpoints after the changed ball | A fix at ball 600 takes tens of milliseconds |
| M5 | The background check reads one consistent read-only snapshot, replays with the match's own code, and compares hashes. It repairs only through a normal "rebuild state" command, and alerts | A wrong state is found and fixed within minutes, and the fix is in the log |
| M6 | Each version of a sport's code is installed in its own folder and runs in its own worker processes. Every ball compares fingerprints | A deploy adds rules for new matches; it can never change a running match |
| M7 | The log gains sent_at and received_at. The match time stays inside the event's input, as typed | Late or offline events keep their real match time |
| M8 | Every sport has a speed test in CI: a full-length synthetic match; it fails if the last 100 events cost more than twice the first 100, or the state passes 16 KB | This is how S3 is enforced |
| M9 | A command makes 2 trips to the database: one statement that locks and reads, one database function that does all the writes | 2 trips keep a ball at about 2 to 3 ms on AWS (estimate), with any number of matches |
Built today, or still to build
Checked in the code on 7 Oct 2026 (branch feat/result-feeds).
| Piece | Today | Agreed design | Status |
|---|---|---|---|
| Apply mode (scorecard and operations) | workflows/engine.py:89-228: lock, key, schema, rules, log row (unit record), writers, outbox | Moves into engine/scorecard.py, same behaviour | Built today |
| Match and side values by hand | results.set_status, set_result, edit_side and others (flows/results/commands.py:114-200), with pins | Kept | Built today |
| A player or set line by hand | No command. Lines are written only by a sport's line actions | results.set_line and writers/lines.py (S12) | Agreed, to build |
| Mode per competition | A workflow class has Mode.FOLD or Mode.APPLY (contract/flows/types.py:61-74), fixed per sport | scoring.mode in competition.meta, override in fixture.meta, default scorecard (S11) | Agreed, to build |
| The log | timeline_item, never changed; corrections point back (ledger.py:220-376) | Gains sent_at, received_at (M7) | Partly built |
| Saved state | scoreboard.snapshot written every ball (projections.py:62-88), but every ball refolds the whole log | scoreboard gains last_seq, rules_id, code_sha, state_hash; the next ball starts from it (M1) | Partly built |
| Step function | Only athletics is incremental=True (athletics/actions.py:27). Cricket (cricket/actions.py:46) and volleyball (volleyball/actions.py:19) read the whole log | EventSport: initial_state, check, step, summary for every events sport (S2) | Partly built |
| Reads per ball | Whole log read twice (pipeline.py:897, 686), line actions over the whole log | One state row, one event (S3) | Agreed, to build |
| State size | Cricket 149 KB at ball 600 (design doc) | Under 16 KB, checked in CI (M2, M8) | Agreed, to build |
| Checkpoints | scoring_checkpoint, every 50, dropped from a corrected seq (checkpoints.py, pipeline.py:781-796, 1155). Used by athletics only | Kept, plus code_sha; every events sport uses them (S4, M4) | Partly built |
| Corrections and undo | core.correct, core.void (pipeline.py:69-70, 1113-1162), then a full refold | Replay from the newest checkpoint, 2 s CPU budget (S4, S7) | Partly built |
| Background check | None. pipeline.rebuild by hand, no lock (pipeline.py:1196-1216) | engine/verify.py, every 5 minutes, hash compare, repair by command (S5, M5) | Agreed, to build |
| Code fingerprint | impl_sha = sha256 of one function's source (components.py:116-120), written at publish (flows_publish.py:110), never read at run time | code_sha = sha256 of the wheel RECORD, checked every ball (S6) | Agreed, to build |
| Versions side by side | One copy of the flows plugin installed | One folder per version, workers per version (M6) | Agreed, to build |
| Sport code isolation | Runs in the web process; budget checked after the call (flows_host.py:244-249); 10 s transaction guard | Worker processes with CPU and wall limits, killable | Agreed, to build |
| Players checked before the write | ledger.append writes, flushes, then checks players (ledger.py:248-255) | Check first (migration step 1) | Agreed, to build |
| Two trips per command | About 35 to 45 trips per ball today (research notes, not re-counted) | 2 trips (M9) | Agreed, to build |
Numbers
| Number | What | Where it came from |
|---|---|---|
| 0.23 ms | Cricket, total per ball at ball 10 (0.07 state + 0.16 row actions) | Measured, laptop, 6 Oct, sport code only |
| 3.1 ms | Same, ball 100 | Measured, laptop, 6 Oct |
| 18.9 ms | Same, ball 300 | Measured, laptop, 6 Oct |
| 69.1 ms | Same, ball 600 (63.2 state + 5.9 row actions) | Measured, laptop, 6 Oct |
| over 1 s | A late ball in a 2,700-ball Test, today's code | Estimate from the curve, not run |
| 3 ms / 97 ms | Reading 600 log rows, median / 99th percentile | Measured, laptop, 6 Oct |
| 149 KB | Cricket state at ball 600 today (148 KB is the delivery list) | Measured, design doc, 6 Oct |
| 0.67 ms / 1.15 ms | Saving a 2 KB state per ball, median / p99 | Measured, laptop, 6 Oct |
| 2.62 ms / 7.55 ms | Saving a 105 KB state per ball, median / p99 | Measured, laptop, 6 Oct |
| 2.6 KB / 47.2 KB | Change log written per ball, small / big state | Measured, laptop, 6 Oct |
| about 1.5 ms | Python JSON read and write of the big state | Measured, design doc |
| 0.0044 ms | sha256 of a 10 KB state | Measured, 6 Oct |
| 0.05 ms | Canonical JSON of the same state | Measured, 6 Oct |
| 2,276 a second | Commands the database took, 8 senders | Measured, laptop, Section 4 |
| 0.3 to 1 ms | One trip server to database on AWS, same zone | Estimate, to measure in Section 12 |
| about 9 | Trips per ball as the SQL is written | Counted from the SQL |
| 2 to 3 ms | One ball on prod with 2 trips | Estimate |
| under 5 ms | Target server time per ball, any ball (S9) | Agreed target |
| under 50 ms | A correction at ball 140 during ball 600 (500 steps) | Estimate |
| under 0.3 s | Background check of a full Test (2,700 steps) | Estimate |
| about 100 MB | Memory per sport worker | Estimate, to be measured |
| 50 | Events between checkpoints | From the code, settings.py:296 |
Read next
- Commands and the workflow engine: the one road around this engine: key, lock, version check, answer.
- Writing a workflow in code: how a sport's commands, validations and actions are written, line by line.
- Sport plugins and rule versions: the step contract per sport, versions side by side, and the catalogue of line keys.
- Bridge and live updates: how a second screen sees the new score.
- Stats, feeds and delivery: what happens after the outbox note.