Temporal and Predictive Systems · Systems Study

State Under Delay

Reconstructing coherent operational reality from asynchronous records, events, and sensor streams.

Constructed benchmark Seven temporal fault classes Event time separated from arrival time Three state dimensions per record

Empirical status. This is a controlled temporal-reconstruction study on a programmatically constructed benchmark in which the authoritative decision-time state of every record is known exactly. Exact reference state is the point of the design: real event streams cannot supply ground-truth decision-time validity labels, so silent temporal failure can only be estimated on them, never measured. The three domain labels denote constructed schema families, and no figure here is a customer record, a production log, an industry delay rate, or a measured Cortonex deployment result. The exact current-source oracle equals benchmark validity by construction; it is a logical ceiling, not a detector.

Abstract

A collection of individually authentic records can fail to describe the state that controlled at a decision time. In some source histories, it can also combine values that were never simultaneously true. The failure arises when sources report different event times, arrival times, revisions, episodes, and validity intervals, yet an operational system combines them as though they were simultaneous. A late event may be newer in processing time but older in event time. A recently delivered snapshot may contain a superseded state. A corrective revision may carry an earlier valid time than the record it replaces. Sensor measurements can arrive after downstream decisions have already been assembled.

We study this problem with a controlled temporal state-reconstruction benchmark containing 24,000 constructed records divided equally among healthcare operations, legal operations, and supply-chain operations schema families. Every case contains three state dimensions, current and prior episodes, current and prior revisions, event times, arrival times, interval-coverage state, source-clock error, and exact decision-time reference state. We inject seven temporal fault classes: delayed arrival, out-of-order burst, clock skew, stale snapshot, revision inversion, missing interval, and compound temporal fault.

The final held-out set contains 4,800 records: 2,000 valid states, 2,800 invalid states, and 2,648 invalid states that the base assembler would release silently. A scalar policy, consisting of the irreversible base hold plus a fitted layer that sees only one processing-time confidence value, holds 258 of 2,800 invalid records, or 9.2%, including 106 of 2,648 base-silent failures, or 4.0%. A trace policy, consisting of the same base hold plus a fitted layer that sees noisy episode, revision, interval, support, lineage, watermark, and clock diagnostics, holds 2,459 invalid records, or 87.8%, including 2,307 base-silent failures, or 87.1%. It false-holds 61 of 2,000 valid records, or 3.05%.

The scalar gate releases 93.6% of the stress suite, but 56.6% of its released records are invalid. The trace gate releases 47.5%, with 15.0% invalidity among released records. The trace result is a controlled observability result under disclosed, deliberately imperfect diagnostic channels. It is not an end-to-end field estimator. An exact current-source oracle separates the benchmark perfectly because it is definitionally equivalent to the validity label. It is not a learned detector, a calibrated probability, or a Cortonex product result.

The central finding is not that one temporal policy solves state reconstruction. It is that recency, arrival order, revision order, event time, and decision-time validity are different relations. A system that collapses them into a single timestamp can remain internally consistent while reconstructing a state that is invalid at the decision boundary and, in some histories, was never jointly true.

Primary held-out results
PolicyInvalid heldBase-silent heldValid false holdsCoverageReleased invalidity
Scalar confidence gate258 of 2,800, 9.2%106 of 2,648, 4.0%49 of 2,000, 2.45%93.6%56.6%
Trace gate2,459 of 2,800, 87.8%2,307 of 2,648, 87.1%61 of 2,000, 3.05%47.5%15.0%
Exact current-source oracle2,800 of 2,800, by construction2,648 of 2,648, by construction0 of 2,000, by construction41.7%0%, by construction

1The operational problem

Operational reality is not stored in one place. It is distributed across transaction systems, documents, event logs, sensor streams, revision ledgers, planning records, and human updates. Each source has its own clock, write path, retry behavior, revision semantics, and delivery latency. The resulting records can all be authentic and locally correct while describing different moments.

Consider an operational decision assembled at time τ\tau. A status event says an item is held. A sensor value says capacity is available. A control ledger says standard handling applies. If the status event describes minute 118, the sensor value describes minute 104, and the control record was superseded at minute 110, their combination may fail to represent the controlling state at the decision time and may not correspond to any state the institution jointly occupied. The error is not fabricated content. It is a temporally invalid join at the decision boundary and can also be an impossible global state.

The problem has several familiar forms:

  • a controlling update arrives after the decision cutoff;
  • an older event arrives after a newer one and wins under processing-time order;
  • an observation-specific or time-varying source-clock error changes reported-time order and can move a superseded observation close to the apparent decision boundary;
  • a newly delivered snapshot contains old state;
  • a higher revision has an earlier valid time than the record it corrects;
  • an interval is missing, so the system silently carries forward the last observed value;
  • several of these conditions occur across different state dimensions.

Distributed-systems research has long separated causal order from physical time [1], and consistent-snapshot work shows that a meaningful global state cannot be obtained by taking arbitrary local states from asynchronous processes [2]. Temporal database research distinguishes when a fact is valid in the modelled world from when it is stored in the database [3]. Stream-processing systems distinguish event time from processing time and use watermarks to reason about temporal completeness under disorder [4,5]. Clock-aware systems expose uncertainty rather than pretending that physical timestamps are exact [6,7]. State-estimation research likewise treats delayed and out-of-sequence measurements as a distinct inference problem [8].

The Cortonex question is narrower and operational:

When an intelligence record is assembled from asynchronous sources, which signals establish that its component facts belong to one coherent decision-time state, and when should the record be held because no such state can be defended?

1.1Contributions and scope

We make six contributions.

First, we define a decision-time observation model that separates event time, arrival time, episode identity, revision precedence, validity interval, source-clock uncertainty, and provenance.

Second, we define strict record validity. A reconstructed state is valid only when its emitted facts are correct, its source episode is current, its controlling revision is current, its validity intervals cover the decision time, and its attached source state supports the emitted facts.

Third, we construct an instrumented temporal benchmark and inject faults into concrete timing, revision, interval, and source-state mechanisms. The benchmark does not assign arbitrary labels after generation.

Fourth, we compare a one-signal scalar gate with an eight-signal temporal trace gate. Their purpose is to measure observability under a fixed protocol, not to establish a universal best classifier.

Fifth, we separate model fitting, probability calibration, threshold selection, and final evaluation. The 4,800-record validation pool is divided into disjoint 2,400-record calibration and 2,400-record threshold-selection subsets. Final results are measured once on a disjoint 4,800-record test set.

Sixth, we report an exact current-source oracle only as a definition-bound ceiling. The oracle is excluded from calibration and ranking claims.

The Cortonex Lab retains the versioned benchmark, record-level outputs, frozen generator, and separate numerical verification materials used to produce the study. Those internal materials are not part of the public release.

1.2Five kinds of quantity

The article contains five epistemically different kinds of quantity.

Assumed design choices include the domain-labelled schema families, exact fault-allocation proportions, 86% material-fault probability, source-lag distributions, diagnostic-noise scales, local-plausibility overlap rates, decision cutoff, 2% validation false-hold budget, and model class. These are not estimates of real institutions.

Constructed observations include source events, prior and current revisions, arrival histories, clock errors, missing intervals, emitted states, diagnostic traces, and exact reference state.

Fitted quantities include logistic-regression coefficients, sigmoid calibration maps, and release thresholds. They are learned only from constructed training and validation records.

Held-out measurements include the 258 of 2,800 and 2,459 of 2,800 invalid-state counts, false-hold counts, Brier scores, calibration errors, risk-coverage values, and fault-level results measured on the 4,800-record test set.

Derived scenarios include the assumed operating-prevalence table and authoritative-refresh budget curve. They transform held-out conditional rates under additional assumptions. They are not deployment forecasts.

2Formal temporal system

2.1Observation state

For entity ii, source jj, and state dimension mm, define an observation

Here:

  • xijmx_{ijm} is the observed value;
  • sijms_{ijm} identifies the source;
  • eijme_{ijm} is the source-reported event-time coordinate, intended to identify when the observation describes the world;
  • aijma_{ijm} is arrival time, when the system receives or stores it;
  • νijm\nu_{ijm} is revision identity;
  • ιijm\iota_{ijm} is episode identity;
  • [bijm,dijm)[b_{ijm},d_{ijm}) is the interval during which the observation is valid;
  • uijm0u_{ijm}\ge 0 is a nonnegative source-clock uncertainty radius.

The corresponding physical event time is represented by an uncertainty interval:

Equation (2) does not create causal order. It states only that a physical timestamp is not necessarily exact. Causal order is represented separately through a happens-before relation \prec [1].

2.2Arrival order is not event order

A processing-time assembler that selects the latest arrival uses

where aτa_\tau is the assembly cutoff in processing time.

Equation (3) answers: which candidate arrived most recently? It does not answer: which candidate controls the state at decision time τ\tau?

An event-time rule instead uses

Equation (4) can still fail when a corrective revision carries an earlier valid time than the record it replaces. A revision-aware rule must treat revision precedence and valid time as separate axes rather than assuming that one timestamp totally orders both.

In Equations (3) and (4), the maximization is defined only when the candidate set is nonempty. An empty candidate set returns \bot and triggers a hold. Equal maximizing timestamps are resolved by a fixed deterministic source order used only to make the benchmark execution single-valued; that order carries no semantic authority.

2.3Decision-time coherence

Let ιi\iota_i^* denote the controlling episode at decision time τ\tau. For each required state dimension mm, define the eligible set

Within that set, the controlling observation is the maximal valid revision:

If Eim(τ)\mathcal{E}_{im}(\tau) is empty for any required dimension, no complete coherent cut exists and the record must be held. If multiple eligible observations share the maximal revision identifier but disagree in value, the state is treated as conflicted rather than resolved by an arbitrary tie-break.

The coherent decision-time cut is

provided the selected observations are causally closed with respect to required dependencies. In practical systems, causal closure may be represented by explicit lineage edges, sequence identifiers, vector or hybrid logical clocks, or application-level revision relations [1,2,6].

A reconstructed state is then

where C^i(τ)\widehat{\mathcal{C}}_i(\tau) is the cut assembled from the observations available to the system at release time.

The crucial distinction is between a cut that is internally coherent and a cut that is authoritative. A delayed controlling update can make the locally available cut coherent but globally stale. No internal transformation can recover an event that has not yet arrived.

2.4Record validity

For benchmark record ii, define five ground-truth conditions:

and

Strict record validity is

Equation (14) is conjunctive. A state can contain the right surface values and still be invalid because it uses the wrong episode, a superseded revision, an interval that does not cover the decision time, or a source record that does not support the emitted state. The constructed benchmark introduces no additional hidden cross-dimensional dependency beyond these conditions. Trace completeness is therefore an observability diagnostic, not a sixth ground-truth validity component.

2.5Base hold and silent temporal failure

The benchmark base assembler holds a record when a required interval is visibly absent, when scalar confidence is below 0.58, or when observed source support is below 0.67. Let Bi=1B_i=1 denote that overt base hold.

A base-silent temporal failure is

Equation (15) identifies the dangerous case: the state is invalid, but the base assembler has no visible reason to stop it.

2.6Fitted release policies

For fitted policy kk, let zi(k)\mathbf{z}_i^{(k)} be its diagnostic signal vector. A logistic model produces an uncalibrated validity score

where σ(u)=(1+eu)1\sigma(u)=(1+e^{-u})^{-1}.

A disjoint calibration subset fits a one-dimensional sigmoid map. In the numerical implementation, qi(k)q_i^{(k)} is clipped to [108,1108][10^{-8},1-10^{-8}] before the logit transform:

The release threshold is selected on a separate threshold subset:

where D^k\widehat D_k is invalid-state detection and F^k\widehat F_k is valid false-hold rate on the threshold-selection subset. The outer maximum in Equation (18) resolves detection ties in favor of the larger observed threshold. The candidate set Θk\Theta_k contains the distinct calibrated scores observed on that subset, together with the endpoints 0 and 1. Both D^k\widehat D_k and F^k\widehat F_k are calculated from the irreversible combined hold in Equation (19), so a fitted layer cannot release a record already held by the base assembler.

Let εnum=1012\varepsilon_{\mathrm{num}}=10^{-12}. At finite implementation precision, calibrated values within this tolerance of a selected threshold are treated as equal to that threshold. This numerical tie rule prevents serialization roundoff from changing a hold decision while preserving a strict comparison outside the equality band.

The final hold is irreversible with respect to the base rule:

Beyond the irreversible base rule, the fitted scalar layer uses only one value, the base assembler's model confidence. The fitted trace layer uses eight diagnostic values:

These correspond to processing-time confidence, episode alignment, revision freshness, decision-time interval coverage, source support, lineage-edge completeness, watermark slack, and source-clock consistency.

2.7Selective risk and coverage

Let Yi(k)(θ)=1Hi(k)(θ)Y_i^{(k)}(\theta)=1-H_i^{(k)}(\theta) denote release. Coverage is

and invalidity among released records is

when at least one record is released. Equations (21) and (22) follow the selective-prediction view that abstention must be evaluated together with retained coverage [9,10].

2.8Probability calibration

For calibrated validity probabilities, the Brier score is

Expected calibration error with bins Br\mathcal{B}_r is

Brier score originates as a proper score for probabilistic forecasts [11]. Reliability diagrams and post-hoc calibration provide complementary evidence, but calibration alone does not establish useful ranking or low released risk [12]. The reported ECE uses ten equal-width bins, [0,0.1),[0.1,0.2),,[0.9,1][0,0.1),[0.1,0.2),\ldots,[0.9,1]; empty bins contribute zero.

2.9Exact current-source oracle

The constructed benchmark contains exact authoritative decision-time state. Define

Under the benchmark definition,

The oracle hold is Hioracle=1OiH_i^{\mathrm{oracle}}=1-O_i. Equation (26) is a logical identity, not empirical detector performance. The oracle is excluded from AUROC, average precision, Brier score, ECE, reliability diagrams, and risk-coverage curves. Where the inherited summary schema retains those fields, their values are null rather than mechanical performance scores.


3Controlled benchmark

3.1Record structure

The benchmark contains 24,000 records, with 8,000 assigned to each of three domain-labelled schema families:

  • healthcare operations;
  • legal operations;
  • supply-chain operations.

The labels provide different typed values and source names. They do not represent real domain evidence, real workflows, or domain-specific performance.

Each entity contains three state dimensions:

  1. a categorical operational status;
  2. a typed operational measure;
  3. a categorical control mode.

Each dimension has a prior value and a current value. Every case also contains a current episode, prior episode, current revision, prior revision, transition time, arrival lag, observation-specific source-clock error, interval state, and exact reference state at decision minute 120. The decision time and processing-time assembly cutoff are conceptually different boundaries, but this normalized benchmark assigns both the coordinate 120: τ=aτ=120\tau=a_\tau=120.

The domain schemas are deliberately simple:

Schema familyStatus examplesOperational measureControl examples
Healthcare operationsREADY, HELD, ESCALATED, PENDINGCapacity indexSTANDARD, RESTRICTED, SURGE, MANUAL
Legal operationsAPPROVED, HELD, COUNSEL_REVIEW, REJECTEDNotice window in daysSTANDARD, PRIVILEGED, COUNSEL_ONLY, EXCEPTION
Supply-chain operationsRELEASED, HELD, CONDITIONAL, ESCALATEDCommitted quantitySTANDARD, QUALITY_HOLD, EXPEDITE, MANUAL

The structured values make state correctness, episode identity, revision identity, interval coverage, and source support exactly checkable without a subjective grader.

3.2Fault assignment

Each record receives one condition according to the following design distribution:

ConditionAssignment probabilityFull benchmark count
Clean execution0.327,680
Delayed arrival0.122,880
Out-of-order burst0.112,640
Clock skew0.112,640
Stale snapshot0.102,400
Revision inversion0.092,160
Missing interval0.092,160
Compound temporal fault0.061,440

For each nonclean record, materiality is sampled with probability 0.86. A material fault changes record validity. A nonmaterial condition preserves the controlling decision-time state while introducing benign temporal disorder or a suspicious diagnostic perturbation used to test specificity.

These probabilities create a fault-rich stress suite. They are not estimates of real operational prevalence.

3.3Fault mechanisms

Delayed arrival. The controlling update is emitted before the decision time but arrives after the decision cutoff. The previous value is carried forward as locally current. Internal revision metadata can remain plausible because the controlling update is absent from the local view.

Out-of-order burst. An older event arrives after the controlling event. Processing-time order selects the superseded value even though the full event history contains the correct state.

Clock skew. An observation-specific timestamp error, representing a changed clock state, device restart, or inconsistent timestamp normalization, moves a superseded event past the controlling event in reported-time order and close to the apparent decision boundary. A constant source offset applied uniformly to both observations would preserve their order and would not create this mechanism. The arrival can be recent and the selected record can appear complete.

Stale snapshot. A newly delivered snapshot contains a prior state, and sometimes a prior episode. Delivery recency therefore points in the wrong direction.

Revision inversion. A higher corrective revision carries an earlier valid time than the record it replaces. Event-time latest selection chooses the lower revision.

Missing interval. A required interval is absent. Some cases expose the gap overtly. Others silently bridge it by carrying forward the last observed value.

Compound temporal fault. Delay, revision, interval, or clock faults occur across two state dimensions. Some compound cases expose a missing field; others remain internally complete.

3.4Diagnostic signal construction

The trace gate does not receive the final validity label or the exact current-source match in Equation (25). It receives bounded constructed diagnostics intended to represent evidence available at the decision cutoff. Several channels use explicit benchmark metadata that a field system would have to infer. The construction deliberately prevents perfect separation.

For episode alignment, revision freshness, interval coverage, source support, and trace completeness, each state dimension first receives a raw local diagnostic in [0,1][0,1]. The record-level raw channel is the arithmetic mean across the three required dimensions. Independent Gaussian measurement noise is then added and the result is clipped to [0,1][0,1].

Diagnostic channelNoise standard deviation
Episode alignment0.055
Revision freshness0.065
Interval coverage0.065
Source support0.045
Trace completeness0.065

Watermark slack uses only the records selected and available at the decision cutoff. For selected dimension mm, let

where a^im\widehat a_{im} is selected arrival time and e^im\widehat e_{im} is selected reported event time. Define LmaxL_{\max} as maximum selected lag, LΔL_{\Delta} as selected-lag spread, and DD as selected reported-event-time spread. The watermark-slack signal is

Here [x]+=max(x,0)[x]_+=\max(x,0), and clip[0,1]\operatorname{clip}_{[0,1]} truncates its argument to the unit interval. No future arrival, canonical current-event timestamp, or exact reference-state value enters Equation (27). It is a constructed completeness heuristic, not a universal watermark formula and not a truth certificate.

Let δim\delta_{im} denote the signed timestamp error of the selected observation for state dimension mm relative to the normalized benchmark clock. In the construction, the selected reported coordinate is the corresponding physical event-time coordinate plus δim\delta_{im}. The error is observation-specific and can therefore represent a changed or inconsistently normalized source-clock state; it is not assumed to be one constant offset shared by every observation from that source. Clock consistency uses its maximum and mean absolute magnitudes, CmaxC_{\max} and C\overline C:

Equation (28) uses known injected observation-level clock error because the benchmark controls the timestamp-generation process. A field system would have to estimate changing offset, drift, restart state, normalization error, or hardware uncertainty from synchronization telemetry, hardware bounds, protocol state, or another clock model. The benchmark does not claim that such estimation is exact.

Outside the material clock-skew mechanism, selected event-time coordinates are normalized to the benchmark clock, while an observation-specific signed timestamp-error variable is retained separately for the clock-consistency diagnostic. In a material clock-skew case, the superseded observation receives an uncorrected error large enough to place its reported event time after the controlling observation while remaining no later than the decision boundary. In a nonmaterial clock-skew case, timestamp error is present, but normalization and revision guards preserve the current state.

To prevent the fitted gate from becoming a disguised oracle, the construction adds class-specific overlap after baseline noise. A selected share of material faults receives locally plausible diagnostic values despite being invalid:

Material fault classLocal-plausibility probability
Delayed arrival0.15
Out-of-order burst0.06
Clock skew0.12
Stale snapshot0.20
Revision inversion0.06
Missing interval0.28
Compound temporal fault0.04

The designated channels are replaced by values sampled from disclosed high-plausibility ranges, generally between 0.70 and 1.00 depending on fault class. Conversely, 10% of nonmaterial temporal perturbations receive one benign suspicious channel sampled uniformly from 0.45 to 0.75. These overlap rates and ranges are design assumptions. They create false negatives and false positives without changing ground-truth validity.

The scalar confidence signal rewards processing-time recency and visible completeness:

where GimissG_i^{\mathrm{miss}} indicates an overt missing field, ζiN(0,0.0352)\zeta_i\sim\mathcal{N}(0,0.035^2), and

Equations (27) through (30), the diagnostic-noise scales, and the overlap assumptions are disclosed so that designed observability remains separate from measured held-out outcomes.

3.5Split and fitting protocol

The benchmark is stratified by domain, fault class, and validity, then divided into:

SplitRecordsUse
Training14,400Fit logistic coefficients
Validation pool4,800Internal model selection only
Calibration subset2,400Fit sigmoid probability calibration
Threshold subset2,400Select release thresholds under the 2% valid false-hold budget
Held-out test4,800Final reported evaluation

The calibration and threshold subsets are disjoint. Test labels are not used to fit coefficients, calibrate probabilities, or select thresholds.

The two principal fitted policies use balanced logistic regression with C=1.0C=1.0. The calibration map in Equation (17) is a one-dimensional logistic regression fitted with C=106C=10^6 and no class weighting on the calibration subset. Thresholds are selected from observed calibrated scores on the threshold subset according to Equation (18).


4Results

4.1Stress-suite composition

The held-out test set contains:

StateRecordsShare of test set
Valid2,00041.67%
Invalid2,80058.33%
Base-silent invalid2,64855.17%
Overt invalid1523.17%

The identities are exact:

and

Among invalid records, 94.57% are base-silent in this stress suite. That percentage is a consequence of the selected fault mix, materiality process, and base-hold definition. It is not an industry estimate.

4.2Temporal faults remain confident

On held-out test records, material delayed-arrival cases have mean scalar confidence 0.843, compared with 0.877 for clean records. Material stale snapshots average 0.886, slightly above the clean-record mean. Processing-time confidence therefore remains high under temporal failure, and a recently delivered stale snapshot can receive more confidence than a clean state.

The observability matrix separates this scalar behavior from the trace channels. Delayed arrivals often preserve episode, revision, interval, and source-support signals in the local view while weakening trace completeness or watermark slack. Clock-skew faults preserve most content-level signals while clock consistency falls. Revision inversions preserve interval and support signals while revision freshness falls. Because the policy receives noisy and deliberately overlapping diagnostics, none of these patterns is perfectly separable.

Figure 1 Temporal observability by fault class Complete constructed benchmark

Mean observed signal values for each constructed temporal fault class, including the definition-bound exact decision-time column.

Heat map of nine temporal diagnostics across the constructed temporal fault classes. High signal is not universally good: a diagnostic can stay high while the decision-time state is already wrong, which is what makes the failure silent.

Figure 1. Temporal observability by fault class. The heat map reports mean processing-time confidence, episode alignment, revision freshness, interval coverage, temporal support, trace completeness, watermark slack, clock consistency, and exact current-source match. The exact-current-source column is a definition-bound oracle signal and must remain analytically separate from the fitted diagnostics.
Figure values

4.3Scalar confidence detects few silent temporal failures

The scalar confidence gate holds 258 of 2,800 invalid records:

It holds 106 of 2,648 base-silent failures:

Processing-time confidence has weak validity discrimination on the held-out stress suite, with AUROC 0.588. Delayed arrivals, stale snapshots, clock-skew faults, and revision inversions can remain complete and recent in the local view.

The scalar gate holds 49 of 2,000 valid records:

The threshold was selected under a 2% false-hold budget on the disjoint threshold subset. Equation (35) is the independent test-set realization and can exceed that validation budget.

4.4Temporal trace separates most fault classes

The trace gate holds 2,459 of 2,800 invalid records:

It holds 2,307 of 2,648 base-silent failures:

It false-holds 61 of 2,000 valid records:

The two-sided nominal 95% Wilson interval for Equation (38) is approximately 2.38% to 3.90%. The interval is a finite-count summary under an iid binomial approximation. The benchmark is stratified and constructed, so it is not a deployment confidence interval.

Fault-level detection is heterogeneous:

Fault classTrace-gate invalid detectionTrace-gate base-silent detection
Delayed arrival59.1%59.1%
Out-of-order burst99.8%99.8%
Clock skew87.4%87.4%
Stale snapshot100.0%100.0%
Revision inversion78.9%78.9%
Missing interval99.7%99.6%
Compound temporal fault99.2%99.1%

Delayed arrivals are the hardest class because the controlling update is absent at the cutoff. Revision inversions also remain difficult when a lower revision has a locally plausible event-time position. Missing intervals are easier in this construction because interval and trace channels usually expose the bridge, although a small residual set remains locally plausible.

Figure 2 Detection by temporal fault class Held-out test set

Base-silent detection for the two fitted policies, with the exact current-source oracle shown as a definition-bound ceiling.

Grouped comparison of detection by fault class for each release policy, with the exact oracle shown as a definition-bound ceiling rather than fitted detector performance. Every value is listed in the figure values table.

Figure 2. Detection by temporal fault class. The default view compares base-silent detection for the scalar and trace gates, with the exact current-source oracle shown only as a deterministic ceiling. Alternate views report all-invalid detection, residual invalidity, and valid false holds.
Figure values

4.5Evidence-channel ladder

The diagnostic ladder fits each row independently under the same train, calibration, threshold, and test protocol.

Evidence available to the policyInvalid detectionBase-silent detectionValid false holdsAUROCBrier
Arrival-time confidence9.2%4.0%2.45%0.5880.2348
Confidence + episode/revision51.3%48.5%1.20%0.7990.1722
+ interval coverage64.1%62.0%1.35%0.9010.1191
+ temporal support/watermark70.8%69.1%1.50%0.9430.0923
Full temporal diagnostic set87.8%87.1%3.05%0.9740.0554
Exact current-source oracle100%, by construction100%, by construction0%, by constructionNot a fitted estimateNot a fitted estimate

The largest selected-operating-point gain occurs when episode and revision diagnostics are added. Interval coverage adds another 12.8 percentage points of invalid detection. Temporal support and watermark slack add 6.7 points. Trace completeness and clock consistency then add 17.1 points and reduce Brier score from 0.0923 to 0.0554.

The ladder is monotone at the selected operating points in this run, but the protocol does not guarantee monotonicity. Each row is fitted, calibrated, and thresholded independently. A higher AUROC does not mathematically require higher detection at one constrained review budget.

Figure 3 Progressive temporal evidence ladder Refit and recalibrated per step

Invalid and base-silent detection as temporal evidence channels are added, each row fitted independently under the same split protocol.

Ladder of independently fitted rows showing invalid and base-silent detection as diagnostic channels are added. Each row is trained, calibrated, thresholded, and evaluated separately, so operating-point behaviour need not improve monotonically. Every value is listed in the figure values table and in the results tables in the article body.

Figure 3. Progressive temporal evidence ladder. The figure shows invalid detection and base-silent detection at each validation-selected operating point, together with false holds, release coverage, AUROC, and Brier score for the fitted rows. The oracle row retains null AUROC and Brier fields because it is a deterministic identity, not a fitted probability policy.
Figure values

4.6Selective risk and release coverage

The scalar gate releases 4,493 of 4,800 records:

Of those releases, 2,542 are invalid:

The trace gate releases 2,280 records:

Of those releases, 341 are invalid:

The trace gate therefore reduces stress-suite invalidity among released records by 41.62 percentage points, but it also routes an additional 2,213 records to hold. The benchmark does not assign a business cost to those holds. A deployment must price review burden, delay cost, and error cost explicitly.

Figure 4 Selective risk versus release coverage Held-out threshold grid

Exact step curves over the fixed threshold grid. The fitted policies cannot reverse the irreversible base hold.

Risk-coverage step curves over the disclosed threshold grid, with the irreversible base hold active at every point. Lower invalidity among released items requires a larger review population. The exact oracle is excluded because it is not a fitted threshold curve. Every value is listed in the figure values table.

Figure 4. Selective risk versus release coverage. The two fitted policies are evaluated on the fixed threshold grid 0.00, 0.01, ..., 1.00. Thresholds that release no records are omitted because Equation (22) is undefined at zero coverage. The irreversible base hold remains active at every displayed point. The exact current-source oracle is excluded because it is not a fitted threshold curve.
Figure values

4.7Calibration

The scalar policy has:

The trace policy has:

The trace policy has a much lower Brier score because its probabilities are both better separated and closer to the validity outcomes. Its ten-bin ECE is slightly higher than the scalar policy's ECE. That is not contradictory. A weak classifier can be calibrated near broad base rates while providing little case-level discrimination. ECE is therefore read together with Brier score, AUROC, operating counts, and risk-coverage behavior.

Figure 5 Reliability diagrams Held-out reliability bins

Mean predicted validity against empirical validity in ten equal-width bins for the two fitted policies. The oracle is excluded.

Reliability diagram comparing mean predicted validity with empirical validity in equal-width held-out probability bins for the two fitted policies. The oracle is excluded because it is not a calibrated probability model. Every value is listed in the figure values table.

Figure 5. Reliability diagrams. The two fitted-policy diagrams compare mean predicted validity with empirical validity in ten equal-width bins. The exact current-source oracle is excluded because it is not a calibrated probability model.
Figure values

4.8Authoritative refresh as a constrained budget

The trace gate cannot detect every delayed or locally plausible state from internal evidence. We therefore evaluate a logical overlay in which exact authoritative current-source comparison is applied to a seeded nested subset of test records. The overlay can add holds but cannot reverse a trace hold.

Let Qi(u)Q_i^{(u)} indicate whether record ii is selected for exact refresh at budget fraction uu. The mixed hold is

At zero refresh, Equation (45) equals the trace gate. At 50% refresh coverage, base-silent detection rises from 87.12% to 93.69%, while residual base-silent failure across the entire test set falls from 7.10% to 3.48%. At full refresh coverage, every invalid record is held by construction.

Full-coverage release is 40.40%, lower than the standalone oracle's 41.67%, because the overlay preserves the trace gate's 61 valid false holds.

Figure 6 Authoritative-refresh budget Random assignment; coverage budget

Base-silent detection and residual risk as a function of the fraction of records granted exact decision-time refresh.

Dual-axis budget curve over increasing exact-recheck coverage. Greater idealized recheck coverage removes residual base-silent failures while preserving holds already imposed by the trace policy. This is an architectural budget ceiling, not measured verifier performance. Every value is listed in the figure values table.

Figure 6. Authoritative-refresh budget curve. The x-axis is the fraction of test records assigned to exact refresh. The lines report base-silent detection, residual base-silent failure over all records, and stress-suite release coverage. This is a logical budget calculation over seeded nested subsets, not a measured verifier operating curve.
Figure values

4.9Record-level traces

Aggregate metrics conceal the mechanisms that matter operationally. The trace explorer therefore presents six explicitly constructed held-out examples.

One delayed-arrival case carries forward a prior operational measure, 0.72 rather than 0.88, because the controlling update arrives after the decision cutoff. Its scalar confidence is 0.798. Its trace probability is 0.23912, about 0.00252 above the selected threshold 0.23660, so the fitted trace gate releases it. This is the clearest residual limit in the study: internal evidence cannot directly inspect an update that has not arrived.

An out-of-order case emits APPROVED rather than COUNSEL_REVIEW after an older legal-status event arrives later in processing time. The trace gate holds it with probability 0.00082 of validity.

A clock-skew case emits READY rather than PENDING after a positive observation-specific timestamp error moves a superseded clinical-status observation to reported time 119.4. Its scalar confidence is 0.961, but clock and revision diagnostics produce a trace hold.

A stale-snapshot case emits capacity 0.59 rather than 0.57 from a newly delivered prior revision. A revision-inversion case emits 0.73 rather than 0.62 because event-time order selects a lower revision. A missing-interval case silently carries forward a 7-day measure instead of the current 21-day measure; its trace probability is 0.22584, below threshold, so it is held.

Figure 7 Constructed temporal trace explorer Held-out examples

One constructed example per temporal fault condition: query, expected state, emitted state, signals, and each gate decision.

Interactive trace explorer over six held-out records, one per temporal fault condition. Each trace shows the authoritative state, selected record, revision state, emitted state, attached source, injected fault, diagnostics, and the release decision under each policy.

Figure 7. Constructed temporal trace explorer. Each example shows the query, expected state, emitted state, descriptive source label, selected source text, authoritative decision-time state, fault mechanism, scalar confidence, trace probability and decision, and exact oracle decision. The examples are constructed benchmark records, not customer records.

Print view shows the delayed arrival trace. The remaining traces are available in the online version.

Figure values

4.10Domain-labelled stability

The trace result is similar across the three schema families:

Schema familyTest recordsInvalid shareTrace invalid detectionTrace valid false holds
Healthcare operations1,60158.90%87.59%3.34%
Legal operations1,60158.15%87.00%3.58%
Supply-chain operations1,59857.95%88.88%2.23%

The similarity is expected because the schemas share one generator and fault model. It is not evidence of domain transfer.

4.11Conditional operating-prevalence scenarios

The stress suite has 58.33% invalid prevalence by design. To show why released risk changes with prevalence, let π\pi be an assumed operating invalid prevalence, dkd_k the held-out invalid-detection rate, and fkf_k the held-out valid false-hold rate.

The implied hold rate is

and invalidity among released records is

At an assumed π=0.05\pi=0.05, the trace-gate scenario implies a 7.29% hold rate, 92.71% release rate, and 0.657% invalidity among released records. The scalar-gate scenario implies a 2.79% hold rate and 4.67% invalidity among released records.

Equations (46) and (47) assume that conditional detection and false-hold rates transfer unchanged into the new environment. That assumption is unverified. These are sensitivity calculations, not production forecasts.


Interactive Prevalence scenario calculator Scenario, not a field estimate

Projection of hold, release, and residual rates from the measured held-out conditional rates at a reader-chosen invalid prevalence.

Scenario calculator over an assumed invalid prevalence chosen by the reader, applying the measured held-out conditional rates of each policy. Scenario projection only: the prevalence is not estimated from the constructed stress suite, and the calculation assumes those rates transfer unchanged.

Conditional operating-prevalence scenarios. Estimated hold rate, release rate, and invalidity among released records under an assumed operational invalid prevalence. This is conditional algebra applied to the held-out operating points, not a forecast.
Figure values

5What the calculations establish

5.1A correct source is not enough

Every source can be authentic. Every transformation can preserve its input. Every citation can accurately quote the selected record. The assembled state can still be wrong because the selected records do not belong to one coherent decision-time cut.

This is a different failure class from unsupported generation. The evidence can support the answer exactly while the answer remains temporally invalid.

5.2Latest is not a coherence operator

The word latest is underspecified. It can mean:

  • greatest arrival time;
  • greatest transaction time;
  • greatest event time;
  • greatest revision number;
  • greatest valid-time start;
  • greatest causal timestamp;
  • most recently observed snapshot.

Those orders need not agree. A production specification that says only latest record has not defined temporal control.

5.3Freshness is a vector

Freshness is not one scalar property. A record can be fresh in delivery time and stale in event time. It can be current in event time and superseded in revision order. It can be current within the wrong episode. It can be recent while its validity interval does not cover the decision time.

A defensible temporal record should therefore preserve, where available:

5.4A watermark is not a truth certificate

Watermarks are valuable because they provide a principled signal about stream completeness under unbounded disorder [4,5]. They do not prove that a source is authoritative, that a clock is correct, that a revision controls the decision, or that no upstream producer will later correct its record.

The benchmark reflects that distinction. The selected-record watermark signal improves observability, but it does not dominate the study and it remains imperfect. The full trace still benefits materially from lineage completeness and clock consistency, while delayed arrivals remain difficult without authoritative refresh.

5.5Revision order can dominate event time

A corrective revision can validly describe an earlier event. If the system selects only the largest event timestamp, it can discard the controlling correction. Revision identity and valid time must therefore remain distinct.

This is the temporal analogue of source lineage. A conclusion must preserve not only which event supported it, but which revision and episode made that event controlling.

5.6Holding is part of reconstruction

A state-reconstruction system does not always have enough information to return a coherent state. When the controlling update may not have arrived, the technically correct output can be a hold, an uncertainty interval, or an explicit statement that no defensible cut is available.

The trace gate's 47.5% stress-suite release coverage is not a proposed production target. It demonstrates the cost of strict temporal validity under an intentionally dense fault distribution. A real operating threshold must be selected against measured prevalence, review capacity, error cost, and latency requirements.


6Architectural implications

6.1Preserve both valid time and transaction time

Systems that retain only storage time cannot reconstruct what was believed at an earlier moment or determine when a fact was valid in the modelled world. Bitemporal storage is not required for every source, but the distinction must survive wherever later reconstruction matters [3].

6.2Treat episode identity as a first-class key

Entity identity alone is insufficient when the same entity can enter multiple episodes, matters, orders, encounters, cases, or operating cycles. A current value from the wrong episode is not current for the decision.

6.3Keep revision precedence explicit

Revision should not be inferred from arrival time or event time. If the source system has an authoritative revision order, preserve it. If it does not, the absence is itself part of the uncertainty state.

6.4Carry clock uncertainty forward

A physical timestamp without an uncertainty model can create false precision. Systems such as Spanner expose uncertainty explicitly [7], while hybrid logical clocks combine causal order with physical-time proximity [6]. The appropriate mechanism depends on the deployment, but clock quality should not disappear during normalization.

6.5Separate temporal support from content support

A source can support the emitted value while failing to establish that the value controlled at τ\tau. Content support asks whether the source says the fact. Temporal support asks whether the source state is eligible for the decision-time cut. Both are required.

6.6Make completeness conditional and inspectable

A watermark, interval ledger, or causal frontier should be retained as evidence, not collapsed into a hidden confidence adjustment. Reviewers need to know whether a state was assembled after all expected sources crossed a frontier, whether one source was late, and which dimensions were carried forward.

6.7Preserve correction history

Late events can require correction of previously released state. A temporal intelligence record should be versioned so a later update can show:

  • the prior reconstructed state;
  • the newly arrived source;
  • the changed controlling cut;
  • the conclusions affected;
  • the owner route and release state.

6.8Distinguish exact refresh from internal inference

An exact comparison against authoritative current state is stronger than inference from local traces, but it may be expensive, unavailable, or itself delayed. It should be treated as a separate architectural channel with an explicit coverage budget, not as an invisible assumption.


7Limitations

This study is a controlled benchmark, not field measurement.

First, the three domain labels are schema families. They do not encode the complexity, semantics, regulation, or error costs of real healthcare, legal, or supply-chain environments.

Second, the benchmark has three state dimensions and one fixed decision time. Real systems can contain thousands of dimensions, nested episodes, overlapping intervals, probabilistic sensors, contradictory authorities, and decisions that evolve continuously.

Third, source lags, clock errors, fault proportions, materiality probability, diagnostic-noise scales, local-plausibility overlap rates, confidence formula, watermark-slack formula, and clock-consistency formula are design choices. Different constructions will produce different rates.

Fourth, the benchmark has exact authoritative state. Many real institutions do not. Sources can disagree about what should count as authority, and the controlling rule can itself change.

Fifth, the raw temporal diagnostics are generated from explicit constructed metadata, then perturbed by disclosed noise and overlap rules. A field system would have to estimate episode identity, revision authority, interval coverage, source support, lineage completeness, watermark state, and clock error from imperfect operational evidence. The reported trace result does not measure that upstream estimation problem.

Sixth, the watermark signal is a bounded heuristic over selected records available at the cutoff. It does not implement a production stream processor's source-specific watermark semantics, and it does not prove completeness or authority.

Seventh, the trace gate is a small diagnostic model. It is not an implementation of a production temporal graph, causal database, stream processor, or state estimator.

Eighth, the benchmark evaluates detection and selective release. It does not measure automatic correction quality, reviewer time, downstream utility, latency, throughput, storage cost, or security properties.

Ninth, the Wilson intervals use an iid binomial approximation. The benchmark is stratified and constructed, so the intervals are descriptive finite-count summaries rather than design-based uncertainty statements.

Tenth, the conditional prevalence calculations assume stable conditional rates under prevalence shift. That assumption can fail under domain shift, changed fault mix, changed source quality, or changed review policy.

Eleventh, exact authoritative refresh is a logical overlay. It does not include the latency, access, permission, or availability costs of reaching an authoritative source.

Twelfth, the article establishes no Cortonex production-performance claim. Field evaluation is required before any deployment-specific reliability statement can be made.

Thirteenth, interval semantics are represented by decision-time coverage diagnostics, and causal structure is represented by per-dimension lineage diagnostics. The benchmark does not evaluate arbitrary interval algebra or reconstruction over a full partial-order graph.

Fourteenth, clock consistency uses known injected observation-specific timestamp error before disclosed noise and overlap. A constant offset applied uniformly to all observations from one source would not reverse their internal order; the material clock-skew mechanism instead represents time-varying error, changed clock state, device restart, or inconsistent timestamp normalization. Real systems may have only uncertain offset and drift estimates or hardware bounds. The benchmark therefore measures the value of a clock-quality channel, not the practical accuracy of a particular clock-estimation system.

8Study specification and verification

The corrected study pipeline was executed twice from the same fixed seed and frozen protocol. The two runs produced byte-identical benchmark records, held-out scores, internal summaries, and environment metadata. A deterministic public-projection step then produced the compact aggregate payload under the unchanged figure schema. The frozen generator produced all source states, fault mechanisms, diagnostics, split assignments, fitted-policy inputs, and reference labels.

A separate audit implementation, which does not import the generator, reconstructed record validity, fault semantics, diagnostic formulas, fitted probabilities from retained coefficients, threshold selection, held-out metrics, calibration, risk-coverage curves, fault summaries, domain summaries, authoritative-refresh scenarios, prevalence scenarios, trace examples, and the public aggregate payload. It completed 104,529 protocol and record-level checks with no failures. A separate public-projection audit completed 2,500 checks over the article-facing payload, exact recursive field-key and element-type compatibility, oracle null semantics, and every aggregate figure value. A content audit completed 193 checks, and all 54 display equations rendered without a MathJax error.

The verification procedure specifically checked:

  • exact benchmark, domain, fault, and split counts;
  • disjoint training, calibration, threshold-selection, and test identifiers;
  • absence of future-arrival and canonical-current-state inputs from the watermark signal;
  • material and nonmaterial fault semantics at record level;
  • record-validity identities in Equation (14);
  • the silent and overt invalid partition in Equations (31) and (32);
  • reconstruction of every fitted probability from retained coefficients;
  • threshold feasibility and maximal invalid detection under Equation (18);
  • every held-out count, rate, Brier score, ECE value, and Wilson interval;
  • fixed-grid risk-coverage points;
  • fault-level and domain-level summaries;
  • authoritative-refresh and conditional-prevalence calculations;
  • exact equality Oi=ViO_i=V_i for every benchmark record;
  • exact recursive compatibility with the previous study's figure-payload schema;
  • consistency between the article, aggregate figure payload, and retained private outputs.

The public figure payload contains compact aggregate values and six explicitly constructed trace examples. It contains no full benchmark, record-level evaluation table, model coefficients, source generator, fault-injection implementation, run log, or environment file.

Data and code availability. This study uses a constructed reference benchmark. The experimental design, benchmark composition, temporal fault conditions, metrics, aggregate results, and limitations are documented in this publication. Record-level benchmark data and the reference implementation are retained by The Cortonex Lab and are not publicly distributed.

9Conclusion

An operational record can be fully sourced and still fail to describe the state that controlled at the decision time. In some histories, it may also describe a combination that was never jointly true. The failure occurs when source identity is preserved but temporal control is not.

The constructed benchmark shows why processing-time confidence is insufficient. Recent arrival can reward stale snapshots. Event-time latest can reject a controlling correction. A source can support a value while failing to establish that the value controlled at the decision time. A watermark can indicate likely completeness without proving authority. A physical timestamp can appear precise while hiding clock uncertainty.

The practical requirement is therefore stronger than freshness. A consequential intelligence record should preserve the temporal relations that make its state defensible: event time, arrival time, validity interval, episode, revision, clock uncertainty, causal context, source support, and release state.

The Cortonex Lab treats reconstruction as a governed systems problem. When the available observations do not define a coherent decision-time cut, the system should not manufacture one by collapsing time into a single field. It should expose the uncertainty, preserve the competing state histories, and hold the record until the temporal boundary can be defended.


Appendix A. Exact benchmark signal definitions

For each diagnostic family cc and state dimension mm, the latent local value dimcd_{imc} lies in [0,1][0,1]. It need not be binary. The raw record-level channel is

and the baseline observed channel is clip[0,1](Dic+ϵic)\operatorname{clip}_{[0,1]}(\overline D_{ic}+\epsilon_{ic}) with the noise scales reported in Section 3.4. Class-specific local-plausibility replacements and benign false-alarm perturbations are applied after this baseline noise. The policy therefore sees a noisy diagnostic, not the exact validity component.

For the selected records available at the decision cutoff, lag, lag spread, and reported-event-time spread are

Maximum and mean absolute injected observation-level timestamp error are

The base hold is

All random perturbations, local-plausibility assignments, benign false-alarm assignments, split assignments, and fitted-model seeds are fixed for the reported benchmark.

Appendix B. Wilson interval

For xx observed events in nn trials, with p^=x/n\widehat p=x/n and z=1.9599639845z=1.9599639845, the two-sided nominal 95% Wilson interval is

For the trace gate's 61 false holds among 2,000 valid test records, Equation (52) gives approximately 2.38% to 3.90%.

Study specification and verification reference benchmark implementation, not production Cortonex software
Version1.4
Benchmark seed2026072802
Benchmark size24,000 constructed records
Splits14,400 train / 4,800 validation / 4,800 test

Empirical status. A controlled temporal reconstruction study on a programmatically constructed benchmark with exact decision-time reference state, chosen so that the validity of every record is known and silent temporal failure can be measured rather than estimated. Every quantity reported here is a benchmark measurement, not a customer record, production log, industry delay rate, or measured Cortonex deployment result.

The protocol was frozen before the final seed was drawn, and the published operating-point metrics were regenerated from that frozen benchmark version and cross-checked through a separate numerical verification implementation. Internal freezing is the assurance used here; the study was not externally preregistered or peer reviewed. Record-level data and the reference implementation are retained by Cortonex and are not publicly distributed.

Cite this study

The Cortonex Lab. State Under Delay: Reconstructing Coherent Operational Reality from Asynchronous Records, Events, and Sensor Streams. Version 1.4. Cortonex Technologies Inc. https://cortonex.com/lab/state-under-delay/

@techreport{cortonexlab-state-under-delay,
  author      = {{The Cortonex Lab}},
  title       = {State Under Delay: Reconstructing Coherent Operational Reality
                 from Asynchronous Records, Events, and Sensor Streams},
  institution = {The Cortonex Lab, Cortonex Technologies Inc.},
  version     = {1.4},
  url         = {https://cortonex.com/lab/state-under-delay/},
  note        = {Controlled constructed temporal benchmark;
                 no production-performance claim.}
}

References

[1] Leslie Lamport. “Time, Clocks, and the Ordering of Events in a Distributed System.” Communications of the ACM, 21(7), 558-565, 1978. DOI: 10.1145/359545.359563.

[2] K. Mani Chandy and Leslie Lamport. “Distributed Snapshots: Determining Global States of Distributed Systems.” ACM Transactions on Computer Systems, 3(1), 63-75, 1985. DOI: 10.1145/214451.214456.

[3] Richard T. Snodgrass and Ilsoo Ahn. “A Taxonomy of Time in Databases.” Proceedings of the 1985 ACM SIGMOD International Conference on Management of Data, 236-246, 1985. DOI: 10.1145/318898.318921.

[4] Tyler Akidau, Robert Bradshaw, Craig Chambers, Slava Chernyak, Rafael J. Fernández-Moctezuma, Reuven Lax, Sam McVeety, Daniel Mills, Frances Perry, Eric Schmidt, and Sam Whittle. “The Dataflow Model: A Practical Approach to Balancing Correctness, Latency, and Cost in Massive-Scale, Unbounded, Out-of-Order Data Processing.” Proceedings of the VLDB Endowment, 8(12), 1792-1803, 2015. DOI: 10.14778/2824032.2824076.

[5] Tyler Akidau, Edmon Begoli, Slava Chernyak, Fabian Hueske, Kathryn Knight, Kenneth Knowles, Daniel Mills, and Dan Sotolongo. “Watermarks in Stream Processing Systems: Semantics and Comparative Analysis of Apache Flink and Google Cloud Dataflow.” Proceedings of the VLDB Endowment, 14(12), 3135-3147, 2021. DOI: 10.14778/3476311.3476389.

[6] Sandeep S. Kulkarni, Murat Demirbas, Deepak Madappa, Bharadwaj Avva, and Marcelo Leone. “Logical Physical Clocks.” Principles of Distributed Systems, Lecture Notes in Computer Science 8878, 17-32, 2014. DOI: 10.1007/978-3-319-14472-6_2.

[7] James C. Corbett, Jeffrey Dean, Michael Epstein, Andrew Fikes, Christopher Frost, J. J. Furman, Sanjay Ghemawat, Andrey Gubarev, Christopher Heiser, Peter Hochschild, Wilson Hsieh, Sebastian Kanthak, Eugene Kogan, Hongyi Li, Alexander Lloyd, Sergey Melnik, David Mwaura, David Nagle, Sean Quinlan, Rajesh Rao, Lindsay Rolig, Yasushi Saito, Michal Szymaniak, Christopher Taylor, Ruth Wang, and Dale Woodford. “Spanner: Google's Globally-Distributed Database.” 10th USENIX Symposium on Operating Systems Design and Implementation, 251-264, 2012.

[8] Keshu Zhang, X. Rong Li, and Yunmin Zhu. “Optimal Update with Out-of-Sequence Measurements.” IEEE Transactions on Signal Processing, 53(6), 1992-2004, 2005. DOI: 10.1109/TSP.2005.847830.

[9] Yonatan Geifman and Ran El-Yaniv. “Selective Classification for Deep Neural Networks.” Advances in Neural Information Processing Systems 30, 4878-4887, 2017.

[10] Yonatan Geifman and Ran El-Yaniv. “SelectiveNet: A Deep Neural Network with an Integrated Reject Option.” Proceedings of the 36th International Conference on Machine Learning, PMLR 97, 2151-2159, 2019.

[11] Glenn W. Brier. “Verification of Forecasts Expressed in Terms of Probability.” Monthly Weather Review, 78(1), 1-3, 1950. DOI: 10.1175/1520-0493(1950)078<0001:VOFEIT>2.0.CO;2.

[12] Chuan Guo, Geoff Pleiss, Yu Sun, and Kilian Q. Weinberger. “On Calibration of Modern Neural Networks.” Proceedings of the 34th International Conference on Machine Learning, PMLR 70, 1321-1330, 2017.