When a web shop fails, the error shows up far from where it started. A checkout page times out because payment is slow, because a link is dropping packets, because a radio is weak. Engineers trace back through metrics, traces and logs until they find the one thing that broke.
Language models are now pitched for that job. But most evaluations grade a free-text answer with another model, and say little about which evidence the answer rests on, or whether the model can tell a network problem from an application one.
Rootline grades by evidence: every line a model sees has an ID, and an answer only counts if it cites lines that exist.
prompt.py · the exact instructionsevery model, every run
# excerpt
You are the on-call engineer for a microservice web shop. Below is telemetry from a 10-minute window; "fault onset" is the midpoint. The window may contain a fault, or the system may be healthy.
Find the root cause using only the evidence below.
Layers: application (the service itself is broken), network (the link between services is degraded), none (healthy).
Rules: give up to 3 hypotheses ranked best first; cite only IDs that appear below; confidence is your probability that the hypothesis is right.