There has never been a better time to turn a webpage into structured data. A model reads the page, you hand it a schema, and out comes clean JSON — a price, a date, a name, a list. It feels solved.
It isn’t. Nothing in that pipeline checks the JSON against the page.
Models are confident, and confidence is not accuracy. They will return a price that never appeared, a date off by a year, a name that belongs to someone else — and they return it in exactly the same shape as a correct answer. Write that straight into a database, a retrieval index, or an agent’s next decision, and you have introduced silent corruption: no error, no exception, just a value that is quietly wrong.
I keep coming back to this because it sits underneath almost everything I build right now. Agents are only as trustworthy as the data they act on, and the open web is the least trustworthy source there is. The interesting work has moved one layer up from extraction: not “can I get the data,” but “can I prove the data is faithful to its source.”
Faithfulness is harder than it looks, which is exactly why it is worth doing well. You have to ground each field back to the span of text it came from. You have to reconcile representations — “$1,299” and 1299.0 are the same; “Q3” and “third quarter” are the same; a rounded number is sometimes fine and sometimes a bug. You have to tell a paraphrase apart from an invention. And you have to be calibrated, because a verifier that cries wolf is worse than no verifier at all.
None of this is glamorous, and an autonomous agent cannot one-shot it from a single prompt. That is the point. The durable parts of this field are the ones that take accumulated craft — a calibrated checker, a labeled benchmark, the long tail of edge cases — and those compound. They get better every time you find a new way for an extraction to lie.
So the rule I try to hold myself to is simple: treat verification as a feature, not an afterthought. Before anything depends on a value, prove it came from somewhere real. It is the least exciting line in the changelog, and the one that keeps the whole system honest.