Reconciliation by Hand
The company collects listings from the largest real estate portals in each market and sells the combined data on. The pipeline has three steps. Collection brings in raw pages. Normalization maps each one onto a standard schema: property type, address, postcode, rooms, area, price. Deduplication merges the records that describe the same property, and the same agent, across portals.
Checking that work is reconciliation by hand: open the record, open the source, compare field by field, move on. It reaches only a small share of the data. Hard cases are settled by whoever happens to review them, and the reasoning leaves with that person.
Shortcuts That Cost Trust
- Handing the whole check to a model. It is faster, but the borderline cases get settled by a machine nobody can question.
- Putting a model inside the matcher. Every merge has to be reproducible and explainable to a customer. A model that can answer differently on the same input breaks that promise.
- Routing on the model's opinion of itself. A model reports its own confidence just as firmly when it is wrong, so routing on that number lets mistakes through.
- Re-visiting every page. Opening the live listing again for every record is slow and expensive. Verification that costs as much as collection will not be run often enough.
- A verdict with no trail. "Checked, correct" with nothing behind it cannot be confirmed or overturned.
Three Levels of Checking
The layer sits beside the pipeline: the crawlers, the normalization and the matcher stay as they are. It reads their output and checks every record, and each check stops at the cheapest level that can resolve it.
- Rules. Deterministic checks compare each extracted value with the raw page stored at collection: exact and normalized matches, units, ranges, references. There is a versioned rule pack per portal and country, and what the rules settle never reaches a model.
- Model. Only what rules cannot settle. The relevant fragment of the page is selected first, and the model returns a verdict, a corrected value and the exact source text it relied on.
- Routing. Evidence decides what goes to a person: whether the cited text is really in the source, whether repeated runs agree, whether two different model families agree, whether the value departs from that portal's usual pattern. Where the signals disagree, the case is borderline by definition.
- Reviewers. Borderline cases reach people in a queue ranked by disagreement and field impact, along with a blind random sample that keeps the automatic levels honest.
- Trail. Every verdict is stored with what was compared, the source fragment, the deciding level and the reason.
Drift monitoring runs alongside, per source and with no model calls, so a portal redesign is flagged before the data goes wrong.
Technology Stack
Who Decides What
Routine comparisons belong to the layer; judgment calls belong to reviewers.
- Borderline and complex cases always go to a person. However confident the automation looks, it leaves them to a reviewer.
- Any case can be checked by hand. A person can open any record, escalated or not, see the reasoning in one place, and confirm or overturn it.
- The model verifies and the matcher merges. The matcher stays deterministic. A contradiction between portals goes to a person instead of being silently corrected.
- People are measured too. The reference set uses two independent reviewers per record and a third to settle disagreements.
Coverage Targets
What follows is the coverage the layer is built to provide.
- Every record checked, not a sample.
- No borderline case settled without a person.
- Any automated verdict can be opened with its evidence and reason, and overturned.
- A matcher with no model components.
- Accuracy measured against the human reference per country, portal and record type, so weak groups stay visible instead of hiding inside an overall figure.
Other Data Businesses With the Same Risk
A business that sells data assembled from third-party sources it does not control carries the same risk:
- reconciliation is done by hand, record by record;
- only a small share of the data ever gets checked;
- some cases need a person's judgment rather than a rule;
- every decision about the data has to be explainable afterwards.
Job boards, vehicle marketplaces, multi-seller product catalogs and company registry aggregators are typical cases. The same layer can sit beside other pipelines, and the share of cases sent to people is set with the business that owns the data. See human-in-the-loop and LLM evaluation in our glossary for the methods, and our big data consulting and data analytics services for how such projects are run.
of records checked, not a sample
countries in the provider's data