Building AI-ready data foundations
Most stalled AI programmes are not model problems. They are data-contract, lineage and ownership problems that were visible long before the first pilot.
- Define data products with owners and contracts before building pipelines.
- Lineage is the difference between a demo and a system a regulator will accept.
- Quality thresholds should be set per use case, not universally.
- Retrieval workloads need governed content, not just governed tables.
Data products beat data lakes
A lake with no ownership becomes an archive. The organisations moving fastest define a small number of data products — customer, transaction, asset, employee — each with a named owner, a documented schema, a freshness commitment and a quality threshold. Consumers build against the contract rather than against the source system, which is what makes downstream AI work repeatable.
Start with two products tied to funded use cases. Establishing the pattern matters more than the coverage in year one.
Lineage is a control, not documentation
When a model output is challenged — by a regulator, an auditor or a customer — the question is where the input came from and what transformed it. Lineage captured automatically from the pipeline answers that in minutes. Lineage maintained by hand answers it in weeks, if at all.
Unstructured content needs the same discipline
Retrieval-augmented systems have moved policy documents, contracts and support transcripts into the critical path. That content needs the same treatment as tabular data: ownership, currency dates, access classification and a removal process. The most common production failure we see is an assistant confidently citing a superseded policy that nobody withdrew from the index.
- Source registry with owner and review date for every indexed corpus.
- Access filtering applied at retrieval time, not at answer time.
- Automated re-indexing tied to document lifecycle events.
- Citation surfaced to the user in every generated answer.