Since the release of ChatGPT, model performance has advanced at an unprecedented pace. Today’s models are better at reasoning, accept longer context, and can execute tasks across different areas. These models are trained on massive amounts of data, enabling them to excel at tackling complex problems. 

However, success with AI agents requires more than an intelligent model. The expression “Garbage In, Garbage Out” still holds in the AI era. Without correct contexts, models will not be able to provide accurate answers. As Rita Sallam, Distinguished VP Analyst at Gartner said, “Agentic AI outcomes depend on context including semantic representations of data. Without context … AI agents cannot operate accurately and are far more likely to hallucinate, introduce bias and produce unreliable results.”*

*Gartner Says Lack of Semantics Causes Inaccurate AI Agents and Wasted Spending - May 2026.

Why do you need to provide the right context?

One common use case that relies heavily on context is analysing data using natural language. Commonly known as Text2SQL, these use cases involve converting natural language questions into SQL queries. This is a critical component in many AI systems, as it provides an efficient way for users to query the database using their instructions. However, this is never a simple task. To generate a correct SQL query, the system must understand the user’s question and the table schema and produce correct syntax. 

As such, there are 3 potential points of failure that could impact the quality of an LLM-developed SQL query:

As illustrated in the table above, improving the model's capability alone may not necessarily improve the quality of the generated SQL. One must use a good model and provide the right context in order to generate the correct query. This brings us to the AI topic that is on everyone’s mind - Context Engineering. 

The Rise of Context Engineering

Context engineering gained traction in mid-2025, as a progression from prompt engineering. 

Taking the definition from the distinguished AI researcher Andrej Karpathy:

Context engineering is the delicate art and science of filling the context window with just the right information for the next step. 

The key words in Andrej’s statement are “Just the right information”. Context engineering is never about pushing all the information into the model, but managing the right amount of information for the LLM to process effectively. We have discussed the drawbacks of pushing as much context to LLM in this blog post here. In essence, too much context will deteriorate LLM models’ performance. 

That’s why it’s important to decide which information provided can help the model carry out its tasks:

* https://trends.google.com/explore?q=Context%2520Engineering&date=today%205-y&geo=Worldwide

* https://x.com/karpathy/status/193790220576560762are6

This is why building context from scratch is challenging: effective context must carefully balance the length with information completeness to avoid hallucinations. 

Effective Context Engineering for Text2SQL

So what does providing the ‘right’ context for Text2SQL actually look like, and how can you package all of it so the agent can access it reliably without drowning in details?

Semantic models provide a governed space for defining metrics, synonyms, and table join keys, rather than reassembling context for every question. Anthropic reported a similar pattern: their analytics agent climbed from no higher than 21% accuracy to consistently above 95% once they gave it curated, well-structured context that pointed the agent to the right tables and definitions.  A semantic model is the same context in governed form.

However, a semantic model only helps if it reaches the agent the right way. If the model is large, the agent probably doesn’t need the entire model for every question. It may only need the definition of a metric, the right join, or a verified query. The same goes for values in the data itself: if a user asks for the “west region,” the agent may need to check how that value is actually represented in the column before generating SQL. The goal is not to include the entire semantic model in the context window, but to make the relevant parts available when needed.

And the hardest part is organizational, not technical. The context that matters most, like how a metric is defined or which exceptions apply, all live in people’s heads, or are scattered in spreadsheets and BI tools. The reality is that capturing this context requires the business and IT to define context together. IT owns the data, while the business owns its meaning. Establishing this shared ownership is what separates the AI that organizations trust from the AI they abandon.

Why is it easier to do it in Dataiku?

The barriers we’ve described to effective context engineering are less about the technology itself than about where the context gets defined and who owns it. This is largely a question of where the semantic layer lives.

In Dataiku, semantic models live above the database rather than inside any one of them. Metadata follows the dataset, so that a definition can be applied regardless of which data warehouse it lives in. 

Aside from where the data sits, it’s important to also consider that context should be managed in a collaborative environment where technical and business users work together on the same platform. The shared ownership that context depends on - IT for the data, the business for its meaning - doesn't require setting up a new process or team. It builds directly on where those groups already collaborate.

None of this removes the real work. The hardest part is still the ongoing effort to keep definitions accurate as the business evolves. But keeping that work in a single, governed, technology-agnostic environment, close to the data and the people who understand it, is what makes it sustainable rather than becoming another source of technical debt.

References

• DCG-SQL: Enhancing In-Context Learning for Text-to-SQL with Deep Contextual Schema Link Graph 

• How to Prompt LLMs for Text-to-SQL: A Study in Zero-shot, Single-domain, and Cross-domain Settings (In-context learning)

• Towards Robustness of Text-to-SQL Models against Synonym Substitution

• How Anthropic Enables Self-Service Data Analytics with Claude

• Dataiku Documentation - Semantic Models