A team at a Fortune 500 company was developing a series of interrelated data models. These data models would drive the product UI, including an AI agent that was key to product adoption and retention. But the data model content — API names and descriptions — didn’t meet standards that would provide good context for the LLM that powered the agent. I made a major improvement in LLM context for the agent by rewriting the content to clearly represent the model’s inherent structure and relationships.
Results
25%
LLM context improvement at release
The situation
Agent performance depended on the data model
The models the team was developing would support agents that answered questions using an organization’s own structured data — such as schedules, records, current status — and could take action based on that data, like processing a request or scheduling something on a user’s behalf. The agent’s performance depended on how clearly the underlying model represented that data.
The Challenge
The structure and relationships inherent in the model weren’t clear
API names are sometimes treated as cosmetic labels. But in large or growing models, poor API names aren’t a cosmetic problem.
If the names and descriptions in a model aren’t perfectly clear, AI is happy to take a stab! But guessing can lead to hallucination, SQL problems, and degraded search.
The names and descriptions that an engineering team drafts can make a solid start. But settling for them as final can make product performance and maintenance suffer later. Garbage in, garbage out.
Well structured data model content also keeps costs down by reducing token usage.
I needed to show the engineers how taking the time to name and define fields precisely was the fastest path to a model that would perform as planned.
[write a sidebar about linguistics and word context and language models. Writing API names that are consistent, meaningful, concise, and that conform to technical constraints: this is a job that calls for linguistic expertise]
“Cate made linguistics and design principles a foundation of her work, resulting in productive discussions and outcomes based on something more solid than opinion.”
Lead Member of Technical Staff, Fortune 500 Company
My approach
I reviewed fields against three criteria, iterating with engineering and product management to resolve questions.
While I worked through these content details, my goal was to bring the big picture into focus: How did the model’s structure and relationships represent the product’s workings?
1. Expose the structure and relationships in the model by writing unambiguous, mutually exclusive names and precise descriptions
For every field, I asked: Do this name and description clearly state what the field does, and do they clearly distinguish it from every other field in the model?
I made generic labels specific, so that a user or agent couldn’t confuse a field with another regardless of the context where it appeared.
A better name and description for a planned field in a data model
Compared to the name Type, the final API name for this field, UserRole, will make the field’s purpose easier to understand when it appears alone or alongside other fields. Names like this — as well as more precise descriptions — also improve LLM context and agent performance.
2. Reinforce clarity by ensuring consistency
The same inconsistencies that confuse a human reader (is “Username” the same as “Account ID”?) can make AI hallucinate.
In a separate pass, I looked across objects in the model to make sure that:
- The same word was used for the same concept everywhere
- Naming patterns were applied across related objects
- Descriptions gave a comparable level of detail throughout
3. Compromise
Once an API ships, renaming a field risks breaking existing integrations. For fields where the ideal name conflicted with something already released, the team and I had to weigh consistency with the existing pattern against the clarity of the new one. We took educated guesses at the future impact of our choices.
“Cate brought order and predictable repetition to a complex job.”
Product Leader, Fortune 500 Company