AI agents gain more from reusable “skills” because those instructions impose a dependable workflow, not because they significantly expand the model’s factual knowledge, according to a study by researchers at Princeton University, UC San Diego and other institutions.

The research, reported by The Decoder, is based on 8,135 controlled test runs comparing agents with and without task-specific skills. It also identifies a serious limitation for teams building agent systems: as a skill library grows, the agent becomes much less likely to retrieve the right instructions. In the reported tests, retrieval precision fell from 29.6% with five skills to 3.3% with 100.

The findings matter as developers increasingly use stored instructions, playbooks and tool procedures to improve AI agents without retraining their underlying models. They suggest that the central engineering problem is not simply how to accumulate more skills, but how to select and apply them reliably.

Why skills improve execution

In the study’s framing, a skill is a compact set of instructions for completing a particular task. It can describe the sequence of actions an agent should take, the tools it should use, the checks it should perform and the mistakes it should avoid.

That makes skills different from a conventional knowledge store. Rather than supplying a new fact, a skill gives an agent a procedural route through a task. It may tell the system how to prepare an environment, call tools in the correct order, validate an intermediate result or format its final output.

The study found that this procedural grounding accounted for 65.7% of the cases in which an agent with a skill outperformed an agent without one. By contrast, directly providing additional knowledge explained only 4.5% of the improvement in the tested cases, according to The Decoder’s account of the research.

For builders, the distinction is important. A model may already know the relevant concepts but still fail because it skips a setup step, invokes a tool incorrectly or produces an unusable output. A well-designed skill can reduce those execution errors by turning an open-ended request into a repeatable workflow.

The result also helps explain why skills have become an attractive alternative to model retraining. Teams can update a procedure, add a validation step or encode a recurring exception without changing model weights. That can make agent behavior easier to revise as products and internal processes change.

Evidence and limits in the study

The research team compared agent behavior on identical tasks with and without a relevant skill across more than 8,000 runs. That controlled setup is stronger than anecdotal demonstrations of an agent completing a task, because it focuses on the contribution of the skill itself.

Still, the reported evidence should be read as a study result, not a guarantee for every agent architecture or workload. The available reporting does not specify every model, benchmark task or retrieval system used in the experiments. The 65.7% and 4.5% figures therefore describe the study’s tested conditions rather than a universal split between procedural and factual benefits.

Skills also introduced new failure modes. In about 10% of cases, the agent reportedly applied a useful playbook mechanically or used it in a situation where it did not fit. A skill can therefore reduce one class of error while creating another: the agent follows the instructions too literally instead of recognizing that the task requires a different approach.

The study also indicates that an exact skill match is not always required. A related skill may provide enough structure to help the agent. That flexibility can be useful in practice, but it makes evaluation more complicated. Teams must test not only whether the correct skill is available, but also whether similar or partially relevant skills cause inappropriate behavior.

The retrieval bottleneck grows with the library

The sharpest warning concerns retrieval. When the tested library increased from five entries to 100, the reported hit rate dropped from 29.6% to 3.3%. The Decoder says especially similar-sounding options made selection more difficult.

This creates a scaling problem for AI agents. A small library can be managed with relatively simple matching, but a production system may accumulate hundreds or thousands of skills covering different teams, software tools, permissions and edge cases. More coverage can then make the system less reliable if the agent cannot distinguish the relevant instruction from nearby alternatives.

The problem is not limited to search quality. Skill names, descriptions and metadata all influence whether retrieval works. Poorly separated procedures may be difficult for both a model and a conventional search system to distinguish. A large library can also contain obsolete or overlapping instructions, increasing the chance that an agent selects a technically plausible but operationally wrong workflow.

That makes skill management a lifecycle problem. Creating a procedure is only the first step. Teams also need mechanisms to test, version, rank, retire and retrieve skills. The study’s conclusion, as reported, is that better self-learning agents will require more reliable methods for creating, finding and applying stored experience—not merely larger collections of it.

What the findings mean for builders and enterprises

For product teams, the immediate lesson is to treat skills as executable operating procedures rather than generic prompt add-ons. A useful skill should specify prerequisites, tool order, checkpoints and conditions under which the agent should stop or ask for help.

Evaluation should measure the full chain. An agent may retrieve a relevant skill but still misuse it, or it may complete a task only because a benchmark happens to contain an unusually clear match. Tests should separately track retrieval accuracy, procedural compliance, inappropriate skill application and recovery when no suitable skill exists.

Enterprise deployments face additional governance questions. Skills may encode access procedures, customer-support policies, financial workflows or internal data handling rules. If an agent retrieves the wrong one, the failure may involve more than a poor answer; it could trigger an incorrect action or expose information to the wrong process. Version control, ownership and audit logs become practical requirements as libraries expand.

The findings also favor selective libraries over indiscriminate accumulation. Adding every successful interaction to long-term memory may increase apparent capability while making retrieval worse. Product teams may get better results by consolidating duplicate skills, using clearer boundaries between procedures and adding explicit negative conditions that explain when a skill should not be used.

For model vendors and agent-platform developers, the research points toward retrieval systems that understand task context, tool state and procedural similarity. It also raises the value of fallback behavior: when confidence is low or several skills look similar, an agent should defer, ask a clarifying question or run a narrower search instead of choosing mechanically.

What to watch next

The next signal will be whether follow-up research tests larger and more diverse skill libraries outside controlled tasks. Results across customer service, coding, research and enterprise operations would show how broadly the reported retrieval decline applies.

Developers should also watch for agent frameworks adding dedicated skill registries, versioning, evaluation suites and confidence-aware retrieval. Those features would indicate that the market is treating skills as managed software components rather than static prompt files.

A further test will be whether systems can learn to reject a nearly relevant skill. The reported cases of mechanical application make refusal, clarification and escalation as important as retrieval itself. Reliable agents will need to know not only what procedure to use, but when no stored procedure is safe to apply.

Creati.ai perspective

This study gives a useful corrective to the assumption that agent capability scales simply by adding more memory. Skills appear most valuable when they make execution explicit, but a growing library can turn that advantage into a selection problem. For AI builders, retrieval quality and procedural boundaries may matter as much as the model’s raw reasoning ability.

The practical architecture suggested by the evidence is selective and testable: small, well-defined skill sets; explicit prerequisites and stop conditions; continuous evaluation of retrieval and misuse; and a safe fallback when instructions conflict or do not fit. Until agents can manage those trade-offs reliably, adding more skills may improve coverage while quietly reducing consistency.