Is skills-based assessment breaking the item bank?
By Whitney Coggeshall
Key takeaways
- Traditional assessment systems, designed for standalone test questions, struggle to accommodate newer skills-based assessments that rely heavily on interactive simulations.
- Organizations need to rethink their existing assessment infrastructure — either adapting current item banks to handle more complex tasks or completely redesigning their approach to assessment management.
- The transition to skills-driven simulations represents a permanent change in how assessments are conducted, making strategic infrastructure planning essential for seamless integration into current programs.
In my recent articles, I’ve argued that skills-based assessment requires us to rethink more than just test content. We’ve talked about design frameworks, validity arguments, and structural models for integrating skills into established programs. But there’s another layer that often goes unexamined, the infrastructure that supports it all.
Most conversations about task-based assessment focus on scoring models, AI, authenticity, and learner experience. Those are important questions. But there’s a quieter, more structural question sitting underneath them. If a meaningful portion of our assessment portfolio becomes task-based and simulation-driven, does the traditional concept of item banking still make sense as the center of our assessment infrastructure?
This is not a technical implementation detail. It is a strategic architecture question.
Traditional item banking was built for a world of discrete, independently scorable items. Task-based simulations operate on a different logic. They behave less like content and more like applications. As we introduce more performance-based tasks into our programs, we are not just changing what we assess. We may be changing the assumptions our systems were built on.
The real question is not “Where do simulations go?”, it’s “Does our item-centric infrastructure still fit the model of assessment we are moving toward?”
To understand why task-based assessment creates tension, it helps to first look at what traditional item banking is designed to do.
What traditional item banking assumes
At its core, an item bank is built around a very specific set of assumptions about how assessment works. The most important one is that the fundamental unit of assessment is a discrete item.
Each item is treated as a self-contained object. It has a prompt, a response format, and a scoring rule. It can be reviewed, approved, tagged with metadata, and stored independently of other items. They also follow a predictable lifecycle:
This model works extremely well for a wide range of assessment types, including multiple choice questions, item sets, and constructed responses.
In this world, the item bank functions as both a content warehouse and an assembly engine. It stores discrete units of assessment and provides the logic to combine them into coherent exam forms.
For decades, this has been a highly effective and scalable way to design, manage, and deliver assessment programs. But it is built on the core idea that assessment is made up of individual, self-contained items.
Task-based simulations assume something different
Task-based simulations are built on a very different set of assumptions about how assessment works. The most important difference is that the fundamental unit is no longer an item.
A simulation is not a single, self-contained prompt, but an experience. It may include multiple steps, inputs, and decision points. It may evolve based on what the candidate does. It may draw on external data or services, or integrate third-party tools. In many cases, it behaves less like a piece of content and more like an application.
Scoring begins to look different as well. Instead of applying a fixed scoring key to a single response, you may need to evaluate sequences of actions, patterns of behavior, or outputs generated across multiple steps.
Even the lifecycle shifts. Rather than following a content lifecycle, simulations operate more like software. They require versioning, updates, bug fixes, security patches, and dependency management over time.
At that point, the comparison becomes difficult to ignore. This is not content warehousing. This is application lifecycle management. And once you start to see simulations this way, the original question becomes more pressing.
If a growing portion of your assessment behaves like applications rather than discrete items, what does that mean for systems built around storing and assembling discrete content?
What this means for item banking
Once you put these two models side by side, the tension becomes clear. Traditional item banking is built to store and assemble discrete, independent units of content. Task-based simulations behave more like applications, with dependencies, logic, and evolving behavior. Both can coexist, but they do not operate on the same underlying assumptions.
So what does that mean in practice? There are a few ways organizations are starting to respond. Broadly speaking, they fall into three strategic paths:
- Leave the item bank largely unchanged and treat simulations as something separate. In this approach, the item bank continues to do what it does best for multiple choice, item sets, and constructed responses. Simulations are managed in a different system, often with their own development, versioning, and delivery infrastructure.
- Expand the concept of item banking itself. Instead of thinking of it as a warehouse for static items, the item bank becomes a broader assessment asset management system. Under this model, everything is treated as an asset, whether it is a multiple-choice question or a full simulation. This requires rethinking what the platform can store, how it handles versioning, and how it separates content from scoring and logic.
- Step back and rethink assessment infrastructure more fundamentally. If simulations become a meaningful portion of the portfolio, it may no longer make sense to organize everything around an item bank at all. Instead, you might move toward a model with multiple systems working together: one for content, one for applications, and a layer that orchestrates how everything is assembled and delivered.
These are not implementation details. They represent different ways of thinking about what sits at the center of your assessment ecosystem. The right path depends on how far you believe this shift toward skills and task-based simulations will go.
The real decision point
At first glance, this can feel like a question of system design. Do simulations live in the item bank? Do they sit alongside it? Do we need new infrastructure altogether?
But those are downstream decisions. The real question is much simpler and much more important. Are simulations a feature, or are they the beginning of a structural shift in how we assess?
If simulations are a feature, then the existing model largely holds. You can introduce them alongside traditional items, manage them in parallel systems, and continue to treat the item bank as the center of your assessment infrastructure. If simulations represent a broader shift, you are no longer just adding a new item type. You are changing what counts as evidence. You are moving from static responses to dynamic performance. You are shifting from content to interaction. And once that happens, the assumptions underlying an item-centric model start to feel less stable.
This does not mean item banking becomes obsolete. Far from it. Discrete items will continue to play a critical role in assessing knowledge and structured reasoning. But it does raise a strategic question: Should item banking remain the center of the system, or become one part of a broader assessment ecosystem?
That is the decision point. And it is not one you need to answer immediately. But it is one you need to start thinking about now, especially if skills-based assessment is more than a short-term experiment.
Infrastructure follows philosophy
In my earlier articles, I argued that skills-based assessment forces us to rethink what we measure and how we build trust in those measurements. This is the next layer of that conversation. Once we change what counts as evidence, the systems that support that evidence have to change as well.
Item banking was designed for a world where assessment was made up of discrete, independent items. That model has served the industry well for decades, and it will continue to play an important role. But as we introduce more task-based simulations and performance-driven experiences, we are operating in a different paradigm.
This does not require an immediate overhaul. Most programs can and should continue to evolve incrementally. But it does require clarity about where things are heading.
If simulations remain a small part of your portfolio, your existing infrastructure will likely hold. If they continue to grow, and if skills become central to how you define readiness, then the systems around them will need to evolve as well. The important thing is not to rush to a solution. It is to recognize the shift early and design with intention.
Author
You may also be interested in
Explore our programs and certificates
CFA Institute offers a diverse range of programs and certificates designed to meet the needs of finance professionals across various career stages and specializations.
