Managing Model Risk Across the AI Lifecycle

Artificial intelligence is expanding the use of models and automated decisioning across financial services, but managing the associated risk requires more than validating a model before it enters production. Model risk can arise from flawed assumptions, data limitations, inappropriate use, performance deterioration, changes in the operating environment, or reliance on third-party technology. Effective management therefore requires controls that extend across the full lifecycle, from development and implementation through ongoing monitoring and eventual retirement.

For financial institutions, the objective is not to apply identical controls to every AI system or model. It is to establish a risk-based framework that identifies material sources of model risk, assigns clear accountability, applies appropriate validation and effective challenge, and gives management and the Board meaningful visibility into limitations and emerging risks. Recent revisions to interagency model risk management guidance, including SR 26-2, emphasize a risk-based approach tailored to model characteristics, materiality, use, and the institution's risk profile.

Table of Contents

• What Is the Issue? Defining Model Risk Across the AI Lifecycle

• The Governance Framework: Managing Risk Across the Lifecycle

• What Can Go Wrong? Managing Model, Vendor, and Aggregate Risk

• What Should Management Consider? Aligning Model Risk With Board Oversight

What Is the Issue? Defining Model Risk Across the AI Lifecycle

Effective AI model risk management begins with a clear understanding of what constitutes a “model” and how risk evolves as a system moves from a controlled development environment to live production. Not all automated systems are models, and applying a monolithic governance structure to every tool is both inefficient and ineffective. The key is to distinguish between complex quantitative systems and simpler deterministic tools, applying proportionate controls based on materiality and risk.

Revised interagency guidance on model risk management (SR 26-2), which supersedes SR 11-7 and SR 21-8, provides a useful reference point. It defines a model as a complex quantitative method applying statistical, economic, or financial theory to process input data into quantitative estimates. This definition intentionally excludes:

• Simple arithmetic calculations and spreadsheets.

• Deterministic, rule-based software that does not rely on underlying statistical or financial theory.

This guidance is expected to be most relevant to banking organizations with total assets over $30 billion, though it may also apply to smaller institutions with significant model risk exposure due to model complexity or non-traditional activities. Importantly, the guidance does not establish enforceable standards or prescriptive requirements; accordingly, noncompliance with the guidance will not, by itself, result in supervisory criticism.

The Lifecycle Context: From Development to Production

Model risk is not a static attribute assessed only at deployment. It is a dynamic variable that changes throughout the AI lifecycle. In the development phase, risks are primarily conceptual, stemming from flawed design assumptions, unrepresentative training data, or theoretical weaknesses. Once a model is implemented in a production environment, new operational risks emerge, including data pipeline failures, concept drift as market conditions change, and the potential for misuse by end-users who may not fully understand the model’s limitations.

Scoping for Advanced AI: Generative and Agentic Systems

The interagency guidance explicitly states that generative and agentic AI are outside its scope. However, this exclusion does not absolve institutions of their responsibility to govern these advanced systems. Instead, it signals that these technologies may require a different, potentially more robust, set of controls than traditional predictive models. The guidance notes that institutions should determine appropriate governance and controls for tools and systems that fall outside its formal definition of a model. For management, this means developing a broader AI governance framework that can accommodate a diverse portfolio of technologies, from credit scoring models to customer service chatbots.

The Governance Framework: Managing Risk Across the Lifecycle

A sound governance framework applies specific controls at each stage of the AI lifecycle. This structured approach ensures that risks are identified, assessed, and mitigated from initial concept through ongoing use, preventing gaps in oversight that can occur when risk management is treated as a one-time, pre-deployment event.

1. Development and Intended Use

The foundation of model risk management is laid during the development stage. Comprehensive documentation of a model’s design, theoretical underpinnings, assumptions, and data sources is essential. Critically, the intended use of the model must be clearly defined and approved. Using a model for a purpose beyond its original design—for example, adapting a fraud detection model for credit underwriting—introduces unvetted risks and requires a new round of analysis, validation, and control implementation before it can be approved.

2. Validation and Effective Challenge

Model validation is the critical process of confirming that a model performs as intended and is suitable for its proposed use. This process involves a rigorous review of the model’s conceptual soundness, data, performance, and limitations, with the nature and rigor of validation aligned to the model's approach, use, and materiality. To be effective, this "effective challenge" must be conducted by qualified individuals with sufficient organizational standing and independence from the model’s developers.

While validation generally occurs before a model’s first use, the regulatory guidance recognizes that urgent business needs may require an exception. In such circumstances, sound practice involves giving greater attention to the model's limitations when considering whether its use is appropriate, informing relevant stakeholders of those limitations, and determining appropriate controls, such as limiting model use or more closely monitoring performance.

3. Implementation and Change Management

Deploying a model into a production environment introduces new risks related to technology integration, data pipelines, and user access. The implementation process must include robust pre-deployment testing to ensure the model operates correctly within the existing IT infrastructure. Furthermore, a formal change management process is required to govern any modifications to the model, its underlying code, or its data inputs. Even minor changes can have unintended consequences, and all modifications should be documented, tested, and approved through the established governance structure.

4. Ongoing Monitoring and Performance

Once in production, models require continuous monitoring to ensure their performance does not degrade over time. This involves tracking key performance metrics, assessing the stability of inputs, and comparing model outcomes against real-world results. Any significant performance degradation or deviation from expected behavior should trigger a review to determine if the model needs to be recalibrated, retrained, or retired. This ongoing oversight is crucial for detecting issues like concept drift, where a model’s predictive power erodes as the relationship between inputs and outputs changes in the real world.

What Can Go Wrong? Managing Model, Vendor, and Aggregate Risk

Even with a robust lifecycle framework, specific risk categories demand focused attention from management. These include the inherent limitations of models, the complexities of third-party dependencies, and the systemic risks that arise when multiple models interact.

Data and Model Limitations

Every model is a simplification of reality and has inherent limitations. These can stem from biases in the training data, flawed assumptions in the model’s design, or an inability to account for rare but high-impact events. A common failure point is the misapplication of a model to populations or market conditions not represented in its development data. Management must ensure these limitations are not only documented but also clearly communicated to end-users and decision-makers to prevent overreliance on model outputs.

The Challenge of Third-Party and Vendor Models

Financial institutions retain responsibility for appropriately governing and overseeing models used in their operations, including models and other products provided by third parties. Reliance on a vendor does not transfer this accountability. While vendors may provide their own testing results, an institution must conduct its own validation and monitoring appropriate to the model's characteristics, materiality, use, and risk. This includes developing a sufficient understanding of the model's conceptual soundness, development data, performance, and limitations, even when the underlying code is proprietary. A robust third-party risk management program is essential for governing these relationships.

Dependencies and Aggregate Model Risk

Aggregate model risk emerges when multiple models are interconnected or share common assumptions, data sources, or dependencies. For example, a credit scoring model and a fraud detection model may both rely on the same customer data feeds. An error or bias in that shared data could simultaneously impact both models, compounding the potential for negative outcomes. Identifying and managing these interdependencies is a critical, and often overlooked, component of a mature model risk management program. It requires maintaining a comprehensive model inventory that allows risk managers to map connections and assess potential systemic vulnerabilities.

What Should Management Consider? Aligning Model Risk With Board Oversight

Effective AI model risk management is not solely the responsibility of technical teams; it requires active engagement from senior management and the Board. Governance should be viewed as an enabler of innovation, providing the transparency and controls needed to adopt sophisticated technologies safely.

Accountability and the Three Lines of Defense

Clear accountability must be established across the three lines of defense. The first line (business units and model developers) owns the risk associated with their models. The second line generally establishes the governance framework and provides independent risk oversight and effective challenge, while validation responsibilities should be clearly defined within the institution's governance structure. The third line (Internal Audit) provides independent assurance over the effectiveness of the first two lines. It is critical that Internal Audit evaluates the rigor of the model risk management process itself, rather than duplicating the validation activities performed by the second line.

Board Reporting and Communication of Limitations

Board reporting on model risk should move beyond simple performance metrics. It must provide a clear-eyed view of the institution’s overall model risk profile, including significant model limitations, the potential impact of model errors on business decisions, and the status of remediation efforts for any identified weaknesses. This level of transparency is essential for the Board to fulfill its oversight responsibilities and make informed strategic decisions. A useful tool for this is an AI governance maturity model, which can help frame progress and identify remaining gaps.

Model Inventory and Materiality

Maintaining an appropriate model inventory is an important component of sound model risk management. This inventory should serve as more than a simple list; it should be a dynamic risk management tool. For models included within the institution's model inventory, each entry should include information appropriate to the model's characteristics and risk, such as its owner, purpose, materiality, key assumptions, validation status, and known limitations. By assigning a materiality rating to each model based on its potential financial and reputational impact, management can prioritize governance resources and ensure the highest-risk models receive the greatest level of scrutiny.

Executive Takeaways

For boards and senior management, overseeing AI model risk requires asking targeted, strategic questions that test the robustness of the institution’s governance framework.

• Does our current model inventory clearly distinguish between models subject to model risk management and simpler deterministic software outside the scope of the model definition?

• Is our model validation function sufficiently independent and empowered to provide effective challenge to model developers and business line owners?

• How are we assessing and managing the aggregate risk created by shared data, assumptions, or dependencies across our portfolio of internal and vendor-supplied models?

• Is our Internal Audit function appropriately focused on evaluating the effectiveness of our model risk management processes, or is it duplicating validation activities?

• Does our board reporting provide a clear and transparent view of key model limitations and their potential impact on strategic and financial outcomes?

How Versapien Can Help

Versapien provides specialized advisory services to help financial institutions design and implement effective AI and model risk management frameworks. Our senior-led teams, with deep experience from Big Four consulting firms, work with boards and executive management to align governance with regulatory expectations and business objectives. We assist clients in developing practical, implementation-focused solutions for model validation, third-party risk management, board reporting, and examination readiness. By integrating AI governance with traditional risk domains, we help organizations adopt innovative technologies safely and sustainably, turning compliance and risk management into a strategic advantage.