• Open Banker
  • Posts
  • Putting the Genie Back in the Bottle: Potential Artificial Intelligence Regulations

Putting the Genie Back in the Bottle: Potential Artificial Intelligence Regulations

Written by Shannon Kelly

Author

Open Banker
July 07, 2026

Shannon Kelly consults in artificial intelligence and model risk management and is an independent Board Director of Celtic Bank, and she previously served as a senior regulator and an executive at large international and regional banks. Shannon holds a Master of Science in mathematics and statistics from Cornell University.

Open Banker curates and shares policy perspectives in the evolving landscape of financial services for free.

With headlines about the power of AI to replace software engineers, lawyers, and other white-collar professions — followed by concerns about AI hallucinations or manipulation of AI — world leaders and policymakers are racing to impose AI governance as a regulatory requirement.

It is widely recognized that untested and unrestrained AI will harm individuals, specific groups, and even society as a whole,1 and the warnings in Pope Leo’s “Magnifica humanitas: On Safeguarding the Human Person in the Time of Artificial Intelligence” are just in time. Pope Leo’s warning received bipartisan support, and was acknowledged as “profound” by Vice President Vance.2

So what is being done about it? 

A Brief History of Time(ly Policymaking)

In March, the White House released “A National Policy Framework for Artificial Intelligence,”3 intended to increase innovation and access to AI tools, as well as prevent harm to individuals (especially children) and communities from AI infrastructure. Then in June, President Trump issued an Executive Order creating a federal framework for voluntary reviews of “covered frontier models” for cyber security while promoting innovation and advancement of US AI capabilities.4

Congress contributed a bipartisan discussion paper termed the Great American AI Act,5 proposed by the US House of Representatives that addresses safety and security and calls for the establishment of a new agency, the Center for AI Standards and Innovation under the Department of Commerce. Internationally, the Financial Stability Board (FSB) released a consultative paper “Sound Practices for Responsible Adoption of Artificial Intelligence (AI),6 ” requesting comments on best practices in AI governance and lifecycle risk management, including data governance, model, cyber, information technology, and third-party risk management. Several states have already enacted AI regulations,7 focusing on transparency, protection of proprietary information, deep fakes, bias, discrimination, and fraud.  Furthermore, AI systems may be subject to current state data privacy and consumer rights statutes8 and any federal data privacy laws in the future. 

These international, federal, and state policy developments will drive broader regulations to prevent harm from AI. But, as is often the case when generalist policymakers develop governance around novel technology, these efforts can be drastically improved by consulting with practitioners to understand the risks and necessary guardrails. Also, such practitioners should be regularly engaged to evolve policies as the technology and risks evolve. Model risk management (MRM) has already developed governance and testing practices across the lifecycle of AI models. AI that produces quantitative estimates qualifies as a model and falls under MRM regulatory expectations for safety and soundness. Policymakers should understand recent changes in regulatory guidance for MRM, considerations for the regulation of generative and agentic AI, and the role self-regulatory organizations can have in implementing them.

Updated Model Risk Management Guidance

On April 17, 2026, federal banking agencies issued revised MRM guidance9 (SR26-2), replacing guidance issued 15 years earlier (SR11-7). While this is only regulatory guidance and thus not legally binding, the first footnote references current safety and soundness rules and states that unsafe and unsound model risk management practices can lead to supervisory action.10

SR26-2 revised the definitions of a model and model risk to focus on the most complex models, using advanced analytical methods (including AI methodologies), and posing material financial risks:

  • “The term ‘model’ refers to a complex quantitative method, system, or approach that applies statistical, economic, or financial theories to process input data into quantitative estimates.”

  • “The term ‘model risk’ refers to the potential for adverse financial consequences associated with models, which may result from decisions made based on model output. Model risk is influenced by a model’s inherent risk, exposure, purpose, and use.”

SR26-2 offers flexible model risk management practices but places accountability on model developers and the MRM function to manage the organization’s model risks across the model lifecycle, ensuring that models are fit for purpose at implementation, continue to perform in production and do not pose a material financial risk. 

In removing specific examples and practices in the previous guidance and referencing safety and soundness regulations, SR26-2 has effectively eliminated the check-box validations that served more as protection from supervisory scrutiny than managing material financial risks.  

  • Model validations cannot simply rely on a battery of standard tests but now must determine under what circumstances the model would be expected to perform satisfactorily, requiring testing across a range of scenarios.11

  • Post implementation, ongoing performance monitoring tests for data drift, model structure shifts, assumption changes, early warning indicators, and statistical performance12 on an ongoing basis to ensure that the model continues to perform satisfactorily.

SR26-2 grants flexibility, but requires a rigorous governance process, and establishes accountability as a key component of governance in managing model lifecycle risks. Per SR26-2: “Sound governance practices delineate the individual(s) responsible for key activities throughout the model lifecycle, from development through validation and ongoing monitoring.” 

Machine Learning AI Models under Updated Model Risk Management Guidance

Machine learning (ML) methods that produce quantitative estimates are models under SR26-2, and they are often used for higher risk and should receive robust validations. Figure 1 below highlights some common but not exhaustive testing areas.

Figure 1: Key Validation Steps for Machine Learning Models

Given the dynamic, complex, and material financial risk posed by many ML applications, monitoring of ML models, especially self-learning models, will involve extensive testing built into the production code to quickly identify and remediate weaknesses. To manage ML model risk:

  • Challenger models in ongoing performance monitoring test new model structures that could replace the primary model when a challenger consistently outperforms.

  • For self-learning ML models with automatic updates, rigorous validation of the update process and monitoring of updates in production are necessary.

Even with robust validations and ongoing performance monitoring, there may be circumstances where a ML model is deteriorating rapidly (e.g., material data drifts or structural changes), and it is necessary to quickly implement a new model. SR26-2 in such circumstances allows a minimum viable validation, conditional approval and implementation of a new model prior to completing full validation testing.13 This strengthens the case for (multiple) challenger ML models in production.

SR26-2 excludes generative and agentic AI that do not produce quantitative estimates.  Guidance and regulations around broader uses of AI will be developed separately, and will likely be multi-disciplinary (across technical, legal, and ethics fields) and span industries (e.g., financial services, commerce, communications, and technology). Some considerations for potential AI regulations are discussed in the following sections.

Generative and Agentic AI Governance Principles

For AI applications like large language models, agentic commerce, and finance, the implementation of continuous monitoring of the input data structure and guardrails around outputs are necessary to prevent bias and discrimination in decisions, minimize hallucinations, and stop harmful actions taken on behalf of individuals. While MRM still applies to generative and agentic AI, the full set of controls must be broader and interdisciplinary (see Figure 2 below) to effectively address the dynamic, fast-paced environment and the level of uncertainty and risks inherent in these technologies.

Figure 2: Disciplines Involved AI Governance

Even if an AI works effectively without material harm, constraints have recently been realized with the costs of AI so that efficiency and energy usage are also critical considerations, and there are now examples where companies are returning to human employees.14 This may be an opportunity to step back and reconsider the design and usage of algorithms with utility, efficiency, governance, and ethics in mind, potentially retrofitting many AI solutions.  

Governance Considerations for Generative AI

Generative AI identifies probabilistic relationships that structure input information to produce relevant content such as text, code, or images (rather than quantitative estimates), leading to generative AI being viewed as a software application versus model.15 However, principles can be drawn from MRM practices to test AI before releasing it into the wild, including testing the efficacy and structure of input information, reliability of the generative AI algorithm, and red teaming,16 or challenging the AI to identify areas of unreliability and vulnerability.  

Similar to the ongoing performance monitoring required of models, continuous monitoring can be implemented for generative AI, and AI agents (in addition to human in-the-loop), such as:17

  • monitoring input information structure and ongoing changes (e.g., biases and poisoning);

  • feedback and correction, by AI trainers, domain experts and other generative AI;

  • guardrails to prevent inappropriate, unacceptable or biased content generation.

The design of generative AI systems with embedded controls, legal, ethical and efficiency considerations may require technology firms to pause and even retrofit or redesign the control infrastructure of their AI algorithms. Also, there must always be individuals accountable for the performance and implementing controls to prevent material financial losses and other harm.

Governance Considerations for Agentic AI 

Agentic AI refers to AI systems that independently execute processes, design the workflows, and make decisions at key steps. These systems can be semi-autonomous (with human oversight) or fully autonomous and adaptable to new information18 (limited human oversight). 

Testing and monitoring of how an AI agent structures data and uses probability theory to make decisions and even design and create workflows can leverage governance from other AI systems, but there are additional aspects to consider for AI agents.19

  • Initial testing/validation and production monitoring of an agentic AI will require human review but also should implement “red team” challenger AI agents to ensure effective function of the primary AI agent or shut off the primary agent (circuit breaker) if needed.

  • Monitoring systems should report key decisions and alternatives offered by a challenger agent (with the level of uncertainty), where decisions were challenged or overturned, and where a challenger agent stopped the primary agent from executing specific decisions. 

  • Rules around when a human needs to be involved in key decisions (with veto power) or even make the final decision must consider multiple dimensions, such as performance requirements versus legal and ethical constraints. 

  • As workflows change autonomously, both human designers and agentic AI challengers should review the changes to ensure that the process continues to perform as intended. 

The level of control over the AI agents, including guardrails and circuit breakers, should be commensurate with the risks. Additionally, as with AI models and generative AI, there must always be individuals accountable for the performance and for establishing the necessary controls to prevent material financial losses and other harms.

Role of Self-Regulatory Organizations in AI Governance

Given the proprietary nature of AI technologies, it is very difficult to conduct effective validation of or monitor third-party vendor AI solutions, as the training data and algorithms are not disclosed. But financial institutions are still accountable for managing the risks under SR26-220 for ML models and as part of third-party risk management for broader AI solutions. 

Vendors often commission independent audits and validations and conduct performance monitoring, and financial institutions conduct validation and ongoing performance monitoring of their implementation. However, this has proven to be burdensome for many financial institutions and suboptimal. 

One potential solution is the central certification of model, AI and technology solutions, with a proposal released in July 2020 by the FDIC.21 A central certification and even the establishment of a self-regulatory organization (SRO) to certify AI models and technologies would be consistent with the new SR26-2 MRM guidance and is also consistent with the recent EOs and proposals from the White House to test AI prior to “releasing into the wild.” 

An SRO might be the only way to ensure effective independent certification/validation of AI models and technologies. No individual private sector institution would be allowed to access the proprietary algorithms and data on which the AI was trained and run, but an SRO (nonprofit) could gain access (with strict non-disclosure agreements) to this proprietary information to conduct the testing necessary for the certification. 

Figure 3: SRO Certification Lifecycle

A central certification would not remove responsibilities from the financial institution using a vendor to test and monitor their implementation of the AI model or technology. Validation/testing and monitoring AI would still be a shared responsibility between the central certifier, the AI technology provider and the financial institution customer. However, the burden of testing would be significantly reduced for individual institutions.

The Urgent Need for AI Regulation and Governance

Between governance practices in MRM (by regulation) and AI governance developed by technology firms, there is a strong body of practice to draw from in developing policies and regulations that ensure AI safety through requirements for rigorous testing and monitoring. Policymakers should engage industry leaders, but also experts in AI technology, quantitative models, regulatory compliance, law (across multiple industries impacted by AI) and ethics to determine what the requirements of AI should be for the betterment of society and individuals.

Policymaking and engagements should commence as soon as possible, as AI is reading the internet of things, rewriting code and workflows in companies, making probability-based decisions from retail purchases and vacations to financial investments as you are reading this article. Most AI systems will likely require some level of retrofitting to build sufficient monitoring, guardrails and circuit breakers into the code, and this will only be harder the more time passes and the AI systems become more complex and indecipherable.

The opinions shared in this article are the author’s own and do not reflect the views of any organization they are affiliated with.

[3]  This federal AI framework is intended to “to protect American rights, support innovation, and prevent a fragmented patchwork of state regulations that would hinder our national competitiveness, while respecting federalism and State rights.”  https://www.whitehouse.gov/wp-content/uploads/2026/03/03.20.26-National-Policy-Framework-for-Artificial-Intelligence-Legislative-Recommendations.pdf

12 Sound Practices:  1) Strategic direction and oversight; 2) governance and accountability, 3) incorporation of AI risks into risk management framework, 4) organisational adaptability, 5) materiality and risk assessment, 6) selection, 7) data governance, 8) explainability and transparency, 9) performance management, 10) human oversight, 11) cyber and ICT risk management and 12) third-party AI risk management.

[8]  California Consumer Privacy Act (CCPA) and the California Privacy Rights Act (CPRA) 

[10]  “This guidance does not set forth enforceable standards or prescriptive requirements; accordingly, non-compliance with this guidance will not result in supervisory criticism against a banking organization.*”   * “See 12 CFR Part 4, Subpart F, Appendix A (OCC); 12 CFR Part 262, Appendix A (Board); 12 CFR Part 302, Appendix A (FDIC). However, supervisory action may result for any violations of law or unsafe or unsound practices stemming from insufficient management of model risk.”

[11]  “Model validation evaluates whether models perform as expected and includes an assessment of a model’s reliability and its limitations.” … “Validation provides insight into the reliability of a given model, based on its underlying assumptions, methods, data, and relevant theories, as appropriate.”

[12]  “Even with sound modeling practices and rigorous validation, material model risk can remain. … “An effective ongoing monitoring plan may also include regularly assessing any model limitations at the development stage and over time, along with procedures for responding to any issues that may occur, before and after a model is approved for use.”

[13]  “Validation generally occurs prior to a model’s first use. However, certain circumstances (e.g., an urgent business need) may necessitate using the model before validation is completed.”

“AI red teaming is a structured, adversarial testing process designed to uncover vulnerabilities in AI systems before attackers do. It simulates real-world threats to identify flaws in models, training data, or outputs. This helps organizations strengthen AI security, reduce risk, and improve system resilience.”

[20]  “Sound practice includes developing an understanding of the vendor model, including its conceptual soundness, design, development data, and performance. Similarly, sound practice involves conducting ongoing monitoring and outcome analysis to assess whether vendor models are accurate, remain fit for purpose, and continue to be reliable.”

[21]  Request for Information on Standard Setting and Voluntary Certification for Models and Third-Party Providers of Technology and Other Services

Open Banker curates and shares policy perspectives in the evolving landscape of financial services for free.

If an idea matters, you’ll find it here. If you find an idea here, it matters. 

Interested in contributing to Open Banker? Send us an email at [email protected].