SEC is worried chatbots could fuel a market panic

The US Securities and Exchange Commission (SEC) has expressed concern about generative AI’s impact on financial markets.

In a speech given to the National Press Club on Monday, SEC Chair Gary Gensler said recent advances in generative AI increase the possibility of institutions relying on the same subset of information to make decisions. 

Gensler said the large demand for data and computing power could mean only a few tech platforms may dominate the field, narrowing the field of AI models companies can use. If a model provides inaccurate or irrelevant information, financial institutions may end up using the same flawed data and making the same bad decisions — creating the risk of something like the 2008 financial crisis, where banks played “follow the leader” based on information from credit raters, or the Twitter-fueled run on Silicon Valley Bank. Gensler compared the potential fallout to something like the 2008 crisis, which he said demonstrated the risks of a “centralized dataset or model” in finance.

“AI may heighten financial fragility as it could promote herding with individual actors making similar decisions because they are getting the same signal from a base model or data aggregator,” Gensler said. He added that the rise of generative AI and other deep-learning models “could exacerbate the inherent network interconnectedness of the global financial system.” 

The financial sector has been using AI systems for a long time. Some insurance companies and creditors deploy algorithms and natural language processing to parse through financial data before deciding loan amounts. Trading firms have relied on AI to check for fraud and check for market signals much faster than humans looking at a computer screen.

Here, Gensler focused on large language models, calling generative AI and LLMs the “most transformative technology of our time.” His speech sometimes conflated this with the more general category of AI tech — though these systems don’t all present the same risks and questions. Gensler also noted that generative AI is not yet widely used in finance.

This is not the first time Gensler sounded the alarm on AI impacting financial markets. While still at MIT, Gensler and co-author Lily Bailey wrote a paper exploring how current regulatory structures cannot address issues arising from using AI in finance. 

Nor is AI regulation a new topic for the SEC. The agency established FinHub, a resources center set up to answer questions around AI, crypto, and other fintech-related issues, in 2018. It has actively pursued cases against companies in emerging technology that it feels have violated the law, especially in the crypto space.

The SEC itself also uses machine learning to aid market surveillance to enforce its policies.  

Gensler said current guidelines around risk management need to be updated to keep up with new and powerful technology, but he noted there may need to be a financial industrywide rethink around how to use it.