Bloomberg AI Researchers Mitigate Risks of “Unsafe” RAG LLMs and GenAI in Finance

There's immense pressure for companies in every industry to adopt AI, but not everyone has the in-house expertise, tools, or resources to understand where and how to deploy AI responsibly. Bloomberg hopes this taxonomy – when combined with red teaming and guardrail systems – helps to responsibly enable the financial industry to develop safe and reliable GenAI systems, be compliant with evolving regulatory standards and expectations, as well as strengthen trust among clients.
Bloomberg AI Researchers Mitigate Risks of "Unsafe" RAG LLMs and GenAI in Finance

Two new academic papers reflect Bloomberg’s commitment to transparent, trustworthy, and responsible AI


From discovering that retrieval augmented generation (RAG)-based large language models (LLMs) are less “safe” to introducing an AI content risk taxonomy meeting the unique needs of GenAI systems in financial services, researchers across Bloomberg’s AI Engineering group, Data AI group, and CTO Office aim to help organizations deploy more trustworthy solutions.

They have published two new academic papers that have significant implications for how organizations deploy GenAI systems more safely and responsibly, particularly in high-stakes domains like capital markets financial services.

In RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models,” Bloomberg researchers found that RAG, a widely-used technique that integrates context from external data sources to enhance the accuracy of LLMs, can actually make models less “safe” and their outputs less reliable.

To determine whether RAG-based LLMs are safer than their non-RAG counterparts, the authors used more than 5,000 harmful questions to assess the safety profiles of 11 popular LLMs, including Claude-3.5-Sonnet, Llama-3-8B, Gemma-7B, and GPT-4o. Comparing the resulting behaviors across 16 safety categories, the findings demonstrate that there were large increases in unsafe responses under the RAG setting. In particular, they discovered that even very “safe” models, which refused to answer nearly all harmful queries in the non-RAG setting, become more vulnerable in the RAG setting [see Figure 3 from the paper].

The change of risk profile from non-RAG to RAG is model dependent. (Figure 3, RAG LLMs are Not Safer: A Safety Analysis of Retrieval-Augmented Generation for Large Language Models, arXiv, 2025.)

This research clearly underscores the need for anyone using RAG LLMs to assess whether their models have any hidden layers of vulnerability and what additional safeguards they might need to add.

“This counterintuitive finding has far-reaching implications given how ubiquitously RAG is used in GenAI applications such as customer support agents and question-answering systems. The average Internet user interacts with RAG-based systems daily,” explained Dr. Amanda Stent, Bloomberg’s Head of AI Strategy & Research in the Office of the CTO. “AI practitioners need to be thoughtful about how to use RAG responsibly, and what guardrails are in place to ensure outputs are appropriate. Our research offers a framework for approaching that so others can evaluate their own solutions and identify any potential blind spots.”

In a related paper, “Understanding and Mitigating Risks of Generative AI in Financial Services,” Bloomberg’s researchers examined how GenAI is being used in capital markets financial services and found that existing general purpose safety taxonomies and guardrail systems fail to account for domain-specific risks.

To close this gap, they introduced a new AI content risk taxonomy that meets the needs of real-world GenAI systems for financial services. It goes beyond what may be addressed by general-purpose safety taxonomies and guardrail systems by addressing risks specific to the financial sector such as confidential disclosure, counterfactual narrative, financial services impartiality, and financial services misconduct.

“There have been strides in academic research addressing toxicity, bias, fairness, and related safety issues for GenAI applications for a broad consumer audience, but there has been significantly less focus on GenAI in industry applications, particularly in financial services,” said David Rabinowitz, Technical Product Manager for AI Guardrails at Bloomberg.

[See Table 1 from the paper]

The categories in Bloomberg’s AI content safety taxonomy for financial services. (Table 1, Understanding and Mitigating Risks of Generative AI in Financial Services, 2025.)

“There’s immense pressure for companies in every industry to adopt AI, but not everyone has the in-house expertise, tools, or resources to understand where and how to deploy AI responsibly,” said Dr. Sebastian Gehrmann, Bloomberg’s Head of Responsible AI. “Bloomberg hopes this taxonomy – when combined with red teaming and guardrail systems – helps to responsibly enable the financial industry to develop safe and reliable GenAI systems, be compliant with evolving regulatory standards and expectations, as well as strengthen trust among clients.”

The RAG safety paper will be presented at the 2025 Annual Conference of the Nations of the Americas Chapter of the Association for Computational Linguistics (NAACL 2025) in Albuquerque, New Mexico later this week. The AI risk taxonomy paper will be presented at the ACM Conference on Fairness, Accountability, and Transparency (FAccT) in Athens, Greece in June. For more details, read the Tech At Bloomberg blog post and both papers:

About AI at Bloomberg
Since 2009, Bloomberg has been building and using artificial intelligence (AI) in the finance domain – including machine learning (ML), natural language processing (NLP), information retrieval (IR), time-series analysis, and generative models – to help process and organize the ever-increasing volume of structured and unstructured financial information. With this technology, Bloomberg is developing new ways for financial professionals and business leaders to derive valuable intelligence and actionable insights from high-quality financial information and make more informed business decisions. Learn more about Bloomberg’s AI solutions at www.bloomberg.com/AIatBloomberg.

About Bloomberg
Bloomberg is a global leader in business and financial information, delivering trusted data, news, and insights that bring transparency, efficiency, and fairness to markets. The company helps connect influential communities across the global financial ecosystem via reliable technology solutions that enable our customers to make more informed decisions and foster better collaboration. For more information, visit Bloomberg.com/company or request a demo.


Recent Content

DeepSeek AI has emerged as a major competitor to OpenAI, offering a low-cost, efficient AI chatbot that has soared to the top of the Apple App Store. Founded in China, DeepSeek’s compute-efficient AI models, aggressive pricing, and open-source approach have disrupted the industry. With AI advancements like DeepSeek-R1 for reasoning tasks and Janus Pro for AI image generation, the startup is reshaping the global AI race—but also raising concerns about cybersecurity, U.S. AI leadership, and regulatory oversight.
Oumi AI, founded by ex-Google and Apple engineers, is the first fully open-source AI platform offering unrestricted access to models, data, and training pipelines. Unlike Llama and DeepSeek-R1, Oumi eliminates AI silos by enabling seamless collaboration across researchers, universities, and enterprises. With backing from MIT, Stanford, and Oxford, Oumi is enabling AI development through transparency, decentralization, and scalable infrastructure—making AI truly accessible to all.
A new Economist Impact report, sponsored by SAS, highlights that AI and digital transformation alone won’t drive government productivity gains. While AI-powered automation can improve efficiency, cybersecurity, and public services, the study underscores that employee engagement, adaptive structures, and cultural readiness are equally important. Based on a survey of 1,550+ public sector employees across 26 countries, the report explores key AI challenges, success stories, and policy recommendations to help governments balance technology and culture for sustainable productivity improvements.
OpenAI has launched ChatGPT Gov, a secure AI chatbot tailored for U.S. government agencies. Designed for enhancing efficiency, security, and AI-powered public services, ChatGPT Gov offers GPT-4o access, secure cloud deployment on Microsoft Azure, and compliance with FedRAMP, CJIS, IL5, and ITAR security standards. With over 90,000 government users already leveraging AI, OpenAI is expanding its role in the public sector while ensuring data privacy, secure deployments, and AI-driven innovation.
NTT DATA has introduced Smart AI Agent™, an AI-powered automation tool designed to accelerate Generative AI adoption and enhance workforce efficiency across industries. With applications in automotive, finance, and manufacturing, the AI agent streamlines workflows and enables businesses to scale AI-driven operations. Targeting $2 billion in revenue by 2027, NTT DATA is positioning itself as a leader in enterprise AI adoption, offering secure, scalable, and intelligent automation solutions to address global workforce challenges.
Artificial Intelligence (AI) took center stage at Davos 2025, influencing discussions on governance, AI agents, and China’s growing AI presence. As we approach MWC 2025 in March, AI is expected to dominate key sessions on AI-driven telecom innovations, security risks, and business applications. With major players like Salesforce, Nvidia, and emerging Chinese startups shaping the landscape, AI’s expanding role in industries and global policies is more critical than ever.

Currently, no free downloads are available for related categories. Search similar content to download:

  • Reset

It seems we can't find what you're looking for.

Download Magazine

With Subscription

Subscribe To Our Newsletter

Scroll to Top