Optimizing RAG Pipelines in Financial Services: Advanced Strategies from Fitch Group

In the world of financial services, handling vast volumes of frequently updated and highly similar documents presents unique challenges. Pablo Vega-Behar, Head of AI Implementation at Fitch Group, delves into how their Emerging Tech team has fine-tuned Retrieval-Augmented Generation (RAG) systems to optimize retrieval accuracy and generation relevance. This blog explores their strategies, including custom chunking techniques, hybrid retrieval methods, and robust development frameworks designed for seamless collaboration between data scientists and machine learning engineers.

Introduction to RAG at Fitch Group

Fitch Group employs RAG pipelines in three applications to enhance information retrieval and analysis. Financial institutions face two primary challenges: managing extensive document collections and navigating the high similarity among financial reports, particularly quarterly filings. While many organizations develop RAG prototypes that appear promising, they often struggle with scaling to production environments. Fitch Group addresses this issue by implementing structured, effective strategies that make RAG systems truly valuable for users.

Fitch Group Overview

Fitch Group, which includes Fitch Ratings and Fitch Solutions, is a global leader in financial information services. Fitch Ratings operates in a regulated environment, focusing on credit ratings and risk assessment, while Fitch Solutions provides broader financial insights. With a team of approximately 3,000 analysts across more than 40 cities, Fitch Group publishes hundreds of thousands of reports annually. AI-powered external products enable clients to efficiently access these reports, making RAG implementation crucial for delivering accurate and relevant data.

Fundamentals of a RAG Pipelines

A basic RAG system comprises key components: a knowledge set, chunking, embedding, indexing, and a vector database. The RAG pipelines operates by retrieving relevant document chunks based on a user query and leveraging a language model to generate a response. Two fundamental components of RAG include:

Retriever: Converts a user’s question into an embedding and identifies the most relevant document chunks stored in a vector database.

Content Generation: Uses the retrieved chunks to generate a precise and contextually accurate answer.

Common RAG applications extend beyond financial services to areas such as chatbots, code assistants, medical record analysis, and literature reviews.

Building a Scalable Production RAG System

To transition from prototype to production, a structured approach is required. Fitch Group’s production RAG pipeline incorporates:

• Data collection, preprocessing, and ingestion
• Retrieval model configuration
• Generation model tuning
• System integration and prompt refinement
• Continuous evaluation and deployment

Choosing the right vector database is also essential. Fitch Group employs Azure AI Search and Elastic Search, balancing enterprise-grade features such as single sign-on (SSO), permissions, and rate limiting with flexibility and scalability.

Custom Strategies for Handling Large Data Sets

• Given the sheer volume of documents, Fitch Group developed tailored strategies to refine retrieval performance:
• Pre-retrieval Techniques: Query routing narrows the search space, leveraging machine learning models trained on real and synthetic question-answer data.
• Query Expansion: Language models refine user queries before embedding, improving retrieval effectiveness.

Post-retrieval Optimization:

• Reranking: Prioritizes the most relevant and recent chunks.
• Context Compression: Reduces retrieved data to focus on key insights.
• Agents: Determine whether additional information is required, rewriting queries when necessary.
• Guardrails: Prevent inappropriate or misleading responses in regulated environments.

Fitch Group’s RAG Applications

Fitch has successfully deployed three RAG-driven applications, each tailored to specific use cases:

FitGPT: An internal tool allowing employees to upload documents and search across Fitch Group’s knowledge base. It integrates multiple LLMs and Azure AI Search.

Fitch Ratings Pro Genie: A subscription-based external tool powered by Elastic Search, GPT-4 Turbo, and intent-based routing for accurate financial analysis.

Credit Sites Genie: A specialized tool handling permissioned document sets with advanced content ingestion and chunking strategies.

Each system employs varying levels of retrieval sophistication, adapting to its respective business requirements.

Architectures of Fitch’s RAG Systems

Fitch Ratings Pro

New research reports undergo chunking and embedding before being stored in Elastic Search. An intent classifier directs queries to specific RAG pipelines optimized for different financial inquiries. Simultaneously, guardrails run in parallel, ensuring compliance with financial regulations.

Credit Sites

This system employs query expansion techniques, hypothetical document embeddings, and an integrated retriever-generator structure to enhance information retrieval. By refining intent classification and post-retrieval ranking, Credit Sites deliver highly relevant results.

Iterative Development and Performance Improvement

Fitch Group follows an iterative approach to RAG development. Their Credit Sites RAG system initially had 30-second response times with a one-third thumbs-up user rating. Through progressive refinements, including optimized retrieval and improved ranking, response times were reduced to under five seconds, and the thumbs-up rate exceeded 80%.

Demonstrating RAG Capabilities

Live demonstrations showcase how Fitch Ratings Pro cites sources for each response, reinforcing credibility and transparency. A comparative analysis between simple RAG and agent-augmented RAG highlights the value of intelligent query rewriting and retrieval refinement.

Implementing Access Control and Permissions

Security is paramount in financial applications. Fitch Group applies permissions at the chunk level, ensuring that users can only access information relevant to their clearance level. Metadata tagging and filtering mechanisms safeguard proprietary data.

Challenges and Future Outlook

Despite significant advancements, RAG pipelines face ongoing challenges:

• Performance and Cost Considerations: Scaling vector search and generation models efficiently.
• Use Case Validation: Ensuring applications justify the investment.
• Future-proofing AI Systems: Developing modular architectures that accommodate evolving retrieval methodologies.

Fitch Group’s long-term vision includes building a generative AI platform to streamline RAG application development. This initiative aims to cut development timelines from eight months to under three months, enhancing agility and responsiveness.

Key Takeaways

• Data quality is critical for effective RAG implementation.
• Vector search alone is insufficient; metadata, filtering, and retrieval agents improve accuracy.
• Balancing complexity and scalability is essential for ensuring a high return on investment.
• Custom development is often necessary to optimize performance in real-world applications.

As the financial sector continues integrating AI-driven document analysis, Fitch Group’s approach offers a roadmap for organizations seeking to maximize the impact of RAG technologies.