Podcast: How to Evaluate LLMs and RAG Applications with Pasquale Antonante
Learn about cutting-edge developments in AI and data science from the experts who know them best on ODSC’s Ai X Podcast. Each week we release an interview with a leading expert, core contributor, experienced practitioner, or acclaimed instructor who is helping to shape the future of the AI industry through their work or research.
In this episode, Pasquale Antonante, Co-Founder & CTO of Relari AI, joins us to discuss evaluation methods for LLM and RAG applications. Since his time as a PhD student at MIT, Pasquale has been interested in understanding reliability in complex AI systems. Now, at Relari AI, they are building an open-source platform to simulate, test, and validate complex generative AI (GenAI) applications.
Inspired by the testing methodologies used in the autonomous vehicle industry, Relari AI’s is creating an innovative approach to improving generative AI and RAG applications.
During this discussion, you’ll hear about topics like the complexity of GenAI workflows, the challenges in evaluating LLM and RAG systems, and various evaluation methods such as reference-based, and synthetic data-based approaches. You’ll also explore metrics like precision, recall, faithfulness, and relevance, and compare GPT auto-evaluators with simulated user feedback.
Lastly, we’ll highlight Relari’s continuous-eval open-source project and explore the future of leveraging synthetic data for LLM finetuning.
Start listening now to get the full impact of Pasquale’s extensive knowledge and expertise in evaluating LLM and RAG applications and don’t forget to subscribe to ODSC’s Ai X Podcast to ensure you never miss an episode. Finally, like what you hear? Leave a review or share it with a friend! You can listen on Spotify, Apple, and SoundCloud.
To take an even deeper dive into AI topics and tools, and their effects on society at large, join us at one of our upcoming conferences, ODSC APAC (August 13th, Virtual), ODSC Europe (September 5–6, Hybrid, or ODSC West (October 29–31, Hybrid).
Originally posted on OpenDataScience.com
Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Interested in attending an ODSC event? Learn more about our upcoming events here.
