10 Can’t-Miss ODSC East 2025 Sessions to Teach You About LLMs and AI Agents

We’re thrilled to introduce you to the leading experts and passionate data and AI practitioners who will be guiding you through an exploration of the latest in AI and data science at ODSC East 2025 this May 13th to 15th! These luminaries come from the companies and institutions at the forefront of innovation. Discover what they will be presenting at ODSC East below.

AI Agents

AI Software Engineering Agents: What Works and What Doesn’t

Robert Brennan | CEO | All Hands AI

AI is reshaping software development, but are autonomous coding agents like Devin and OpenHands a game-changer or just hype? This session cuts through the noise, exploring where AI-driven tools excel, where they fall short, and how to use them effectively without drowning in tech debt. Walk away with a clear roadmap for leveraging AI in 2025 — and knowing which tasks are still best left to human developers.

Evals for Supercharging Your AI Agents

Aditya Palnitkar | Staff Software Engineer | Meta

Testing and monitoring LLMs are often overlooked — but they’re critical to improving performance and development speed. This session dives into how a world-class evaluation system can accelerate your LLM projects, from generating high-quality datasets to using LLMs as judges for multi-turn scenarios. Learn how to create benchmarks, catch hallucinations, select meaningful metrics, and monitor AI agent failure modes, turning evaluation into a key driver for success in your AI applications.

Beyond the Prompt: Architecting Reliable Enterprise LLM Agents

Vivek Muppalla | Director of AI Engineering | Cohere

Creating high-performance LLM agents for enterprise use is challenging, especially when balancing accuracy, safety, and scalability. This session offers a deep dive into the critical decision-making process for building enterprise agents, using a customer support use case as an example. Learn how to select frameworks, ensure safety through human-in-the-loop, define robust tools, establish effective evaluation criteria, and improve model performance with synthetic training data. Perfect for anyone tackling AI at scale in demanding enterprise environments.

Blueprint for Impactful Agentic AI in the Enterprise

Matan-Paul Shetrit | Director of Product | Writer

As agentic AI begins to show promise in business, there’s a disconnect between cutting-edge AI architectures and the real-world challenges of productizing these solutions in complex enterprise environments. Join Matan-Paul Shetrit, Director of Product at Writer, as he outlines a strategic approach to successfully integrate agentic AI into your business processes. Learn how to map workflows, establish robust data infrastructure, ensure governance, and use orchestration layers to guide AI agents for seamless integration and maximum ROI. This session will equip business leaders with actionable insights for driving AI transformation in their organizations.

LLMs

Building a Multimodal AI Assistant: Build an AI Application Using Advanced RAG for Cross-Modal Data Retrieval

Suman Debnath | Principal AI/ML Advocate | Amazon Web Services

Take your AI systems to the next level with cutting-edge Retrieval-Augmented Generation (RAG) techniques! This session breaks down how to integrate LlamaIndex with visual language models and embedding techniques to enable seamless retrieval across text, tables, and images. Learn how to overcome RAG limitations, structure large datasets, and implement query fusion strategies — essential skills for developers and data scientists building powerful, multimodal AI assistants.

Hybrid Text Classification: Labeling with LLMs and Dense Neural Networks

Mohammad Soltanieh-ha, PhD | Clinical Assistant Professor | Boston University

Labeling text data doesn’t have to be slow or expensive. This hands-on workshop shows you how to combine premium LLMs with open-source tools to streamline labeling, generate embeddings, and train neural network classifiers — reducing costs without sacrificing accuracy. Walk away with practical techniques to accelerate text classification and optimize your machine learning pipeline.

Hill Climbing: Best Practices for Evaluating LLMs

Rajiv Shah, PhD | Machine Learning Engineer | Contextual AI

Unlock the full potential of large language models with a systematic evaluation strategy. This hands-on tutorial covers essential techniques — from benchmarking with EleutherAI’s LM Evaluation Harness to leveraging model-as-a-judge approaches and human feedback. Learn how to modularize tasks, implement unit tests, and refine your models with an iterative process. Walk away with practical tools, curated Jupyter notebooks, and a roadmap for building robust LLM evaluation pipelines.

Systematically Improving RAG Applications

Jason Liu | Founder | 567 Labs

Stop relying on anecdotal testing and start measuring retrieval performance with precision. This session introduces a powerful framework for RAG evaluation, leveraging synthetic data to rapidly iterate and improve retrieval systems. Learn how to scale test cases from 50 to 1,000+, optimize query performance, and implement specialized search indices — critical insights for anyone building high-performing AI retrieval systems.

Adaptive RAG Systems with Knowledge Graphs: Building Reinforcement-Learning-Driven AI Applications

David vonThenen | Senior AI/ML Engineer | DigitalOcean

Take your RAG systems to the next level by integrating knowledge graphs with reinforcement learning for continuous performance improvement. This workshop provides hands-on experience in building adaptive AI applications that evolve with real-time feedback. Learn how to create, query, and maintain knowledge graphs, implement feedback loops, and retrain models for smarter, more relevant responses — skills that will help you deploy high-performance, self-improving AI systems. Perfect for AI developers and data scientists ready to build next-gen, adaptive AI solutions.

Entity-Resolved Knowledge Graphs: Taking your Retrieval-Augmented Generation to the Next Level

Dr. Clair Sullivan | Founder and CEO | Clair Sullivan & Associates, LLC

Overcome one of the biggest challenges in building accurate, reliable LLM-powered applications: mitigating hallucinations. This session dives into the transformative impact of entity resolution (ER) on knowledge graphs (KGs), showing how consolidating duplicate entities leads to clearer, more accurate relationships in RAG systems. Learn practical techniques for building and implementing entity-resolved knowledge graphs (ERKGs) to boost LLM performance, streamline complex queries, and elevate your graph-based AI applications to the next level.

Great! How can I see these sessions live?

ODSC East 2025 coming up this May 13th-15th in Boston, MA, in addition to virtually, is the best AI conference for AI builders and data scientists there is. Come learn from experts representing the biggest names in AI like Google, Microsoft, Amazon, and others, network with hundreds of other like-minded individuals, and get hands-on with everything you need to excel in the field.

Register now while tickets are 40% off, and use this link for an additional 10% off!