Podcast: DBRX and Open Source Mixture of Experts LLMs with Hagay Lupesko
Learn about cutting-edge developments in AI and Data Science from the experts who know them best on ODSC’s Ai X Podcast. Each week we release an interview with a leading expert, core contributor, experienced practitioner, or acclaimed instructor who is helping to shape the future of the AI industry through their work or research.
In today’s episode of ODSC’s Ai X Podcast, we speak with Hagay Lupesko, Senior Director of Engineering in the Mosaic AI team at Databricks and one of the key architects behind Databricks’ groundbreaking large language model, DBRX.
Previously Haguy was the VP of Engineering at Mosaic ML, which was acquired by Databricks in 2023. Hagay has also held AI engineering leadership roles at Meta, AWS, and GE Healthcare.
Our topic today is the the open-source DBRX large language model, which stands out in the LLM AI landscape due to its innovative use of the Mixture of Experts (MoE) architecture. This architecture allows DBRX to efficiently scale by distributing tasks across 64 combinable experts, allowing the model to select the most suitable set of internal configurations from a pool of experts for each specific task. This results in faster processing and potentially better performance compared to traditional LLM architectures.
Start listening now to get the full impact of Hagay’s knowledge and insights into DBRX and Mixture of Experts, and don’t forget to subscribe to ODSC’s Ai X Podcast to ensure you never miss an episode. Finally, like what you hear? Leave a review or share it with a friend! You can listen on Spotify, Apple, SoundCloud, and Castbox.
Originally posted on OpenDataScience.com
Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Interested in attending an ODSC event? Learn more about our upcoming events here.
