Podcast: Small Language Models with Luca Antiga
Learn about cutting-edge developments in AI and data science from the experts who know them best on ODSC’s Ai X Podcast. Each week we release an interview with a leading expert, core contributor, experienced practitioner, or acclaimed instructor who is helping to shape the future of the AI industry through their work or research.
In this episode, Luca Antiga, CTO of Lightning AI and ODSC East 2024 speaker, joins us to discuss Small Language Models (SLMs).
You’ll explore the differences between SLMs and Large Language Models (LLMs), examine the importance of data quality in training SLMs, and talk about key SLM frameworks such as TinyLama.
You’ll also touch on the process of combining SLMs with Retrieval Augmented Generation (RAG), address hallucinations within SLMs in particular, and how to deploy SLMs into production.
Start listening now to get the full impact of Luca’s extensive knowledge and expertise in evaluating SLMs and don’t forget to subscribe to ODSC’s Ai X Podcast to ensure you never miss an episode. Finally, like what you hear? Leave a review or share it with a friend! You can listen on Spotify, Apple, and SoundCloud.
To take an even deeper dive into AI topics and tools, and their effects on society at large, join us at one of our upcoming conferences, ODSC APAC (August 13th, Virtual), ODSC Europe (September 5–6, Hybrid, or ODSC West (October 29–31, Hybrid).
Show Notes:
Learn more about Luca Antiga, CTO at Lightning AI:
https://www.linkedin.com/in/lantiga/
Learn more about Lightning AI and PyTorch Lightning:
https://lightning.ai/docs/pytorch/stable/
https://github.com/Lightning-AI
Learn more about Small Language Models
https://opendatascience.com/the-growth-of-small-language-models/
https://opendatascience.com/6-small-language-models-to-get-the-job-done-with-ease/
MMLU (Multi-task Language Understanding) Benchmark:
https://arxiv.org/pdf/2009.03300
MiniCPM: Unveiling the Potential of Small Language Models with Scalable Training Strategies
https://ar5iv.labs.arxiv.org/html/2404.06395
Small Language Model Frameworks:
https://github.com/jzhang38/TinyLlama
https://github.com/Lightning-AI/litgpt
Why Larger Language Models Do In-context Learning Differently?
https://arxiv.org/abs/2405.19592
Learn more about RAG:
Learn more about dottxt:
Learn more about structured text generation:
https://github.com/outlines-dev/outlines
Learn more about HyperTalk:
https://en.wikipedia.org/wiki/HyperTalk
Learn more about crewAI:
https://github.com/joaomdmoura/crewAI
Learn more about Lightning AI Studios:
Learn more about the Concurrent Versions System:
Originally posted on OpenDataScience.com
Read more data science articles on OpenDataScience.com, including tutorials and guides from beginner to advanced levels! Subscribe to our weekly newsletter here and receive the latest news every Thursday. You can also get data science training on-demand wherever you are with our Ai+ Training platform. Interested in attending an ODSC event? Learn more about our upcoming events here.
