llm
Subscribe to the podcast
Get The Stack Overflow Podcast at your favorite listening service.
Generating text with diffusion (and ROI with LLMs)
Two guests for the price of one! This episode has two interviews recorded at AWS re:Invent back in December.
Settle down, nerds. AI is a normal technology
Ryan welcomes Anil Dash, writer and former Stack Overflow board member, back to the show to discuss how AI is not a magical technology, but rather the normal next step in computing’s evolution. They explore the importance of democratizing access to technology, the unique challenges that LLMs’ non-determinism poses, and how developers can keep Stack Overflow’s ethos of community alive in a world of AI.
Last week in AWS re:Invent with Corey Quinn
Ryan sits down with Corey Quinn, Chief Cloud Economist at Duckbill, at AWS re:Invent to get Corey’s patented snarky take on all the happenings from the conference.
The server-side rendering equivalent for LLM inference workloads
Ryan is joined by Tuhin Srivastava, CEO and co-founder of Baseten, to explore the evolving landscape of AI infrastructure and inference workloads, how the shift from traditional machine learning models to large-scale neural networks has made GPU usage challenging, and the potential future of hardware-specific optimizations in AI.
Attention isn’t all we need; we need ownership too
Ryan welcomes Illia Polosukhin, co-author of the original "Attention Is All You Need" Transformers paper and co-founder of NEAR, on the show to talk about the development and impact of the Transformers model, his perspective on modern AI and machine learning as an early innovator of the tech, and the importance of decentralized, user-owned AI utilizing the blockchain.
Why you need diverse third-party data to deliver trusted AI solutions
Diverse, high-quality data is a prerequisite for reliable, effective, and ethical AI solutions.
Getting rid of the pain for developers on Shopify
Ryan welcomes Glen Coates, VP of Product at Shopify, to dive into the intricacies of managing a developer-focused product, the challenges of backwards compatibility, and the implications of AI and LLMs in Shopify's development environment.
Improving on a 30-year-old hardware architecture
At HumanX 2025, Ryan chatted with Rodrigo Liang, cofounder and CEO of SambaNova, about reimagining 30-year-old hardware architecture for the AI era.
How self-supervised learning revolutionized natural language processing and gen AI
Self-supervised learning is a key advancement that revolutionized natural language processing and generative AI. Here’s how it works and two examples of how it is used to train language models.
“Translation is the tip of the iceberg”: A deep dive into specialty models
Olga Beregovaya, VP of AI at Smartling, joins Ryan and Ben to explore the evolution and specialization of language models in AI.
Variants of LoRA
Want to train a specialized LLM on your own data? The easiest way to do this is with low rank adaptation (LoRA), but many variants of LoRA exist.
Writing tests with AI, but not LLMs
How Diffblue leverages machine learning techniques to write effective unit tests.
Breaking up is hard to do: Chunking in RAG applications
A look at some of the current thinking around chunking data for retrieval-augmented generation (RAG) systems.
Four approaches to creating a specialized LLM
Wondering how to go about creating an LLM that understands your custom data? Start here.
Even high-quality code can lead to tech debt
Ben talks with Eran Yahav, a former researcher on IBM Watson who’s now the CTO and cofounder of AI coding company Tabnine. Ben and Eran talk about the intersection of software development and AI, the evolution of program synthesis, and Eran’s path from IBM research to startup CTO. They also discuss how to balance the productivity and learning gains of AI coding tools (especially for junior devs) against very real concerns around quality, security, and tech debt.
Your docs are your infrastructure
Fabrizio Ferri-Benedetti, who spent many years as a technical writer for Splunk and New Relic, joins Ben and Ryan for a conversation about the evolving role of documentation in software development. They explore how documentation can (and should) be integrated with code, the importance of quality control, and the hurdles to maintaining up-to-date documentation. Plus: Why technical writers shouldn’t be afraid of LLMs.
A student of Geoff Hinton, Yann LeCun, and Jeff Dean explains where AI is headed
Ben and Ryan are joined by Matt Zeiler, founder and CEO of Clarifai, an AI workflow orchestration platform. They talk about how the transformer architecture supplanted convolutional neural networks in AI applications, the infrastructure required for AI implementation, the implications of regulating AI, and the value of synthetic data.
One of the world’s biggest web scrapers has some thoughts on data ownership
Or Lenchner, CEO of Bright Data, joins Ben and Ryan for a deep-dive conversation about the evolving landscape of web data. They talk through the challenges involved in data collection, the role of synthetic data in training large AI models, and how public data access is becoming more restrictive. Or also shares his thoughts on the importance of transparency in data practices, the likely future of data regulation, and the philosophical implications of more people using AI to innovate and solve problems.
No code, only natural language: Q&A on prompt engineering with Professor Greg Benson
Will prompt engineering replace the coder’s art or will software engineers who understand code still have a place in future software lifecycles?
A brief summary of language model finetuning
Here's a (brief) summary of language model finetuning, the various approaches that exist, their purposes, and what we know about how they work.
Tragedy of the (data) commons
Ben chats with Shayne Longpre and Robert Mahari of the Data Provenance Initiative about what GenAI means for the data commons. They discuss the decline of public datasets, the complexities of fair use in AI training, the challenges researchers face in accessing data, potential applications for synthetic data, and the evolving legal landscape surrounding AI and copyright.
Masked self-attention: How LLMs learn relationships between tokens
Masked self-attention is the key building block that allows LLMs to learn rich relationships and patterns between the words of a sentence. Let’s build it together from scratch.
Detecting errors in AI-generated code
Ben chats with Gias Uddin, an assistant professor at York University in Toronto, where he teaches software engineering, data science, and machine learning. His research focuses on designing intelligent tools for testing, debugging, and summarizing software and AI systems. He recently published a paper about detecting errors in code generated by LLMs. Gias and Ben discuss the concept of hallucinations in AI-generated code, the need for tools to detect and correct those hallucinations, and the potential for AI-powered tools to generate QA tests.
The world’s largest open-source business has plans for enhancing LLMs
Ben and Ryan talk to Scott McCarty, Global Senior Principal Product Manager for Red Hat Enterprise Linux, about the intersection between LLMs (large language models) and open source. They discuss the challenges and benefits of open-source LLMs, the importance of attribution and transparency, and the revolutionary potential for LLM-driven applications. They also explore the role of LLMs in code generation, testing, and documentation.