Maintainable

Drawing from his experience at PagerDuty, Sentry, and now FusionAuth, he joins Robby to explore what happens when software becomes trusted infrastructure. Their conversation touches on API design, naming things, product management, self-hosted software, and why customers rarely upgrade as quickly as we’d like them to.

They also discuss why “boring” software is often the most successful, how engineering teams can better communicate technical debt by connecting it to customer outcomes, and why estimates and scope creep are often influenced as much by human psychology as technical complexity. David shares lessons from supporting long-lived authentication systems and explains why stability, compatibility, and predictable upgrades matter more than constantly shipping flashy new features.

The conversation takes an unexpected turn into naval history, where David draws parallels between successful fleets and successful software teams. Clear incentives, strong communication, and dependable checklists consistently outperform heroics. As AI becomes part of the software development process, those lessons feel even more relevant. Teams that make expectations explicit, align around customer outcomes, and invest in reliable processes will be better positioned for whatever comes next.

Episode Highlights

[00:00:50] What Makes Software Truly Maintainable: David explains why fitness for purpose is at the heart of maintainable software.

[00:04:32] API Design Decisions That Last for Years: David shares lessons from PagerDuty about naming, breaking changes, and the long-term consequences of API decisions.

[00:10:44] What FusionAuth Is and Why Authentication Is Different: David explains FusionAuth and why authentication is a problem many teams are better off not solving themselves.

[00:16:57] The Realities of Maintaining Self-Hosted Software: Supporting self-hosted deployments means balancing upgrades, compatibility, security, and customer control.

[00:24:49] Why Customers Rarely Upgrade the Way Product Teams Expect: David explains why customers often settle into a product’s existing capabilities and may not rush to adopt new features.

[00:27:37] What “Boring Software” Really Means: David makes the case that successful software should focus on making users successful rather than giving teams opportunities to build what they find exciting.

[00:29:56] Connecting Technical Debt to Business Outcomes: David discusses why engineering teams need to connect technical debt to customer outcomes and broader business needs.

[00:39:22] Estimates, Optimism, and Scope Creep: David explores how estimates can become persuasion tools and how optimism can unintentionally expand project scope.

[00:43:15] Naval Warfare Lessons for Software Teams: David draws unexpected parallels between naval history, incentives, communication, and software development.

[00:48:29] Alignment, Checklists, and Operational Excellence: David explains why aligned teams with reliable processes consistently outperform teams that rely on individual heroics.

[00:51:21] AI, Explicit Instructions, and the Future of Software Development: David discusses how AI is changing the skills, expectations, and checklists developers need to work effectively.

[00:53:29] Where to Learn More About FusionAuth: David shares where listeners can learn more about FusionAuth and its work around authentication and compliance.

Thanks to Our Sponsors!

Your test coverage says 90%, but that might be misleading. Undercover CI looks at your Ruby pull requests and shows you which parts of your changes weren't tested- not just overall coverage, but what changed and what got missed, down to the method level. Visit undercover-ci.com and use code MAINTAINABLE for 15% off your first billing cycle. Free for public repos. Private repos with unlimited users also available.

[Mailtrap]Mailtrap is a modern email delivery platform built for developers. Native SDKs, a secure Email API and SMTP, and a free tier with 4,000 emails a month. When you need help, you'll reach real people on 24/7 support, not an AI chatbot. Try Mailtrap for free!

Resources & Links

FusionAuth

FusionAuth Blog

PagerDuty

Sentry

curl

Book Recommendations

Castles of Steel: Britain, Germany, and the Winning of the Great War at Sea by Robert K. Massie

The Last Stand of the Tin Can Sailors: The Extraordinary World War II Story of the U.S. Navy's Finest Hour by James D. Hornfischer

Subscribe to Maintainable on:
Apple Podcasts
Spotify
Or search "Maintainable" wherever you stream your podcasts.
Keep up to date with the Maintainable Podcast by joining the newsletter.

Machine learning systems can degrade even when the underlying code has not changed. Diana Pfeil of Sunbeam Consulting joins Robby Russell on Maintainable to explain how changing data, model behavior, and non-deterministic outputs create a different kind of maintenance challenge.

Rather than asking whether a system is simply correct or incorrect, teams need reliable ways to measure confidence in its behavior.

Diana introduces evals as a way to test AI-generated outputs that may be phrased differently each time. She and Robby discuss using LLMs to judge other LLM outputs, reviewing production traces, sampling unusual cases, protecting sensitive data, and keeping humans involved when automated checks cannot provide enough confidence.

They also explore how prompts should be versioned and tested against representative examples.

The conversation turns to the operational costs that come with AI features. Model providers can deprecate dependencies quickly, prompts may behave differently after an upgrade, and teams must continue monitoring systems that might once have been considered finished.

Diana encourages organizations to ask what can now be automated, how accurate the result needs to be, and whether the benefit justifies the additional maintenance work.

Diana also makes the case for starting with the simplest model or deterministic process that can solve the problem. Teams can add complexity once the baseline proves insufficient, but designing around imagined future requirements often creates the wrong system.

Her advice for engineers trying to introduce AI internally is equally direct: build a small prototype that solves a real problem, then let the result make the case.

Episode Highlights

[00:00:50] Maintaining Probabilistic Software: Diana explains why maintaining AI systems involves the code, changing data, model behavior, and confidence in the output.

[00:02:35] What Are Evals?: Robby asks how teams test AI-generated results when the correct response may be worded differently each time.

[00:04:38] Using an LLM as a Judge: Diana describes using one model to evaluate another and why the judge must be calibrated against human decisions.

[00:06:39] Monitoring AI in Production: Diana introduces human review, traces, observability, and production sampling.

[00:08:44] Recognizing Input Drift: A meeting-notes example shows how changing inputs can degrade an otherwise unchanged system.

[00:12:06] Maintaining Models and Prompts: Diana outlines the code, model, prompt, and data-pipeline changes teams may need to make.

[00:14:24] Versioning and Testing Prompts: Robby asks how prompt experimentation fits into source control and repeatable testing.

[00:18:32] Building Confidence in a Black Box: Diana explains how evals and production reviews help teams avoid regressions.

[00:21:16] Diana’s Machine Learning Background: Diana shares her path from recommendation systems at Amazon to startup leadership and consulting.

[00:22:26] How Sunbeam Consulting Helps Teams: Diana describes advising leaders on AI strategy and helping teams build machine learning products.

[00:24:28] Finding Useful Automation Opportunities: Diana explains how teams can identify previously unstructured work that may now be practical to automate.

[00:27:40] The Cost of Automated Decisions: Robby and Diana compare human error with the oversight and infrastructure required by AI systems.

[00:30:14] AI Is Not Free to Maintain: Diana discusses model deprecations, vendor dependencies, and the ongoing support required after launch.

[00:36:45] Keeping Up With Rapidly Changing Tools: Diana explains why teams need room to experiment without constantly disrupting established workflows.

[00:38:49] Why Simpler Models Often Win: Diana makes the case for starting with a baseline before introducing more sophisticated approaches.

[00:45:18] Selling an AI Idea Without the Buzzwords: Diana recommends building a useful prototype and allowing the result to make the case.

[00:46:30] The Inner Game of Tennis: Diana recommends W. Timothy Gallwey’s book about learning, judgment, and performance.

Resources Mentioned

Sunbeam Consulting

Diana Pfeil on LinkedIn

Pydantic

Amazon Bedrock Guardrails

Claude Code

Cursor

OpenAI Codex

The Inner Game of Tennis by W. Timothy Gallwey

Thanks to Our Sponsors!

Your test coverage says 90%, but that might be misleading. Undercover CI looks at your Ruby pull requests and shows you which parts of your changes weren't tested- not just overall coverage, but what changed and what got missed, down to the method level. Visit undercover-ci.com and use code MAINTAINABLE for 15% off your first billing cycle. Free for public repos. Private repos with unlimited users also available.

Mailtrap is a modern email delivery platform built for developers. Native SDKs, a secure Email API and SMTP, and a free tier with 4,000 emails a month. When you need help, you'll reach real people on 24/7 support, not an AI chatbot. Try Mailtrap for free!

Subscribe to Maintainable on:
Apple Podcasts
Spotify
Or search "Maintainable" wherever you stream your podcasts.
Keep up to date with the Maintainable Podcast by joining the newsletter.

What does it take to keep a product healthy after more than 15 years of continuous evolution?

In this episode, Robby Russell talks with Chris Coyier, co-founder of CodePen, about the long game of maintaining software. Chris shares how CodePen has evolved over time, the trade-offs involved in migrating parts of the platform from Rails to Go, and the challenges of balancing maintenance work with the desire to build what's next.

They also explore the human side of maintainability, the role of technical debt in shaping priorities, and why small teams often have to make very intentional decisions about where to invest their limited time and attention.

Whether you're maintaining a side project, stewarding a legacy application, or helping a team navigate change, this conversation offers practical insights into building software that lasts.

Key Topics

Defining what "well-maintained software" really means

Why maintainability is often more of a people problem than a code problem

The origin story of CodePen

Supporting a product that has evolved over 15 years

Balancing maintenance work with product evolution

Gradually migrating from Rails to Go

Using GraphQL across multiple implementations

Technical debt and its many interpretations

Team size, communication overhead, and organizational design

Simplifying software by embracing browser capabilities

Links & Resources

ChrisCoyier.net

Chris Coyier on Bluesky

CodePen

ShopTalk Show

CSS-Tricks

Book Recommendation

Understanding Comics: The Invisible Art (Goodreads) by Scott McCloud

Thanks to Our Sponsors!

Your test coverage says 90%, but that might be misleading. Undercover CI looks at your Ruby pull requests and shows you which parts of your changes weren't tested- not just overall coverage, but what changed and what got missed, down to the method level. Visit undercover-ci.com and use code MAINTAINABLE for 15% off your first billing cycle. Free for public repos. Private repos with unlimited users also available.

Turn hours of debugging into just minutes! AppSignal is a performance monitoring and error-tracking tool designed for Ruby, Elixir, Python, Node.js, Javascript, and other frameworks. It offers six powerful features with one simple interface, providing developers with real-time insights into the performance and health of web applications. Keep your coding cool and error-free, one line at a time! Use the code maintainable to get a 10% discount for your first year. Check them out!

Subscribe to Maintainable on:
Apple Podcasts
Spotify
Or search "Maintainable" wherever you stream your podcasts.
Keep up to date with the Maintainable Podcast by joining the newsletter.

Sally Lait joins Robby Russell on Maintainable to explore software maintainability through a different lens… not just code quality, but how teams work together over time.

Sally is a fractional technology leader and advisor with more than two decades in the industry. You can follow her on LinkedIn or Mastodon.

They start with a familiar question: what makes software well maintained? Structure and standards matter, but Sally shifts the focus to signals around the edges… documentation, onboarding speed, knowledge sharing, and especially how confident people feel making changes.

That confidence becomes the thread throughout the conversation.

Teams with high confidence move faster and adapt more easily. Teams with low confidence hesitate, avoid parts of the system, and struggle to make progress… regardless of what the code looks like.

Robby and Sally also dig into why maintenance work often struggles to get traction. It rarely speaks for itself. Leaders need to connect it to outcomes the business already cares about… risk, hiring, delivery speed, and long-term sustainability.

Sally references a LeadDev panel she moderated on why maintenance still feels “stuck in 2015”: Why Software Maintenance Is Stuck in 2015.

They also discuss modernizing legacy systems and moving away from long-standing in-house software… work that is rarely just technical. It requires trust, clear communication, and navigating the emotional attachment teams have to what they’ve built.

The episode closes with advice for engineers joining older codebases: stay curious, build relationships early, and use onboarding gaps as opportunities to improve things for the next person.

Episode Highlights

[00:01:02] What Makes Software Maintainable: Technical quality matters, but cultural signals often tell the deeper story.

[00:05:45] Why Progress Still Feels Slow: Even with improvements, teams can feel stuck due to perception gaps.

[00:07:30] Communicating Small Wins: Lack of visibility into incremental progress impacts morale and confidence.

[00:12:40] Influencing Without Manipulating: Maintenance work needs to be framed in business terms.

[00:16:00] Technical Debt as a Hiring Problem: Outdated systems affect recruiting and retention.

[00:20:22] Modernizing a Siloed System: Unlocking legacy data required both technical and organizational change.

[00:26:55] Building Trust for Change: Surprise proposals fail… alignment takes time.

[00:32:39] Letting Go of “Our Baby”: Replacing systems involves emotional and cultural dynamics.

[00:46:25] Joining an Older Codebase: Practical advice for onboarding and building confidence quickly.

Resources Mentioned

Sally Lait

Sally Lait on LinkedIn

Sally Lait on Mastodon

Why Software Maintenance Is Stuck in 2015 (LeadDev Panel)

Lara Hogan

The Murderbot Diaries by Martha Wells

Death of the Author by Nnedi Okorafor

Sally’s Reading & Reviews Site

Thanks to Our Sponsors!

Your test coverage says 90%, but that might be misleading. Undercover CI looks at your Ruby pull requests and shows you which parts of your changes weren't tested- not just overall coverage, but what changed and what got missed, down to the method level. Visit undercover-ci.com and use code MAINTAINABLE for 15% off your first billing cycle. Free for public repos. Private repos with unlimited users also available.

Turn hours of debugging into just minutes! AppSignal is a performance monitoring and error-tracking tool designed for Ruby, Elixir, Python, Node.js, Javascript, and other frameworks. It offers six powerful features with one simple interface, providing developers with real-time insights into the performance and health of web applications. Keep your coding cool and error-free, one line at a time! Use the code maintainable to get a 10% discount for your first year. Check them out!

Subscribe to Maintainable on:
Apple Podcasts
Spotify
Or search "Maintainable" wherever you stream your podcasts.
Keep up to date with the Maintainable Podcast by joining the newsletter.

Software maintenance is often framed as a technical problem. Refactoring code, fixing bugs, or upgrading dependencies. In this conversation, Robby Russell talks with Rein Henrichs about a different lens, one centered on understanding.

Rein is a Principal Software Engineer at Procore, where he works within a large, long-lived system used across the construction industry. Rather than focusing on tooling, Rein emphasizes that well-maintained software is software that makes sense to the people maintaining it.

To explain this, Rein introduces the idea of the line of representation, drawing on the work of Richard Cook. Engineers do not interact directly with systems. They rely on representations such as logs, dashboards, and code. These are approximations, not reality, echoing ideas from Plato’s Allegory of the Cave.

When those representations break down, teams lose shared understanding, what Rein describes as “common ground.” This often shows up as weak signals. Subtle indicators that something is not quite right. They are easy to ignore, but over time they lead to confusion and slower decision-making.

Incidents make this especially visible. Rein explains how teams build alignment under pressure, highlighting that the role of an incident commander is coordination, not control. Clear communication matters as much as technical correctness.

The conversation also explores how large systems behave in practice. They rarely fail completely. Instead, they degrade in multiple ways at once. While SLOs can help teams respond to customer-facing issues, they do not capture internal clarity or alignment.

Rein references W. Edwards Deming to highlight a common trap. Not everything that matters can be measured. High-performing teams often rely on judgment, experience, and shared context.

Toward the end, Rein connects these ideas to The Field Guide to Understanding Human Error by Sidney Dekker, challenging the idea that incidents are simply caused by mistakes. Instead, they emerge from the same behaviors that usually lead to success, just under different conditions.

For teams working in complex systems, the takeaway is straightforward. Maintaining software depends on maintaining understanding.

Links & Resources

Procore

Rein Henrichs on LinkedIn

Concepts & References

How Complex Systems Fail – Richard Cook

The Field Guide to Understanding Human Error – Sidney Dekker

W. Edwards Deming

Gerald Weinberg – Secrets of Consulting

Referenced in this Conversation

Kent Beck: You’re Ignoring Optionality and Paying for It

Charity Majors: Deploys Are Just the Beginning

Heidi Helfand: The Art and Wisdom of Changing Teams

Thanks to Our Sponsor!

Turn hours of debugging into just minutes! AppSignal is a performance monitoring and error-tracking tool designed for Ruby, Elixir, Python, Node.js, Javascript, and other frameworks.

It offers six powerful features with one simple interface, providing developers with real-time insights into the performance and health of web applications.

Keep your coding cool and error-free, one line at a time! 

Use the code maintainable to get a 10% discount for your first year. Check them out!

Subscribe to Maintainable on:
Apple Podcasts
Spotify
Or search "Maintainable" wherever you stream your podcasts.
Keep up to date with the Maintainable Podcast by joining the newsletter.