Videos of My Talks

Neal Ford and I gave a talk at USI in Paris (2010) on some aspects of why agile works (as opposed to how). This probes at some of the core forces that makes agile effective, rather than looking at techniques. In particular we look at role of communication and feedback and how they interplay in agile environments.

Sadly the video appears to be truncated and cuts off in the middle of the talk. I haven't been able to figure out how to get the full video posted.

#

I gave this talk ten years after we wrote the agile manifesto. I set out the historical context in which we wrote the manifesto, explain why the semantic diffusion we've experienced since is an inevitable consequence of its success, argue that those who say they don't care about agile are usually wrong, and highlight two areas where we see some interesting new activity in agile thinking.

Further Reading

• Manifesto for Agile Software Development
• Agile Guide

#

In 2014 Dave "Pragmatic" Thomas got irritated with the state of the agile software world saying Agile is Dead. The organizers of the goto conference in Aarhus, who have long been active explorers of agile styles felt that was a good opportunity to bring together him and myself, as two of the authors of the manifesto, together with some people who have been on the receiving end - using and extending agile approaches in the field.

Further Reading

• Dave's original post
• The New Methodology: My original article that explores the essential nature of agile thinking.

#

I was asked to do a fourteen minute minute keynote at OSCON to explain why architecture was important. I decided the best thing was to begin by exploring the meaning of that awkward term, guided by my favorite mailing list post from Ralph Johnson. Once I'd done that I moved onto why it matters by focusing on the economic argument of the Design Stamina Hypothesis.

Further Reading

• I previously ransacked Ralph's email about the meaning of architecture in my 2003 column for IEEE Software.
• Design Stamina Hypothesis
• To explore a little bit of the "how", see this slightly longer talk on Agile and Architecture by Molly Dishman and myself.
• A slightly deeper talk on the economics of software design
• The word architecture comes from construction, but it is a inappropriate metaphor.

#

Increasingly we are seeing organizations move to a world of autonomous teams focused on business capabilities. This helps software development be responsive, and focused on business outcomes. But what role does this leave for architecture, both within a team and across the wider organization?

We argue that architecture is still important, but must be based on guidance rather than command and control. Cultivating this thinking inside and between teams requires work in several areas:

• understanding the business context
• prioritizing cross-functional requirements
• developing a set of architectural principles
• using practices that encourage alignment, such as a technology radar
• keeping records of decisions as part of the product

#

Often people justify effort on software design by pointing to the need for greater craftsmanship and quality. My view is that such moralizing arguments are wrong, that instead we should focus on the economics. Most software efforts slow over time, as the weight of bad design decisions slow down our teams. Attention to design can reduce or even reverse this.

I've found that the metaphor of Technical Debt is a good way to think about the consequences of bad design - do we pay interest or pay off the principal. Some argue that technical debt isn't the result of sloppy design, but I point out that technical debt comes from various causes, and even the best teams will generate some.

Further Reading

• Is High Quality Software Worth the Cost?
• Design Stamina Hypothesis
• Technical Debt
• Technical Debt Quadrant

#

At QCon San Francisco 2008 Rebecca Parsons and I gave a talk about how agile approaches work with enterprise architecture groups. At the moment there's a lot of distrust and conflict between agile project teams and architecture groups. We dig into why this is so, and explore ways that these groups can work together.

#

I was invited to give a keynote for O'Reilly's Software Architecture conference in early 2020, just before Covid-19 closed down the world for a while (and O'Reilly's conference business for good). I struggled to come up with a topic, so I asked my Thoughtworks colleagues at a radar meeting - what was the topic that was most notably absent in conversations about software architecture. Surprisingly I got a clear single answer: business value. I was able to take twenty minutes to explain why it was important, and some techniques we can use to address it

#

A discussion between me and Badri Janakiraman- one of Thoughtworks's most senior developers about the architecture of Rails applications. We start by talking about the notion of Hexagonal Architecture and the role of the database in enterprise applications, particularly Ruby on Rails app. These principles colors the decision between using the Active Record or Data Mapper patterns to organize collaboration with the database. We then move onto talking about how to work with a full-stack application framework like Rails, with the choice between treating it as a platform or as a suite of components.

Further Reading

• P of EAA Book
• P of EAA Catalog

#

Software architecture is an ill-defined concept, that makes an inappropriate terminological borrowing from the building industry. We consider architecture to be the selection of the most important attributes of a system, with a focus on those things that are hard to change. Architecture is something that can evolve as a system grows, but only by putting effort and attention to ensuring it's looked after. We can do that through a combination of initial visioning and on-going efforts.

(Dallas video includes Q&A leading to 65 minutes total.)

#

Continuous Delivery is now becoming a central practice for effective software delivery organizations. This talk explains the essentials of how it works, the role of a deployment pipeline, the difference between continuous delivery and continuous deployment, and some vital ingredients. It also covers the three main benefits of Continuous Delivery: reducing deployment risk, believable progress, and user feedback.

Further Reading

• Guide
• Book
• Definition
• Deployment Pipeline

#

The title software architect comes with many connotations, and often these are not good. Developers think of hand-waivers who inhabit ivory towers and have forgotten how to write code. Project managers think of technologists who are chasing perfection in initiatives that are serving obscure technical purposes. Yet, for the success of any software project architecture is crucial, particularly with the current interest in microservice architectures.

We argue that we can support good architecture without a traditional architecture role, introducing techniques to get good designs and sustainable applications.

#

A 20 minute introductory talk on microservices. I cover the definition of microservices, compare it to a more monolithic approach, and outline important things you have to get right before you should deploy a microservice application.

Further Reading

• Guide overview and links to more articles, books, and videos about microservices
• Definition of Microservices
• Microservice Trade-Offs
• Microservice Prerequisites

#

My colleague Jim Webber has built quite a reputation for taking a lightweight and business-oriented approach to integration in the enterprise. He also has a reputation for being a very robust and entertaining speaker. So I was as nervous as I was excited to share the stage with him for a keynote at QCon 2008. He put together a wonderfully funny presentation with some serious tidbits of meat woven through it. We then just dove in and did it - possibly helped by the pre-talk pint. We talk about the history of Enterprise Integration, the growth of systems that think they are strong but are really just fat, the role of agile thinking, the influence of the web (including Jim's unique theory on why it was invented), and how this leads to Guerilla SOA.

#

I grew up in the Iron Age, where new servers had to be ordered as physical machines, but now we live in the Cloud Age where new servers can be spun up on demand in minutes. To take advantage of the speed and flexibility of the cloud age, we have to rethink how we manage infrastructure.

Infrastructure as Code keeps infrastructure definitions in an executable form, which can then be managed like any other code artifact and kept in version control. This provides more accurate documentation, and infrastructure that can be subject to the same build and test disciplines as application code. This allows us to scale to larger infrastructure configurations while maintaining a greater degree of consistency, reduces the risk of change, and allows us to rapidly support new demand.

Further Reading

• Bliki

#

Our keynote at QCon London 2012 looks at the role data is playing in our lives (and that it's doing more than just getting bigger). We start by looking at how the world of data is changing: its growing, becoming more distributed and connected. We then move to the industry's response: the rise of NoSQL, the shift to service integration, the appearance of event sourcing, the impact of clouds and new analytics with a greater role for visualization. We take a quick look at how data is being used now, with a particular emphasis from Rebecca on data in the developing world. Finally we consider what does all this mean to our personal responsibilities as software professionals.

Further Reading

• Infodeck

#

Event Sourcing is an approach which handles updates by storing an event that describes the update and then processing that event to change the current application state. The event log then becomes the authoritative store of information, allowing you to delete any application states and rebuild them from the event store. Essentially this is the approach taken by version-control systems. Using event sourcing opens up several advantages in audit, ability to query historic states, debugging, and distribution.

Further Reading

• Article My 2005 article explaining the technique in more detail

#

There's much talk about schemaless databases these days, but there is almost always a schema. An implicit schema seems flexible, but is usually worse because it makes harder to figure out how to use data. When people want schemalessness, what they usually need is Variable State, which is useful for custom fields and non-uniform data structures.

Further Reading

• Infodeck

#

The origins of the term "NoSQL" were a hashtag for a twitter meetup, but they turned into the most serious challenge to the twenty year hegemony of relational databases. Due to the accidental nature of their name, they cover a wide range of with not much definition - but it's useful to group many of them under the moniker of Aggregate-Oriented Databases.

NoSQL databases introduce raise issues around consistency, but it's worthwhile remembering that even with ACID transactions we still often have to manage concurrent updates in our applications. The ability of many NoSQL databases to support distributed data further complicates consistency, leading to the CAP theorem's trade-off between consistency and availability (and response time). This trade-off is fundamentally a business decision, not a technical one.

NoSQL databases are a serious choice for modern data needs, but not the only choice. We are now in a period of Polyglot Persistence where we have to choose our data storage technology based on our specific data access needs.

This is my most popular talk (with over 750,000 views for the original goto Aarhus video).

Further Reading

• Guide
• Book
• Definition

#

Most NoSQL databases force people to think differently about consistency than you find in the relational world. Aggregate-oriented databases naturally remove some of need for transactions in relational systems. Database transactions do not prevent us from having to deal with the problems in concurrent updates. Adding distribution to our data increases the kind of consistency issues we need to deal with. The CAP theorem is primarily about the trade-off between consistency and availability (indeed latency) in a distributed system - a trade-off that is primarily a business decision.

(This talk is the consistency part of my Introduction to NoSQL talk, and duplicates material from that talk.)

Further Reading

• Book

#

Software professionals cannot consider ourselves to be merely order takers who do whatever our funders require, we are responsible for how our software affects our users and the broader society. Even if we think we have nothing to hide, our privacy is necessary to protect the bothersome people who prevent corruption and allow society to progress. The movement of email to online services has led to worrying concentration of email provision, which makes it easier to carry out mass surveillance of an important form of our communications. Even seemingly innocuous interception can lead to serious problems because this information about us is valuable to corporations, even if innocuous to governments.

We need to reduce these problems by working to widen the use of encryption for email, so that the cost of mass surveillance becomes prohibitive. The challenge for this is primarily a challenge of user-experience and software packaging, not something that requires great understanding of cryptography.

(The first 12 minutes of this talk is a compressed version of my Not Just Code Monkeys talk.)

Further Reading

• Privacy is important to all of us, because it protects the bothersome people that keep our society free.
• Tim Bray's talk at the same conference

#

I grew up in the Iron Age, where new servers had to be ordered as physical machines, but now we live in the Cloud Age where new servers can be spun up on demand in minutes. To take advantage of the speed and flexibility of the cloud age, we have to rethink how we manage infrastructure.

Infrastructure as Code keeps infrastructure definitions in an executable form, which can then be managed like any other code artifact and kept in version control. This provides more accurate documentation, and infrastructure that can be subject to the same build and test disciplines as application code. This allows us to scale to larger infrastructure configurations while maintaining a greater degree of consistency, reduces the risk of change, and allows us to rapidly support new demand.

Further Reading

• Bliki

#

Event Sourcing is an approach which handles updates by storing an event that describes the update and then processing that event to change the current application state. The event log then becomes the authoritative store of information, allowing you to delete any application states and rebuild them from the event store. Essentially this is the approach taken by version-control systems. Using event sourcing opens up several advantages in audit, ability to query historic states, debugging, and distribution.

Further Reading

• Article My 2005 article explaining the technique in more detail

#

Non-deterministic tests are tests that sometimes pass, sometimes fail, without any change in the underlying code. Left untreated, they make your whole test suite useless. First you need to quarantine them, and then fix them. Common causes include lack of test isolation, asynchrony, and talking to remote services

Further Reading

• Article

#

Often people justify effort on software design by pointing to the need for greater craftsmanship and quality. My view is that such moralizing arguments are wrong, that instead we should focus on the economics. Most software efforts slow over time, as the weight of bad design decisions slow down our teams. Attention to design can reduce or even reverse this.

I've found that the metaphor of Technical Debt is a good way to think about the consequences of bad design - do we pay interest or pay off the principal. Some argue that technical debt isn't the result of sloppy design, but I point out that technical debt comes from various causes, and even the best teams will generate some.

Further Reading

• Is High Quality Software Worth the Cost?
• Design Stamina Hypothesis
• Technical Debt
• Technical Debt Quadrant

#

There's much talk about schemaless databases these days, but there is almost always a schema. An implicit schema seems flexible, but is usually worse because it makes harder to figure out how to use data. When people want schemalessness, what they usually need is Variable State, which is useful for custom fields and non-uniform data structures.

Further Reading

• Infodeck

#

Refactoring is a vital technique to preserve the health of a code base. Often it's described in the context of Test-Driven Development, but there many places in a developer's workflow that refactoring can be used. It's particularly important to use it opportunistically, so that problems can be fixed as soon as they become apparent. The value of regular refactoring lies not in some call to craftsmanship, but for business reasons to ensure the future flow of valuable software.

Further Reading

• Infodeck

#

Most NoSQL databases force people to think differently about consistency than you find in the relational world. Aggregate-oriented databases naturally remove some of need for transactions in relational systems. Database transactions do not prevent us from having to deal with the problems in concurrent updates. Adding distribution to our data increases the kind of consistency issues we need to deal with. The CAP theorem is primarily about the trade-off between consistency and availability (indeed latency) in a distributed system - a trade-off that is primarily a business decision.

(This talk is the consistency part of my Introduction to NoSQL talk, and duplicates material from that talk.)

Further Reading

• Book

#

A 20 minute introductory talk on microservices. I cover the definition of microservices, compare it to a more monolithic approach, and outline important things you have to get right before you should deploy a microservice application.

Further Reading

• Guide overview and links to more articles, books, and videos about microservices
• Definition of Microservices
• Microservice Trade-Offs
• Microservice Prerequisites

#

It's been over a decade since we wrote the Manifesto for Agile Software Development, and the agile meme has been more successful than we ever could have hoped for. But like any success, there is the regular danger of Semantic Diffusion. I try to combat this disease by describing the essence of agile software development: preferring adaptive planning to predictive planning and favoring people over process.

I then describe the Agile Fluency model, which I find an effective way to think about how agile teams become proficient, and the steps you typically go through as you become a more skillful user of agile techniques.

Further Reading

• The New Methodology: My original introduction to the essence of Agile Methods
• Agile Fluency Model

#

Continuous Delivery is now becoming a central practice for effective software delivery organizations. This talk explains the essentials of how it works, the role of a deployment pipeline, the difference between continuous delivery and continuous deployment, and some vital ingredients. It also covers the three main benefits of Continuous Delivery: reducing deployment risk, believable progress, and user feedback.

Further Reading

• Guide
• Book
• Definition
• Deployment Pipeline

#

This is a tricky talk to describe. Usually I like a title and abstract to describe what the talk is about - but this talk is a journey, and I don't want to tell you where I'm going, but instead to explore the ground with me. I will say that it starts with my biggest problem with most adoption of agile software development - the nature of the interaction between users, analysts, and programmers. It goes on to explore these roles, raising questions about the relationship of programmers to users, our responsibilities to them, and finally the Two Great Challenges that I think programmers need to face up to.

#

When people talk about agile methods, they often focus on the product and project management side of things. Delivering small releases, learning from each one, allows teams to change direction quickly as we learn what our users need. This allows to build software that provides value quickly in uncertain environments.

There's a lot of value in this way of working, but to make it work, we need a code base that supports rapid change, not just to its details, but also to its overall architecture. To build such a codebase, we need several technical practices, ones that underpin the delivering zone of the agile fluency model.

• Self-testing code allows to confidently detect faults when we make changes
• Refactoring allows us to change the code quickly without introducing bugs
• Continuous Integration allows all members of team to make changes without interfering with each other
• YAGNI, keeps the software as simple as it can be, making it easier to extend

#

Further Reading

• Guide Page
• Book

#

Thoughtworks often organizes "Quarterly Technology Briefings" - open talks for the community in cities where we have offices. In this QTB in Toronto, Scott Shaw and I talk about how to build new relationships between IT and business. It explains why we think IT departments should be disbanded.

#

For a talk at QCon London 2009 I surveyed Thoughtworks use of Ruby from 2006-2008 in which time we did 41 projects. My talk covers our views on Ruby's producitivity, speed, and maintainability. I conclude that Ruby should be taken seriously as a development environment.

Further Reading

• Article

#

My colleague Zack Exley and I talk about the software that was used by the 2008 Obama presidential campaign. An aspect I found particularly interesting was the way in which the software enabled and interacted with the organizational approach to the campaign.

Further Reading

• Article

#

Mobile is still a smaller part of traffic than the traditional web, but its share is growing, so we need to think about our strategy for developing effective mobile applications. We discuss thinking about a product vision, separating the styles of user engagement into "Lean-forward", "Lean-back", and "Look-down" styles; while integrating them into a transmedia application. We talk about why its more important to focus on value than on traffic, the laser and cover-your-bases platform strategies, and opine that Android, iOS, and the Web are the three viable platform choices. Giles finishes with a case-study of our work with a major airline.

Further Reading

• Supporting Multiple Mobile Platforms
• Selecting a Mobile Implementation Strategy
• Other articles about mobile on martinfowler.com

#

We give a one-hour overview of Continuous Delivery. Topics include the justification of Continuous Delivery, the deployment pipeline, continuous integration, devops, and deployment strategies. The highlight is Jez's personification of a release candidate as a hero in a greek myth.

Further Reading

• Guide
• Book

#

All the talk people make about transforming a business into a digital organization is all very well, but it can't happen unless you have a technical organization that can do its job well.

Nicole Forsgren has done research that correlates IT performance to Organizational Performance, and how her research on Devops identified three important indicators of IT performance: Deployment Frequency, Deployment Lead Time, and Mean Time To Recover. A simple example of lottery tickets illustrates the monetary value of rapid cycle times.

The characteristics we've observed about high-performing technical teams include: using Continuous Delivery, organized in a business oriented manner, technology led, and operating in a climate of trust. To go far you need to go in the right direction, but also take care of your vehicle.

#