Much modern web server code talks to upstream services which return JSON data, do a little munging of that JSON data, and send it over to rich client web pages using fashionable single page application frameworks. Talking to people working with such systems I hear a fair bit of frustration of how much work they need to do to manipulate these JSON documents. Much of this frustration could be avoided by encapsulating a combination of loading strategies.

It's now common in many programming environments to represent data structures as a composite of lists and hashmaps. Most major languages now provide standard versions of these data structures, together with a rich range of operations, in particular Collection Pipelines, to manipulate them. These data structures are very flexible, allowing us to represent most forms of hierarchy in a manner that's easy to process and manipulate.

When I was starting out on my writing career, I began with writing articles for technical magazines. Now, when I write article length pieces, they are all written for the web. Paper magazines still exist, but they are a shrinking minority, probably doomed to extinction. Yet despite the withering of paper magazines, many of the assumptions of paper magazines still exact a hold on writers and publishers. This has particularly risen up in some recent conversations with people working on articles I want to publish on my site.

When an organization takes on an agile mindset, enterprise architecture doesn't go away, but the role of enterprise architects changes. Enterprise Architects no longer make choices, but help others make the right choice and then radiate that information. Enterprise Architects still need to form a vision, but then need to build bridges between teams to build communities of learning. These will allow teams to explore new approaches and learn from each other, with Enterprise Architects as partners in that growth.

Most of our software logic is written in our programming languages, these give us the best environment to write and evolve such logic. But there are circumstances when it's useful to move that logic into a data structure that our imperative code can interpret - what I refer to as an adaptive model. Here I'll show some product selection logic in JavaScript and show how it can be refactored to a simple production rule system encoded in JSON. This JSON data allows us to share this selection logic between devices using different programming languages and to update this logic without updating the code on these devices.

There isn't a simple dichotomy of remote versus co-located work, instead there are several patterns of distribution for teams each of which has different trade-offs and effective techniques suitable for them. While it's impossible to determine conclusive evidence, my sense is that most groups are more productive working in a co-located manner. But you can build a more productive team by using a distributed working model, because it gives you access to a wider talent pool.

As a program grows in size it's important to split it into modules, so that you don't need to understand all of it to make a small modification. Often these modules can be supplied by different teams and combined dynamically. In this refactoring essay I split a small program using Presentation-Domain-Data layering. I then refactor the dependencies between these modules to introduce the Service Locator and Dependency Injection patterns. These apply in different languages, yet look different, so I show these refactorings in both Java and a classless JavaScript style.

A required interface is an interface that is defined by the client of an interaction that specifies what a supplier component needs to do so that it can be used in that interaction.

The notion of changing software development from laboriously crafting code to building powerful systems by simple assembly of components has been a target since I entered our profession. It's target that is sometimes glimpsed, but never really attained - although many technologies have dangled the carrot of industrial reuse.

One of the most common ways to modularize an information-rich program is to separate it into three broad layers: presentation (UI), domain logic (aka business logic), and data access. So you often see web applications divided into a web layer that knows about handling HTTP requests and rendering HTML, a business logic layer that contains validations and calculations, and a data access layer that sorts out how to manage persistent data in a database or remote services.

Andrew Koenig first coined the term “antipattern” in an article in JOOP, which is sadly not available on the internet. The essential idea (as I remember it ) was that an antipattern was something that seems like a good idea when you begin, but leads you into trouble. Since then the term has often been used just to indicate any bad idea, but I think the original focus is more useful.

Alignment maps are organizational information radiators that help visualize the alignment of ongoing work with business outcomes. The work may be regular functionality addition or technical work such as re-architecting or repaying technical debt or improving the build and deployment pipeline. Team members use alignment maps to understand what business outcomes their day-to-day work is meant to improve. Business and IT sponsors use them to understand how ongoing work relates to the business outcomes they care about.

The loop is the classic way of processing collections, but with the greater adoption of first-class functions in programming languages the collection pipeline is an appealing alternative. In this article I look at refactoring loops to collection pipelines with a series of small examples.

Agile software development has broken down some of the silos between requirements analysis, testing and development. Deployment, operations and maintenance are other activities which have suffered a similar separation from the rest of the software development process. The DevOps movement is aimed at removing these silos and encouraging collaboration between development and operations.

Many development teams have found the microservices architectural style to be a superior approach to a monolithic architecture. But other teams have found them to be a productivity-sapping burden. Like any architectural style, microservices bring costs and benefits. To make a sensible choice you have to understand these and apply them to your specific context.

Collection pipelines are a programming pattern where you organize some computation as a sequence of operations which compose by taking a collection as output of one operation and feeding it into the next. (Common operations are filter, map, and reduce.) This pattern is common in functional programming, and also in object-oriented languages which have lambdas. This article describes the pattern with several examples of how to form pipelines, both to introduce the pattern to those unfamiliar with it, and to help people understand the core concepts so they can more easily take ideas from one language to another.

Doctor Who is a long-running TV series, but you don't have to invest too much time to start enjoying it. It's easy to cherry-pick excellent stand-alone episodes.

A summary of how Tor works and how you can use it. It also covers The Tor Browser Bundle, Hidden Services, Tails and looks at some of the controversies around Tor.

In the last few months, I’ve heard repeatedly that the only way to get to a successful microservices architecture is by starting with a monolith first. To paraphrase Simon Brown: If you can’t build a well-structured monolith, what makes you think you can build a well-structured set of microservices? The most recent – and, as usual, very convincing – rendering of this argument comes from Martin Fowler on this very site. As I had a chance to comment on an earlier draft, I had some time to think about this. And I did, especially because I usually find myself in agreement with him, and some others whose views I typically share seemed to agree with him, too.

I’m firmly convinced that starting with a monolith is usually exactly the wrong thing to do.

As I hear stories about teams using a microservices architecture, I've noticed a common pattern.

1. Almost all the successful microservice stories have started with a monolith that got too big and was broken up
2. Almost all the cases where I've heard of a system that was built as a microservice system from scratch, it has ended up in serious trouble.

This pattern has led many of my colleagues to argue that you shouldn't start a new project with microservices, even if you're sure your application will be big enough to make it worthwhile. .

Yagni originally is an acronym that stands for “You Aren't Gonna Need It”. It is a mantra from ExtremeProgramming that's often used generally in agile software teams. It's a statement that some capability we presume our software needs in the future should not be built now because “you aren't gonna need it”.

The microservices architectural style has been the hot topic over the last year. At the recent O'Reilly software architecture conference, it seemed like every session talked about microservices. Enough to get everyone's over-hyped-bullshit detector up and flashing. One of the consequences of this is that we've seen teams be too eager to embrace microservices, not realizing that microservices introduce complexity on their own account. This adds a premium to a project's cost and risk - one that often gets projects into serious trouble.

Kent Beck came up with his four rules of simple design while he was developing ExtremeProgramming in the late 1990's. I express them like this.

When I write code that deals with external services, I find it valuable to separate that access code into separate objects. Here I show how I would refactor some congealed code into a common pattern of this separation.

Data Lake is a term that's appeared in this decade to describe an important component of the data analytics pipeline in the world of Big Data. The idea is to have a single store for all of the raw data that anyone in an organization might need to analyze. Commonly people use Hadoop to work on the data in the lake, but the concept is broader than just Hadoop.

Running the martinfowler.com site is a large part of my job at Thoughtworks. Traditionally it has got more traffic than our main site, although that is happily set to change as our main site improves. My site is a vehicle for us to influence the industry as part of our pillar 2 work.

As with any bit of new architectural terminology, it's hard to get a decent definition of what microservices are, so this talk kicks off by tackling that, based on the article by James and I that helped fuel the interest. I then compare microservices to SOA, compare the architecture to a more monolithic approach, and outline important things you have to get right before you should deploy a microservice application.

I've often been involved in discussions about deliberately increasing the diversity of a group of people. The most common case in software is increasing the proportion of women. Two examples are in hiring and conference speaker rosters where we discuss trying to get the proportion of women to some level that's higher than usual. A common argument against pushing for greater diversity is that it will lower standards, raising the spectre of a diverse but mediocre group.

A simple example of how it can be easier to make a change by first refactoring the code to make the change easy.