Products Over Projects

Ability to Reorient Quickly

It used to be sufficient for IT/engineering to be setup to respond within a year of receipt of market intelligence or feedback. With “digital” going mainstream, such is no longer the case. Let’s take an online retail example. Broadly, the capabilities of a basic, online retail platform may be classified as catalog, order management, payment and fulfillment.

In projects-mode, it might be the case at a point in time, that owing to priorities, we have active projects affecting order management, payment and fulfillment but not catalog. Now if were to receive unexpected, catalog related feedback from the market or from business stakeholders, we wouldn’t have a team in place ready to act on the feedback. Usually, any new feedback would by default be subject to business case and project approval processes followed by prioritization for staffing. Even if we could suspend one of the active projects and redeploy the team to catalog, the team would likely be new to catalog and therefore unable to hit the ground running.

By contrast, in product-mode, we’d have a long-lived team (or teams) dedicated to each of catalog, order management etc. and always ready to act on new feedback. It would only require the decision of the team’s product owner and their business counterpart to divert some team capacity to act on new information. This ability to quickly reorient a ready team is the first component of better responsiveness.

Reduced End-to-End Cycle Time

Cycle time is a common measure of responsiveness. Included inside overall cycle time is the time to reorient and the time to execute. Let’s examine the latter component now.

Most enterprise DevOps initiatives don’t alter the separation between the “change” and “run” organizations at all. Usually, both organizations adopt some new tooling, some deployment and infrastructure automation, perhaps with some “cloud” thrown in for good measure. They may even realize benefits such as more frequent, automated and reliable deployments.

However, due to persistence of the handoff from dev to centralized ops, the end-to-end cycle time of delivering a change doesn’t change much. Product-mode teams don’t suffer this handoff when they deploy and run what they build. They can therefore achieve the full potential of DevOps adoption. Colocation of application level ops with dev also promotes better end-to-end iteration and better understanding between developers and site reliability engineers.

Note that product-mode teams usually continue to rely on other specialist ops teams that provide a self-service build & deployment infrastructure, manage data centers and/or take care of non-application level security. These teams may also be run in product-mode even though they are not business capability aligned.

Ability to Truly Iterate

COOs often complain about the cost of the tech organization. Yet, they miss one of the biggest opportunities to save money because its root cause is outside tech. Big ticket projects are conceived and funded at one go without an effective mechanism for validating actual benefits. These projects regularly miss the mark in terms of delivering benefits. If organizations had the habit of tracking their benefits realization ratio (actual benefits/projected benefits), they'd be shocked in most cases. A truly iterative development process would be able to fail cheap and save money.

Although the notion of iterative development has been around for a while, in practice, iteration only goes as far as a sprint showcase in most cases. Most Scrum teams are bound to scope-based sprint commitments and are expected to deliver scope on plan rather than solve problems. In order to solve a problem iteratively, teams need to be able to respond to feedback (usage analytics, user surveys, stakeholder testimony) of earlier versions of the solution. Product-mode teams, being stable and long-lived can choose to solve a problem in a truly iterative manner.

Project teams can’t easily move out of scope-delivery for two reasons. First, they are usually only meant to “build” the solution, not run it. So, they build a version of the solution over one or more releases, hand it over to a team in the “run” organization and move on to the next project. Second, the way project funding works, funds are made available to sustain a team for a duration of time much shorter than the lifespan of the underlying problem being addressed.

Example: A retirement calculator

For example, in a financial services company, a project was approved to develop a retirement calculator that would nudge prospective customers into buying retirement products or improving contributions in existing plans. The project team was funded only for the time it would take to build the calculator, not address the underlying problem, which was to improve the uptake of retirement products. Another team would embed the calculator on the company website and a third team would set up the necessary digital campaign to drive traffic to the calculator. At least three different managers (supervisors) would closely track the completion of work with a business stakeholder operating at arm’s length after having specified what to build or at best, having participated in a day-long inception/hothouse. If the delivered solution missed its mark in the market, it would have to wait for another round of funding and staffing to find its way into an improved solution.

In terms of cost and time per percent increase in conversion, a product-mode setup would, in all likelihood, compare favorably to a project-mode setup. The only way to verify this reliably for any given organization is to try product-mode for a couple of problems and assess the results.

Knowledge Retention

The truth about software is that no amount of documentation, handover, and knowledge transfer can make up for a 100% churn in team. Yet this is exactly what a project model brings about. Tech capability does not reside in maturity models, process templates, documentation, code, or the infrastructure. It resides and grows in a team.

Knowledge grows and stays better within stable, long-lived teams that work in the same general area for several years. This contrasts with project teams that ramp up and down every few months. Unmaintainable code results—at least partly—from unmaintained teams.

Although documentation is useful and necessary, in organizations that prefer “working software over comprehensive documentation”, it is no substitute for the sort of knowledge retention that occurs within product-mode teams.

Product-mode teams allow team members to focus on a given area (business-aligned capability) for much longer than a typical project duration. This helps build knowledge, improves understanding of problems and trade-offs, and enriches the quality of interaction with business SMEs and stakeholders.

Do product-mode teams lose knowledge when a team member leaves? Not if they work based on collective ownership of overall scope. Knowledge retention is at risk when team members individually own different features or sub-areas.

Architectural Integrity

The incentives involved in project-mode put pressure on teams to neglect medium-term architectural integrity in favor of (often perceived) short term feature delivery. Since the team doesn't face the consequences of that trade off, they don't benefit from the feedback loop that appears when there is longer term ownership. Project teams are known to contribute (inadvertently) to architectural debt by being unaware of enterprise architecture guidelines or simply ignoring them in the narrow pursuit of timely project delivery. In the medium-term, this phenomenon compromises stability of systems and degrades speed of delivery. A couple of examples:

• They may end up performing a low effort integration with a system meant to be sunsetted instead of taking on a higher effort integrating with an upcoming successor. This compromises the sunsetting roadmap and increases the cost of maintaining an application landscape with multiple systems that do similar things.
• They may end up performing direct database integration with a downstream system instead of integrating after exposing its capabilities via API. In turn, this hurts any evolution of the downstream system that requires a breaking change to its database schema.

Product-mode teams, on the other hand, don’t let other teams integrate inappropriately with the systems they own as they are more keenly aware of the consequences. That’s the advantage of having “build+run” teams own systems as opposed to projects-mode where run-only teams own them.

Ownership of Code and Systems

Team level ownership of code and systems greatly helps a team’s ability to operate in ideate-build-run mode. Ownership allows a team evolve systems faster and in a more controlled manner. But what about the ideal of collective ownership? It is great to aim for collective ownership within teams since the alternative of individual ownership is problematic. Between teams though, a clear demarcation of ownership allows for accountable autonomy.

An internal open source model is used at some of the most advanced tech companies. Anyone in any team can propose changes to almost any area of code by sending a pull request to the custodian (similar to core committer team in open source projects) team.

Although this model can co-exist with team level ownership, it tends to create a default expectation that dependent teams will self-service the changes they need. It is a common expectation in open source projects (“Found a bug? Great. Please submit a fix.”) but unrealistic at most mainstream enterprise Digital/Engineering/IT departments. For one, there is seldom the level of domain knowledge to effectively make changes to disparate systems. Besides, it requires all code to be maintained at a self-serviceable level of documentation, build (as in, simply checkout and build) and test readiness—a high bar for most organizations. It is one reason why internal open source initiatives have failed to take off in most mainstream enterprises that have tried it. For such organizations, it is pragmatic to keep the default expectation not as one of self-service but rather that of a negotiated dependency. Any change must by default, be agreed with and prioritized and delivered by the owning team.

Team Motivation and Dynamics

Teams take time to work together effectively as a unit. Tuckman’s theory of group development asserts that a team goes through a series of suboptimal phases (forming, storming, and norming) before it finds its groove in the performing phase. A project model of staffing runs the risk of disbanding teams just as they enter the norming or performing phase. Product-mode teams, being stable and long-lived can benefit from a long performing phase.

Studies have identified autonomy, mastery, purpose and belongingness as key intrinsic motivators. Product-mode teams tend to have greater autonomy and purpose than typical, scope-delivery project teams. Therefore, the introduction of product-mode teams is usually welcomed by the troops even as management takes its time to adjust to a new normal.

Economies of Flow and Iteration

In product development, our greatest waste is not unproductive engineers, but work products sitting idle in process queues.

-- Donald G. Reinertsen

The arguments so far may sound contrary to conventional IT wisdom. But the business landscape has changed dramatically, roughly since the time smartphones hit the market. Although it has never been sensible to treat IT/engineering as a cost center, it is indefensible now. It has never been more counterproductive to manage IT with a primary focus on cost-efficiency. It is not even enough to create a high-speed “digital” front-end organization on top of sluggish back-end IT—Bimodal/Two-speed IT is based on a false premise and its perpetrators have begun acknowledging it.

Conventional IT wisdom is outdated on several fronts. One such front is the norm of organizing IT teams for economies of scale to maximize utilization of IT specialists. In some ways, a project-based operating model reflects this norm by, one, the use of temporary change teams and two, the use of shared services for design, testing, deployment etc. On the other hand, economies of flow and iteration matter more to the goal of responsiveness.

Economies of scale come about when the unit cost of processing reduces with an increase in scale of processing. On the other hand, economies of flow come about when the unit time of processing (end to end cycle time for a batch size of one – single piece flow) reduces with an improvement in flow of processing. Build-only project teams supported by lots of shared services may help achieve economies of scale whereas ideate-build-run product-mode teams help achieve economies of flow.

Similarly, economies of iteration come about when the effort need to achieve business benefit reduces with an improvement in the ability to iterate with small changes. Build-only project teams suffer too many handoffs to iterate effectively. Besides, they don’t live long enough to carry out a sustained series of real feedback based iterations. Product-mode teams, by design, don’t suffer these disadvantages.

Staff Utilization

A common concern goes like this: “What happens when a product-mode team runs out of work?”. This never happens if the team is responsible for a big enough area like catalog or fulfilment. Besides, team sizes are reviewed periodically (e.g. once a year) to adjust for changing, relative priorities. Some places use a core-plus-flex model of a core team of employees and a flex top-up with contractors. Still, if a given business capability area does not have a robust or important enough roadmap, it is folded (along with a core team) into a neighboring capability until the next review.

Sometimes, the underlying question is: “What if we (the PMO or equivalent) don’t prioritize anything in a given team’s area of work?” And the answer is that a product-mode team is meant to be an empowered, semi-autonomous unit and therefore, not entirely dependent on prioritization from outside. Only cross-cutting initiatives are prioritized externally. For roadmap work, team level product owners are empowered to prioritize in concert with business stakeholders. A rule of thumb is to aim for two-thirds roadmap work and no more than one-third initiative work. Portfolio management thus undergoes radical change in product-mode.

What about utilization of different roles in a cross-functional team? What happens when there isn’t enough work to keep all the different roles busy? Well, people take on work that is adjacent to their skills, e.g. a business analyst may perform exploratory testing, a QA might help improve documentation, etc. In the beginning, they may have to shadow others to gain an initial level of skill but soon enough they go from being a specialist to being a generalizing specialist.

While on the topic of utilization, it is important to note that optimizing for utilization is detrimental to reducing end-to-end cycle time. You don’t improve the flow of traffic on a highway by improving its utilization (i.e. introducing more vehicles). A predominant focus on utilization is a relic of “IT as cost-center” mindset. It is out-of-place for a digital business.

Insularity

Aren’t stable, long-lived teams a recipe for insular attitudes and general stagnation? It is a risk but usually, there is always some churn in tech teams and this provides some injection of new thought. Besides, many organizations sponsor communities of practice—informal, internal networks aligned to specific competencies such as customer experience, A/B testing, etc.—that bring people across teams together.

New silos

In projects-mode, we encounter silos such as product management, architecture, design, development, testing, operations and support. In product-mode, will we end up with new, business-capability aligned silos such as catalog, order management, payment and fulfillment? Perhaps, but they are nowhere as bad as activity-oriented silos. Product-mode teams, being outcome-oriented, may at worst result in silos that still work towards a business outcome. Silos aligned to business capabilities are not as bad as silos aligned to stages of a product development value stream.

That said, we can mitigate the risk of new silos somewhat by setting up product-mode teams along business aligned capability boundaries defined by service experiences such as “issue to resolution” or business value streams such as “order to cash”. If this gets too big for a single team, then we segment into multiple teams (squads within a tribe, in terms of the naming schema made popular by Spotify) but try to keep them co-located and appoint a service experience champion or business value stream owner to work with segment level product owners.

As a countermeasure, for initiatives that cut across multiple product-mode teams, it is recommended to appoint silo-penetrating, priority-negotiating, dependency-managing solution champions who work with product owners and business stakeholders. Another good practice is to offer people an option to change teams after a longish stint, say 18 to 24 months.

Ultimately, silo-busting in product-mode is about ensuring alignment along with autonomy. The old school approach achieves alignment by sacrifing autonomy. Instead, we use techniques such as alignment maps to get the best of both worlds.

Tiered Teams

The online retail example described earlier suggested product-mode teams such as catalog, order-management, checkout and fulfillment. The retirement products example suggested product-mode teams such as on-board, enrol, save and withdraw. This brings up the question whether these teams have end to end responsibility with respect to the enterprise application stack, i.e. from market-facing front-end systems all the way to back-end systems that support back-office functions (Tiers 3 and 4 in the picture below). The answer is “not always” for reasons below.

Figure 1: Simplified example of tiered, product-mode teams in online retail.

Even though end-to-end responsibility helps the cause of outcome-orientation, it is often unrealistic given the shape of systems. For instance, online retail platforms typically have, at a minimum, a webapp and mobile app as customer shopfronts, a call center app as a support front-end and a store-management app as another front-end for store managers. Typically, each of these front-end systems has it own dependency footprint on underlying capabilities such as catalog, order-management, checkout and fulfillment. Usually, there are no clear, front-end system boundaries for an end-to-end catalog (or any other capability) team. For instance, it might be possible to avoid a tier-4 store-management apps team if it is only dependent on a single tier-3 capability such as catalog. The picture shown assumes that store-management has dependencies on more than just catalog. Therefore, we resort to separate product-mode teams in tiers 3 and 4 as shown. Higher level tiers are consumers of lower level tiers.

A new trend is to decompose front-end systems into micro-frontends that line up with tier-3 capabilities. That makes it easier to have end-to-end teams that cut across tiers 3 and 4. But most places that have moved/are moving to microservices aren't yet planning a move to micro-frontends just yet.

Sometimes, the existence of legacy systems supporting multiple tier-2B capabilities may lead to a situation where a tier-3 team also depends on a tier-2B team with corresponding skills. In theory, it is possible to embed tier-2B people into corresponding tier-3 teams. In practice, however, this may be constrained by scarce tier-2B capacity or by practical challenges of shared configuration management, test environments, etc.

Tiers 1 and 2A are enablers for continuous delivery, DevOps, and Lean product development. They are more applicable across business domains. For example, the pipeline infrastructure team is responsible for maintaining Jenkins/GoCD/TeamCity... infrastructure. They are not responsible for attending to broken builds of higher tier teams. Thus, the higher tiers may have one or more dependencies on lower tiers. Irrespective of tier, each team is product-mode in the sense that they are stable, long-lived, cross-functional, outcome-oriented, ideate-build-run teams. The lower tier teams treat the higher tier teams as their (internal) customers in a problem-solving (not scope-delivery) manner.

Squads and Tribes

Often, a single area like catalog may be too big for a single team and it’s okay to break it up into multiple product-mode teams. In this case, the catalog tribe has multiple squads, each with its own outcomes, roadmap and systems under custody. This structure also allows local sharing (internal to tribe) of roles that may be hard to embed within each squad. Note that squads are meant to be much longer lived than typical project teams although squad members may be deliberately asked to rotate within the tribe at say, 18 to 24-month intervals in order to gain breadth of knowledge and avoid becoming single points of failure.

But where is the Product?

To reiterate a point made in the beginning, it is not necessary to be building products to operate in product-mode. That said, if we go beyond the conventional definition of a product as something that’s sold in the market, then it possible to look at things like catalog as if it were a product. Once a capability like catalog is contained within a set of systems owned by a team and exposed via internal applications or well-documented API, it becomes a reusable and self-serviceable, product-like capability catering to internal customers. The same applies to lower tier product-mode teams.