Divert the Flow

Map data sources

If we are going to extract and re-implement a Critical Aggregator we first need to understand how it is linked to the rest of the legacy estate. This means analyzing and understanding the ultimate source of data used for the aggregation. It is important to remember here that we need to get to the ultimate upstream system. For example while we might treat a mainframe, say, as the source of truth for sales information, the data itself might originate in in-store till systems.

Creating a diagram showing the aggregator alongside the upstream and downstream dependencies is key. A system context diagram, or similar, can work well here; we have to ensure we understand exactly what data is flowing from which systems and how often. It's common for legacy solutions to be a data bottleneck: additional useful data from (newer) source systems is often discarded as it was too difficult to capture or represent in legacy. Given this we also need to capture which upstream source data is being discarded and where.

User requirements

Obviously we need to understand how the capability we plan to “divert” is used by end users. For Critical Aggregator we often have a very large mix of users for each report or metric. This is a classic example of where Feature Parity can lead to rebuilding a set of “bloated” reports that really don't meet current user needs. A simplified set of smaller reports and dashboards might be a better solution.

Parallel running might be necessary to ensure that key numbers match up during the initial implementation, allowing the business to satisfy themselves things work as expected.

Capture how outputs are produced

Ideally we want to capture how current outputs are produced. One technique is to use a sequence diagram to document the order of data reception and processing in the legacy system, or even just a flow chart. However there are often diminishing returns in trying to fully capture the existing implementation, it not unusual to find that key knowledge has been lost. In some cases the legacy code might be the only “documentation” for how things work and understanding this might be very difficult or costly.

One author worked with a client who used an export from a legacy system alongside a highly complex spreadsheet to perform a key financial calculation. No one currently at the organization knew how this worked, luckily we were put in touch with a recently retired employee. Unfortunately when we spoke to them it turned out they'd inherited the spreadsheet from a previous employee a decade earlier, and sadly this person had passed away some years ago. Reverse engineering the legacy report and (twice 'version migrated') excel spreadsheet was more work than going back to first principles and defining from fresh what the calculation should do.

While we may not be building to feature parity in the replacement end point we still need key outputs to 'agree' with legacy. Using our aggregation example we might now be able to produce hourly sales reports for stores, however business leaders still need the end of month totals and these need to correlate with any existing numbers. We need to work with end users to create worked examples of expected outputs for given test inputs, this can be vital for spotting which system, old or new, is 'correct' later on.

Delivery and Testing

We've found this pattern lends itself well to an iterative approach where we build out the new functionality in slices. With Critical Aggregator this means delivering each report in turn, taking them all the way through to a production like environment. We can then use Parallel Running to monitor the delivered reports as we build out the remaining ones, in addition to having beta users giving early feedback.

Our experience is that many legacy reports contain undiscovered issues and bugs. This means the new outputs rarely, if ever, match the existing ones. If we don't understand the legacy implementation fully it's often very hard to understand the cause of the mismatch. One mitigation is to use automated testing to inject known data and validate outputs throughout the implementation phase. Ideally we'd do this with both new and legacy implementations so we can compare outputs for the same set of known inputs. In practice however due to availability of legacy test environments and complexity of injecting data we often just do this for the new system, which is our recommended minimum.

It's common to find “off system” workarounds in legacy aggregation, clearly it's important to try and track these down during migration work. The most common example is where the reports needed by the leadership team are not actually available from the legacy implementation, so someone manually manipulates the reports to create the actual outputs they see - this often takes days. As no-one wants to tell leadership the reporting doesn't actually work they often remain unaware this is how really things work.

Go Live

Once we are happy functionality in the new aggregator is correct we can divert users towards the new solution, this can be done in a staged fashion. This might mean implementing reports for key cohorts of users, a period of parallel running and finally cutting over to them using the new reports only.

Monitoring and Alerting

Having the correct automated monitoring and alerting in place is vital for Divert the Flow, especially when dependencies are still in legacy systems. You need to monitor that updates are being received as expected, are within known good bounds and also that end results are within tolerance. Doing this checking manually can quickly become a lot of work and can create a source of error and delay going forwards. In general we recommend fixing any data issues found in the upstream systems as we want to avoid re-introducing past workarounds into our new solution. As an extra safety measure we can leave the Parallel Running in place for a period and with selective use of reconciliation tools, generate an alert if the old and new implementations start to diverge too far.