Note: this article is part 8 of a series called Accelerated Velocity. This part can be read stand-alone, but I recommend that you read the earlier parts so as to have the overall context.
Most startups are, by necessity and by design, minimalistic when it comes to feature development. They build their delivery stack (web site or API), a few tools needed to manage delivery (control panel, CMS) and then race to market and scramble to meet customer requests. Long term architecture thinking is often reduced to a few hasty sketches and technical debt mitigation is a luxury buried deep in the “someday” queue.
At some point success catches up and the tech debt becomes really painful. Engineers spend crazy amounts of time responding to production issues which they could have used to develop new capabilities. New features take longer and longer to implement. The system collapses under new load. At this point tweaks won’t save the day. An enterprise architecture strategy and runway is needed.
What is an architecture runway? In short it’s a foundational set of capabilities aligned to the big picture architecture strategy that enable rapid development of new features. (SAFe describes it well here.) In plain english – it’s investing in foundational capabilities so features come faster.
The anchor of the architecture runway is, of course, the architecture itself. I’m not going to wade into the dogmatic debate about “what is software architecture”; rather, I’ll simply state that a good architecture creates and maintains order and adaptability within a complex system. The architecture itself should be guided by a strategy and long-term view on how the enterprise architecture will evolve to meet the needs of the business in a changing market and tech-space.
In developing an architecture strategy and runway, architects should start with the current state. At the very least, create a simple diagram that gives context to everyone on the team as to what pieces and parts are in the system and how they play together. Once the “as is” architecture is identified and documented, the architects can roll up their sleeves and develop the “to be” picture, identify the gaps between the two states, and then develop a strategy for moving towards the “to be”. The strategy can be divided into discreet epics / projects, and construction of the runway can begin.
Bonial’s Architecture Runway
Success had caught up to Bonial in 2014. Given the alternative I think everyone would agree that that’s the right problem to have, but it was a problem none-the-less. The majority of the software was packaged into a single, huge executable called “Portal3,” which contained all of the business logic for the web sites, mobile APIs, content publishing system and a couple dozen batch jobs. There were a few ancillary systems for online marketing and some assorted scripts, but they were largely “rogue” projects which didn’t add to the overall enterprise coherence. While this satisfied the immediate needs and had fueled impressive early growth and business success, it wasn’t ready for the next phase.
One of my first hires at Bonial was Al Villegas, an experienced technologist who I asked to focus on enterprise architecture. He was a great fit as he had the right mix of broad systems perspective and a roll-up-his-sleeves / lead-from-the-front mentality. He and I collaborated on big-picture “as-is” and “to-be” diagrams that highlighted the full spectrum of enterprise domains and showed clearly where we needed to invest going forward. Fortunately we version and save the diagrams, so here are the originals:
These pictures served several purposes: (1) they gave us an anchor point for defining and prioritizing long-term platform initiatives, (2) they let us identify the domains that were misaligned, underserved or needed the most work, and (3) they gave every engineer additional context as they developed their solutions on a day-to-day basis.
Then the hard work started. We would have loved to do everything at once, but given the realities of resource constraints and business imperatives we had to prioritize which runways to develop first. As described in other articles of this series, we focussed early on our monitoring frameworks and breaking up the monolith. In parallel we also started a multi-phase, long-term initiative to overhaul our tracking architecture and data pipelines. Later we moved our software and data platforms to AWS in phases and adopted relevant AWS IaaS and SaaS capabilities, often modifying or greatly simplifying elements of the architecture in the process. Across the span of this period, we continually refined and improved our APIs, moving to a REST-based, event-driven micro-services model from the dedicated/custom approach previously used. We also invested in an SDLC runway, building tools on top of the already mature devops capabilities to further accelerate the development process.
The end result is a massive acceleration effect. For example, we recently implemented a first release of a complex new feature involving sophisticated machine-learning personalization algorithms, new APIs and major UI changes across iOS, Android and web. The implementation phase was knocked out in a couple of sprints. How? In part because the cross-functional team had available a rich toolbox of capabilities that had been laid down as part of the architecture runway: REST APIs, a flexible new content publishing system, a massive data-lake with realtime streaming, a powerful SDLC / staging system that made spinning up new production systems easy, etc. The absence any of these capabilities would have added immensely to the timeline.
The architecture continues to evolve. We’ve recently added realtime machine learning and AI capabilities as well as integrations with a number of external partners, both of which have extended the architecture and brought both new capabilities and new (and welcome) challenges. We are continually updating the “as is” picture, adapting the architecture strategy to match the needs of the business, and investing into new runway.
And the cycle continues.
Closing Thoughts
- Companies should start with a simple single solution – that’s fine, it’s important to live to fight another day. But eventually you’ll need a defined architecture and runway.
- Start with a “big picture” to give everyone context and drill down from there.
- Don’t forget the business systems: sales force automation, order management, CRM, billing, etc. As much as everyone likes to focus on product delivery, it’s the enterprise systems that run the business.
- Create a long-term architectural vision to help guide the big, long-term investments.