Many people in the United States remember the rollout of the US government\u2019s HealthCare.gov website. This site was a crucial part of the new Affordable Care Act, AKA \u201cObamacare.\u201d Hundreds of thousands of US citizens needed to access the site to make critical decisions on how their healthcare<\/a> insurance would work.<\/p>\n Unfortunately, the site\u2019s rollout was an epic failure. The teams building the site missed the mark on capacity planning. Thousands of citizens flooded the site to enroll for insurance, and the site couldn\u2019t support the load. As a result, the site was unusable for several months while the responsible teams scrambled to mitigate the numerous performance problems.<\/strong><\/p>\n Everyone associated with the US site\u2019s release effort suffered significant reputational damage. Worse yet, the end-users — US citizens in need of insurance — could not make critical health and financial decisions for months.<\/p>\n The reasons for the system\u2019s failures were numerous, but one central theme was clear: The teams had completely missed the basic performance needs for the site.<\/strong><\/p>\n Few projects have the size, scope and impact of Healthcare.gov. However, every project supports some critical aspect of an organization\u2019s business. (If the project doesn\u2019t, you shouldn\u2019t build it, but that\u2019s a different post.) We use \u201cnormal\u201d testing and quality assurance activities to ensure each project meets requirements.<\/p>\n We should add performance testing<\/a> to nearly every project to ensure those systems behave correctly when under user load.<\/p>\n We traditionally performance testing is viewed as a highly technical activity. That\u2019s 100 percent true, but it\u2019s not the complete picture. At its root, performance testing is a vital business activity that helps your organization understand if your most critical user activities are successful \u2014 are you able to complete your most basic mission?<\/strong><\/p>\n Perhaps that mission is selling products to customers. Perhaps it\u2019s helping people find healthcare choices. Regardless, every organization should ensure its systems support these mission-critical activities. Failing to do so creates an extreme risk to your company\u2019s mission \u2014 and most likely revenue.<\/p>\n In the past, only a few projects used performance testing to ensure those mission-critical features. Thankfully, the industry is changing.<\/p>\n Traditional approaches for performance testing used teams of specialists who were far removed from the actual delivery teams. Those teams spent weeks or months detailing performance requirements documentation, then would take one version of the system and work in isolation.<\/p>\n Weeks, sometimes months later, the performance team would release a lengthy report detailing system performance and calling out particular areas of concern. Numerous problems with this approach resulted in significant impacts on the delivery cycle.<\/strong><\/p>\n Modern performance testing approaches this work in a much more agile, collaborative fashion. Performance specialists are either embedded in the delivery teams, or at least work in close, constant collaboration. Performance specialists focus on one or two use cases at a time, often tied to work underway in the current iteration.<\/p>\n Traditional performance testing has been done through large, complex enterprise-level toolsets. Users could rarely integrate these tools into the larger development release cycle.<\/p>\n Modern performance testing still uses complex tools \u2014 it\u2019s the nature of the domain \u2014 however, they\u2019re often coupled with code-level tests checking simple measurements around timing.<\/strong> This second approach is key, as it enables quick measurements even on developers\u2019 local systems versus having to push everything to a specific performance testing environment. Again, fast feedback is central to good performance testing.<\/p>\n Continuous Integration\/Continuous Delivery<\/a> (DevOps<\/a>) has been a game-changer for modernizing performance testing. CI\/CD pipelines enable teams to include segments of load testing runs in build cycles as an automated set of performance-related regression checks. These performance regression tests aren\u2019t the complete set of the load suite, but they\u2019re comprehensive enough to ensure no performance degradation happens as development work continues.<\/strong><\/p>\n As noted in many works such as Jez Humble\u2019s book Continuous Delivery, companies like Etsy, Flickr and Netflix personify the success of these practices. Flickr, an online photo-sharing service, has been doing multiple deploys to production since the early 2000s. Etsy used them as a model and extended the DevOps model to include basic performance testing as part of their release model \u2013 thereby ensuring the user experience was always hitting expectations. Netflix\u2019s online stream service can\u2019t survive without a complex, continuous performance testing approach as a first-citizen for their release process.<\/p>\n Organizations moving to modern performance practices see measurable results. You can identify major architectural issues early in the project, dramatically reducing risk.<\/strong> Stakeholders and executives can have faith that their most critical use cases are solid or quickly have information to make appropriate changes in direction.<\/p>\nWhy Performance Testing Matters<\/h2>\n
Performance Testing Is a Business Activity<\/h2>\n
Traditional Performance Testing Lags Delivery<\/h2>\n
\n
Modern Performance Testing is Different<\/h2>\n
Fast Feedback, Risk Mitigation, Value Maximization<\/h2>\n