On election night in America, millions of people will follow the election returns on The Washington Post’s website. There they will find the very latest results, presented clearly, and loading smoothly, on any device is key to its readers. We worked with The Washington Post to build an infrastructure that ingests data from tens of thousands of election races and immediately shares that information with hundreds of thousands of viewers simultaneously. To ensure that this data could be quickly understood at a glance, we built beautiful visualizations that integrate seamlessly across The Washington Post’s website.

Beautiful and informative

We worked closely with The Washington Post’s Graphics team to design an experience that pushes the boundaries of live data reporting. The real-time data app engages users by showing the latest returns, developments, and context needed to understand the data. When new information is available, unobtrusive notifications or “freshies” alert users. Locations are subtlety highlighted when a race is tight or final results are announced, guiding readers to relevant areas on the map. The result is an instantly understandable and deeply informative app.

The application allows Washington Post readers to follow the election from any device, working fluidly on both mobile and desktop browsers, so they never miss a moment of election coverage, no matter where they are.

Data at scale

The Washington Post’s election tool works on a massive scale. The data pipeline ingests Associated Press (AP) data on tens of thousands of races, processes it all within seconds, and displays the outputs to hundreds of thousands of Washington Post viewers. To accomplish this, we built a robust pipeline from the AP, and optimized it to allow it to scale and maximize reporting speed.

The app and visualizations load quickly regardless of the number of people viewing the site. We accomplish this with a flat architecture and scaleable cloud services. We use Dockerized processes deployed to AWS ECS and country-wide caching with AWS CloudFront. A worker continuously polls the AP data service for new information and updates static JSON files with the current state of each race. We run the worker on AWS ECS so that we could quickly create a new instance in the event of any failure. Static JSON files allow us to distribute the data at scale instantly. To minimize load times for all users, we use AWS CloudFront cache locations around the country. Utilizing this static API is fundamental to the overall speed of the system, as it means that we didn’t need to build an API server that can manage the huge number of requests on election nights.

The Washington Post used this platform for every primary and caucus night. At peak, the platform successfully handled hundreds of thousands of requests per minute. Also, the system was resilient enough to support deploys during live usage. This allowed us to change the app over the course of the evening to accommodate new edge cases and address unforeseen issues.

Integrated into The Post’s existing publishing tools

To accommodate The Post’s evolving reporting needs, with the unpredictability of the 2016 election, we created a highly modular and customizable system. To do this, we built flexible design components and gave The Washington Post team the ability to mix and match these components as they needed. We integrated the election application with The Washington Post’s publishing system, exposing configuration parameters so that The Post staff could modify the display and features of each embedded component. We also built the components in a way that they intelligently interact with each other if multiple elements are deployed to the same page. To improve load time we built a top-level controller to allow modules to share data rather than having each request the data separately. All this makes for a smart and adaptable system.