State of Technology

HOME DEPOT

Opportunity

Home Depot was looking for a way to track the agile health, productivity and overall happiness of developers and product teams.

Hypothesis

If we were to better understand which metrics were needed, and how best to gather accurate data, a dashboard would help executives and management better lead their teams, improving the performance of technical teams.

Goal

To research leadership need, data availability, to build a dashboard as a means to display the metrics needed to understand how teams were functioning and performing.

Roles

  • Research how managers, directors and executives understand how their teams are performing with two UX teammates.

  • Compile and synthesize findings to inform what kind of information is needed and how to display it

  • Use findings to aid in prioritizing data, and which data pipelines to build or improve

  • Use research to inform design sprints and create concepts to build and test

  • Use research and survey information to build an semi-annual report on the State of Technology (SOT) in collaboration with two UX teammates

Research

Interview many teams within the technology umbrella to better understand what metrics they use, and how they use them. Initially the scope centered mostly around engineering managers, but then as leaders they realized they also wanted metrics for executives, the scope was changed.

First round of research based on team manager need

Scope Change

Executives also wanted metrics on the teams, but with additional information to help them better understand context of how teams were performing and their effect on larger outcomes. The scope changed from being just focused on team managers to also including executive management. More research interviews with VP and executives were conducted.

Challenges

  • The development teams feared how these metrics would be used, so there needed to be an educational element for management on how use more them more effectively.

  • Home Depot technology is segmented, employing many types of software, and a variety of use patterns to build and deploy that software. These variances in work pattern made data inconsistent and hard to analyze. This variability and lack of visibility, made it difficult for executive management to understand how their teams were functioning.

  • Managers would often have their own systems and methods for understanding how their teams were performing, but those methods didn’t often easily translate into something easily understandable by upper management. Part of the goal was to use existing methods combined with new information streams to gain desired data. Some of those information sources were difficult to obtain.

  • In addition to the manager dashboard, we were asked to include a dashboard for upper/executive level understanding. They often needed additional data to create a bigger picture to help inform them, which necessitated a different setup of metrics on the dashboard. The systems and methods managers used for understanding how their teams were performing, didn’t easily translate into something easily understandable by upper/executive management. Part of the goal was to combine existing metrics with new methods needed to gain desired information.

  • Pushed to create a dashboard released in phases that would customizable for various types of management, along with education on what these metrics mean, their source, and how to improve them by improving your processes. We started with very a very simple dash to get data that was immediately available in front of team managers, and would iterate as new data sets became available.

  • Get a system in place that defined the best practices, and made information readily accessible in a common site.

There were a number of challenges:

Breakdown of information needed for management levels

Round 1 Design Sprint based only on team manager needs

Round 1 concept for manager dash

Round 2 design sprint to include all levels of management

Round 2 design concepts used for additional research with VPs

Additional Ask

My data partners and I were asked to pull together semi-annual reports showing team DORA metrics, additional performance information, along with findings from a internal tech employee satisfaction survey, as an interim solution to the dashboard.

Outcome

After several changes in scope from just managers, to adding directors, and then executives, the scope became too large, so we began to move forward with just a manager, and senior manager dashboard, in small interations. Not long after we brought on a new CTO, and this dashboard and the data pipelines necessary to build it, were put on hold while they explored outside vendors that could possibly scrub data from the existing systems and patterns to build some metrics for managers to use. This exploration is still underway.

Second round of dashboard (Phase 1), built out of existing components for quick iteration. Project put on hold in initial phases of build.