What data should I report to my boss / my CEO / my non-technical cofounders?

Your boss, CEO, or technical cofounders rightfully demand some transparency into the activity of technical teams. But reporting on engineering activity is notoriously hard for many reasons:

• A significant portion of engineering work is invisible to outsiders.

• It’s difficult to communicate on the value of technical work, especially to a non-technical audience.

• It’s easy to get bogged down into reporting on the efficiency of the process rather than on results.

A practical approach to effective reporting

There’s a very pernicious trap to keep in mind when reporting up: you will get challenged on what you report. It’s part of you role as an engineering leader to draw the line between what you expect to be challenged on, and what you consider to be your area of autonomy. That generally means that you’ll want to communicate on results and leading indicators of success, and leave the “how” at the door. Naturally, the symmetrical applies for your relationship with your direct reports.

The example of quality

Let’s illustrate this point with a specific example that I’ve heard on many occasions: quality and code coverage. Your organization certainly cares about quality (or, at the very least, should) but how is quality defined? Observable proxies of quality might be found in your number of incidents, or in the number of calls to customer support.

Code coverage itself is not a measure of quality: it’s one of the many ways a team may measure how it intends to reach its desired level of quality. Reporting a level of 40% of code coverage is rarely more than an invitation to ask for bringing that number up to 50% next quarter. Worse: what good is 100% of code coverage when incidents are frequent and our customers are upset.

Don’t get me wrong, this doesn’t make code coverage a bad metric to follow! However, it certainly doesn’t match level of details you should aim for when reporting up. Only when the results are not what we want them to be should we dig deeper into the how: reporting prematurely on details of the process will only prevent more valuable conversations from happening. So where do we draw this line, and what makes good reporting?

The actionability test

One easy test is to ask whether the data you’re presenting is actionable by your audience. Sure data is great, and people will have opinions on data, but is it actually going to inform any decision? Are we somehow going to do things differently when presented with this data? At the core of this topic is the idea of focusing on outcomes.

To our earlier example, it is unlikely that code coverage will inform any decision for an engineering director or VP. The reason for this is that an organization rarely value code coverage: it values quality. We should therefore be satisfied as long as quality standards are met, regardless if this is achieved with 5% or 90% of code coverage. As a VP of Engineering, only when quality standards aren’t met I may then start digging into the reasons and possibly take action to reverse that trend.

This example is applicable to many process-oriented metrics: companies value time-to-market, not pull request review time; companies value progress toward its goals, not number of code changes; companies value customer satisfaction, not number of bugs closed.

How Echoes can help you communicate

Echoes helps you materialize the line between the intended outcomes (what the organization values) and the how. At its core, Echoes holds the source of truth for the outcomes you care about, and help you effortlessly tie day-to-day efforts to these intents.

For example, we can define quality as one of our intended outcomes of interest:

Once defined, Echoes will publish this outcome to your integrations of choice, for example here as GItHub labels:

Expressing the intent behind work becomes as simple as adding labels to pull requests to express why this change exists in the first place:

With this data Echoes will surface you how much efforts are actually invested in quality throughout the organization:

Combined with observable measures of quality (such as the number of incidents, or the number of customer calls to support), Echoes will soon be capable of giving you a sense of impact.

Conclusion

We hope this post will help you determine the right level of details to communicate and be challenged, both when reporting up, and when receiving updates from your direct reports. As many organizations strive to become more outcomes focused, it is crucial to keep a similar focus in our communication and not fall into the trap of communicating on details of the “how”.