Are we the best that we could be?
Are we making the most of what we have?

This is a natural concern for engineering leaders, and for most people really: finding accomplishment in doing our job well. Every engineering leader has at heart to build a great organization, but how are they to answer these questions in a space as complex and heterogenous as software?

Comparing to others

This is benchmarking: we search for comparable organizations and compare a set of chosen performance metrics. Unfortunately:

• The perfectly comparable organization doesn’t exist. Companies of similar size may have very different challenges based on the industry they evolve in. Companies within a same industry may have different setups based on their market and product maturity.
  
  

• The perfectly comparable metrics don't exist either. Sure, DORA metrics come to mind as a widely recognized industry-standard for measuring the performance of engineering organization. However they say nothing about the value created and the business impact of engineering efforts.

Comparing to our potential

This is the most interesting but the most complex, because it requires building a realistic picture of what could be (realistic in that as much as every CEO wants the new feature tomorrow, there sadly are practical limitations to achieving that).

The best we can do is to identify the next thing standing between us and our potential, make that step, and repeat this process until we start seeing diminishing returns. Most likely, our conditions change fast enough that we won’t ever run out of things to improve before hitting a ceiling.

This is what you should be asking from your engineering metrics: not only to tell you who you are today, but what you could become tomorrow. There are no values in the numbers, only in the decisions they inform.