Get data that matters from GitHub and unlock your team’s potential

Hook

Yet another talk about measuring developer productivity… This time, let’s discuss how to get metrics tailored to your unique need and context! Join us to learn how WP Media went from zero metrics to a tailored data service that empowers our engineering teams. Dive into a practical, simple, real-world solution that can revolutionize your approach to measuring and enhancing productivity. Don’t miss this chance to make metrics work for you and your team!

Summary

Obtaining meaningful and useful metrics about engineering efficiency is a common struggle for tech leaders: for instance, DORA metrics might not be relevant for all teams and context, getting them accurately can be complicated or required disrupting changes in your way of using JIRA or GitHub, and finally those metrics are rarely well understood by teammates who have a hard time finding ways to improve them.

With this observation in mind when I joined WP Media, the fact that the team was only using GitHub for backlog and sprint management and no efficiency metrics in place at all, I had the freedom to explore new approaches. I decided to build our own engineering metric data service, and the results have been great: we get data adapted to our context and ways of working, our KPIs and SLAs are based on this data and our teammates have adapted tooling to understand and navigate the metrics! During this talk, I reflect on the challenge of accessing insightful productivity metrics for your tech team and how to make them useful for every teammate, extensively leveraging my experience of building this custom service at WP Media through examples.

I engage the talk by quickly discussing the challenges of measuring engineering efficiency. While there is an extensive bibliography on this topic, no commonly accepted best practice emerges and several crucial aspects are often left out such as implementation and how to empower teams through those metrics. I provide real-life examples of those challenges with my personal experience.

I explain key elements about how we use GitHub for backlog and sprint management at WP Media, and how we built a simple Django app (Python) to gather raw data, that we can then use to build relevant metrics. This section provides some entry points and strategies about how APIs such as the one from GitHub (and similar tools) can be combined with a custom back-end service to build observability for engineering efficiency.

Finally, I go through the different ways we are using the gathered raw data to showcase the advantages of a custom system. This covers how we derive refined and customized engineering SLAs and KPIs for our teams, tools to explore the data and examples of automated feedbacks the service provides to the developers so that they can autonomously improve and reflect on their own ways of working. I also share the tools that we use so that anyone can easily try and reproduce the methods presented here.

Main takeaways

• Limits of state-of-the-art metrics (LoC, DORA, SPACE)
• Common challenges of measuring engineering efficiency
• Using GitHub as a backlog & sprint management tool
• Building a simple Django app to gather useful metrics from GitHub GraphQL API
• Deriving, tracking and sharing custom metrics with Metabase
• Providing insights and build ownership around KPIs in teams
• Continuous feedback & improvement with automated metric-based alerts
• Metrics as improvement instead of evaluation

Targeted audience

This talk is primarily designed for developers, team leaders, and engineering managers. More senior tech profiles can also benefit from this talk. Attendees with previous experience of developing in a team with an issue tracker or project management tool (JIRA, GitHub, etc.) will fully benefit from this talk.