What are Engineering Metrics?
Engineering Metrics are measures used to understand the software development workflow from an engineering perspective.
What are the benefits of Engineering Metrics?
Keep an eye on
Work in Progress
Aging Time to Merge
The Aging Time to Merge visually aggregates open Pull Requests into three phases:
Open
the Pull Request was opened, and the team has not interacted or reviewed it.Engaged
one or more people commented on the Pull Request. However, it’s not approved, nor changes were requested.Reviewed
the Pull Request received at least one review (be it an approval or request changes)
Understand the team’s Engagement
Engagement Volume
SourceLevel breaks down the team’s engagement into actions. The chart measures the volume of Pull Requests opened in a week in bars. The lines represent the number of comments, approvals, and request changes in the week.
It’s an excellent tool to understand how engaged the team is. The number of comments should be proportional to the team’s size. Besides, the chart is useful for monitoring rework. A high number of request changes usually indicates code needs severe interventions.
Time to Engage
Accelerating deliveries requires measuring how much time the team spends on each step of the development flow. The Time to Engage chart displays how much time it takes for the first engagement to happen. Be it a comment, an approval, or a request change, the first action defines the starting point of the engagement.
The chart displays the 75th and 95th percentile of this number. We don’t use medians. It means that SourceLevel calculates the Time to First Engagement taking into account 75% and 95% of Pull Requests, respectively.
Accelerate and increase Deliveries
Throughput with Time to Merge
Throughput is the number of Pull Requests merged in a week. SourceLevel ignores closed Pull Requests as they didn’t add value for the final user. The Time to Merge tells how many days each pull request remained open.
Finding the correlation between the amount of work done by the time it took is the best way to determine what accelerates or slows down the process. Plus, the numbers can give you the confidence to forecast future deliveries.
Time to Merge Histogram
Histograms are an accurate representation of the distribution of numerical data. In practice, it’s a bar chart that shows the number of merged pull requests (Y-Axis) grouped by the number of days they stayed open (X-Axis). The higher the bars on the left, the faster pull requests get merged.
Examples of Engineering Metrics
Opened PRs
Opened PRs is the number of Pull Requests opened in the selected period Sum
Time to Discuss
Time to Discuss is the time elapsed from the first engagement (comment or review) to the last one. It considers only Pull Requests closed or merged in the selected period and ignores comments and reviews performed by the Pull Request’s author. 75th PCTL
Time from Last Engagement to Close
Time from Last Engagement to Close is the time elapsed from the last engagement (comment or review) to its closing. It considers only Pull Requests closed in the selected period and ignores merged ones. 75th PCTL
Reviewed PRs
Reviewed PRs is the number of Pull Requests that received at least one review (approve or request change) from someone other than the author in the selected period. Sum
Time to Merge
Time to Merge is the time elapsed between opening a Pull Request and merging it. It considers only Pull Requests merged in the period and ignores closed ones. 75th PCTL
Merged PRs
Merged PRs is the number of Pull Requests merged in the selected period. It doesn’t include merely closed Pull Requests. Sum
Added Lines of Code per PR
Added Lines of Code per PR is the 75th percentile of LOCs added in the Pull Requests of the selected period. 75th PCTL
Removed Lines of Code per PR
Removed Lines of Code per PR is the 75th percentile of LOCs removed in the Pull Requests of the selected period. 75th PCTL
Commits per PR
Commits per PR is the 75th percentile of commits in the Pull Requests of the selected period. 75th PCTL
Reviews per PR
Reviews per PR is the 75th percentile of reviews (approve or request change) in the Pull Requests of the selected period. 75th PCTL
Time to First Engagement
Time to First Engagement is the time elapsed between opening a Pull Request and its first engagement (comment or review). It considers only Pull Requests closed or merged in the selected period. 75th PCTL
Engaged PRs
Engaged PRs is the number of Pull Requests that received at least one comment or review by someone other than the author in the selected period. Sum
Time to First Review
Time to First Review is the time elapsed between opening a Pull Request and its first review (approve or request change). It considers only Pull Requests closed or merged in the selected period. 75th PCTL
Time to Close
Time to Close is the time elapsed between opening a Pull Request and closing it. It considers only Pull Requests closed in the selected period. 75th PCTL
Closed PRs
Closed PRs is the number of Pull Requests closed in the selected period. It doesn’t include Merged PRs. Sum
Time from Last Review to Merge
Time from Last Review to Merge is the time elapsed from the last review (approve or request change) to its merge. It considers only Pull Requests closed in the selected period and ignores closed ones. 75th PCTL
Approvals
Approves is the sum of approves given in the selected period. Sum
Request Changes
Request Changes is the sum of request changes given in the selected period. Sum
Comments
Comments is the number of comments (including inline comments) published in the selected period. Sum
Total Changed Lines of Code per PR
Total Changed Lines of Code per PR is the 75th percentile of LOCs changed in the Pull Requests of the selected period. 75th PCTL
Comments per PR
Comments per PR is the 75th percentile of comments (including inline comments) in the Pull Requests of the selected period. 75th PCTL
FAQ
Why doesn’t SourceLevel measure averages?
SourceLevel prefers using percentiles instead of averages because averages hide too much information when working with metrics that vary too much, such as velocity.
SourceLevel provides two percentiles for comparison: the 75th and 95th.
What do 75th and 95th percentile mean?
Let’s see an example to understand better what they mean. If your organization’s Time to Merge is 14 hours, it means that 75% of all Pull Requests merged in the period was merged up to 14 hours.
The same applies to the 95th percentile. So whatever the metric is, the 95th percentile means that 95% of all analyzed data are below that number.
Why does SourceLevel use 75th and 95th percentiles?
The 75th percentile gives more visibility to the engineering team. Managers don’t want an average team. They want a team to be as productive as it can.
The 95th percentile shows an even bigger picture of the teamwork, excluding the bottom 5% (outliers).