Trends and data from source control
81Case study: moving to the pull request workflow and incorporating quality engineering
That makes it a lot easier to see the variance. As you can tell from figure 4.20, there wasn’t much change. Even though there’s a big dip in bugs from sprint 18 to 19, bugs weren’t decreasing over time; there was just a big jump in bugs in sprint 18. After dis-cussing the situation, the team decided to add more data points to the mix. To see how much collaboration was happening in the pull requests, they began adding com-ments to their graphs. That resulted in the chart shown in figure 4.21. To keep things consistent, we divided comments by two.
Figure 4.21 shows that there weren’t many comments along with the pull requests, which implies there wasn’t much collaboration going on. Because the bug trend wasn’t changing, it looked like the changes to their process weren’t having any effect yet. The workflow itself wasn’t producing the change the development team wanted;
they needed to make a bigger impact on their process. To do this they decided that developers should act like the QM team when they were put on a pull request. The perspective they needed wasn’t just “is this code going to solve the problem?” but “is this code well built and what can go wrong with it?” There was some concern about developers accomplishing less if they had to spend a lot of time commenting on other developers’ code and acting like the QM team. They moved one of their QM members over to the development team as a coach, and the team agreed that if this would result in fewer bugs then the effort spent up front was time well spent. They started taking the time to comment on each other’s code and ended up iterating quite a bit more on tasks before checking them in. A few sprints of this process resulted in the graph shown in figure 4.22.
Figure 4.22 shows that as collaboration between development and quality increased—in this case shown through comments in pull requests—the number of bugs went down. This was great news for the team, so they decided to take the process
30
Sprint 23
16
9
2
15 16 17 18 19
Pull requests Commits
Comments Bugs SCM stats against bugs
Not much commenting is happening on this team.
Figure 4.21 Adding comments to the graph and finding an ugly trend
one step further. The development managers brought in another member of the QM team to work with the developers on code reviews and quality checks to avoid throw-ing code over the wall to the whole QM team.
Over time, commits and pull requests started increasing as well. As the development team started thinking with a quality mindset, they started writing better code and pro-ducing fewer bugs. The combined QM and development teams found and fixed many issues before deploying their code to the test environment.
4.6 Summary
Source control is where your code is being written and reviewed and is a great source to complement PTS data for better insight into how your team is operating. Using the pull request workflow and distributed version control can give you a lot more data
Test engineers
For a long time the role of the quality department in software engineering has involved checking to make sure features were implemented to spec. That’s not an engineering discipline, and as a result many people in the QA/QM space were not engineers. To truly have an autonomous team, quality engineering has to be a significant part of the team. The role of the quality engineer, a.k.a. QE, a.k.a. SDET, a.k.a. test engineer, has started to become more and more popular. But as quality moves from one state to another in the world of software engineering, this role isn’t clearly defined, and often you have either someone with an old quality background who recently learned to write code or you have an expert in testing technology. Neither of these works; you need a senior engineer with a quality mindset. This topic could fill another book, so we’ll leave it at that.
30
Sprint 23
16
9
2
15 16 17 18 19 20 21
Pull requests Commits
Comments Bugs SCM stats against bugs
Everything is trending in the right direction. Yay!
Figure 4.22 Everything is trending in the right direction!
83 Summary
than non-distributed SCMs. Often web-based DVCS systems, like GitHub, will have built-in charts and graphs you can use to get a picture of how your team is using them.
■ Teams use source control management to manage their codebase.
■ Here are some things you can learn from SCM data alone:
■ Who is changing the code?
■ How much change is happening in the codebase?
■ Here are some questions you can answer with just SCM data:
■ Who is working on what?
■ Who is helping whom?
■ How much effort is going into the work?
■ Use pull requests with DVCS to obtain the richest data out of your SCM.
■ Look for these key trends from SCM:
■ Relationship between pull requests, commits, and comments.
■ Denied pull requests versus merged pull requests.
■ CLOC over time.
■ SCM data against PTS data to see how they affect each other.
■ DVCSs are superior to centralized ones for a variety of reasons:
■ They provide much richer data than centralized SCM systems.
■ They can improve the development process by using recommended flows.
■ They tend to have RESTful APIs for easier data collecting.
■ Pull requests combined with comments and code reviews add another dimen-sion to the team’s collaborative process.
■ GitHub with its Pulse and Graph tabs contains a lot of useful information about the health of the project.
■ Visualization techniques are available for centralized VCS through commercial products.
84