Evaluating Atoms of Confusion in the Context of Code Reviews

Abstract

Code review is a popular software engineering practice. Success of code reviews can be threatened by confusion experienced by code reviewers. For instance, on the one hand, research has studied the reasons for confusion in code reviews, and on the other hand, it also has analyzed source code patterns, so called "atoms of confusion", that have been shown to lead to misunderstanding in the lab setting. However, to the best of our knowledge, there is no research which tried to investigate the possible cause and effect relationship between atoms of confusion and confusion in code reviews. Another important aspect still not studied is how those atoms of confusion evolve across pull requests. In this emerging results paper, we report an exploratory case study to provide a deeper understanding of atoms of confusion, more specifically, whether atoms of confusion are related to confusion in code reviews and how they persist across pull requests. With the help of an existing tool for the detection of atoms of confusion, and a manual analysis of code reviews comments, we observed that statistical analysis did not show any relationship between atoms of confusion and presence of confusion comments in code reviews. Additionally, we found evidence that atoms of confusion are mostly not being removed in pull requests. Based on the results, we formulate hypotheses on atoms of confusion in the code review context, that should be confirmed or rejected by future studies.