Predicting developers' negative feelings about code review

Auteur-es

Egelman, Carolyn D.; Murphy-Hill, Emerson; Kammer, Elizabeth; Hodges, Margaret Morrow; Green, Collin; Jaspan, Ciera; Lin, James

Nombre Auteurs

7

Titre

Predicting developers' negative feelings about code review

Année de publication

2020

Référence (APA)

Egelman, C. D., Murphy-Hill, E., Kammer, E., Hodges, M. M., Green, C., Jaspan, C., & Lin, J. (2020). Predicting developers’ negative feelings about code review. Proceedings of the ACM/IEEE 42nd International Conference on Software Engineering, 174‑185. https://doi.org/10.1145/3377811.3380414

Mots-clés

code review, interpersonal con ict

URL

https://dl.acm.org/doi/10.1145/3377811.3380414

doi

https://doi.org/10.1145/3377811.3380414

Accessibilité de l'article

Open access

Champ

Software Engineering

Type contenu (théorique Applicative méthodologique)

Applicative

Méthode

To study negative behaviors in code reviews, we combine qualitative and quantitative methods using surveys and log data from Google. We developed three log-based metrics, informed by interviews with a diverse group of 14 developers, to detect feelings of pushback in code reviews and validated those metrics through a survey.
Analysis of developer interviews
Analysis of developer survey responses

Cas d'usage

Google developers

Objectifs de l'article

This paper seeks to understand and measure negative experiences in code review. Measurement enables understanding of the prevalence of the bad experiences, whether negative experiences are occurring at different rates in subpopulations of developers, and whether initiatives aimed at reducing negative experiences, like codes of conduct, are working.

Question(s) de recherche/Hypothèses/conclusion

Research question(s) : We asked the following research questions: 1) How frequent are negative experiences with code review? 2) What factors are associated with pushback occuring? 3) What metrics detect author-perceived pushback?

Hypothesis(es) : In the process, developers may have negative interpersonal interactions with their peers, which can lead to frustration and stress; these negative interactions may ultimately result in developers abandoning projects.

Conclusion(s) : Our results suggest that such negative experiences, which we call “pushback”, are relatively rare in practice, but have negative repercussions when they occur. Our metrics can predict feelings of pushback with high recall but low precision, making them potentially appropriate for highlighting interactions that may bene t from a self-intervention

Cadre théorique/Auteur.es

Interpersonal conflict (Schieman et Reid, 2008)

Developer's professional experiences (Singh Arneja, 2015 ; Dietrich, 2018 ; Ranzhin, 2019 ; Graziotin et al., 2019)

Concepts clés

Interpersonal conflict

Données collectées (type source)

Developer log-based metrics data from Google
Responses to interviews with Google developers
Responses to a survey of Google developers

Définition des émotions

No definition

Use of sentiment categories/groups

Ampleur expérimentation (volume de comptes)

14 one-hour semi-structured interviews
2,500 contacted, 1,317 responded to the first section; 606 authors and 573 reviewers responded to the second section on their own code reviews; and 1,182 responded to the third section on third-party code reviews. 78% of survey respondents worked in an office in the USA; 16% in Europe, the Middle East or Africa; 4% in the Asia-Pacific region; and 2% in the Americas outside the USA.

Technologies associées

Google’s code review process

Mention de l'éthique

About the objective of research :
Unfortunately, we have little systematic understanding of what makes a code review go bad. This is important for three reasons. First, from an ethical perspective, we should seek to make the software engineering process fair and inclusive.

About the interviews :
Given the sensitivity of the interview topics, we conducted interviews with informed consent and according to a strict ethical protocol.1
1. We note specifically that we provided participants the option to participate with or without audio recording, and we paused recording when starting to discuss topics the interviewer or participant identified as potentially sensitive. We also provided materials on organizational resources for dealing with workplace concerns, and described the responsibilities the researchers and notetakers had related to reporting policy violations to reduce any ambiguity about what to expect during or after the session.

Finalité communicationnelle

While our predictions are far from perfect, we believe such predictions are needed to support future interventions designed to help reduce pushback so that we can monitor the effectiveness of those interventions. Detecting pushback may also help identify any subpopulations where pushback may be more prevalent.

Résumé

During code review, developers critically examine each others' code to improve its quality, share knowledge, and ensure conformance to coding standards. In the process, developers may have negative interpersonal interactions with their peers, which can lead to frustration and stress; these negative interactions may ultimately result in developers abandoning projects. In this mixed-methods study at one company, we surveyed 1,317 developers to characterize the negative experiences and cross-referenced the results with objective data from code review logs to predict these experiences. Our results suggest that such negative experiences, which we call "pushback", are relatively rare in practice, but have negative repercussions when they occur. Our metrics can predict feelings of pushback with high recall but low precision, making them potentially appropriate for highlighting interactions that may benefit from a self-intervention.