[Development] User Experiment: Risk Assessment in Code Review

x224yu at uwaterloo.ca x224yu at uwaterloo.ca
Wed Nov 16 17:25:16 CET 2022


Hello there QT developers,


My name is Eve and I'm a master’s student working with Prof. Shane McIntosh and Prof. Mike Godfrey at the University of Waterloo.  We're conducting a research study on the effectiveness of risk assessment during code review. Specifically, we're studying whether risk assessment can improve a reviewer’s awareness of risky changes, and if this assessment can affect their performance in identifying defects during code review.


We'd like to perform a study on the Qt codebase since Qt is a large, well-known open-source system with a high rate of code review coverage. Also, Qt has been widely used as a target system in prior related studies, so the results from this work could help to improve the overall understanding of Qt and its development practices.


For our next step, we'd like to recruit developers from Qt-Base to participate in a short user experiment.  We expect it will take each participant about 30-60 minutes in total, and the experiment be performed online and asynchronously using a simple web application.  Each participant will be asked to examine three recent code changes (i.e., commits) taken from Qt-Base; they will then be asked to rank the commits by perceived riskiness and also to perform a code review on each commit.


Once the experiment is complete, we'll share our results with the Qt community (and anyone else who is interested).  Additionally, if the experiment goes well, we'll release our risk assessment tool as a (free) plug-in to Gerrit.  We're hoping that this "carrot" might make you feel more positive about taking part in the study and furthering research into code review.  Have we managed to convince you to take part?  If so, please use the link below to sign up.

Sign-up link: https://forms.gle/zG16V9jESHZt6Vvx8

[https://lh4.googleusercontent.com/M4bTt6Kua93-eh4nVRyIRwvPUE_V0pMXoY1LEpBhBLtQ-E2SbBEZSztbApF0GdTQgeLWvlqgwLw=w1200-h630-p]<https://forms.gle/zG16V9jESHZt6Vvx8>
Studying the Impact of Risk Assessment Analytics on Risk Awareness and Code Review Performance<https://forms.gle/zG16V9jESHZt6Vvx8>
forms.gle




This study has been reviewed by and received ethics clearance from the University of Waterloo Research Ethics Board. Please note that all your experiment data will be anonymized. You will be assigned a randomly generated participation ID to access the experiment and your identifiable information will not be collected and used in any dataset or publication.


Best,

Eve

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.qt-project.org/pipermail/development/attachments/20221116/7809d924/attachment-0001.htm>


More information about the Development mailing list