UBC Computer Science organizes conference on reproducibility and replicability of computer science research
Two papers, one keynote presentation and one poster by UBC Computer Science researchers at ACM REP 2025
As the field of computer science expands, researchers are publishing more and more scientific papers. With increasingly vast amounts of studies, scientists must ensure that their results are validated and that the knowledge they generate is true.
Replication, which is arriving at published conclusions through independent experimentation, is the cornerstone of the scientific method. In computational fields, replicability is often conflated with reproducibility, which is obtaining identical results to those published using the same code and data. While simply running someone else's code on the same data seems simple, it is surprisingly difficult due to subtle and often undocumented assumptions within the code.
This year, from July 29 – 31, 2025, researchers from across the globe congregated at the UBC Vancouver campus to discuss the latest advances in reproducibility and replicability in computational research for the ACM Conference on Reproducibility and Replicability (REP).
Several UBC Computer Science professors served as conference organizers, including Dr. Thomas Pasquier, who was the General Chair, and Dr. Nguyen Phong Hoang, who was the Local Arrangements Chair.
UBC Computer Science Department Co-Head Dr. Margo Seltzer gave the keynote presentation, “If Correct Science is the Goal, is Reproduction the Answer?" in which she challenged the scientific community to think deeply about what researchers need from the scientific process and to consider the importance of reproducibility. She then urged her colleagues to focus on the hard problems such as ensuring that what appears in published works matches the artifacts that researchers release, which helps others to verify or refute the fundamental breakthroughs claimed in publications.
Dr. Pasquier and collaborators published a paper, “On the Reproducibility of Provenance-based Intrusion Detection that uses Deep Learning,” in which they attempt to reproduce the experimental results from eight intrusion-detection systems. The researchers detailed and categorized the challenges that prevent other researchers from reproducing studies in intrusion detection, such as missing documentation, unavailable datasets and vague preprocessing steps.
UBC Computer Science graduate students Joseph Wonsil and Rúbia Guerra from Dr. Seltzer’s group led a paper, “Raising the Reproducibility Bar," in which they propose that the reproducibility community explore new technologies such as large language models and learn from user experience research to create comprehensible artifacts.
Dr. Seltzer’s group also presented a poster, led by UBC Computer Science graduate student Joseph Wonsil and UBC alum Nichole Boufford, called “Experience with Reproducibility and Consistency in Writing an Academic Paper,” that describes how to produce a paper with consistent and reproducible results using standard tools from the reproducibility literature. Lastly, they propose a research agenda to help create improved tools that make these techniques more accessible to researchers.