Tutorial at GECCO 2021

Tutorial “Replicability and Reproducibility in Evolutionary Optimization”, L. Paquete and M. López-Ibáñez, was accepted to GECCO 2021

Abstract:

The reproducibility of experimental results is one of the corner-stones of experimental science, yet often, published experimental results are neither replicable nor reproducible due to insufficient details, missing datasets or software implementations. The Association for Computing Machinery (ACM) distinguishes among “repeatability”, “replicability” and “reproducibility” and it has instituted different badges to be attached to research articles in ACM publications depending on the level of reproducibility. The new ACM journal “Transactions on Evolutionary Learning and Optimization (TELO)” uses these badges to encourage the publication of reproducible results.

The tutorial will be structured in two main parts. The first part will introduce basic concepts in reproducible research, the motivation behind it, and potential pitfalls illustrated from real-world examples. This part will also explain in detail the ACM standards for reproducibility and explain how the process runs at the ACM TELO journal. The second part will describe several techniques for improving reproducibility, ranging from trivial but surprisingly effective to somewhat more technical and laborious. Techniques presented will be demonstrated during the session step-by-step.