Netflix created and deployed a tool inspired by Certus research

Recently, Netflix created and deployed a tool which schedules the execution of test cases on a variety of devices. For that, they took inspiration from our Certus research paper “Reinforcement Learning for Automatic Test Case Prioritization and Selection in Continuous Integration” by Helge Spieker, Arnaud Gotlieb, Dusica Marijan and Morten Mossige.

 

They nicely gave us credit for it in  https://medium.com/netflix-techblog/lerner-using-rl-agents-for-test-case-scheduling-3e0686211198

In this paper, we introduced Retecs, a new method for automatically learning test case selection and prioritization in continuous integration with the goal to minimize the round-trip time between code commits and developer feedback on failed test cases. The method exploited reinforcement learning to select and prioritize test cases according to their duration, previous last execution and failure. By applying Retecs on data extracted from an ABB’s use case, we showed that reinforcement learning was able to automatically perform adaptive test case selection and prioritization in continuous integration.