Machine Learning REPA:
Reproducibility, Experiments and Pipelines Automation

Lessons learned trying implement ML REPA ideas in day-to-day work
Talk video

Cool stories about the engineering part of DS projects and simple howto's to improve your engineering skills. Agenda: Clean Code (Both Notebooks and Py), Linters 101, Refactoring and Patterns, Code Review - Practical Issues.
Target audience: junior/middle DS
Talk video

There's more to productionizing a model than packing it into a runnable container. Saving the entire training process is also a must. By keeping the model creation process reproducible we guarantee that experimental results and knowledge won't be lost. In OK.ru we developed a formal procedure whereby every model is trained in its own controlled and reproducible environment and then manually reviewed before getting put into production
Talk video

Recently, automation in software development has reached an unprecedented level of adoption. Infrastructure as code (IaC) principles, supported by tools like Terraform, Puppet, and Ansible, have become a mainstream approach to automating software building, testing, and deployment. In contrast, data science and machine learning projects frequently involve many manual steps, including data transfer and processing, model training and evaluation, and provisioning resources like cloud compute and storage. Each manual step lowers the overall reproducibility of a project and creates another hurdle to productionizing a project.
Talk video

Cool stories about the engineering part of DS projects and simple howto's to improve your engineering skills. Agenda: Clean Code (Both Notebooks and Py), Linters 101, Refactoring and Patterns, Code Review - Practical Issues.
Target audience: junior/middle DS
Talk video

In the world of winning pipelines, it is not surprising to describe DAG computations as code or config. Such a pipeline allows you to version it, work on it as a team. But which one to choose? Historically Airflow has dominated, but is it really that good? What are the benefits of the more modern Prefect? We will analyze all this in the report
Talk video

We will go thought motivation for testing in DS and common ways DS test their code. Then will be a gentle introduction to PyTest Framework. While PyTest is enough for many cases, Hypothesis for property-based testing will also be mentioned
Talk video

With the development of AI technology in all industries, the problem of reproducibility became highly important: it sharply affects the timing and cost of the development. Reproducibility also impacts on quality and performance of the final AI model, which is supposed to be implemented into production. This problem appears because of a lack of development process organization, unstructured workflow, and sometimes even low information sharing. To find a solution, we will discuss the common Model Lifecycle by overviewing the AI development process step by step, define what is Model Lifecycle Management System and how it can help a data scientist to control the results and make a model reproducible.
Talk video

DVC is an open source Version Control System for Machine Learning Projects. Along with git, DVC allows you to version your source code, your datasets, and even your pipelines. The goal of this talk is to go through DVC showing how this tool and the workflow it brings can make your life as a Data Scientist or Machine Learning Engineer much easier
Talk video

Continuous integration is a key practice from DevOps that has been demonstrated to speed up development cycles, but hasn't yet found widespread adoption in data science and machine learning projects. Yet continuous integration has many potential applications to machine learning, particularly in model training and testing in production-like environments
Talk video

One day or another every Data Science team faces a great bunch of routine tasks. Typically DS work includes creating a scope of similar models exploiting the very same client data. Designing your first model is very fun, third model is just fun, twentieth model makes you think about existential problems.

There is no doubt that several ready-made solutions to automate monotonous fit-predict already exist, however none of them meets all our demands completely. So we had no choice but to design and implement yet another framework (called Siegfried as far as Raiffeisen is Vienna bank) with Bamboo, Spark and Airflow. The framework allows to design, compare, test, deploy ML models at short term of two weeks and also increases the bus factor from 1 to 1.42
Talk video

The experience of running production machine learning systems is still limited. Best practices are yet to be shaped, and many operational aspects become an afterthought. One of the crucial but often overlooked components is model monitoring
Talk video

We will go thought motivation for testing in DS and common ways DS test their code. Then will be a gentle introduction to PyTest Framework. While PyTest is enough for many cases, Hypothesis for property-based testing will also be mentioned
Talk video

With the development of AI technology in all industries, the problem of reproducibility became highly important: it sharply affects the timing and cost of the development. Reproducibility also impacts on quality and performance of the final AI model, which is supposed to be implemented into production. This problem appears because of a lack of development process organization, unstructured workflow, and sometimes even low information sharing. To find a solution, we will discuss the common Model Lifecycle by overviewing the AI development process step by step, define what is Model Lifecycle Management System and how it can help a data scientist to control the results and make a model reproducible.
Talk video