ai_repo

Important information: This repository has now been archived.

---

Examples

🔢 MNIST classification with a MLP

python projects/classify_mnist/train.py task=mlp

🤸 Acrobot score optimization with neuroevolution

python -m projects.ne_control_score task=acrobot

Table of Contents

👀 I. Introduction

📖 II. Overview

🌳 III. Rough repository tree

I. Introduction

This repo aims to:

Reduce code & configuration boilerplate with:

• Hydra for task configuration.
• Hydra-zen for Hydra structured configuration management.
• Lightning for PyTorch code.

Simplify machine learning workflows:

• Hyperparameter optimization with AutoRL-Sweepers & Optuna.
• SLURM job definition, queuing and monitoring with Submitit through its Hydra Launcher plugin.
• Docker / Apptainer environment containerization for both regular & SLURM-based execution.
• Transition from regular execution to SLURM-based execution by only swapping container technology and as little as a single Hydra configuration field.

Automate workspace & coding processes:

• Package upgrades through Renovate.
• Docstring documentation generation with Sphinx.
• Pre-commit formatting & linting hooks with pre-commit.
• Documentation/Docker image validation/deployment, formatting, linting, type-checking & unit tests upon contribution to the main branch using GitHub Actions.

Facilitate collaboration through:

• An object-oriented structure for code sharing & reusability.
• A mono-repository workspace with task/experiment-specific subdirectories.
• A very informative & clear to navigate Python API reference thanks to Autodoc and plugins like sphinx-autodoc-typehints and sphinx-paramlinks.
• Shared logging with Weights & Biases.

Promote high-quality and reproducible code by:

• Linting with Ruff, formatting with Black, unit-testing with pytest.
• Type-checking with Mypy (static) & Beartype (dynamic).
• DType & Shape type hinting for PyTorch tensors using jaxtyping & NumPy arrays using nptyping. Fully type checkable at runtime with Beartype.
• Providing a common Development Container recipe with the above features enabled + documentation preview with esbonio.

Smoothen up rough edges by providing:

• Extensive documentation on how to install/execute on regular & SLURM-based systems.
• Unassuming guides on how to contribute to the codebase.
• Tutorials on how to facilitate code transport across machines.

II. Overview

Service

🔍 Overview

A service refers to a Python package located at common/SERVICE_NAME/. Each service is meant to sketch out a form of execution.

📂 Examples

• Optimization

• Inference

Engine

🔍 Overview

An engine refers to a Python package located at common/SERVICE_NAME/ENGINE_NAME. Each engine is meant to drive a specific type of execution.

📂 Examples

• Deep Learning

• Lightning Checkpoint Inference

Project

🔍 Overview

A project refers to a Python package located at projects/PROJECT_NAME/.

📂 Examples

• MNIST classification

• Neuroevolution Control Score Optimization

Task

🔍 Overview

A task is a work unit specified by a Hydra .yaml config file located in projects/PROJECT_NAME/task/TASK_NAME.yaml.

📂 Examples

• Neuroevolution Control Score Optimization on Acrobot

• Learning rate random search for MNIST classification with a MLP on the Beluga SLURM cluster

Subtask

A subtask is a sub-work unit of a task (ex: a model training run with a specific set of hyper-parameters).

Rough repository tree

ai_repo/
├─ .github/                  <-- Config files for GitHub Actions (tests, containers, etc)
├─ common/                   <-- Code common to various `projects`
│  ├─ infer/                 <-- Model inference
│  │  └─ lightning/          <-- Inference from Lightning checkpoints
│  └─ optim/                 <-- Model optimization
│     ├─ dl/                 <-- Deep Learning
│     │  ├─ datamodule/      <-- Lightning DataModules
│     │  ├─ litmodule/       <-- Lightning Modules
│     │  │  └─ nnmodule/     <-- PyTorch Modules
│     │  └─ train.py
│     └─ ne/                 <-- Neuroevolution
│        ├─ agent/
│        ├─ net/
│        ├─ space/           <-- Where agents evolve
│        └─ evolve.py
├─ docs/                     <-- Documentation
└─ projects/                 <-- Contains all existing projects
   │
   │                             ******************************************
   └─ my_new_dl_project/     <-- ******** Your new project folder *********
      ├─ task/               <-- *********** Your task folder *************
      │  └─ config.yaml      <-- ****** Your task configuration file ******
      ├─ __main__.py         <-- ************ Your entry-point ************
      ├─ datamodule.py       <-- ******* Your Lightning DataModule ********
      ├─ litmodule.py        <-- ********* Your Lightning Module **********
      └─ nnmodule.py         <-- ********** Your PyTorch Module ***********
                                 ******************************************