Skip to content

Commit

Permalink
Add Cookiecutter MLOps Package template
Browse files Browse the repository at this point in the history
  • Loading branch information
fmind committed Jul 28, 2024
1 parent 8ad7a41 commit 309e726
Show file tree
Hide file tree
Showing 4 changed files with 24 additions and 17 deletions.
4 changes: 4 additions & 0 deletions docs/0. Overview/0.6. Resources.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ description: Access additional resources like the MLOps Python Package, blog pos

This course is supplemented with a variety of resources aimed at enriching your MLOps learning journey. A key resource is the **[MLOps Python Package](https://github.com/fmind/mlops-python-package)**, designed to exemplify how to structure an MLOps codebase efficiently. This package incorporates the dataset featured in the course, providing a holistic view of what your end project could resemble.

Another important resource is the **[Cookiecutter MLOps Package](https://github.com/fmind/cookiecutter-mlops-package)** which generalizes the concepts provided in the [MLOps Python Package](https://github.com/fmind/mlops-python-package) and this course, allowing you to quickly create a new MLOps project with the same structure.

For those seeking to deepen their knowledge in specific areas, the course creators have also contributed insights through [personal blog posts](https://fmind.medium.com/). These articles explore subjects like setting up [Visual Studio Code for MLOps activities](https://fmind.medium.com/how-to-configure-vs-code-for-ai-ml-and-mlops-development-in-python-%EF%B8%8F%EF%B8%8F-8582d8c6ea54) or [employing Pydantic for robust data validation](https://fmind.medium.com/make-your-mlops-code-base-solid-with-pydantic-and-pythons-abc-aeedfe9c3e65). Such resources offer additional insights and actionable advice to enhance your understanding and skills.

## Can you suggest a new project resource?
Expand All @@ -18,6 +20,8 @@ To propose a new project resource, please forward your suggestions to the course

## Additional vendor resources

- **[MLOps Python Package](https://github.com/fmind/mlops-python-package)**
- **[Cookiecutter MLOps Package](https://github.com/fmind/cookiecutter-mlops-package)**
- [MLOps: Continuous delivery and automation pipelines in machine learning](https://cloud.google.com/architecture/mlops-continuous-delivery-and-automation-pipelines-in-machine-learning)
- [The Big Book of MLOps](https://www.databricks.com/sites/default/files/2023-10/2023-10-eb-big-book-of-mlops-2nd-edition-v2-102723-final.pdf)
- [Practitioners guide to MLOps: A framework for continuous delivery and automation of machine learning](https://services.google.com/fh/files/misc/practitioners_guide_to_mlops_whitepaper.pdf)
Expand Down
33 changes: 19 additions & 14 deletions docs/6. Sharing/6.4. Templates.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,8 @@ description: Discover the power of code templates for MLOps projects, learn how

[A code template](https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-template-repository) provides a standardized framework for initiating new projects, particularly beneficial in environments where multiple projects share common elements. This structure usually includes predefined configurations for common tools like linters, unit testing frameworks, and code formatters. By doing so, each new project can be customized with specific details such as the project name, description, and operating environment, while maintaining a consistent approach to software development.

For instance, the authors of this course provide the **[Cookiecutter MLOps Project](https://github.com/fmind/cookiecutter-mlops-package)** for free to quickly generate new MLOps Projects based on the principles described in this course. This section explains how to leverage similar code templates and adapt them for your organization.

## Why do you need to create code templates?

- **Align code base and best practices:** Creating a code template helps enforce uniformity and adherence to best practices across all projects within an organization
Expand All @@ -26,7 +28,7 @@ As AI/ML projects become more standardized and akin to an assembly line producti
cookiecutter [template-directory]
```

This command processes the template directory or repository containing a [`cookiecutter.json`](https://github.com/audreyfeldroy/cookiecutter-pypackage/blob/master/cookiecutter.json) file and potentially other template files, prompting the user for input on defined variables.
This command processes the template directory or repository containing a [`cookiecutter.json`](https://github.com/fmind/cookiecutter-mlops-package/blob/main/cookiecutter.json) file and potentially other template files, prompting the user for input on defined variables.

### Cruft

Expand Down Expand Up @@ -55,36 +57,38 @@ Variables in Cookiecutter templates are managed using [Jinja2](https://jinja.pal
project_name = "{{ cookiecutter.project_name }}"
```

The [`cookiecutter.json`](https://github.com/audreyfeldroy/cookiecutter-pypackage/blob/master/cookiecutter.json) file is where all default values for the variables in a template are defined. When a new project is generated, Cookiecutter prompts the user to input values for these variables or accept the defaults as specified in the JSON file.
The [`cookiecutter.json`](https://github.com/fmind/cookiecutter-mlops-package/blob/main/cookiecutter.json) file is where all default values for the variables in a template are defined. When a new project is generated, Cookiecutter prompts the user to input values for these variables or accept the defaults as specified in the JSON file.

Here is an example of defining variables in the [`cookiecutter.json`](https://github.com/audreyfeldroy/cookiecutter-pypackage/blob/master/cookiecutter.json) file, which includes project metadata and configuration defaults:
Here is an example of defining variables in the [`cookiecutter.json`](https://github.com/fmind/cookiecutter-mlops-package/blob/main/cookiecutter.json) file, which includes project metadata and configuration defaults:

```json
{
"full_name": "Template Author",
"email": "author@localhost",
"github_username": "author",
"project_name": "Python Boilerplate",
"project_slug": "{{ cookiecutter.project_name.lower().replace(' ', '_').replace('-', '_') }}",
"project_short_description": "This is a template for Python projects.",
"version": "0.1.0"
"user": "fmind",
"name": "MLOps Project",
"repository": "{{cookiecutter.name.lower().replace(' ', '-')}}",
"package": "{{cookiecutter.repository.replace('-', '_')}}",
"license": "MIT",
"version": "0.1.0",
"description": "TODO",
"python_version": "3.12",
"mlflow_version": "2.14.3"
}
```

## How should you structure a cookiecutter template repository?

[A cookiecutter template repository](https://github.com/audreyfeldroy/cookiecutter-pypackage/tree/master) typically consists of two primary levels of structure:
[A cookiecutter template repository](https://github.com/fmind/cookiecutter-mlops-package) typically consists of two primary levels of structure:

1. **Template Files and Folders:** These are the directories and files that will be generated and are identifiable by their names containing cookiecutter variables (e.g., [`{{cookiecutter.project_slug}}`](https://github.com/audreyfeldroy/cookiecutter-pypackage/tree/master/%7B%7Bcookiecutter.project_slug%7D%7D)). All such template files and folders are contained within a single root directory (e.g, [`{{cookiecutter.project_slug}}`](https://github.com/audreyfeldroy/cookiecutter-pypackage/tree/master/%7B%7Bcookiecutter.project_slug%7D%7D)) to facilitate easy generation.
1. **Template Files and Folders:** These are the directories and files that will be generated and are identifiable by their names containing cookiecutter variables (e.g., [`{{cookiecutter.project_slug}}`](https://github.com/fmind/cookiecutter-mlops-package/tree/master/%7B%7Bcookiecutter.repository%7D%7D). All such template files and folders are contained within a single root directory (e.g, [`{{cookiecutter.repository}}`](https://github.com/fmind/cookiecutter-mlops-package/tree/master/%7B%7Bcookiecutter.repository%7D%7D)) to facilitate easy generation.

2. **Supporting Files:** These include additional resources such as documentation, scripts for automating setup tasks, or configuration files necessary for the template itself but not part of the generated project files. These files reside outside the main template folder.

Refer to the [cookiecutter-pypackage template by cookiecutter's author](https://github.com/audreyfeldroy/cookiecutter-pypackage) for a practical example of how to structure a template repository.
Refer to the [cookiecutter-mlops-package template by this course authors](https://github.com/fmind/cookiecutter-mlops-package) for a practical example of how to structure a template repository.

**Initializing this template package**:

```bash
cookiecutter gh:audreyfeldroy/cookiecutter-pypackage
cookiecutter gh:fmind/cookiecutter-mlops-package
```

For advanced structuring techniques and best practices, refer to the [Advanced Usage section of Cookiecutter documentation](https://cookiecutter.readthedocs.io/en/stable/advanced/index.html).
Expand Down Expand Up @@ -177,5 +181,6 @@ for path in REMOVE_PATHS:

## Templates additional resources

- **[Cookiecutter MLOps Project](https://github.com/fmind/cookiecutter-mlops-package)**
- [Creating a GitHub template repository](https://docs.github.com/en/repositories/creating-and-managing-repositories/creating-a-template-repository)
- [Python Package template with cookiecutter](https://github.com/audreyfeldroy/cookiecutter-pypackage/blob/master/cookiecutter.json)
2 changes: 1 addition & 1 deletion pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,12 @@
name = "mlops-coding-course"
version = "0.1.0"
description = "Learn how to create, develop, and maintain an MLOps code base."
package-mode = false
repository = "https://github.com/MLOps-Courses/mlops-coding-course"
documentation = "https://mlops-coding-course.fmind.dev/"
authors = ["Médéric HURIER <[email protected]>"]
readme = "README.md"
license = "CC BY"
package-mode = false

# DEPENDENCIES

Expand Down
2 changes: 0 additions & 2 deletions tasks.py
Original file line number Diff line number Diff line change
Expand Up @@ -4,8 +4,6 @@

from invoke.context import Context
from invoke.tasks import task
from pathlib import Path
import shutil

# %% TASKS

Expand Down

0 comments on commit 309e726

Please sign in to comment.