Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updated readme file with better design, less text #1265

Merged
merged 11 commits into from
Nov 14, 2024
136 changes: 91 additions & 45 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,55 +1,90 @@
# Model Compression Toolkit (MCT)

Model Compression Toolkit (MCT) is an open-source project for neural network model optimization under efficient, constrained hardware.

This project provides researchers, developers, and engineers tools for optimizing and deploying state-of-the-art neural networks on efficient hardware.

Specifically, this project aims to apply quantization to compress neural networks.
<div align="center" markdown="1">

<img src="https://github.com/sony/model_optimization/raw/main/docsrc/images/mct_block_diagram.svg" width="10000">
# Model Compression Toolkit (MCT)

MCT is developed by researchers and engineers working at Sony Semiconductor Israel.
**Open-source project for neural network model optimization, providing researchers, developers, and engineers with advanced quantization and compression tools for deploying state-of-the-art neural networks on efficient, constrained hardware.**
______________________________________________________________________

</div>
<div align="center">
<p align="center">
<a href="#getting-started">Getting Started</a> •
<a href="#tutorials-and-examples">Tutorials</a> •
<a href="#supported-features">High level features and techniques</a> •
<a href="#resources">Resources</a> •
<a href="#contributions">Community</a> •
<a href="#license">License</a>
</p>
<p align="center">
<a href="https://sony.github.io/model_optimization#prerequisites"><img src="https://img.shields.io/badge/pytorch-2.1%20%7C%202.2%20%7C%202.3-blue" /></a>
<a href="https://sony.github.io/model_optimization#prerequisites"><img src="https://img.shields.io/badge/TensorFlow-2.12%20%7C%202.13%20%7C%202.14%20%7C%202.15-blue" /></a>
<a href="https://sony.github.io/model_optimization#prerequisites"><img src="https://img.shields.io/badge/python-3.9%20%7C3.10%20%7C3.11-blue" /></a>
<a href="https://github.com/sony/model_optimization/releases"><img src="https://img.shields.io/github/v/release/sony/model_optimization" /></a>
<a href="https://github.com/sony/model_optimization/blob/main/LICENSE.md"><img src="https://img.shields.io/badge/license-Apache%202.0-blue" /></a>

</p>
</div>

__________________________________________________________________________________________________________

## Getting Started
### Quick Installation
Pip install the model compression toolkit package in a Python>=3.9 environment with PyTorch>=2.1 or Tensorflow>=2.12.
```
pip install model-compression-toolkit
```
For installing the nightly version or installing from source, refer to the [installation guide](https://github.com/sony/model_optimization/blob/main/INSTALLATION.md).

**Important note**: In order to use MCT, you’ll need to provide a floating point .pt or .keras model as an input.

## Table of Contents
### Tutorials and Examples

- [Getting Started](https://github.com/sony/model_optimization?tab=readme-ov-file#getting-started)
- [Supported features](https://github.com/sony/model_optimization?tab=readme-ov-file#supported-features)
- [Results](https://github.com/sony/model_optimization?tab=readme-ov-file#results)
- [Troubleshooting](https://github.com/sony/model_optimization?tab=readme-ov-file#trouble-shooting)
- [Contributions](https://github.com/sony/model_optimization?tab=readme-ov-file#contributions)
- [License](https://github.com/sony/model_optimization?tab=readme-ov-file#license)
Our [tutorials](https://github.com/sony/model_optimization/blob/main/tutorials/README.md) section will walk you through the basics of deploying your first model; Covering various compression techniques for both Keras and PyTorch models.
Access interactive notebooks for hands-on learning with popular models/tasks or move on to [Resources](#resources) section.
ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved

### Supported Quantization flows</div>
MCT supports various quantization flows as appears below.
ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
<div align="center">
<p align="center">

## Getting Started
Quantization Method | Complexity | Computational Cost | Tutorial
-------------------- | -----------|--------------------|---------
ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
PTQ (Post Training Quantization) | Low | Low (~1-10 CPU minutes) | <a href="https://colab.research.google.com/github/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_post_training_quantization.ipynb"><img src="https://img.shields.io/badge/Pytorch-green"/></a> <a href="https://colab.research.google.com/github/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/keras/example_keras_post-training_quantization.ipynb"><img src="https://img.shields.io/badge/Keras-green"/></a>
GPTQ (parameters fine-tuning using gradients) | Moderate | Moderate (~2-3 GPU hours) | <a href="https://colab.research.google.com/github/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/keras/example_keras_mobilenet_gptq.ipynb"><img src="https://img.shields.io/badge/Keras-green"/></a>
QAT (Quantization Aware Training) | High | High (~12-36 GPU hours) | <a href="https://colab.research.google.com/github/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/keras/example_keras_qat.ipynb"><img src="https://img.shields.io/badge/Keras-green"/></a>

This section provides an installation and a quick starting guide.
</p>
</div>

### Installation
For each flow, **Quantization core** utilizes various algorithms and hyper-parameters for optimal [hardware-aware](https://github.com/sony/model_optimization/blob/main/model_compression_toolkit/target_platform_capabilities/README.md) quantization results.
For further details, please see [Supported features and algorithms](#supported-features). User can either provide their own representative dataset, or utilize the [Data Generation](#data-generation-) capability.

ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
To install the latest stable release of MCT, run the following command:
```
pip install model-compression-toolkit
```
<div align="center">
<p align="center">

For installing the nightly version or installing from source, refer to the [installation guide](https://github.com/sony/model_optimization/blob/main/INSTALLATION.md).
<img src="/docsrc/images/mctFlow.png" width="800">
</p>
ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
</div>

### Resources
* [User Guide](https://sony.github.io/model_optimization/docs/index.html) contains detailed information about MCT and guides you from installation through optimizing models for your edge AI applications.

### Quick start & tutorials
* MCT's [API Docs](https://sony.github.io/model_optimization/docs/api/api_docs/) is seperated per quantization methods:

ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
Explore the Model Compression Toolkit (MCT) through our tutorials,
covering compression techniques for Keras and PyTorch models. Access interactive [notebooks](https://github.com/sony/model_optimization/blob/main/tutorials/README.md)
for hands-on learning. For example:
* [Keras MobileNetV2 post training quantization](https://github.com/sony/model_optimization/blob/main/tutorials/notebooks/imx500_notebooks/keras/example_keras_mobilenetv2_for_imx500.ipynb)
* [Post training quantization with PyTorch](https://github.com/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_post_training_quantization.ipynb)
* [Data Generation for ResNet18 with PyTorch](https://github.com/sony/model_optimization/blob/main/tutorials/notebooks/mct_features_notebooks/pytorch/example_pytorch_data_generation.ipynb).
* [Post-training quantization](https://sony.github.io/model_optimization/docs/api/api_docs/index.html#ptq) | PTQ API docs
* [Gradient-based post-training quantization](https://sony.github.io/model_optimization/docs/api/api_docs/index.html#gptq) | GPTQ API docs
* [Quantization-aware training](https://sony.github.io/model_optimization/docs/api/api_docs/index.html#qat) | QAT API docs

* [Debug](https://sony.github.io/model_optimization/docs/guidelines/visualization.html) – modify optimization process or generate explainable report

* [Release notes](https://github.com/sony/model_optimization/releases)


### Supported Versions

Currently, MCT is being tested on various Python, Pytorch and TensorFlow versions:

<details id="supported-versions">
<summary>Supported Versions Table</summary>
ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved

| | PyTorch 2.1 | PyTorch 2.2 | PyTorch 2.3 | PyTorch 2.4 |
|-------------|----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------|
Expand All @@ -65,6 +100,7 @@ Currently, MCT is being tested on various Python, Pytorch and TensorFlow version
| Python 3.10 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python310_keras215.yml) |
| Python 3.11 | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras212.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras213.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras214.yml) | [![Run Tests](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml/badge.svg)](https://github.com/sony/model_optimization/actions/workflows/run_tests_python311_keras215.yml) |

</details>

## Supported Features
MCT offers a range of powerful features to optimize neural network models for efficient deployment. These supported features include:
Expand All @@ -83,7 +119,7 @@ MCT supports different quantization methods:
| Quantization Method | Complexity | Computational Cost |
|-----------------------------------------------|------------|-----------------------------|
| PTQ | Low | Low (order of minutes) |
| GPTQ (parameters fine-tuning using gradients) | Mild | Mild (order of 2-3 hours) |
| GPTQ (parameters fine-tuning using gradients) | Moderate | Moderate (order of 2-3 hours) |
| QAT | High | High (order of 12-36 hours) |

ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved

ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
Expand Down Expand Up @@ -130,15 +166,14 @@ For more details, we highly recommend visiting our project website where experim


## Results
### Keras
Graph of [MobileNetV2](https://keras.io/api/applications/mobilenet/) accuracy on ImageNet vs average bit-width of weights, using
single-precision quantization, mixed-precision quantization, and mixed-precision quantization with GPTQ.

<img src="https://github.com/sony/model_optimization/raw/main/docsrc/images/mbv2_accuracy_graph.png">
<p align="center">
<img src="/docsrc/images/Classification.png" width="225">
<img src="/docsrc/images/SemSeg.png" width="225">
<img src="/docsrc/images/PoseEst.png" width="225">
<img src="/docsrc/images/ObjDet.png" width="225">

For more results, please see [1]

### Pytorch
### Pytorch
We quantized classification networks from the torchvision library.
In the following table we present the ImageNet validation results for these models:

Expand All @@ -148,6 +183,14 @@ In the following table we present the ImageNet validation results for these mode
| ResNet-18 [3] | 69.86 | 69.63 |69.53|
| SqueezeNet 1.1 [3] | 58.128 | 57.678 ||

### Keras
MCT can quantize an existing 32-bit floating-point model to an 8-bit fixed-point (or less) model without compromising accuracy.
Below is a graph of [MobileNetV2](https://keras.io/api/applications/mobilenet/) accuracy on ImageNet vs average bit-width of weights (X-axis), using
single-precision quantization, mixed-precision quantization, and mixed-precision quantization with GPTQ.

<img src="https://github.com/sony/model_optimization/raw/main/docsrc/images/mbv2_accuracy_graph.png">

ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
For more results, please see [1]

ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved
#### Pruning Results

Expand All @@ -158,23 +201,26 @@ Results for applying pruning to reduce the parameters of the following models by
| ResNet50 [2] | 75.1 | 72.4 |
| DenseNet121 [3] | 74.44 | 71.71 |


## Trouble Shooting
## Troubleshooting and Community

If the accuracy degradation of the quantized model is too large for your application, check out the [Quantization Troubleshooting](https://github.com/sony/model_optimization/tree/main/quantization_troubleshooting.md)
for common pitfalls and some tools to improve quantization accuracy.
ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved

Check out the [FAQ](https://github.com/sony/model_optimization/tree/main/FAQ.md) for common issues.
Check out the [FAQ](https://github.com/sony/model_optimization/tree/main/FAQ.md) for common issues.

You are welcome to ask questions and get support on our [issues section](https://github.com/sony/model_optimization/issues) and manage community discussions under [discussions section](https://github.com/sony/model_optimization/discussions).


## Contributions
MCT aims at keeping a more up-to-date fork and welcomes contributions from anyone.

*You will find more information about contributions in the [Contribution guide](https://github.com/sony/model_optimization/blob/main/CONTRIBUTING.md).
*Checkout more our [Contribution guide](https://github.com/sony/model_optimization/blob/main/CONTRIBUTING.md) for more details.

ServiAmirPM marked this conversation as resolved.
Show resolved Hide resolved

## License
[Apache License 2.0](https://github.com/sony/model_optimization/blob/main/LICENSE.md).
MCT is licensed under Apache License Version 2.0. By contributing to the project, you agree to the license and copyright terms therein and release your contribution under these terms.

<a href="https://github.com/sony/model_optimization/blob/main/LICENSE.md"><img src="https://img.shields.io/badge/license-Apache%202.0-blue" /></a>

## References

Expand Down
Binary file added docsrc/images/Classification.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docsrc/images/ObjDet.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docsrc/images/PoseEst.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docsrc/images/SemSeg.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
1 change: 1 addition & 0 deletions docsrc/images/colab-badge.svg
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docsrc/images/mctFlow.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Loading