Skip to content

Commit

Permalink
docs: clean formats
Browse files Browse the repository at this point in the history
  • Loading branch information
yuxizama committed Jun 17, 2024
1 parent e8fc22f commit 509d9da
Show file tree
Hide file tree
Showing 6 changed files with 19 additions and 19 deletions.
2 changes: 1 addition & 1 deletion docs/built-in-models/encrypted_dataframe.md
Original file line number Diff line number Diff line change
Expand Up @@ -98,7 +98,7 @@ During the pre-processing and post-processing stages, the `ValueError` can happe

- A column contains values outside the allowed range for integers
- Too many unique strings
- Unsupported data type by *Concrete ML
- Unsupported data type by Concrete ML
- Unsupported data type by the operation attempted

## Example workflow
Expand Down
10 changes: 5 additions & 5 deletions docs/built-in-models/linear.md
Original file line number Diff line number Diff line change
@@ -1,11 +1,11 @@
# Linear models

This document introduces some [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `regression` and `classification` that *Concrete ML provides.
This document introduces some [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `regression` and `classification` that Concrete ML provides.

## Supported models
The use of the following models in FHE is very similar to the use of scikit-learn's [API](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model). They are also compatible with some of scikit-learn's main workflows, such as `Pipeline()` and `GridSearch()`.

| *Concrete ML | scikit-learn |
| Concrete ML | scikit-learn |
| :--------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------: |
| [LinearRegression](../references/api/concrete.ml.sklearn.linear_model.md#class-linearregression) | [LinearRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression) |
| [LogisticRegression](../references/api/concrete.ml.sklearn.linear_model.md#class-logisticregression) | [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) |
Expand Down Expand Up @@ -38,7 +38,7 @@ For a more detailed comparison of the impact of such pre-processing, please ref

## Pre-trained models

You can convert an already trained scikit-learn linear model to a *Concrete ML one by using the [`from_sklearn_model`](../references/api/concrete.ml.sklearn.base.md#classmethod-from_sklearn_model) method. See [the following example](linear.md#loading-a-pre-trained-model).
You can convert an already trained scikit-learn linear model to a Concrete ML one by using the [`from_sklearn_model`](../references/api/concrete.ml.sklearn.base.md#classmethod-from_sklearn_model) method. See [the following example](linear.md#loading-a-pre-trained-model).

## Example

Expand Down Expand Up @@ -90,15 +90,15 @@ print(
## Model accuracy
The figure below compares the decision boundary of the FHE classifier and a scikit-learn model executed in clear. You can find the complete code in the [LogisticRegression notebook](../tutorials/ml_examples.md).

The overall accuracy scores are identical (93%) between the scikit-learn model (executed in the clear) and the *Concrete ML one (executed in FHE). In fact, quantization has little impact on the decision boundaries, as linear models can use large precision numbers when quantizing inputs and weights in *Concrete ML. Additionally, as the linear models do not use [Programmable Boostrapping](/getting-started/concepts.md#cryptography-concepts), the FHE computations are always exact, irrespective of the [PBS error tolerance setting](../explanations/advanced_features.md#approximate-computations). This ensures that the FHE predictions are always identical to the quantized clear ones.
The overall accuracy scores are identical (93%) between the scikit-learn model (executed in the clear) and the Concrete ML one (executed in FHE). In fact, quantization has little impact on the decision boundaries, as linear models can use large precision numbers when quantizing inputs and weights in Concrete ML. Additionally, as the linear models do not use [Programmable Boostrapping](/getting-started/concepts.md#cryptography-concepts), the FHE computations are always exact, irrespective of the [PBS error tolerance setting](../explanations/advanced_features.md#approximate-computations). This ensures that the FHE predictions are always identical to the quantized clear ones.

![Sklearn model decision boundaries](../figures/logistic_regression_clear.png) ![FHE model decision boundaries](../figures/logistic_regression_fhe.png)



## Loading a pre-trained model

An alternative to the example above is to train a scikit-learn model in a separate step and then to convert it to *Concrete ML.
An alternative to the example above is to train a scikit-learn model in a separate step and then to convert it to Concrete ML.

```
from sklearn.linear_model import LogisticRegression as SKlearnLogisticRegression
Expand Down
4 changes: 2 additions & 2 deletions docs/built-in-models/nearest-neighbors.md
Original file line number Diff line number Diff line change
@@ -1,8 +1,8 @@
# Nearest neighbors
This document introduces the nearest neighbors non-parametric classification models that *Concrete ML provides with a scikit-learn interface through the `KNeighborsClassifier` class.
This document introduces the nearest neighbors non-parametric classification models that Concrete ML provides with a scikit-learn interface through the `KNeighborsClassifier` class.


| *Concrete ML | scikit-learn |
| Concrete ML | scikit-learn |
| :---------------------------------------------------------------------------------------------------: | --------------------------------------------------------------------------------------------------------------------- |
| [KNeighborsClassifier](../references/api/concrete.ml.sklearn.neighbors.md#class-kneighborsclassifier) | [KNeighborsClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html) |

Expand Down
8 changes: 4 additions & 4 deletions docs/built-in-models/neural-networks.md
Original file line number Diff line number Diff line change
@@ -1,16 +1,16 @@
# Neural networks
This document introduces the simple built-in neural networks models that *Concrete ML provides with a scikit-learn interface through the `NeuralNetClassifier` and `NeuralNetRegressor` classes.
This document introduces the simple built-in neural networks models that Concrete ML provides with a scikit-learn interface through the `NeuralNetClassifier` and `NeuralNetRegressor` classes.

## Supported models

| *Concrete ML | scikit-learn |
| Concrete ML | scikit-learn |
| :-------------------------------------------------------------------------------------------: | ------------------------------------------------------------------------------------------------------------ |
| [NeuralNetClassifier](../references/api/concrete.ml.sklearn.qnn.md#class-neuralnetclassifier) | [MLPClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) |
| [NeuralNetRegressor](../references/api/concrete.ml.sklearn.qnn.md#class-neuralnetregressor) | [MLPRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html) |

The neural network models are implemented with [skorch](https://skorch.readthedocs.io/en/stable/index.html), which provides a scikit-learn-like interface to Torch models (more [here](../explanations/inner-workings/external_libraries.md#skorch)).

*Concrete ML models are multi-layer, fully-connected, networks with customizable activation functions and have a number of neurons in each layer. This approach is similar to what is available in scikit-learn when using the `MLPClassifier`/`MLPRegressor` classes. The built-in models train easily with a single call to `.fit()`, which will automatically quantize weights and activations. These models use Quantization Aware Training, allowing good performance for low precision (down to 2-3 bits) weights and activations.
Concrete ML models are multi-layer, fully-connected, networks with customizable activation functions and have a number of neurons in each layer. This approach is similar to what is available in scikit-learn when using the `MLPClassifier`/`MLPRegressor` classes. The built-in models train easily with a single call to `.fit()`, which will automatically quantize weights and activations. These models use Quantization Aware Training, allowing good performance for low precision (down to 2-3 bits) weights and activations.

While `NeuralNetClassifier` and `NeuralNetClassifier` provide scikit-learn-like models, their architecture is somewhat restricted to make training easy and robust. If you need more advanced models, you can convert custom neural networks as described in the [FHE-friendly models documentation](../deep-learning/fhe_friendly_models.md).

Expand Down Expand Up @@ -44,7 +44,7 @@ concrete_classifier = NeuralNetClassifier(**params)

The [Classifier Comparison notebook](../tutorials/ml_examples.md) shows the behavior of built-in neural networks on several synthetic data-sets.

The folowing figure shows the *Concrete ML neural network trained with Quantization Aware Training in an FHE-compatible configuration and compares it to the floating-point equivalent trained with scikit-learn.
The folowing figure shows the Concrete ML neural network trained with Quantization Aware Training in an FHE-compatible configuration and compares it to the floating-point equivalent trained with scikit-learn.

![Comparison neural networks](../figures/neural_nets_builtin.png)

Expand Down
2 changes: 1 addition & 1 deletion docs/built-in-models/training.md
Original file line number Diff line number Diff line change
Expand Up @@ -28,7 +28,7 @@ model.fit(X_binary, y_binary, fhe="execute")
```

{% hint style="info" %}
Training on encrypted data provides the highest level of privacy but is slower than training on clear data. Federated learning is an alternative approach, where data privacy can be ensured by using a trusted gradient aggregator, coupled with optional _differential privacy_ instead of encryption. *Concrete ML can import linear models, including logistic regression, that are trained using federated learning using the [`from_sklearn` function](linear.md#pre-trained-models).
Training on encrypted data provides the highest level of privacy but is slower than training on clear data. Federated learning is an alternative approach, where data privacy can be ensured by using a trusted gradient aggregator, coupled with optional _differential privacy_ instead of encryption. Concrete ML can import linear models, including logistic regression, that are trained using federated learning using the [`from_sklearn` function](linear.md#pre-trained-models).
{% endhint %}

## Training configuration
Expand Down
12 changes: 6 additions & 6 deletions docs/built-in-models/tree.md
Original file line number Diff line number Diff line change
@@ -1,20 +1,20 @@
# Tree-based models
This document introduces several [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `classification` and `regression` tree models that *Concrete ML provides.
This document introduces several [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `classification` and `regression` tree models that Concrete ML provides.

## Supported models



| *Concrete ML | scikit-learn |
| Concrete ML | scikit-learn |
| :--------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------: |
| [DecisionTreeClassifier](../references/api/concrete.ml.sklearn.tree.md#class-decisiontreeclassifier) | [DecisionTreeClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier) |
| [DecisionTreeRegressor](../references/api/concrete.ml.sklearn.tree.md#class-decisiontreeregressor) | [DecisionTreeRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html#sklearn.tree.DecisionTreeRegressor) |
| [RandomForestClassifier](../references/api/concrete.ml.sklearn.rf.md#class-randomforestclassifier) | [RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier) |
| [RandomForestRegressor](../references/api/concrete.ml.sklearn.rf.md#class-randomforestregressor) | [RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html#sklearn.ensemble.RandomForestRegressor) |

*Concrete ML also supports [XGBoost](https://xgboost.ai/)'s `XGBClassifier` and `XGBRegressor`:
Concrete ML also supports [XGBoost](https://xgboost.ai/)'s `XGBClassifier` and `XGBRegressor`:

| *Concrete ML | XGboost |
| Concrete ML | XGboost |
| :-------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------: |
| [XGBClassifier](../references/api/concrete.ml.sklearn.xgb.md#class-xgbclassifier) | [XGBClassifier](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBClassifier) |
| [XGBRegressor](../references/api/concrete.ml.sklearn.xgb.md#class-xgbregressor) | [XGBRegressor](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBRegressor) |
Expand Down Expand Up @@ -113,13 +113,13 @@ print(f"{(y_pred_fhe == y_pred_clear[:N_TEST_FHE]).sum()} "
# 1 examples over 1 have an FHE inference equal to the clear inference
```

We can plot and compare the decision boundaries of the *Concrete ML model and the classical XGBoost model executed in the clear. Here we show a 6-bit model to illustrate the impact of quantization on classification. You will find similar plots in the [Classifier Comparison notebook](../tutorials/ml_examples.md).
We can plot and compare the decision boundaries of the Concrete ML model and the classical XGBoost model executed in the clear. Here we show a 6-bit model to illustrate the impact of quantization on classification. You will find similar plots in the [Classifier Comparison notebook](../tutorials/ml_examples.md).

![Comparison of clasification decision boundaries between FHE and plaintext models](../figures/xgb_comparison_pipeline.png)

## Quantization parameters

When using a sufficiently high bit width, quantization has little impact on the decision boundaries of the *Concrete ML FHE decision tree model, as quantization is done individually on each input feature. It means FHE models can achieve similar accuracy levels as floating point models. Using 6 bits for quantization is effective in reaching or even exceeding floating point accuracy.
When using a sufficiently high bit width, quantization has little impact on the decision boundaries of the Concrete ML FHE decision tree model, as quantization is done individually on each input feature. It means FHE models can achieve similar accuracy levels as floating point models. Using 6 bits for quantization is effective in reaching or even exceeding floating point accuracy.

To adjust the number of bits for quantization, use the `n_bits` parameter. Setting `n_bits` to a low value may introduce artifacts, potentially reducing accuracy. However, the execution speed in FHE could improve. This adjustment allows you to manage the accuracy/speed trade-off. Additionally, you can recover some accuracy by increasing the `n_estimators` parameter.

Expand Down

0 comments on commit 509d9da

Please sign in to comment.