diff --git a/docs/built-in-models/encrypted_dataframe.md b/docs/built-in-models/encrypted_dataframe.md index 01b384320..c2e915726 100644 --- a/docs/built-in-models/encrypted_dataframe.md +++ b/docs/built-in-models/encrypted_dataframe.md @@ -98,7 +98,7 @@ During the pre-processing and post-processing stages, the `ValueError` can happe - A column contains values outside the allowed range for integers - Too many unique strings -- Unsupported data type by *Concrete ML +- Unsupported data type by Concrete ML - Unsupported data type by the operation attempted ## Example workflow diff --git a/docs/built-in-models/linear.md b/docs/built-in-models/linear.md index 9d2fe5242..bb5df2a86 100644 --- a/docs/built-in-models/linear.md +++ b/docs/built-in-models/linear.md @@ -1,11 +1,11 @@ # Linear models -This document introduces some [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `regression` and `classification` that *Concrete ML provides. +This document introduces some [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `regression` and `classification` that Concrete ML provides. ## Supported models The use of the following models in FHE is very similar to the use of scikit-learn's [API](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model). They are also compatible with some of scikit-learn's main workflows, such as `Pipeline()` and `GridSearch()`. -| *Concrete ML | scikit-learn | +| Concrete ML | scikit-learn | | :--------------------------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------------------------------------------------------------: | | [LinearRegression](../references/api/concrete.ml.sklearn.linear_model.md#class-linearregression) | [LinearRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LinearRegression.html#sklearn.linear_model.LinearRegression) | | [LogisticRegression](../references/api/concrete.ml.sklearn.linear_model.md#class-logisticregression) | [LogisticRegression](https://scikit-learn.org/stable/modules/generated/sklearn.linear_model.LogisticRegression.html#sklearn.linear_model.LogisticRegression) | @@ -38,7 +38,7 @@ For a more detailed comparison of the impact of such pre-processing, please ref ## Pre-trained models -You can convert an already trained scikit-learn linear model to a *Concrete ML one by using the [`from_sklearn_model`](../references/api/concrete.ml.sklearn.base.md#classmethod-from_sklearn_model) method. See [the following example](linear.md#loading-a-pre-trained-model). +You can convert an already trained scikit-learn linear model to a Concrete ML one by using the [`from_sklearn_model`](../references/api/concrete.ml.sklearn.base.md#classmethod-from_sklearn_model) method. See [the following example](linear.md#loading-a-pre-trained-model). ## Example @@ -90,7 +90,7 @@ print( ## Model accuracy The figure below compares the decision boundary of the FHE classifier and a scikit-learn model executed in clear. You can find the complete code in the [LogisticRegression notebook](../tutorials/ml_examples.md). -The overall accuracy scores are identical (93%) between the scikit-learn model (executed in the clear) and the *Concrete ML one (executed in FHE). In fact, quantization has little impact on the decision boundaries, as linear models can use large precision numbers when quantizing inputs and weights in *Concrete ML. Additionally, as the linear models do not use [Programmable Boostrapping](/getting-started/concepts.md#cryptography-concepts), the FHE computations are always exact, irrespective of the [PBS error tolerance setting](../explanations/advanced_features.md#approximate-computations). This ensures that the FHE predictions are always identical to the quantized clear ones. +The overall accuracy scores are identical (93%) between the scikit-learn model (executed in the clear) and the Concrete ML one (executed in FHE). In fact, quantization has little impact on the decision boundaries, as linear models can use large precision numbers when quantizing inputs and weights in Concrete ML. Additionally, as the linear models do not use [Programmable Boostrapping](/getting-started/concepts.md#cryptography-concepts), the FHE computations are always exact, irrespective of the [PBS error tolerance setting](../explanations/advanced_features.md#approximate-computations). This ensures that the FHE predictions are always identical to the quantized clear ones. ![Sklearn model decision boundaries](../figures/logistic_regression_clear.png) ![FHE model decision boundaries](../figures/logistic_regression_fhe.png) @@ -98,7 +98,7 @@ The overall accuracy scores are identical (93%) between the scikit-learn model ( ## Loading a pre-trained model -An alternative to the example above is to train a scikit-learn model in a separate step and then to convert it to *Concrete ML. +An alternative to the example above is to train a scikit-learn model in a separate step and then to convert it to Concrete ML. ``` from sklearn.linear_model import LogisticRegression as SKlearnLogisticRegression diff --git a/docs/built-in-models/nearest-neighbors.md b/docs/built-in-models/nearest-neighbors.md index 831c84249..bba59e3b9 100644 --- a/docs/built-in-models/nearest-neighbors.md +++ b/docs/built-in-models/nearest-neighbors.md @@ -1,8 +1,8 @@ # Nearest neighbors -This document introduces the nearest neighbors non-parametric classification models that *Concrete ML provides with a scikit-learn interface through the `KNeighborsClassifier` class. +This document introduces the nearest neighbors non-parametric classification models that Concrete ML provides with a scikit-learn interface through the `KNeighborsClassifier` class. -| *Concrete ML | scikit-learn | +| Concrete ML | scikit-learn | | :---------------------------------------------------------------------------------------------------: | --------------------------------------------------------------------------------------------------------------------- | | [KNeighborsClassifier](../references/api/concrete.ml.sklearn.neighbors.md#class-kneighborsclassifier) | [KNeighborsClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neighbors.KNeighborsClassifier.html) | diff --git a/docs/built-in-models/neural-networks.md b/docs/built-in-models/neural-networks.md index 9509a16b8..4283b0449 100644 --- a/docs/built-in-models/neural-networks.md +++ b/docs/built-in-models/neural-networks.md @@ -1,16 +1,16 @@ # Neural networks -This document introduces the simple built-in neural networks models that *Concrete ML provides with a scikit-learn interface through the `NeuralNetClassifier` and `NeuralNetRegressor` classes. +This document introduces the simple built-in neural networks models that Concrete ML provides with a scikit-learn interface through the `NeuralNetClassifier` and `NeuralNetRegressor` classes. ## Supported models -| *Concrete ML | scikit-learn | +| Concrete ML | scikit-learn | | :-------------------------------------------------------------------------------------------: | ------------------------------------------------------------------------------------------------------------ | | [NeuralNetClassifier](../references/api/concrete.ml.sklearn.qnn.md#class-neuralnetclassifier) | [MLPClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPClassifier.html) | | [NeuralNetRegressor](../references/api/concrete.ml.sklearn.qnn.md#class-neuralnetregressor) | [MLPRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.neural_network.MLPRegressor.html) | The neural network models are implemented with [skorch](https://skorch.readthedocs.io/en/stable/index.html), which provides a scikit-learn-like interface to Torch models (more [here](../explanations/inner-workings/external_libraries.md#skorch)). -*Concrete ML models are multi-layer, fully-connected, networks with customizable activation functions and have a number of neurons in each layer. This approach is similar to what is available in scikit-learn when using the `MLPClassifier`/`MLPRegressor` classes. The built-in models train easily with a single call to `.fit()`, which will automatically quantize weights and activations. These models use Quantization Aware Training, allowing good performance for low precision (down to 2-3 bits) weights and activations. +Concrete ML models are multi-layer, fully-connected, networks with customizable activation functions and have a number of neurons in each layer. This approach is similar to what is available in scikit-learn when using the `MLPClassifier`/`MLPRegressor` classes. The built-in models train easily with a single call to `.fit()`, which will automatically quantize weights and activations. These models use Quantization Aware Training, allowing good performance for low precision (down to 2-3 bits) weights and activations. While `NeuralNetClassifier` and `NeuralNetClassifier` provide scikit-learn-like models, their architecture is somewhat restricted to make training easy and robust. If you need more advanced models, you can convert custom neural networks as described in the [FHE-friendly models documentation](../deep-learning/fhe_friendly_models.md). @@ -44,7 +44,7 @@ concrete_classifier = NeuralNetClassifier(**params) The [Classifier Comparison notebook](../tutorials/ml_examples.md) shows the behavior of built-in neural networks on several synthetic data-sets. -The folowing figure shows the *Concrete ML neural network trained with Quantization Aware Training in an FHE-compatible configuration and compares it to the floating-point equivalent trained with scikit-learn. +The folowing figure shows the Concrete ML neural network trained with Quantization Aware Training in an FHE-compatible configuration and compares it to the floating-point equivalent trained with scikit-learn. ![Comparison neural networks](../figures/neural_nets_builtin.png) diff --git a/docs/built-in-models/training.md b/docs/built-in-models/training.md index 2a49dbed4..05954069d 100644 --- a/docs/built-in-models/training.md +++ b/docs/built-in-models/training.md @@ -28,7 +28,7 @@ model.fit(X_binary, y_binary, fhe="execute") ``` {% hint style="info" %} -Training on encrypted data provides the highest level of privacy but is slower than training on clear data. Federated learning is an alternative approach, where data privacy can be ensured by using a trusted gradient aggregator, coupled with optional _differential privacy_ instead of encryption. *Concrete ML can import linear models, including logistic regression, that are trained using federated learning using the [`from_sklearn` function](linear.md#pre-trained-models). +Training on encrypted data provides the highest level of privacy but is slower than training on clear data. Federated learning is an alternative approach, where data privacy can be ensured by using a trusted gradient aggregator, coupled with optional _differential privacy_ instead of encryption. Concrete ML can import linear models, including logistic regression, that are trained using federated learning using the [`from_sklearn` function](linear.md#pre-trained-models). {% endhint %} ## Training configuration diff --git a/docs/built-in-models/tree.md b/docs/built-in-models/tree.md index 7f624f489..2947eb3b4 100644 --- a/docs/built-in-models/tree.md +++ b/docs/built-in-models/tree.md @@ -1,20 +1,20 @@ # Tree-based models -This document introduces several [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `classification` and `regression` tree models that *Concrete ML provides. +This document introduces several [scikit-learn](https://scikit-learn.org/stable/)'s linear models for `classification` and `regression` tree models that Concrete ML provides. ## Supported models -| *Concrete ML | scikit-learn | +| Concrete ML | scikit-learn | | :--------------------------------------------------------------------------------------------------: | :--------------------------------------------------------------------------------------------------------------------------------------------------------------: | | [DecisionTreeClassifier](../references/api/concrete.ml.sklearn.tree.md#class-decisiontreeclassifier) | [DecisionTreeClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html#sklearn.tree.DecisionTreeClassifier) | | [DecisionTreeRegressor](../references/api/concrete.ml.sklearn.tree.md#class-decisiontreeregressor) | [DecisionTreeRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeRegressor.html#sklearn.tree.DecisionTreeRegressor) | | [RandomForestClassifier](../references/api/concrete.ml.sklearn.rf.md#class-randomforestclassifier) | [RandomForestClassifier](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html#sklearn.ensemble.RandomForestClassifier) | | [RandomForestRegressor](../references/api/concrete.ml.sklearn.rf.md#class-randomforestregressor) | [RandomForestRegressor](https://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestRegressor.html#sklearn.ensemble.RandomForestRegressor) | -*Concrete ML also supports [XGBoost](https://xgboost.ai/)'s `XGBClassifier` and `XGBRegressor`: +Concrete ML also supports [XGBoost](https://xgboost.ai/)'s `XGBClassifier` and `XGBRegressor`: -| *Concrete ML | XGboost | +| Concrete ML | XGboost | | :-------------------------------------------------------------------------------: | :----------------------------------------------------------------------------------------------------: | | [XGBClassifier](../references/api/concrete.ml.sklearn.xgb.md#class-xgbclassifier) | [XGBClassifier](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBClassifier) | | [XGBRegressor](../references/api/concrete.ml.sklearn.xgb.md#class-xgbregressor) | [XGBRegressor](https://xgboost.readthedocs.io/en/stable/python/python_api.html#xgboost.XGBRegressor) | @@ -113,13 +113,13 @@ print(f"{(y_pred_fhe == y_pred_clear[:N_TEST_FHE]).sum()} " # 1 examples over 1 have an FHE inference equal to the clear inference ``` -We can plot and compare the decision boundaries of the *Concrete ML model and the classical XGBoost model executed in the clear. Here we show a 6-bit model to illustrate the impact of quantization on classification. You will find similar plots in the [Classifier Comparison notebook](../tutorials/ml_examples.md). +We can plot and compare the decision boundaries of the Concrete ML model and the classical XGBoost model executed in the clear. Here we show a 6-bit model to illustrate the impact of quantization on classification. You will find similar plots in the [Classifier Comparison notebook](../tutorials/ml_examples.md). ![Comparison of clasification decision boundaries between FHE and plaintext models](../figures/xgb_comparison_pipeline.png) ## Quantization parameters -When using a sufficiently high bit width, quantization has little impact on the decision boundaries of the *Concrete ML FHE decision tree model, as quantization is done individually on each input feature. It means FHE models can achieve similar accuracy levels as floating point models. Using 6 bits for quantization is effective in reaching or even exceeding floating point accuracy. +When using a sufficiently high bit width, quantization has little impact on the decision boundaries of the Concrete ML FHE decision tree model, as quantization is done individually on each input feature. It means FHE models can achieve similar accuracy levels as floating point models. Using 6 bits for quantization is effective in reaching or even exceeding floating point accuracy. To adjust the number of bits for quantization, use the `n_bits` parameter. Setting `n_bits` to a low value may introduce artifacts, potentially reducing accuracy. However, the execution speed in FHE could improve. This adjustment allows you to manage the accuracy/speed trade-off. Additionally, you can recover some accuracy by increasing the `n_estimators` parameter.