docs: document KNN, PoT, Hybrid models #260

andrei-stoian-zama · 2023-09-20T12:33:52Z

Not finished, the examples won't work yet, it should be done when the KNN + Hybrid model PRs are merged.

Closes https://github.com/zama-ai/concrete-ml-internal/issues/3975
Closes https://github.com/zama-ai/concrete-ml-internal/issues/3976
Closes https://github.com/zama-ai/concrete-ml-internal/issues/3977

kcelia · 2023-09-21T16:02:05Z

docs/built-in-models/nearest-neighbors.md

+```python
+from concrete.ml.sklearn import KNeighborsClassifier
+
+concrete_classifier = KNeighborsClassifier(n_bits=2,     n_neighbors=3)


too much space

docs/built-in-models/nearest-neighbors.md

docs/built-in-models/neural-networks.md

RomanBredehoft

a few questions, but apart from that all good for me

docs/built-in-models/neural-networks.md

kcelia · 2023-09-21T16:33:40Z

docs/built-in-models/nearest-neighbors.md

+
+The FHE inference latency of this model is heavily influenced by the `n_bits`, the dimensionality of the data. Furthermore, the size of the dataset has a linear impact on the complexity of the data and the number of nearest neighbors, `n_neighbors`, also plays a role.
+
+The KNN computation executes in FHE in $$O(Nlog^2k)$$ steps, where $$N$$ is the training dataset size and $$k$$ is `n_neighbors`. Each step requires several PBS of the precision required to represent the distances between test vectors and the training dataset.


I suggest:

The concrete KNN algorithm operates in O(Nlog⁡2k) steps, where N .....

Each step needs several PBS of the precision required to represent the distances between query vector and the training dataset.

by the way, I find "Each step requires several PBS of the precision required", a bit unclear.

So maybe that way:

Each step requires several PBSs to effectively represent the precision of distances between query vector and the training dataset.

We don't talk about the computation of the labels for the k-nearest neighbors ?

docs/built-in-models/nearest-neighbors.md

kcelia · 2023-09-21T17:04:04Z

docs/built-in-models/neural-networks.md

@@ -89,4 +82,6 @@ You can give weights to each class to use in training. Note that this must be su

 ### Overflow errors

-The `n_hidden_neurons_multiplier` parameter influences training accuracy as it controls the number of non-zero neurons that are allowed in each layer. Increasing `n_hidden_neurons_multiplier` improves accuracy, but should take into account precision limitations to avoid an overflow in the accumulator. The default value is a good compromise that avoids an overflow in most cases, but you may want to change the value of this parameter to reduce the breadth of the network if you have overflow errors. A value of 1 should be completely safe with respect to overflow.
+The `n_accum_bits` parameter influences training accuracy as it controls the number of non-zero neurons that are allowed in each layer. Increasing `n_accum_bits` improves accuracy, but should take into account precision limitations to avoid an overflow in the accumulator. The default value is a good compromise that avoids an overflow in most cases, but you may want to change the value of this parameter to reduce the breadth of the network if you have overflow errors.


jfrery

Thanks for handling this! I have a few comments.

jfrery · 2023-09-22T07:12:03Z

docs/advanced-topics/advanced_features.md

+An example of such implementation is available in [evaluate_torch_cml.py](../../use_case_examples/cifar/cifar_brevitas_training/evaluate_one_example_fhe.py) and [CifarInFheWithSmallerAccumulators.ipynb](../../use_case_examples/cifar/cifar_brevitas_finetuning/CifarInFheWithSmallerAccumulators.ipynb)



jfrery · 2023-09-22T07:17:12Z

docs/advanced-topics/hybrid-models.md

+However, not all applications can be easily converted to FHE computation and the computation cost of FHE may make a full conversion exceed latency requirements.
+


I find the sentence a bit hard to follow. I propose

Suggested change

However, not all applications can be easily converted to FHE computation and the computation cost of FHE may make a full conversion exceed latency requirements.

For certain applications, transitioning to FHE computation may not be straightforward due to inherent complexities. Additionally, the computational overhead of FHE might cause latency issues that surpass acceptable thresholds.

jfrery · 2023-09-22T07:21:15Z

docs/advanced-topics/hybrid-models.md

+Hybrid models are a compromise between on-premise or on-device deployment and full cloud deployment. Hybrid deployment means parts of the model are executed on the client side and parts are executed in FHE on the server side. Concrete ML supports hybrid deployment of neural network models such as MLP, CNN and Large Language-Models.
+


I feel like on-prem and on-device is the same? I propose this:

Suggested change

Hybrid models are a compromise between on-premise or on-device deployment and full cloud deployment. Hybrid deployment means parts of the model are executed on the client side and parts are executed in FHE on the server side. Concrete ML supports hybrid deployment of neural network models such as MLP, CNN and Large Language-Models.

Hybrid models provide an effective balance between on-device deployment and cloud-based deployment. This approach entails executing parts of the model directly on the client side, while other parts are securely processed with FHE (Fully Homomorphic Encryption) on the server side. Concrete-ML facilitates the hybrid deployment of various neural network models, including but not limited to, MLP (Multilayer Perceptron), CNN (Convolutional Neural Network), and Large Language Models.

jfrery · 2023-09-22T07:24:08Z

docs/advanced-topics/hybrid-models.md

+black-box model stealing attacks rely on knowledge distillation
+or on differential methods. As a general rule, the difficulty
+to steal a machine learning model is proportional to the size of the model, in terms of numbers of parameters and model depth.
+{% endhint %}


I am not sure what users should do with that information. Should we point them to the litterature maybe?

I wanted a "disclaimer" - simply making a model hybrid doesn't fix all problems.

I would maybe link to some literature on KD attacks indeed !

Maybe worth creating an issue and properly documenting this in a second time?

docs/advanced-topics/hybrid-models.md

jfrery · 2023-09-22T07:27:15Z

docs/advanced-topics/hybrid-models.md

+The [`save_and_clear_private_info`](../developer-guide/api/concrete.ml.torch.hybrid_model.md#method-save_and_clear_private_info) function serializes the FHE circuits
+corresponding to the various parts of the model that were chosen to be moved
+server-side. Furthermore it saves all necessary information required
+to serve these sub-models with FHE, using the [`FHEModelDev`](../developer-guide/api/concrete.ml.deployment.fhe_client_server.md#class-fhemodeldev) class.


Also something important is that the save_and_clear_private_info removed all the IP related weights.

ok, I'll add that

jfrery · 2023-09-22T07:28:41Z

docs/built-in-models/neural-networks.md

@@ -56,13 +48,14 @@ The figure above right shows the Concrete ML neural network, trained with Quanti
 ### Architecture parameters

 - `module__n_layers`: number of layers in the FCNN, must be at least 1. Note that this is the total number of layers. For a single, hidden layer NN model, set `module__n_layers=2`
- `module__activation_function`: can be one of the Torch activations (e.g., nn.ReLU, see the full list [here](../deep-learning/torch_support.md#activations))
+- `module__activation_function`: can be one of the Torch activations (e.g., nn.ReLU, see the full list [here](../deep-learning/torch_support.md#activations)). Neural networks with `nn.ReLU` activation benefit from specific optimizations that make them around 10x faster than networks with other activation functions.


docs/built-in-models/neural-networks.md

jfrery · 2023-09-22T07:30:16Z

docs/getting-started/showcase.md

+         <td><a href="../../use_case_examples/cifar/cifar_brevitas_with_model_splitting">use_case_examples/cifar_brevitas_with_model_splitting</a></td>
         <!--- end -->


Hmm weird I should have done that I guess my bad.

jfrery · 2023-09-22T07:31:10Z

README.md


- [FHE neural network splitting for client/server deployment](use_case_examples/cifar_brevitas_with_model_splitting): we explain how to split a computationally-intensive neural network model in two parts. First, we execute the first part on the client side in the clear, and the output of this step is encrypted. Next, to complete the computation, the second part of the model is evaluated with FHE. This tutorial also shows the impact of FHE speed/accuracy trade-off on CIFAR10, limiting PBS to 8-bit, and thus achieving 62% accuracy.
+- [FHE neural network splitting for client/server deployment](use_case_examples/cifar/cifar_brevitas_with_model_splitting): we explain how to split a computationally-intensive neural network model in two parts. First, we execute the first part on the client side in the clear, and the output of this step is encrypted. Next, to complete the computation, the second part of the model is evaluated with FHE. This tutorial also shows the impact of FHE speed/accuracy trade-off on CIFAR10, limiting PBS to 8-bit, and thus achieving 62% accuracy.


…st neighbor classification Closes zama-ai/concrete-ml-internal#3975 Closes zama-ai/concrete-ml-internal#3976 Closes zama-ai/concrete-ml-internal#3977

fd0r · 2023-09-22T08:22:09Z

Needs to be rebased

RomanBredehoft · 2023-09-22T12:18:47Z

docs/advanced-topics/hybrid-models.md

+)
+
+
+models_dir = Path(os.path.abspath('')) / "compiled_models"


Path(os.path.abspath('')) is identical to Path('.').resolve() right ? (avoid importing os)

I like os.path :)

fd0r

Looks good to me!

Still some open comments though

andrei-stoian-zama requested a review from a team as a code owner September 20, 2023 12:33

cla-bot bot added the cla-signed label Sep 20, 2023

kcelia reviewed Sep 21, 2023

View reviewed changes

RomanBredehoft reviewed Sep 21, 2023

View reviewed changes

docs/built-in-models/nearest-neighbors.md Outdated Show resolved Hide resolved

RomanBredehoft reviewed Sep 21, 2023

View reviewed changes

docs/built-in-models/nearest-neighbors.md Outdated Show resolved Hide resolved

RomanBredehoft reviewed Sep 21, 2023

View reviewed changes

docs/built-in-models/neural-networks.md Outdated Show resolved Hide resolved

RomanBredehoft reviewed Sep 21, 2023

View reviewed changes

docs/built-in-models/neural-networks.md Show resolved Hide resolved

RomanBredehoft reviewed Sep 21, 2023

View reviewed changes

docs/built-in-models/neural-networks.md Outdated Show resolved Hide resolved

RomanBredehoft previously approved these changes Sep 21, 2023

View reviewed changes

kcelia reviewed Sep 21, 2023

View reviewed changes

docs/built-in-models/neural-networks.md Show resolved Hide resolved

kcelia reviewed Sep 21, 2023

View reviewed changes

docs/built-in-models/nearest-neighbors.md Show resolved Hide resolved

kcelia reviewed Sep 21, 2023

View reviewed changes

andrei-stoian-zama dismissed RomanBredehoft’s stale review via dbeed7e September 21, 2023 20:50

andrei-stoian-zama force-pushed the docs/add_new_features_3975 branch from 78dba60 to dbeed7e Compare September 21, 2023 20:50

jfrery reviewed Sep 22, 2023

View reviewed changes

andrei-stoian-zama added 3 commits September 22, 2023 10:03

docs: detail hybrid deployment, optimization of built-in NNs, K-neare…

f8db02f

…st neighbor classification Closes zama-ai/concrete-ml-internal#3975 Closes zama-ai/concrete-ml-internal#3976 Closes zama-ai/concrete-ml-internal#3977

docs: hybrid models code working

4194dff

chore: fix links

ae4c9e8

andrei-stoian-zama added 2 commits September 22, 2023 10:41

chore: fix format

39f2cf0

chore: address the reviews

0859e99

andrei-stoian-zama force-pushed the docs/add_new_features_3975 branch from dbeed7e to 0859e99 Compare September 22, 2023 12:11

RomanBredehoft reviewed Sep 22, 2023

View reviewed changes

andrei-stoian-zama requested review from jfrery, kcelia and RomanBredehoft September 22, 2023 12:57

fd0r approved these changes Sep 25, 2023

View reviewed changes

andrei-stoian-zama merged commit 68a0b4c into main Sep 25, 2023

andrei-stoian-zama deleted the docs/add_new_features_3975 branch September 25, 2023 09:04

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

docs: document KNN, PoT, Hybrid models #260

docs: document KNN, PoT, Hybrid models #260

andrei-stoian-zama commented Sep 20, 2023

kcelia Sep 21, 2023

RomanBredehoft left a comment

kcelia Sep 21, 2023

kcelia Sep 21, 2023

kcelia Sep 21, 2023

jfrery left a comment

jfrery Sep 22, 2023

jfrery Sep 22, 2023

RomanBredehoft Sep 22, 2023

jfrery Sep 22, 2023

jfrery Sep 22, 2023

andrei-stoian-zama Sep 22, 2023

RomanBredehoft Sep 22, 2023

fd0r Sep 25, 2023

jfrery Sep 22, 2023

andrei-stoian-zama Sep 22, 2023

jfrery Sep 22, 2023

jfrery Sep 22, 2023

jfrery Sep 22, 2023

fd0r commented Sep 22, 2023

RomanBredehoft Sep 22, 2023 •

edited

Loading

andrei-stoian-zama Sep 25, 2023

fd0r left a comment •

edited

Loading


		The FHE inference latency of this model is heavily influenced by the `n_bits`, the dimensionality of the data. Furthermore, the size of the dataset has a linear impact on the complexity of the data and the number of nearest neighbors, `n_neighbors`, also plays a role.

		The KNN computation executes in FHE in $$O(Nlog^2k)$$ steps, where $$N$$ is the training dataset size and $$k$$ is `n_neighbors`. Each step requires several PBS of the precision required to represent the distances between test vectors and the training dataset.

		An example of such implementation is available in [evaluate_torch_cml.py](../../use_case_examples/cifar/cifar_brevitas_training/evaluate_one_example_fhe.py) and [CifarInFheWithSmallerAccumulators.ipynb](../../use_case_examples/cifar/cifar_brevitas_finetuning/CifarInFheWithSmallerAccumulators.ipynb)

		However, not all applications can be easily converted to FHE computation and the computation cost of FHE may make a full conversion exceed latency requirements.

	However, not all applications can be easily converted to FHE computation and the computation cost of FHE may make a full conversion exceed latency requirements.
	For certain applications, transitioning to FHE computation may not be straightforward due to inherent complexities. Additionally, the computational overhead of FHE might cause latency issues that surpass acceptable thresholds.

		Hybrid models are a compromise between on-premise or on-device deployment and full cloud deployment. Hybrid deployment means parts of the model are executed on the client side and parts are executed in FHE on the server side. Concrete ML supports hybrid deployment of neural network models such as MLP, CNN and Large Language-Models.

		<td><a href="../../use_case_examples/cifar/cifar_brevitas_with_model_splitting">use_case_examples/cifar_brevitas_with_model_splitting</a></td>
		<!--- end -->


		- [FHE neural network splitting for client/server deployment](use_case_examples/cifar_brevitas_with_model_splitting): we explain how to split a computationally-intensive neural network model in two parts. First, we execute the first part on the client side in the clear, and the output of this step is encrypted. Next, to complete the computation, the second part of the model is evaluated with FHE. This tutorial also shows the impact of FHE speed/accuracy trade-off on CIFAR10, limiting PBS to 8-bit, and thus achieving 62% accuracy.
		- [FHE neural network splitting for client/server deployment](use_case_examples/cifar/cifar_brevitas_with_model_splitting): we explain how to split a computationally-intensive neural network model in two parts. First, we execute the first part on the client side in the clear, and the output of this step is encrypted. Next, to complete the computation, the second part of the model is evaluated with FHE. This tutorial also shows the impact of FHE speed/accuracy trade-off on CIFAR10, limiting PBS to 8-bit, and thus achieving 62% accuracy.

docs: document KNN, PoT, Hybrid models #260

docs: document KNN, PoT, Hybrid models #260

Conversation

andrei-stoian-zama commented Sep 20, 2023

Choose a reason for hiding this comment

RomanBredehoft left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

jfrery left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fd0r commented Sep 22, 2023

RomanBredehoft Sep 22, 2023 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

fd0r left a comment • edited Loading

Choose a reason for hiding this comment

RomanBredehoft Sep 22, 2023 •

edited

Loading

fd0r left a comment •

edited

Loading