Skip to content

Commit

Permalink
reviews (to be squashed)
Browse files Browse the repository at this point in the history
  • Loading branch information
bcm-at-zama committed Apr 8, 2024
1 parent 9e756a1 commit b16828a
Showing 1 changed file with 1 addition and 1 deletion.
2 changes: 1 addition & 1 deletion concrete-ml-inference-on-endpoints-fhe.md
Original file line number Diff line number Diff line change
Expand Up @@ -147,7 +147,7 @@ Of course, this is just an example of the entrypoint's usage. Developers are enc
### Under the hood
Please note that all of this is done thanks to the flexibility of [custom handlers](https://huggingface.co/docs/inference-endpoints/en/guides/custom_handler) and we express our gratitude to the Hugging Face developers for offering such flexibility. The mechanism is defined in `handler.py`. As explained in the Hugging Face documentation, you can define the `__call__` method of `EndpointHandler` pretty much as you want: In our case, we have defined a `method` parameter, which can be `save_key` (to save FHE evaluation keys), `append_key` (to save FHE evaluation keys piece by piece if the key is too large to be sent in one single call) and finally `inference` (to run FHE inferences). These methods are used to set the evaluation key once, and then run all the inferences, one by one, as seen in `play_with_endpoint.py`.
Please note that all of this is done thanks to the flexibility of [custom handlers](https://huggingface.co/docs/inference-endpoints/en/guides/custom_handler), and we express our gratitude to the Hugging Face developers for offering such flexibility. The mechanism is defined in `handler.py`. As explained in the Hugging Face documentation, you can define the `__call__` method of `EndpointHandler` pretty much as you want: In our case, we have defined a `method` parameter, which can be `save_key` (to save FHE evaluation keys), `append_key` (to save FHE evaluation keys piece by piece if the key is too large to be sent in one single call) and finally `inference` (to run FHE inferences). These methods are used to set the evaluation key once and then run all the inferences, one by one, as seen in `play_with_endpoint.py`.
### Limits
Expand Down

0 comments on commit b16828a

Please sign in to comment.