diff --git a/docs/.gitbook/assets/gitbook_banner.png b/docs/.gitbook/assets/gitbook_banner.png
new file mode 100644
index 000000000..0222c542d
Binary files /dev/null and b/docs/.gitbook/assets/gitbook_banner.png differ
diff --git a/docs/README.md b/docs/README.md
index 7ac3b4b22..8cd4e6607 100644
--- a/docs/README.md
+++ b/docs/README.md
@@ -6,7 +6,7 @@ description: An Open-source ecosystem for Validity and ZK ML.
Orion is an open-source, community-driven framework dedicated to Provable Machine Learning. It provides essential components and a new ONNX runtime for building verifiable Machine Learning models using [STARKs](https://starkware.co/stark/).
-
+
### 🤔 What is ONNX Runtime?
diff --git a/docs/SUMMARY.md b/docs/SUMMARY.md
index 9a781c038..a31015203 100644
--- a/docs/SUMMARY.md
+++ b/docs/SUMMARY.md
@@ -21,7 +21,6 @@
* [fp.new](framework/numbers/fixed-point/fp.new.md)
* [fp.new\_unscaled](framework/numbers/fixed-point/fp.new\_unscaled.md)
* [fp.from\_felt](framework/numbers/fixed-point/fp.from\_felt.md)
- * [fp.from\_unscaled\_felt](framework/numbers/fixed-point/fp.from\_unscaled\_felt.md)
* [fp.abs](framework/numbers/fixed-point/fp.abs.md)
* [fp.ceil](framework/numbers/fixed-point/fp.ceil.md)
* [fp.floor](framework/numbers/fixed-point/fp.floor.md)
diff --git a/docs/academy/tutorials/implement-new-operators-in-orion.md b/docs/academy/tutorials/implement-new-operators-in-orion.md
index 1f6e53ab5..9fa351cb1 100644
--- a/docs/academy/tutorials/implement-new-operators-in-orion.md
+++ b/docs/academy/tutorials/implement-new-operators-in-orion.md
@@ -12,7 +12,7 @@ Throughout this tutorial, any concept that is directly explained in the official
Orion repo uses Scarb, a Cairo package manager. You can find all information about Scarb and Cairo installation [here](../../framework/get-started.md#installations).
-The repository is structured as follows:
+The repository is structured as follows:
```
.
@@ -30,19 +30,16 @@ The repository is structured as follows:
│ ├── numbers.cairo
│ ├── operators
│ ├── operators.cairo
-│ ├── performance
-│ ├── performance.cairo
│ ├── tests
│ ├── tests.cairo
│ └── utils.cairo
└── target
```
-In the `src` directory, you'll find four distinct folders:
+In the `src` directory, you'll find four distinct folders:
-* [`numbers`](../../framework/numbers/): This folder contains a complete implementation of Signed Integer and Fixed Point.
-* [`operators`](../../framework/operators/): This directory includes a set of functions and operations used in calculating neural network models.
-* [`performance`](../../framework/performance/): Here, you'll find a set of functions designed to enhance the performance of models.
+* [`numbers`](../../framework/numbers/): This folder contains a complete implementation of Signed Integer and Fixed Point.
+* [`operators`](../../framework/operators/): This directory includes a set of functions and operations used in calculating neural network models.
* `tests`: This is the location where we'll test our code.
In this tutorial we will focus on `operators` directory, as we will implement a new operator from scratch.
@@ -55,9 +52,9 @@ Orion operators represent specific computations or operations performed by machi
Ensuring compatibility with ONNX operators facilitates integration into the ONNX ecosystem. This enables researchers and developers to pre-train models using their preferred framework, before executing verifiable inferences with Orion.
-We implemented two different types of operators, each having their own trait:
+We implemented two different types of operators, each having their own trait:
-* [`tensor (TensorTrait)`](../../framework/operators/tensor/): This represents a full implementation of multi-dimensional arrays.
+* [`tensor (TensorTrait)`](../../framework/operators/tensor/): This represents a full implementation of multi-dimensional arrays.
* [`nn (NNTrait)`](../../framework/operators/neural-network/) - These are operators designed for building neural networks.
{% hint style="info" %}
@@ -110,43 +107,9 @@ trait NNTrait {
#### Step 2: Add the business logic
-In the `src/operators/nn/functional` directory, create a new file named `softmax.cairo` and include the following code:
+In the `src/operators/nn/functional` directory, create a new file named `softmax.cairo`.
-```rust
-//In softmax.cairo
-mod softmax_u32;
-mod softmax_i32;
-```
-
-Subsequently, create a `softmax` directory containing the following files: `softmax_u32.cairo` and `softmax_i32.cairo`.
-
-The resulting directory structure should look as follows:
-
-```
-nn
-├── core.cairo
-├── functional
-│ ├── softmax
-│ │ ├── softmax_i32.cairo
-│ │ └── softmax_u32.cairo
-│ ├── softmax.cairo
-│ [...]
-├── functional.cairo
-├── implementations
-│ ├── impl_nn_i32.cairo
-│ └── impl_nn_u32.cairo
-└── implementations.cairo
-```
-
-The `functional` folder is where all the business logic resides. The two lines in `softmax.cairo` instruct the compiler to include `softmax_u32.cairo` and `softmax_i32.cairo`.
-
-As you can see, Orion currently supports two implementations for `NNTrait`: `i32` and `u32`. Therefore, we need to develop logic for both implementations.
-
-{% hint style="warning" %}
-`TensorTrait` supports more types, you will therefore need to add the necessary logic to all supported types if you decide to implement a new tensor operator.
-{% endhint %}
-
-Now that the files are set up, let's proceed to code the business logic.
+The `functional` folder is where all the business logic resides. All functions should be implemented with generic type.
A softmax function can be implemented as follows:
@@ -154,41 +117,22 @@ A softmax function can be implemented as follows:
So we can leverage the [`exp`](../../framework/operators/tensor/tensor.exp.md) and [`reduce_sum`](../../framework/operators/tensor/tensor.reduce\_sum.md) operators from `TensorTrait` to implement softmax.
-Here's the implementation in `softmax_i32.cairo`:
+Here's the implementation in `softmax.cairo`:
```rust
-//In softmax_i32.cairo
-
-use orion::numbers::signed_integer::i32::i32;
-use orion::numbers::fixed_point::core::FixedType;
use orion::operators::tensor::core::{Tensor, TensorTrait};
-use orion::operators::tensor::implementations::{
- impl_tensor_i32::Tensor_i32, impl_tensor_fp::{Tensor_fp, FixedTypeTensorDiv}
-};
/// Cf: NNTrait::softmax docstring
-fn softmax_i32(z: @Tensor, axis: usize) -> Tensor {
- let exp_tensor = z.exp();
- let sum = exp_tensor.reduce_sum(axis, true);
- let softmax = exp_tensor / sum;
-
- return softmax;
-}
-```
-
-Similarly, implement the logic in `softmax_u32.cairo`:
-
-```rust
-//In softmax_u32.cairo
-
-use orion::numbers::fixed_point::core::FixedType;
-use orion::operators::tensor::core::{Tensor, TensorTrait};
-use orion::operators::tensor::implementations::{
- impl_tensor_u32::Tensor_u32, impl_tensor_fp::{Tensor_fp, FixedTypeTensorDiv}
-};
-
-/// Cf: NNTrait::softmax docstring
-fn softmax_u32(z: @Tensor, axis: usize) -> Tensor {
+fn softmax<
+ T,
+ impl TTensor: TensorTrait,
+ impl TTensor: TensorTrait,
+ impl TTensorDiv: Div>,
+ impl TCopy: Copy,
+ impl TDrop: Drop,
+>(
+ z: @Tensor, axis: usize
+) -> Tensor {
let exp_tensor = z.exp();
let sum = exp_tensor.reduce_sum(axis, true);
let softmax = exp_tensor / sum;
@@ -199,39 +143,28 @@ fn softmax_u32(z: @Tensor, axis: usize) -> Tensor {
#### Step 3: Add softmax to the implementations
-Now, we need to add the softmax function into the different representations. In `nn/implementations/impl_nn_i32.cairo`, import the business logic and add the softmax implementation.
+Now, we need to add the softmax function into the different representations. In `nn/implementations/nn_fp8x23.cairo`, import the business logic and add the softmax implementation.
```rust
-// In impl_nn_i32.cairo
-use core::option::OptionTrait;
-use orion::operators::tensor::core::Tensor;
-use orion::numbers::signed_integer::i32::i32;
-use orion::operators::nn::core::NNTrait;
-use orion::numbers::fixed_point::core::FixedType;
-use orion::operators::nn::functional::softmax::softmax_i32::softmax_i32;
-
-impl NN_i32 of NNTrait {
+// In nn_fp8x23.cairo
+
+impl FP8x23NN of NNTrait {
// [...]
- fn softmax(tensor: @Tensor, axis: usize) -> Tensor {
- softmax_i32(tensor, axis)
+ fn softmax(tensor: @Tensor, axis: usize) -> Tensor {
+ functional::softmax::softmax(tensor, axis)
}
}
```
-Do the same for `u32` implementation.
+Do the same for all other fixed point implementations (`FP16x16NN`, `FP32x32NN`, `FP64x64NN`). As softmax only support fixed point tensors, it should panic for other implementations. Below an example with `U32NN`.
```rust
-// In impl_nn_u32.cairo
-use core::option::OptionTrait;
-use orion::numbers::fixed_point::core::FixedType;
-use orion::operators::tensor::core::Tensor;
-use orion::operators::nn::core::NNTrait;
-use orion::operators::nn::functional::softmax::softmax_u32::softmax_u32;
-
-impl NN_u32 of NNTrait {
+// In nn_u32.cairo
+
+impl U32NN of NNTrait {
// [...]
- fn softmax(tensor: @Tensor, axis: usize) -> Tensor {
- softmax_u32(tensor, axis)
+ fn softmax(tensor: @Tensor, axis: usize) -> Tensor {
+ panic(array!['not supported!'])
}
}
```
@@ -250,7 +183,7 @@ trait NNTrait {
/// # NNTrait::softmax
///
/// ```rust
- /// fn softmax(tensor: @Tensor, axis: usize) -> Tensor;
+ /// fn softmax(tensor: @Tensor, axis: usize) -> Tensor;
/// ```
///
/// Applies the Softmax function to an n-dimensional input Tensor rescaling them so that the elements of the n-dimensional output Tensor lie in the range \[0,1] and sum to 1.
@@ -268,18 +201,31 @@ trait NNTrait {
///
/// A Tensor of fixed point numbers with the same shape than the input Tensor.
///
+ /// ## Type Constraints
+ ///
+ /// Constrain input and output types to fixed point tensors.
+ ///
/// ## Examples
///
/// ```rust
- /// use orion::operators::nn::core::NNTrait;
- /// use orion::operators::nn::implementations::impl_nn_u32::NN_u32;
+ /// use array::{ArrayTrait, SpanTrait};
+ ///
+ /// use orion::operators::tensor::{TensorTrait, Tensor, FP8x23};
+ /// use orion::operators::nn::{NNTrait, FP8x23NN};
+ /// use orion::numbers::{FP8x23, FixedTrait};
+ ///
+ /// fn softmax_example() -> Tensor {
+ /// let tensor = TensorTrait::::new(
+ /// shape: array![2, 2].span(),
+ /// data: array![
+ /// NNTrait::new(0, false),
+ /// NNTrait::new(1, false),
+ /// NNTrait::new(2, false),
+ /// NNTrait::new(3, false),
+ /// ]
+ /// .span(),
+ /// );
///
- /// fn softmax_example() -> Tensor {
- /// // We instantiate a 2D Tensor here.
- /// // [[0,1],[2,3]]
- /// let tensor = u32_tensor_2x2_helper();
- ///
- /// // We can call `softmax` function as follows.
/// return NNTrait::softmax(@tensor, 1);
/// }
/// >>> [[2255697,6132911],[2255697,6132911]]
@@ -287,15 +233,15 @@ trait NNTrait {
/// // [[0.2689, 0.7311],[0.2689, 0.7311]]
/// ```
///
- fn softmax(tensor: @Tensor, axis: usize) -> Tensor;
+ fn softmax(tensor: @Tensor, axis: usize) -> Tensor;
}
````
Voilà! We have successfully implemented the softmax function in `NNTrait`!
-### How to test the Orion Operator?
+### How to test the Orion Operator?
-Now, let's proceed to testing the softmax operator we've just implemented. When testing an operator in Orion, you should ensure to test across all types of implementation.
+Now, let's proceed to testing the softmax operator we've just implemented. When testing an operator in Orion, you should ensure to test across all types of implementation.
Since softmax employs fixed points for intermediate calculations and returns a tensor of `FixedType`, it is essential to test it across all fixed point implementations. As of now, Orion supports two fixed point implementations: [`FP16x16`](../../framework/numbers/fixed-point/#data-types) and [`FP8x23`](../../framework/numbers/fixed-point/#data-types).
@@ -317,7 +263,7 @@ def softmax(x: np.ndarray, axis: int = -1) -> np.ndarray:
return tmp / s
```
-Finally, we create a Softmax class, containing tests for each dtypes.
+Finally, we create a Softmax class, containing tests for each dtypes.
```python
import numpy as np
@@ -331,136 +277,68 @@ def softmax(x: np.ndarray, axis: int = -1) -> np.ndarray:
return tmp / s
class Softmax(RunAll):
-
- # Define tests for i32 dtype
+
@staticmethod
- def softmax_i32():
- # Softmax returns a FixedType.
- # So we test here with fp8x23 implementation.
- def fp8x23():
- # Create a random numpy array:
- x = np.random.randint(-3, 3, (2, 2)).astype(np.int32)
- # Ddefine the expected result:
- y = softmax(x, 0)
-
- # Convert the input and output to Tensor class, similar to Orion's Tensor struct:
- x = Tensor(Dtype.I32, x.shape, x.flatten())
- # Convert the floats values in `y` to fixed points with `to_fp` method:
- y = Tensor(Dtype.FP8x23, y.shape, to_fp(
- y.flatten(), FixedImpl.FP8x23))
-
- # Define the name of the generated folder.
- name = "softmax_i32_fp8x23"
- # Invoke `make_node` method to generate Cairo representation of `x` and `y`:
- make_node([x], [y], name)
- # Invoke `make_test` method to generate corresponding Cairo tests:
- make_test(
- [x], # List of input tensors.
- y, # The expected output result.
- "NNTrait::softmax(@input_0, 0)", # The code signature.
- name, # The name of the generated folder.
- Trait.NN # The trait, if the function is present in either the TensorTrait or NNTrait.
- )
+ # We test here with fp8x23 implementation.
+ def fp8x23():
+ # Create a random numpy array:
+ x = np.random.randint(-3, 3, (2, 2)).astype(np.float64)
+ # Ddefine the expected result:
+ y = softmax(x, 0)
- # Test here with fp16x16 implementation.
- def fp16x16():
- x = np.random.randint(-3, 3, (2, 2)).astype(np.int32)
- y = softmax(x, 1)
-
- x = Tensor(Dtype.I32, x.shape, x.flatten())
- y = Tensor(Dtype.FP16x16, y.shape, to_fp(
- y.flatten(), FixedImpl.FP16x16))
-
- name = "softmax_i32_fp16x16"
- make_node([x], [y], name)
- make_test([x], y, "NNTrait::softmax(@input_0, 1)",
- name, Trait.NN)
-
- fp8x23()
- fp16x16()
-
- @staticmethod
- def softmax_i8():
- def fp8x23():
- x = np.random.randint(-3, 3, (2, 2)).astype(np.int8)
- y = softmax(x, 1)
-
- x = Tensor(Dtype.I8, x.shape, x.flatten())
- y = Tensor(Dtype.FP8x23, y.shape, to_fp(
- y.flatten(), FixedImpl.FP8x23))
-
- name = "softmax_i8_fp8x23"
- make_node([x], [y], name)
- make_test([x], y, "NNTrait::softmax(@input_0, 1)",
- name, Trait.NN)
-
- def fp16x16():
- x = np.random.randint(-3, 3, (2, 2)).astype(np.int8)
- y = softmax(x, 0)
-
- x = Tensor(Dtype.I8, x.shape, x.flatten())
- y = Tensor(Dtype.FP16x16, y.shape, to_fp(
- y.flatten(), FixedImpl.FP16x16))
-
- name = "softmax_i8_fp16x16"
- make_node([x], [y], name)
- make_test([x], y, "NNTrait::softmax(@input_0, 0)",
- name, Trait.NN)
-
- fp8x23()
- fp16x16()
-
+ # Convert the input and output to Tensor class, similar to Orion's Tensor struct:
+ x = Tensor(Dtype.FP8x23, x.shape, to_fp(x.flatten(), FixedImpl.FP8x23))
+ # Convert the floats values in `y` to fixed points with `to_fp` method:
+ y = Tensor(Dtype.FP8x23, y.shape, to_fp(y.flatten(), FixedImpl.FP8x23))
+
+ # Define the name of the generated folder.
+ name = "softmax_fp8x23"
+ # Invoke `make_node` method to generate Cairo representation of `x` and `y`:
+ make_node([x], [y], name)
+ # Invoke `make_test` method to generate corresponding Cairo tests:
+ make_test(
+ [x], # List of input tensors.
+ y, # The expected output result.
+ "NNTrait::softmax(@input_0, 0)", # The code signature.
+ name, # The name of the generated folder.
+ Trait.NN # The trait, if the function is present in either the TensorTrait or NNTrait.
+ )
+
+ # We test here with fp16x16 implementation.
@staticmethod
- def softmax_u32():
- def fp8x23():
- x = np.random.randint(0, 3, (2, 2)).astype(np.int32)
- y = softmax(x, 1)
-
- x = Tensor(Dtype.U32, x.shape, x.flatten())
- y = Tensor(Dtype.FP8x23, y.shape, to_fp(
- y.flatten(), FixedImpl.FP8x23))
-
- name = "softmax_u32_fp8x23"
- make_node([x], [y], name)
- make_test([x], y, "NNTrait::softmax(@input_0, 1)",
- name, Trait.NN)
-
- def fp16x16():
- x = np.random.randint(0, 3, (2, 2)).astype(np.int32)
- y = softmax(x, 0)
-
- x = Tensor(Dtype.U32, x.shape, x.flatten())
- y = Tensor(Dtype.FP16x16, y.shape, to_fp(
- y.flatten(), FixedImpl.FP16x16))
-
- name = "softmax_u32_fp16x16"
- make_node([x], [y], name)
- make_test([x], y, "NNTrait::softmax(@input_0, 0)",
- name, Trait.NN)
-
- fp8x23()
- fp16x16()
+ def fp16x16():
+ x = np.random.uniform(-3, 3, (2, 2)).astype(np.float64)
+ y = softmax(x, 1)
+
+ x = Tensor(Dtype.FP16x16, x.shape, to_fp(
+ x.flatten(), FixedImpl.FP16x16))
+ y = Tensor(Dtype.FP16x16, y.shape, to_fp(
+ y.flatten(), FixedImpl.FP16x16))
+
+ name = "softmax_fp16x16"
+ make_node([x], [y], name)
+ make_test([x], y, "NNTrait::softmax(@input_0, 1)",
+ name, Trait.NN)
```
Once set up, you can generate tests and data by executing `scarb run nodegen softmax`.
-The above code will generate 6 test files located in `src/tests/nodes`. As an example, here's the content of the `softmax_i32_fp8x23.cairo` generated file:
+The above code will generate 6 test files located in `src/tests/nodes`. As an example, here's the content of the `softmax_fp8x23.cairo` generated file:
```rust
-// softmax_i32_fp8x23.cairo
+// softmax_fp8x23.cairo
mod input_0;
mod output_0;
-use orion::operators::nn::core::NNTrait;
-use orion::numbers::fixed_point::core::FixedTrait;
-use orion::operators::nn::implementations::impl_nn_i32::NN_i32;
-use orion::numbers::fixed_point::implementations::fp8x23::core::FP8x23Impl;
-use orion::operators::tensor::implementations::impl_tensor_fp::FP8x23Tensor::FPTensorPartialEq;
+use orion::operators::nn::NNTrait;
+use orion::numbers::FixedTrait;
+use orion::operators::nn::FP8x23NN;
+use orion::operators::tensor::FP8x23TensorPartialEq;
use orion::utils::assert_eq;
#[test]
#[available_gas(2000000000)]
-fn test_softmax_i32_fp8x23() {
+fn test_softmax_fp8x23() {
let input_0 = input_0::input_0();
let z = output_0::output_0();
@@ -472,8 +350,6 @@ fn test_softmax_i32_fp8x23() {
If you'd like to expand the tests with additional cases, feel free to edit the generated Cairo file.
-
-
You're now ready to prepare your Pull Request. Please ensure you thoroughly read the [Contribution Guidelines](../../framework/contribute.md) before making your first PR. Your contribution is greatly appreciated, and we sincerely value your interest 🫶.
Orion leverages Cairo to guarantee the reliability of inference, providing developers with a user-friendly framework to build complex and verifiable machine learning models. We invite the community to join us in shaping a future where trustworthy AI becomes a reliable resource for all.
diff --git a/docs/academy/tutorials/mnist-classification-with-orion.md b/docs/academy/tutorials/mnist-classification-with-orion.md
index 143051e97..bb85956d5 100644
--- a/docs/academy/tutorials/mnist-classification-with-orion.md
+++ b/docs/academy/tutorials/mnist-classification-with-orion.md
@@ -2,7 +2,7 @@
-Orion is a dedicated Cairo-based library designed specifically to build machine learning models for ValidityML. Its purpose is to facilitate verifiable inference. Orion exclusively operates with 8-bit quantized models, an approach intended to optimize performance. In this tutorial, you will be guided on how to train your model using Quantized Aware Training using MNIST dataset, how to convert your pre-trained model to Cairo 1, and how to perform inference with Orion.
+Orion is a dedicated Cairo-based library designed specifically to build machine learning models for ValidityML. Its purpose is to facilitate verifiable inference. For better performance we will operate with an 8-bit quantized model. In this tutorial, you will be guided on how to train your model using Quantized Aware Training using MNIST dataset, how to convert your pre-trained model to Cairo 1, and how to perform inference with Orion.
{% hint style="info" %}
You can find all the code and the notebook in the dedicated [repository.](https://github.com/gizatechxyz/orion\_tutorials)
@@ -385,30 +385,28 @@ tensors = {
Now let's generate Cairo files for each tensor in the object.
-
# Create the directory if it doesn't exist
+```python
+# Create the directory if it doesn't exist
os.makedirs('src/generated', exist_ok=True)
for tensor_name, tensor in tensors.items():
with open(os.path.join('src', 'generated', f"{tensor_name}.cairo"), "w") as f:
f.write(
- "use array::ArrayTrait;\n" +
- "use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};\n" +
- "use orion::operators::tensor::implementations::impl_tensor_i32::Tensor_i32;\n" +
- "use orion::numbers::fixed_point::core::FixedImpl;\n" +
- "use orion::numbers::signed_integer::i32::i32;\n\n" +
- "fn {0}() -> Tensor<i32> ".format(tensor_name) + "{\n" +
- " let mut shape = ArrayTrait::<usize>::new();\n"
+ "use array::ArrayTrait;\n" +
+ "use orion::operators::tensor::{TensorTrait, Tensor, I32Tensor};\n" +
+ "use orion::numbers::i32;\n\n" +
+ "\nfn {0}() -> Tensor ".format(tensor_name) + "{\n" +
+ " let mut shape = ArrayTrait::::new();\n"
)
for dim in tensor.shape:
f.write(" shape.append({0});\n".format(dim))
f.write(
- " let mut data = ArrayTrait::<i32>::new();\n"
+ " let mut data = ArrayTrait::::new();\n"
)
for val in np.nditer(tensor.flatten()):
- f.write(" data.append(i32 {{ mag: {0}, sign: {1} }});\n".format(abs(int(val)), str(val < 0).lower()))
+ f.write(" data.append(i32 {{ mag: {0}, sign: {1} }});\n".format(abs(int(val)), str(val < 0).lower()))
f.write(
- "let extra = ExtraParams { fixed_point: Option::Some(FixedImpl::FP16x16(())) }; \n" +
- " TensorTrait::new(shape.span(), data.span(), Option::Some(extra))\n" +
+ " TensorTrait::new(shape.span(), data.span())\n" +
"}\n"
)
@@ -416,7 +414,7 @@ with open(os.path.join('src', 'generated.cairo'), 'w') as f:
for param_name in tensors.keys():
f.write(f"mod {param_name};\n")
-
+```
Your Cairo files are generated in `src/generated` directory.
@@ -434,27 +432,25 @@ Here is a file we generated: `fc1_bias.cairo`
```rust
use array::ArrayTrait;
-use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};
-use orion::operators::tensor::implementations::impl_tensor_i32::Tensor_i32;
-use orion::numbers::fixed_point::core::FixedImpl;
-use orion::numbers::signed_integer::i32::i32;
+use orion::operators::tensor::{TensorTrait, Tensor, I32Tensor};
+use orion::numbers::i32;
+
fn fc1_bias() -> Tensor {
let mut shape = ArrayTrait::::new();
shape.append(10);
let mut data = ArrayTrait::::new();
- data.append(i32 { mag: 1300, sign: true });
- data.append(i32 { mag: 7644, sign: false });
- data.append(i32 { mag: 472, sign: true });
- data.append(i32 { mag: 3601, sign: false });
- data.append(i32 { mag: 5538, sign: false });
- data.append(i32 { mag: 5476, sign: false });
- data.append(i32 { mag: 3879, sign: false });
- data.append(i32 { mag: 3268, sign: true });
- data.append(i32 { mag: 1979, sign: false });
- data.append(i32 { mag: 1435, sign: false });
-let extra = ExtraParams { fixed_point: Option::Some(FixedImpl::FP16x16(())) };
- TensorTrait::new(shape.span(), data.span(), Option::Some(extra))
+ data.append(i32 { mag: 1287, sign: false });
+ data.append(i32 { mag: 3667, sign: true });
+ data.append(i32 { mag: 2954, sign: false });
+ data.append(i32 { mag: 7938, sign: false });
+ data.append(i32 { mag: 3959, sign: false });
+ data.append(i32 { mag: 5862, sign: true });
+ data.append(i32 { mag: 4886, sign: false });
+ data.append(i32 { mag: 4992, sign: false });
+ data.append(i32 { mag: 10126, sign: false });
+ data.append(i32 { mag: 2237, sign: true });
+ TensorTrait::new(shape.span(), data.span())
}
```
@@ -482,15 +478,6 @@ The second concept Orion introduced is the [Tensor](../../framework/operators/te
struct Tensor {
shape: Span,
data: Span
- extra: Option
-}
-```
-
-`ExtraParams` is a struct containing additional parameters for the tensor. The `fixed_point` parameter specifies the fixed-point implementation to be used when a tensor operation uses fixed points.
-
-```rust
-struct ExtraParams {
- fixed_point: Option
}
```
@@ -527,7 +514,8 @@ It should return a `Tensor`.
```rust
use orion::operators::tensor::core::Tensor;
-use orion::numbers::signed_integer::i32::i32;
+use orion::numbers::signed_integer::{integer_trait::IntegerTrait, i32::i32};
+use orion::operators::nn::{NNTrait, I32NN};
fn fc1(i: Tensor, w: Tensor, b: Tensor) -> Tensor {
// ...
@@ -539,35 +527,30 @@ To build the first layer, we need a [Linear](../../framework/operators/neural-ne
```rust
use orion::operators::tensor::core::Tensor;
use orion::numbers::signed_integer::{integer_trait::IntegerTrait, i32::i32};
-use orion::operators::nn::core::NNTrait;
-use orion::numbers::fixed_point::core::FixedType;
-use orion::operators::nn::implementations::impl_nn_i32::NN_i32;
+use orion::operators::nn::{NNTrait, I32NN};
fn fc1(i: Tensor, w: Tensor, b: Tensor) -> Tensor {
- let x = NNTrait::linear(i, w, b, true); // `true` because we want to quantize the result
- NNTrait::relu(@x, IntegerTrait::new(0, false))
+ let x = NNTrait::linear(i, w, b);
+ NNTrait::relu(@x)
}
```
#### Dense Layer 2
-In a similar way, we can build the second layer `fc2`, which contains a [Linear](../../framework/operators/neural-network/nn.linear.md) function and a [Softmax](../../framework/operators/neural-network/nn.softmax.md) from [NNTrait](../../framework/operators/neural-network/). Because after a softmax the output tensor lies in the interval \[0,1], floating points are represented by [fixed points](../../framework/numbers/fixed-point/). This is why `fc2` returns `Tensor`.
+In a similar way, we can build the second layer `fc2`, which contains a [Linear](../../framework/operators/neural-network/nn.linear.md) function and a [Softmax](../../framework/operators/neural-network/nn.softmax.md) from [NNTrait](../../framework/operators/neural-network/). We could convert the tensor to fixed point in order to perform softmax, but for this simple tutorial it's not necessary.
```rust
use orion::operators::tensor::core::Tensor;
use orion::numbers::signed_integer::{integer_trait::IntegerTrait, i32::i32};
-use orion::operators::nn::core::NNTrait;
-use orion::numbers::fixed_point::core::FixedType;
-use orion::operators::nn::implementations::impl_nn_i32::NN_i32;
+use orion::operators::nn::{NNTrait, I32NN};
fn fc1(i: Tensor, w: Tensor, b: Tensor) -> Tensor {
- let x = NNTrait::linear(i, w, b, true);
- NNTrait::relu(@x, IntegerTrait::new(0, false))
+ let x = NNTrait::linear(i, w, b);
+ NNTrait::relu(@x)
}
-fn fc2(i: Tensor, w: Tensor, b: Tensor) -> Tensor {
- let x = NNTrait::linear(i, w, b, true);
- NNTrait::softmax(@x, 0)
+fn fc2(i: Tensor, w: Tensor, b: Tensor) -> Tensor {
+ NNTrait::linear(i, w, b)
}
```
@@ -638,7 +621,7 @@ fn mnist_nn_test() {
}
```
-Finally, let's make a prediction. The input data represents the digit 7. The probability at index 7 must therefore be close to 1.
+Finally, let's make a prediction. The input data represents the digit 7. So the index 7 should have the highest probability.
```rust
use core::array::SpanTrait;
@@ -651,7 +634,7 @@ use mnist_nn::generated::fc1_weights::fc1_weights;
use mnist_nn::generated::fc2_bias::fc2_bias;
use mnist_nn::generated::fc2_weights::fc2_weights;
-use orion::operators::tensor::implementations::impl_tensor_fp::Tensor_fp;
+use orion::operators::tensor::I32Tensor;
#[test]
#[available_gas(99999999999999999)]
@@ -665,21 +648,13 @@ fn mnist_nn_test() {
let x = fc1(input, fc1_weights, fc1_bias);
let x = fc2(x, fc2_weights, fc2_bias);
- assert(*x.data.at(0).mag == 0, 'proba x is 0 -> 0');
- assert(*x.data.at(1).mag == 0, 'proba x is 1 -> 0');
- assert(*x.data.at(2).mag == 0, 'proba x is 2 -> 0');
- assert(*x.data.at(3).mag == 0, 'proba x is 3 -> 0');
- assert(*x.data.at(4).mag == 0, 'proba x is 4 -> 0');
- assert(*x.data.at(5).mag == 0, 'proba x is 5 -> 0');
- assert(*x.data.at(6).mag == 0, 'proba x is 6 -> 0');
- assert(*x.data.at(7).mag > 62259, 'proba x is 7 -> 1'); // 62259 represents ONE in fp16x16.
- assert(*x.data.at(8).mag == 0, 'proba x is 8 -> 0');
- assert(*x.data.at(9).mag == 0, 'proba x is 9 -> 0');
-}
+ let x = *x.argmax(0, Option::None(()), Option::None(())).data.at(0);
+ assert(x == 7, 'should predict 7');
+}
```
-Test your model by running `scarb cairo-test -f mnist_nn_test`.
+Test your model by running `scarb test`.
```sh
testing mnist_nn ...
@@ -693,5 +668,3 @@ Bravo 👏 You can be proud of yourself! You just built your first Neural Networ
Orion leverages Cairo to guarantee the reliability of inference, providing developers with a user-friendly framework to build complex and verifiable machine learning models. We invite the community to join us in shaping a future where trustworthy AI becomes a reliable resource for all.
-
-[^1]: Maybe for this to make it a bit cleaner we could use triple quotes so we don't have to add "\n" on each line
diff --git a/docs/academy/tutorials/verifiable-linear-regression-model-in-orion.md b/docs/academy/tutorials/verifiable-linear-regression-model-in-orion.md
index e2c475a76..65e610f14 100644
--- a/docs/academy/tutorials/verifiable-linear-regression-model-in-orion.md
+++ b/docs/academy/tutorials/verifiable-linear-regression-model-in-orion.md
@@ -145,7 +145,7 @@ name = "verifiable_linear_regression"
version = "0.1.0"
[dependencies]
-orion = { git = "https://github.com/gizatechxyz/orion.git", branch = "einsum-impl" }
+orion = { git = "https://github.com/gizatechxyz/orion.git"}
[scripts]
test = "scarb cairo-test -f linear_regression_test"
@@ -165,11 +165,9 @@ def generate_cairo_files(data, name):
with open(os.path.join('src', 'generated', f"{name}.cairo"), "w") as f:
f.write(
"use array::ArrayTrait;\n" +
- "use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};\n" +
- "use orion::operators::tensor::implementations::impl_tensor_fp::Tensor_fp;\n" +
- "use orion::numbers::fixed_point::core::{FixedTrait, FixedType, FixedImpl};\n"
- "use orion::numbers::fixed_point::implementations::fp16x16::core::{FP16x16Impl, FP16x16PartialEq };\n"+
- "fn {0}() -> Tensor ".format(name) + "{\n" +
+ "use orion::operators::tensor::{FP16x16Tensor, TensorTrait, Tensor};\n" +
+ "use orion::numbers::{FixedTrait, FP16x16, FP16x16Impl};\n"
+ "\nfn {0}() -> Tensor ".format(name) + "{\n" +
" let mut shape = ArrayTrait::new();\n"
)
for dim in data.shape:
@@ -180,19 +178,13 @@ def generate_cairo_files(data, name):
for val in np.nditer(data.flatten()):
f.write(" data.append(FixedTrait::new({0}, {1} ));\n".format(abs(int(val * 2**16)), str(val < 0).lower()))
f.write(
- "let extra = ExtraParams { fixed_point: Option::Some(FixedImpl::FP16x16(())) }; \n" +
- "let tensor = TensorTrait::::new(shape.span(), data.span(), Option::Some(extra)); \n \n" +
+ "let tensor = TensorTrait::::new(shape.span(), data.span()); \n \n" +
"return tensor;\n\n"+
"}\n"
)
with open(os.path.join('src', 'generated.cairo'), 'w') as f:
for param_name in tensor_name:
f.write(f"mod {param_name};\n")
-
-
-generate_cairo_files(X, 'X_values')
-generate_cairo_files(y, 'Y_values')
-
```
The X\_values and y\_values tensor values will now be generated under `src/generated` directory.
@@ -209,52 +201,41 @@ This will tell our compiler to include the separate modules listed above during
```rust
use array::ArrayTrait;
-use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};
-use orion::operators::tensor::implementations::impl_tensor_i32::Tensor_i32;
-use orion::numbers::signed_integer::i32::i32;
-
-use orion::numbers::fixed_point::core::{FixedTrait, FixedType, FixedImpl};
-use orion::operators::tensor::implementations::impl_tensor_fp::Tensor_fp;
-use orion::numbers::fixed_point::implementations::fp16x16::core::{FP16x16Impl, FP16x16Into, FP16x16PartialEq };
+use orion::operators::tensor::{FP16x16Tensor, TensorTrait, Tensor};
+use orion::numbers::{FixedTrait, FP16x16, FP16x16Impl};
-fn X_values() -> Tensor {
+fn X_values() -> Tensor {
let mut shape = ArrayTrait::new();
- shape.append(150);
+ shape.append(150);
let mut data = ArrayTrait::new();
- data.append(FixedTrait::new_unscaled(10, true ));
- data.append(FixedTrait::new_unscaled(9, true ));
-
-// data has been truncated (only showing the first and last 2 values out of the 150 values)
-
- data.append(FixedTrait::new_unscaled(24, false ));
- data.append(FixedTrait::new_unscaled(25, false ));
-let extra = ExtraParams { fixed_point: Option::Some(FixedImpl::FP16x16(())) };
-let tensor = TensorTrait::::new(shape.span(), data.span(), Option::Some(extra));
-
-return tensor;
-
+ data.append(FixedTrait::new(32768, true));
+ data.append(FixedTrait::new(32328, true));
+ // data has been truncated (only showing the first and last 2 values out of the 150 values)
+ data.append(FixedTrait::new(32328, false));
+ data.append(FixedTrait::new(32768, false));
+ let tensor = TensorTrait::::new(shape.span(), data.span());
+
+ return tensor;
}
-
```
Since Cairo does not come with built-in signed integers we have to explicitly define it for our X and y values. Luckily, this is already implemented in Orion for us as a struct as shown below:
```rust
-// Example of a FixedType.
-struct FixedType {
- mag: u128,
+// Example of a FP16x16.
+struct FP16x16 {
+ mag: u32,
sign: bool
}
```
-For this tutorial, we will use FixedType numbers where the magnitude represents the absolute value and the boolean indicates whether the number is negative or positive. To replicate the OLS functions, we will conduct our operations using FixedType Tensors which are also represented as a structs in Orion.
+For this tutorial, we will use FP16x16 numbers where the magnitude represents the absolute value and the boolean indicates whether the number is negative or positive. To replicate the OLS functions, we will conduct our operations using FP16x16 Tensors which are also represented as a structs in Orion.
```rust
struct Tensor {
shape: Span,
data: Span
- extra: Option
}
struct ExtraParams {
@@ -263,40 +244,34 @@ struct ExtraParams {
```
-A `Tensor` in Orion takes a shape, a span array of the data and an extra parameter. For our tutorial, the ExtraParams specifies that the Tensor is associated with using fp16x16 format. In a 16x16 fixed-point format, there are 16 bits dedicated to the integer part of the number and 16 bits for the fractional part of the number. This format allows us to work with a wide range of values and a high degree of precision for conducting the OLS Tensor operations.
-
-```rust=
-let extra = ExtraParams { fixed_point: Option::Some(FixedImpl::FP16x16(())) };
-
-```
+A `Tensor` in Orion takes a shape and a span array of the data and an extra parameter. In a 16x16 fixed-point format, there are 16 bits dedicated to the integer part of the number and 16 bits for the fractional part of the number. This format allows us to work with a wide range of values and a high degree of precision for conducting the OLS Tensor operations.
### Implementing OLS functions using Orion
-At this stage, we will be reproducing the OLS functions now that we have generated our X and Y Fixedpoint Tensors. We will begin by creating a separate file for our linear regression functions file named `lin_reg_func.cairo` to host all of our linear regression functions.
+At this stage, we will be reproducing the OLS functions now that we have generated our X and Y fixed point Tensors. We will begin by creating a separate file for our linear regression functions file named `lin_reg_func.cairo` to host all of our linear regression functions.
#### Computing the mean
```rust
-fn calculate_mean(tensor_data: Tensor) -> FixedType {
-
- let tensor_size = FP16x16Impl::new_unscaled(tensor_data.data.len(), false);
+/// Calculates the mean of a given 1D tensor.
+fn calculate_mean(tensor_data: Tensor) -> FP16x16 {
+ let tensor_size = FixedTrait::::new_unscaled(tensor_data.data.len(), false);
let cumulated_sum = tensor_data.cumsum(0, Option::None(()), Option::None(()));
- let sum_result = cumulated_sum.data[tensor_data.data.len() - 1];
- let mean = FP16x16Div::div(*sum_result, tensor_size);
+ let sum_result = cumulated_sum.data[tensor_data.data.len() - 1];
+ let mean = *sum_result / tensor_size;
return mean;
}
-
```
-The above function takes in a FixedType Tensor and computes its corresponding mean value. We break the steps down by first calculating the cumulative sum of the tensor values using the `cumsum` built-in orion operator. We then divide the result by the length of the tensor size and return the output as a Fixedtype number.
+The above function takes in an FP16x16 Tensor and computes its corresponding mean value. We break the steps down by first calculating the cumulative sum of the tensor values using the `cumsum` built-in orion operator. We then divide the result by the length of the tensor size and return the output as a fixed point number.
#### Computing the deviation from the mean
```rust
-fn deviation_from_mean(tensor_data: Tensor ) -> Tensor {
-
+/// Calculates the deviation of each element from the mean of the provided 1D tensor.
+fn deviation_from_mean(tensor_data: Tensor) -> Tensor {
let mean_value = calculate_mean(tensor_data);
let mut tensor_shape = array::ArrayTrait::new();
@@ -304,21 +279,22 @@ fn deviation_from_mean(tensor_data: Tensor ) -> Tensor {
let mut deviation_values = array::ArrayTrait::new();
- let mut i:u32 = 0;
+ let mut i: u32 = 0;
loop {
- if i >= tensor_data.data.len() {
- break();
+ if i >= tensor_data.data.len() {
+ break ();
}
let distance_from_mean = *tensor_data.data.at(i) - mean_value;
deviation_values.append(distance_from_mean);
i += 1;
- };
- let extra = ExtraParams { fixed_point: Option::Some(FixedImpl::FP16x16(())) };
- let distance_from_mean_tensor = TensorTrait::::new(tensor_shape.span(), deviation_values.span(), Option::Some(extra));
+ };
+
+ let distance_from_mean_tensor = TensorTrait::::new(
+ tensor_shape.span(), deviation_values.span()
+ );
return distance_from_mean_tensor;
}
-
```
The following deviation\_from\_mean function calculates the deviation from the mean for each element of a given tensor. We initially calculate the tensor's mean value and store it under the variable mean\_value. We then create a for loop to iterate over each element in the tensor values and calculate the deviation from the mean which we will append the result to `deviation_values` array. Finally, we create a new tensor named distance\_from\_mean\_tensor by passing the deviation\_values array and the tensor shape.
@@ -332,20 +308,18 @@ $$
$$
```rust
-
-fn compute_beta(x_values: Tensor, y_values: Tensor ) -> FixedType {
-
+/// Calculates the beta value for linear regression.
+fn compute_beta(x_values: Tensor, y_values: Tensor) -> FP16x16 {
let x_deviation = deviation_from_mean(x_values);
let y_deviation = deviation_from_mean(y_values);
let x_y_covariance = x_deviation.matmul(@y_deviation);
let x_variance = x_deviation.matmul(@x_deviation);
- let beta_value = FP16x16Div::div(*x_y_covariance.data.at(0), *x_variance.data.at(0));
+ let beta_value = *x_y_covariance.data.at(0) / *x_variance.data.at(0);
return beta_value;
}
-
```
We can now compute the beta value for our linear regression utilising the previous deviation\_from\_mean function. We first calculate both the deviation of x values and y values from the mean and store them in separate variables as tensors. To calculate the covariance, we use the built-in Orion `matmul` operator to multiply x\_deviation by y\_deviation tensors. Similarly, we compute the X variance by multiplying x\_deviation tensor by itself. Finally, we divide the `x_y_covariance` by the `x_variance` to get an approximate gradient value for our regression model.
@@ -354,17 +328,17 @@ We can now compute the beta value for our linear regression utilising the previo
```rust
/// Calculates the intercept for linear regression.
-fn compute_intercept(beta_value:FixedType, x_values: Tensor, y_values: Tensor) -> FixedType {
-
+fn compute_intercept(
+ beta_value: FP16x16, x_values: Tensor, y_values: Tensor
+) -> FP16x16 {
let x_mean = calculate_mean(x_values);
let y_mean = calculate_mean(y_values);
- let mx= FP16x16Mul::mul(beta_value, x_mean);
+ let mx = beta_value * x_mean;
let intercept = y_mean - mx;
return intercept;
}
-
```
Calculating the y-intercept is fairly simple, we just need to substitute the calculated beta, y\_mean and x\_mean values and rearrange for the intercept value as previously shown in the Python implementation section.
@@ -374,56 +348,46 @@ Calculating the y-intercept is fairly simple, we just need to substitute the cal
Now that we have implemented all the necessary functions for the OLS method, we can finally test our linear regression model. We begin by creating a new separate test file named `test.cairo` and import all the necessary Orion libraries including our `X_values` and `y_values` found in the generated folder. We also import all the OLS functions from `lin_reg_func.cairo` file as we will be relying upon them to construct the regression model.
```rust
-use core::array::SpanTrait;
-use traits::Into;
use debug::PrintTrait;
-use array::ArrayTrait;
+
use verifiable_linear_regression::generated::X_values::X_values;
use verifiable_linear_regression::generated::Y_values::Y_values;
-use verifiable_linear_regression::lin_reg_func::{calculate_mean, deviation_from_mean, compute_beta, compute_intercept, predict_y_values, compute_mse, calculate_r_score};
-
-use orion::operators::tensor::math::cumsum::cumsum_i32::cumsum;
-use orion::operators::tensor::implementations::{impl_tensor_u32::Tensor_u32, impl_tensor_fp::Tensor_fp};
-use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};
-use orion::operators::tensor::math::arithmetic::arithmetic_fp::core::{add, sub, mul, div};
-use orion::numbers::fixed_point::core::{FixedTrait, FixedType, FixedImpl};
-use orion::numbers::fixed_point::implementations::fp16x16::core::{
- FP16x16Impl, FP16x16Add, FP16x16AddEq, FP16x16Print, FP16x16PartialEq, FP16x16Sub,
- FP16x16SubEq, FP16x16Mul, FP16x16MulEq, FP16x16Div, FP16x16DivEq, FP16x16PartialOrd, FP16x16Neg
+use verifiable_linear_regression::lin_reg_func::{
+ calculate_mean, deviation_from_mean, compute_beta, compute_intercept, predict_y_values,
+ compute_mse, calculate_r_score
};
-use orion::operators::tensor::linalg::matmul::matmul_fp::core::matmul;
#[test]
#[available_gas(99999999999999999)]
fn linear_regression_test() {
- //Data Retrieval
+ // Fetching the x and y values
let y_values = Y_values();
let x_values = X_values();
- //Beta Calculation
- let beta_value = compute_beta(x_values,y_values );
+ // (*x_values.data.at(18)).print();
+
+ let beta_value = compute_beta(x_values, y_values);
// beta_value.print(); // calculated gradient value
- //Intercept Calculation
- let intercept_value = compute_intercept(beta_value, x_values, y_values );
+ let intercept_value = compute_intercept(beta_value, x_values, y_values);
// intercept_value.print(); // calculated intercept value
- //Prediction Phase
- let y_pred = predict_y_values(beta_value, x_values, y_values );
+ let y_pred = predict_y_values(beta_value, x_values, y_values);
- //Evaluation
let mse = compute_mse(y_values, y_pred);
// mse.print(); // mean squared error ouput
+
let r_score = calculate_r_score(y_values, y_pred);
- // r_score.print(); // accuracy of model 0.97494506835
+ r_score.print(); // accuracy of model around 0.97494506835
assert(beta_value.mag > 0, 'x & y not positively correlated');
assert(r_score.mag > 0, 'R-Squared needs to be above 0');
- assert(r_score.mag < 65536, 'R-Squared has to be below 65536'); // 65536 represents ONE in fp16x16.
+ assert(
+ r_score.mag < 65536, 'R-Squared has to be below 65536'
+ ); // 65536 represents ONE in fp16x16.
assert(r_score.mag > 32768, 'Accuracy below 50% ');
}
-
```
Our model will get tested under the `linear_regression_test()` function which will follow the following steps:
diff --git a/docs/framework/get-started.md b/docs/framework/get-started.md
index 5d24463f5..833df5885 100644
--- a/docs/framework/get-started.md
+++ b/docs/framework/get-started.md
@@ -3,7 +3,7 @@
In this section, we will guide you to start using Orion successfully. We will help you install Cairo 1.0 and add Orion dependency in your project.
{% hint style="info" %}
-Orion supports **Cairo v2.1.1** and **Scarb 0.6.2**
+Orion supports **Cairo v2.2.0** and **Scarb 0.7.0**
{% endhint %}
## 📦 Installations
@@ -58,10 +58,8 @@ You can now use the `orion` in your files:
```rust
use array::{ArrayTrait, SpanTrait};
-use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};
-use orion::operators::tensor::implementations::impl_tensor_i32::{Tensor_i32};
-use orion::operators::nn::core::NNTrait;
-use orion::operators::nn::implementations::impl_nn_i32::NN_i32;
+use orion::operators::tensor::{TensorTrait, Tensor, I32Tensor};
+use orion::operators::nn::{NNTrait, I32NN};
use orion::numbers::signed_integer::i32::{i32, IntegerTrait};
fn relu_example() -> Tensor {
@@ -72,8 +70,8 @@ fn relu_example() -> Tensor {
IntegerTrait::new(2, false),
IntegerTrait::new(1, true),
IntegerTrait::new(2, true),
- ].span(),
- extra: Option::None(())
+ ]
+ .span(),
);
return NNTrait::relu(@tensor);
@@ -82,4 +80,4 @@ fn relu_example() -> Tensor {
## 🔭 Discover the Orion APIs
-
⚙️ Operators
A set of standardized math functions that are used in the computation of neural network models.
diff --git a/docs/framework/numbers/fixed-point/README.md b/docs/framework/numbers/fixed-point/README.md
index 54960e168..9f5f18551 100644
--- a/docs/framework/numbers/fixed-point/README.md
+++ b/docs/framework/numbers/fixed-point/README.md
@@ -1,15 +1,15 @@
# Fixed Point
{% hint style="info" %}
-This library has been modified from [cubit](https://github.com/influenceth/cubit) library by [influenceth](https://github.com/influenceth) and adjusted to match with Q8.23 and Q16.16 fixed points.
+This library has been modified from [cubit](https://github.com/influenceth/cubit) library by [influenceth](https://github.com/influenceth) and adjusted to match with other fixed point implementations.
{% endhint %}
-This API provides basic some operations for signed fixed point Q8.23 and Q16.16 numbers. Fixed point numbers are represented as a struct `FixedType` with a magnitude and a sign.
+This API provides basic some operations for signed fixed point numbers. Fixed point numbers are represented as a struct with a magnitude and a sign.
The magnitude represents the absolute value of the number, and the sign indicates whether the number is positive or negative.
```rust
-struct FixedType {
+struct FP8x23 {
mag: u32,
sign: bool
}
@@ -17,12 +17,14 @@ struct FixedType {
### Data types
-Orion supports currently one fixed point type.
+Orion supports currently these fixed point types:
-| Data type | dtype |
-| --------- | ----------- |
-| Q8.23 | `FixedType` |
-| Q16.16 | `FixedType` |
+| Data type | dtype |
+| --------- | --------- |
+| Q8.23 | `FP8x23` |
+| Q16.16 | `FP16x16` |
+| Q32.32 | `FP32x32` |
+| Q64.64 | `FP64x64` |
### **`Fixed` Trait**
@@ -32,40 +34,40 @@ use orion::numbers::fixed_point::core::FixedTrait;
`Fixed` trait defines the operations that can be performed on a fixed point.
-| function | description |
-| --- | --- |
-| [`fp.new`](fp.new.md) | Constructs a new fixed point instance. |
-| [`fp.new_unscaled`](fp.new\_unscaled.md) | Creates a new fixed point instance with the specified unscaled magnitude and sign. |
-| [`fp.from_felt`](fp.from\_felt.md) | Creates a new fixed point instance from a felt252 value. |
-| [`fp.abs`](fp.abs.md) | Returns the absolute value of the fixed point number. |
-| [`fp.ceil`](fp.ceil.md) | Returns the smallest integer greater than or equal to the fixed point number. |
-| [`fp.exp`](fp.exp.md) | Returns the value of e raised to the power of the fixed point number. |
-| [`fp.exp2`](fp.exp2.md) | Returns the value of 2 raised to the power of the fixed point number. |
-| [`fp.floor`](fp.floor.md) | Returns the largest integer less than or equal to the fixed point number. |
-| [`fp.ln`](fp.ln.md) | Returns the natural logarithm of the fixed point number. |
-| [`fp.log2`](fp.log2.md) | Returns the base-2 logarithm of the fixed point number. |
-| [`fp.log10`](fp.log10.md) | Returns the base-10 logarithm of the fixed point number. |
-| [`fp.pow`](fp.pow.md) | Returns the result of raising the fixed point number to the power of another fixed point number. |
-| [`fp.round`](fp.round.md) | Rounds the fixed point number to the nearest whole number. |
-| [`fp.sqrt`](fp.sqrt.md) | Returns the square root of the fixed point number. |
-| [`fp.acos`](fp.acos.md) | Returns the arccosine (inverse of cosine) of the fixed point number. |
-| [`fp.acos_fast`](fp.acos\_fast.md) | Returns the arccosine (inverse of cosine) of the fixed point number faster with LUT. |
-| [`fp.asin`](fp.asin.md) | Returns the arcsine (inverse of sine) of the fixed point number. |
-| [`fp.asin_fast`](fp.asin\_fast.md) | Returns the arcsine (inverse of sine) of the fixed point number faster with LUT. |
-| [`fp.atan`](fp.atan.md) | Returns the arctangent (inverse of tangent) of the input fixed point number. |
-| [`fp.atan_fast`](fp.atan\_fast.md) | Returns the arctangent (inverse of tangent) of the input fixed point number faster with LUT. |
-| [`fp.cos`](fp.cos.md) | Returns the cosine of the fixed point number. |
-| [`fp.cos_fast`](fp.cos\_fast.md) | Returns the cosine of the fixed point number fast with LUT. |
-| [`fp.sin`](fp.sin.md) | Returns the sine of the fixed point number. |
-| [`fp.sin_fast`](fp.sin\_fast.md) | Returns the sine of the fixed point number faster with LUT. |
-| [`fp.tan`](fp.tan.md) | Returns the tangent of the fixed point number. |
-| [`fp.tan_fast`](fp.tan\_fast.md) | Returns the tangent of the fixed point number faster with LUT. |
-| [`fp.acosh`](fp.acosh.md) | Returns the value of the inverse hyperbolic cosine of the fixed point number. |
-| [`fp.asinh`](fp.asinh.md) | Returns the value of the inverse hyperbolic sine of the fixed point number. |
-| [`fp.atanh`](fp.atanh.md) | Returns the value of the inverse hyperbolic tangent of the fixed point number. |
-| [`fp.cosh`](fp.cosh.md) | Returns the value of the hyperbolic cosine of the fixed point number. |
-| [`fp.sinh`](fp.sinh.md) | Returns the value of the hyperbolic sine of the fixed point number. |
-| [`fp.tanh`](fp.tanh.md) | Returns the value of the hyperbolic tangent of the fixed point number. |
+| function | description |
+| ---------------------------------------- | ------------------------------------------------------------------------------------------------ |
+| [`fp.new`](fp.new.md) | Constructs a new fixed point instance. |
+| [`fp.new_unscaled`](fp.new\_unscaled.md) | Creates a new fixed point instance with the specified unscaled magnitude and sign. |
+| [`fp.from_felt`](fp.from\_felt.md) | Creates a new fixed point instance from a felt252 value. |
+| [`fp.abs`](fp.abs.md) | Returns the absolute value of the fixed point number. |
+| [`fp.ceil`](fp.ceil.md) | Returns the smallest integer greater than or equal to the fixed point number. |
+| [`fp.exp`](fp.exp.md) | Returns the value of e raised to the power of the fixed point number. |
+| [`fp.exp2`](fp.exp2.md) | Returns the value of 2 raised to the power of the fixed point number. |
+| [`fp.floor`](fp.floor.md) | Returns the largest integer less than or equal to the fixed point number. |
+| [`fp.ln`](fp.ln.md) | Returns the natural logarithm of the fixed point number. |
+| [`fp.log2`](fp.log2.md) | Returns the base-2 logarithm of the fixed point number. |
+| [`fp.log10`](fp.log10.md) | Returns the base-10 logarithm of the fixed point number. |
+| [`fp.pow`](fp.pow.md) | Returns the result of raising the fixed point number to the power of another fixed point number. |
+| [`fp.round`](fp.round.md) | Rounds the fixed point number to the nearest whole number. |
+| [`fp.sqrt`](fp.sqrt.md) | Returns the square root of the fixed point number. |
+| [`fp.acos`](fp.acos.md) | Returns the arccosine (inverse of cosine) of the fixed point number. |
+| [`fp.acos_fast`](fp.acos\_fast.md) | Returns the arccosine (inverse of cosine) of the fixed point number faster with LUT. |
+| [`fp.asin`](fp.asin.md) | Returns the arcsine (inverse of sine) of the fixed point number. |
+| [`fp.asin_fast`](fp.asin\_fast.md) | Returns the arcsine (inverse of sine) of the fixed point number faster with LUT. |
+| [`fp.atan`](fp.atan.md) | Returns the arctangent (inverse of tangent) of the input fixed point number. |
+| [`fp.atan_fast`](fp.atan\_fast.md) | Returns the arctangent (inverse of tangent) of the input fixed point number faster with LUT. |
+| [`fp.cos`](fp.cos.md) | Returns the cosine of the fixed point number. |
+| [`fp.cos_fast`](fp.cos\_fast.md) | Returns the cosine of the fixed point number fast with LUT. |
+| [`fp.sin`](fp.sin.md) | Returns the sine of the fixed point number. |
+| [`fp.sin_fast`](fp.sin\_fast.md) | Returns the sine of the fixed point number faster with LUT. |
+| [`fp.tan`](fp.tan.md) | Returns the tangent of the fixed point number. |
+| [`fp.tan_fast`](fp.tan\_fast.md) | Returns the tangent of the fixed point number faster with LUT. |
+| [`fp.acosh`](fp.acosh.md) | Returns the value of the inverse hyperbolic cosine of the fixed point number. |
+| [`fp.asinh`](fp.asinh.md) | Returns the value of the inverse hyperbolic sine of the fixed point number. |
+| [`fp.atanh`](fp.atanh.md) | Returns the value of the inverse hyperbolic tangent of the fixed point number. |
+| [`fp.cosh`](fp.cosh.md) | Returns the value of the hyperbolic cosine of the fixed point number. |
+| [`fp.sinh`](fp.sinh.md) | Returns the value of the hyperbolic sine of the fixed point number. |
+| [`fp.tanh`](fp.tanh.md) | Returns the value of the hyperbolic tangent of the fixed point number. |
### Arithmetic & Comparison operators
@@ -75,26 +77,26 @@ use orion::numbers::fixed_point::core::FixedTrait;
```rust
fn add_fp_example() {
- // We instantiate two fixed point from felt here.
+ // We instantiate two fixed point from here.
// a = 1
// b = 2
- let a = Fixed::from_unscaled_felt(1);
- let b = Fixed::from_unscaled_felt(2);
+ let a = Fixed::new_unscaled(1, false);
+ let b = Fixed::new_unscaled(2, false);
// We can add two fixed point as follows.
let result = a + b;
- assert(result == Fixed::from_unscaled_felt(3), 'invalid result');
+ assert(result == Fixed::new_unscaled(3), 'invalid result');
}
```
```rust
fn compare_fp_example() -> bool {
- // We instantiate two fixed point from felt here.
+ // We instantiate two fixed point from here.
// a = 42
// b = -10
- let a = Fixed::from_unscaled_felt(42);
- let b = Fixed::from_unscaled_felt(-10);
+ let a = Fixed::new_unscaled(42, false);
+ let b = Fixed::new_unscaled(10, true);
// We can compare two fixed point as follows.
return a > b;
diff --git a/docs/framework/numbers/fixed-point/fp.from_unscaled_felt.md b/docs/framework/numbers/fixed-point/fp.from_unscaled_felt.md
deleted file mode 100644
index bac4023fd..000000000
--- a/docs/framework/numbers/fixed-point/fp.from_unscaled_felt.md
+++ /dev/null
@@ -1,2 +0,0 @@
-# fp.from\_unscaled\_felt
-
diff --git a/docs/framework/operators/neural-network/README.md b/docs/framework/operators/neural-network/README.md
index 34d5a954c..19dfbf5a3 100644
--- a/docs/framework/operators/neural-network/README.md
+++ b/docs/framework/operators/neural-network/README.md
@@ -10,12 +10,12 @@ use orion::operators::nn;
Orion supports currently these `NN` types.
-| Data type | dtype |
-| ------------------------- | -------------- |
-| 32-bit integer (signed) | `Tensor` |
-| 8-bit integer (signed) | `Tensor` |
-| 32-bit integer (unsigned) | `Tensor` |
-| Fixed point (signed) | `Tensor` |
+| 8-bit integer (signed) | `Tensor` |
+| 32-bit integer (unsigned) | `Tensor` |
+| Fixed point (signed) | `Tensor` |
### NN**Trait**
@@ -31,4 +31,3 @@ Orion supports currently these `NN` types.
| [`nn.softsign`](nn.softsign.md) | Applies the Softsign function element-wise. |
| [`nn.softplus`](nn.softplus.md) | Applies the Softplus function element-wise. |
| [`nn.linear`](nn.linear.md) | Performs a linear transformation of the input tensor using the provided weights and bias. |
-
diff --git a/docs/framework/operators/tensor/README.md b/docs/framework/operators/tensor/README.md
index 5b0e7a097..5e614e62a 100644
--- a/docs/framework/operators/tensor/README.md
+++ b/docs/framework/operators/tensor/README.md
@@ -2,47 +2,32 @@
A Tensor represents a multi-dimensional array of elements.
-A `Tensor` represents a multi-dimensional array of elements and is depicted as a struct containing both the tensor's shape, a flattened array of its data and extra parameters. The generic Tensor is defined as follows:
+A `Tensor` represents a multi-dimensional array of elements and is depicted as a struct containing both the tensor's shape and a flattened array of its data. The generic Tensor is defined as follows:
```rust
struct Tensor {
shape: Span,
data: Span
- extra: Option
}
```
-`ExtraParams` is a struct containing additional parameters for the tensor.
-
-`fixed_point` extra param indicates the implementation of the fixed point to be used with the tensor, if fixed points are required in certain operations (e.g. `tensor.exp()`).
-
-```rust
-struct ExtraParams {
- fixed_point: Option
-}
-```
-
-**ExtraParams**
-
-
Params
dtype
default
desciption
fixed_point
Option<FixedImpl>
FP16x16()
Specifies the type of Fixed Point a Tensor can supports.
-
### Data types
Orion supports currently these tensor types.
-| Data type | dtype |
-| ------------------------- | -------------- |
-| 32-bit integer (signed) | `Tensor` |
-| 8-bit integer (signed) | `Tensor` |
-| 32-bit integer (unsigned) | `Tensor` |
-| Fixed point (signed) | `Tensor` |
+| 8-bit integer (signed) | `Tensor` |
+| 32-bit integer (unsigned) | `Tensor` |
+| Fixed point (signed) | `Tensor` |
***
### Tensor**Trait**
```rust
-use orion::operators::tensor::core::TensorTrait;
+use orion::operators::tensor::TensorTrait;
```
`TensorTrait` defines the operations that can be performed on a Tensor.
@@ -106,22 +91,15 @@ Element-wise add.
```rust
use array::{ArrayTrait, SpanTrait};
-
-use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};
-use orion::operators::tensor::implementations::impl_tensor_u32::{Tensor_u32, u32TensorAdd};
-
+use orion::operators::tensor::{TensorTrait, Tensor, U32Tensor, U32TensorAdd};
fn element_wise_add_example() -> Tensor {
// We instantiate two 3D Tensors here.
let tensor_1 = TensorTrait::new(
- shape: array![2, 2, 2].span(),
- data: array![0, 1, 2, 3, 4, 5, 6, 7].span(),
- extra: Option::None(())
+ shape: array![2, 2, 2].span(), data: array![0, 1, 2, 3, 4, 5, 6, 7].span(),
);
let tensor_2 = TensorTrait::new(
- shape: array![2, 2, 2].span(),
- data: array![0, 1, 2, 3, 4, 5, 6, 7].span(),
- extra: Option::None(())
+ shape: array![2, 2, 2].span(), data: array![0, 1, 2, 3, 4, 5, 6, 7].span(),
);
// We can add two tensors as follows.
@@ -134,22 +112,17 @@ Add two tensors of different shapes but compatible in broadcasting.
```rust
use array::{ArrayTrait, SpanTrait};
-
-use orion::operators::tensor::core::{TensorTrait, Tensor, ExtraParams};
-use orion::operators::tensor::implementations::impl_tensor_u32::{Tensor_u32, u32TensorAdd};
-
+use orion::operators::tensor::{TensorTrait, Tensor, U32Tensor, U32TensorAdd};
fn broadcasting_add_example() -> Tensor {
// We instantiate two 3D Tensors here.
let tensor_1 = TensorTrait::new(
shape: array![2, 2, 2].span(),
data: array![0, 1, 2, 3, 4, 5, 6, 7].span(),
- extra: Option::None(())
);
let tensor_2 = TensorTrait::new(
shape: array![1, 2, 1].span(),
data: array![10, 100].span(),
- extra: Option::None(())
);
// We can add two tensors as follows.
