Skip to content

Commit

Permalink
Merge pull request #2 from warlock-labs/feat/basic-operations
Browse files Browse the repository at this point in the history
  • Loading branch information
GrugCapital authored Jun 20, 2024
2 parents 226ea7b + a60d892 commit 26eb65d
Show file tree
Hide file tree
Showing 10 changed files with 1,635 additions and 537 deletions.
4 changes: 2 additions & 2 deletions .github/workflows/CI.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,13 +3,13 @@ name: CI
on:
push:
branches:
- master
- main
tags:
- 'v*.*.*'
pull_request:
types: [ opened, synchronize, reopened ]
branches:
- master
- main
env:
CARGO_TERM_COLOR: always

Expand Down
6 changes: 3 additions & 3 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -8,13 +8,13 @@ license = "MIT"
readme = "README.md"
repository = "https://github.com/warlock-labs/dimensionals"
name = "dimensionals"
version = "0.1.1"
version = "0.2.1"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
num = "0.4.3"
num-traits = "0.2.19"

[lib]

Expand All @@ -23,4 +23,4 @@ criterion = "0.5.1"

[[bench]]
name = "benchmarks"
harness = false
harness = false
24 changes: 14 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,9 @@

# dimensionals

Dimensionals is a Rust library for working with n-dimensional data. It provides a flexible and efficient multidimensional array implementation with a generic storage backend.
Dimensionals is a Rust library for working with n-dimensional data. It provides
a flexible and efficient multidimensional array implementation with a generic
storage backend.

## Motivations

Expand All @@ -18,12 +20,10 @@ The key motivations behind Dimensionals are:

## Features

- Generic over element type and number of dimensions
- Efficient storage using a linear memory layout
- Index and mutable index operations
- Arithmetic operations for 1D and 2D arrays
- Convenient macros for array creation
- Extensible with custom storage backends
- Generic over an element type, number of dimensions, and storage backend
- Iterators, slices, indexing, and other standard Rust traits
- Ergonomic and idiomatic `std::ops` implementations for arithmetic operations
- Convenient macros for vector and matrix creation

## Usage

Expand Down Expand Up @@ -55,6 +55,7 @@ For more examples and usage details, see the [API documentation](https://docs.rs

The following features and improvements are planned for future releases:

- Arithmetic operations for 1D and 2D arrays
- SIMD support for improved performance on CPU.
- GPU support for offloading computations to compatible GPUs.
- Comprehensive scalar, vector, matrix, and tensor algebra operations.
Expand All @@ -64,21 +65,24 @@ The following features and improvements are planned for future releases:

## Performance

The `LinearArrayStorage` backend stores elements in a contiguous `Vec<T>` and computes element indices on the fly. This provides good cache locality for traversals, but may not be optimal for sparse or very high dimensional arrays.
The `LinearArrayStorage` backend stores elements in a contiguous `Vec<T>` and computes element indices on the fly. This
provides good cache locality for traversals, but may not be optimal for sparse or very high dimensional arrays.

Alternative storage backends can be implemented by defining a type that implements the `DimensionalStorage` trait.

## Contributing

Contributions are welcome! Please feel free to submit issues, feature requests, or pull requests on the [GitHub repository](https://github.com/warlock-labs/dimensionals).
Contributions are welcome! Please feel free to submit issues, feature requests, or pull requests on
the [GitHub repository](https://github.com/warlock-labs/dimensionals).

## License

This project is licensed under the [MIT License](https://choosealicense.com/licenses/mit/).

## Acknowledgements

This project is inspired by and builds upon ideas from several existing multidimensional array libraries in Rust and other languages.
This project is inspired by and builds upon ideas from several existing multidimensional array libraries in Rust and
other languages.

## Contact

Expand Down
65 changes: 65 additions & 0 deletions TODO.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,65 @@
- [ ] Use safe rust in iterators
- [ * ] Use safe rust in indexing
- [ ] Add tensor macro for creating tensors
- [ ] Remove the need for phantom data markers
- [ ] Move shape data to type-system such that it is known at compile time
- [ * ] Support common arithmetic operations
- [ * ] Use safe rust in arithmetic operations
- [ ] Support reshaping
- [ ] Support appending
- [ ] Support removing
- [ ] Linear algebra functions
- [ ] Support for common statistical functions
- [ ] Support for geometric functions like Brownian motion
- [ ] Support for GPU offloading
- [ ] Support for SIMD
- [ ] Matrix multiplication
- [ ] Support Apache Arrow or safetensors storage backend?
- [ ] Support Pola.rs integration
- [ ] Support plotly-rs integration
- [ ] Support argmin-rs integration
- [ ] Support for rayon
- [ ] Feature flags for enabling/disabling features
- [ ] no-std
- [ ] Support for WebAssembly
- [ ] Support for WebGPU
- [ ] Support for SVM target


The goal is for this library to have an intuitive interface for working with
n dimensional/tensor data, while interfacing with libraries like Polars, and
by proxy plotly-rs. It should have linear algebra functionality, and integration
with argmin-rs for optimization. It should have common statistical functions
such that it is able to work in a statistical computing environment.

The signature for creating a Dimensional is ugly in that N must be speficied twice, a generalized
builder pattern, or obviation of the need to store twice would be ideal.

In lib.rs:

The TODO for the tensor macro is still present. This is not a bug, but a reminder for future implementation.


In core.rs:

The PhantomData<T> in the Dimensional struct is still present but unused. You might consider removing it if it's not needed for type invariance.


In iterators.rs:

The mutable iterator still uses unsafe code. While this is not necessarily a bug, it's worth noting that it introduces potential safety risks if not handled carefully.


In storage.rs:

No significant issues found. The implementation looks correct and well-tested.



Overall, the code appears to be functioning as intended. The main points to consider are:

Removing unused PhantomData if it's not needed.
Potentially finding a safe alternative to the unsafe code in the mutable iterator, if possible.
Implementing the tensor macro in the future, as noted in the TODO.

These are not critical issues, but rather areas for potential future improvement. The library as it stands should work correctly for its intended purpose.
3 changes: 3 additions & 0 deletions benches/benchmarks.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
use criterion::{black_box, criterion_group, criterion_main, Criterion};
use dimensionals::{Dimensional, LinearArrayStorage};

// TODO: This needs meaningful benchmarks for common operations useful in
// quantitive situations

fn bench_dimensional_array_creation_zeros(c: &mut Criterion) {
let shape = [1000, 1000];
c.bench_function("dimensional_array_creation_zeros", |b| {
Expand Down
Loading

0 comments on commit 26eb65d

Please sign in to comment.