Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support of different data types and multiple dimensions #7

Open
versilov opened this issue Jun 11, 2018 · 8 comments
Open

Support of different data types and multiple dimensions #7

versilov opened this issue Jun 11, 2018 · 8 comments

Comments

@versilov
Copy link
Owner

@michalmuskala can you have a look at new structure of the Matrex.Array object in this branch: https://github.com/versilov/matrex/blob/array/lib/matrex/array.ex

I've added types, arbitrary dimensions and strides. These are now stored in Elixir structure, and data field contents only pure data, without dimensions. Exactly like you suggested in the post on ElixirForum.

I plan to merge this new structure into the Matrex structure.

Thanks in advance!
@michalmuskala
Copy link

Some remarks:

  • fiil_data can be replaced by :binary.copy
  • one idea could be to precompute things like ones and zeros at compile-time and have them as literals. This could be interesting from the performance point of view
  • :"foo_#{to_string(bar)}" is the same as :"foo_#{bar}"
  • I'm not really sure, I understand the "strides" thing. But it seems that most of the time you actually use it to traverse it as a collection - a list might be a better use there instead of a tuple

But yeah, in general this looks good to me. I like that you were able to satisfy some functions directly in elixir without going to the C side with that extra information.

@versilov
Copy link
Owner Author

Oh, I forgot to mention, that branch is a bit messy, cuz I used it for experiments on the concept. Sorry for that!

  1. fill_data will be replaced by NIF, I think.
  2. Did not understand, how we can precompute ones and zeros, can you explain, please?
  3. OK, will update.
  4. The strides I borrowed from the NumPy. When I benchmarked against NumPy, I noticed, that they do matrix transposing in very short constant time. Little investigation revealed, that they just swap strides component for that. That impressed me very much and I decided to incorporate it into Matrix right away. Though, for now they are used only for enumerating and are not taken into consideration, when doing element-wise operations, I think it would be nice to have them in place for later extensions.

I also thought to use lists instead of tuples for strides and shape.

@michalmuskala
Copy link

About the ones and zeros - you're right. I somehow overlooked the fact that we don't know the sizes 🤦‍♂️

@versilov
Copy link
Owner Author

@michalmuskala, can we find someone experienced to make a code review for my C code for multi-typed arrays functionality?

@doctorcorral
Copy link

Sorry if not the right place to rise this but, I think it would be nice for Matrex.Array to incorporate ATen for handling tensor functionality right away. Popular frameworks like PyTorch are useful because of its easy handling of tensors and tensor autodifferentiation, which is implemented on top of Autograd, which is implemented on top of ATen.

@versilov
Copy link
Owner Author

versilov commented Jul 1, 2019

@doctorcorral that's a nice idea. Does ATen have external libraries as dependencies?

@elcritch
Copy link
Contributor

elcritch commented Jan 6, 2020

Curious, what's the status of this issue? I'd be interested in double's and/or multi-dimensional data.

elcritch added a commit to elcritch/matrex that referenced this issue Jan 7, 2020
@elcritch
# This is a combination of 22 commits.
719d848 
# This is the 1st commit message:

porting numerix stats libraries

# This is the commit message versilov#2:

adding power function

# This is the commit message versilov#3:

fixing power

# This is the commit message versilov#4:

porting statistics

# This is the commit message versilov#5:

porting over statistics

# This is the commit message versilov#6:

fix apply

# This is the commit message versilov#7:

fix algos

# This is the commit message versilov#8:

fixing algos

# This is the commit message versilov#9:

test

# This is the commit message versilov#10:

updating stats test

# This is the commit message versilov#11:

switch type to matrex

# This is the commit message versilov#12:

fixing more tests

# This is the commit message versilov#13:

matrex

# This is the commit message versilov#14:

updates

# This is the commit message versilov#15:

removing experiment

# This is the commit message versilov#16:

try as row matrix by def

# This is the commit message versilov#17:

add vector pattern match

# This is the commit message versilov#18:

adding pseudo-vector type

# This is the commit message versilov#19:

fix order for column-wise vector

# This is the commit message versilov#20:

fix order for column-wise vector

# This is the commit message versilov#21:

fix order for column-wise vector

# This is the commit message versilov#22:

removing extras deps
@versilov
Copy link
Owner Author

@elcritch, this branch is fully functional, except displaying multi-dimensional matrices in console is not working properly.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@elcritch @michalmuskala @versilov @doctorcorral