Skip to content

Latest commit

 

History

History
126 lines (98 loc) · 4.39 KB

meeting_00016_20201215.md

File metadata and controls

126 lines (98 loc) · 4.39 KB
Raw

Rust ML WG Meeting 00016

Meeting Info

Date: 20201215

Start time: 1600 ET

Zoom link: https://1password.zoom.us/j/98269163669?pwd=U2E4TDltOHM5Vk00azJ4QnF4YVJaZz09

  • Meeting ID: 982 6916 3669
  • Passcode: 485260

Participants

Agenda

  • Short update on Fwumious Wabbit (by Andraz)

  • chrism wrote a small prototype for itegrating a convolutional with fully-connected layer

    • Still double-checking that the backwards pass between the two layers types are working (has been talking with Benard, who leads juice devel)
    • Could use another set of eyes on the linear algebra if anyone's interested before moving

Minutes

  • just chatting about chrism's work and other people's interests as a pre-show :)

Chrism small prototype for integrating fully-connected convolutional layer

  • tsuga
  • generalizing away from fully-connected layers into convolution layers
    • can do horizontal line detection
      • with sliding window and matrix multiplation
  • working on adding convolutonal layers
  • next step
    • move way from NIST into CIFAR10
  • less confident about backwards passes
    • needs some eyes on the linear alegabra for the backward passes from fully-connected into convolutional layer
  • Could you take an input and put through edge detection
    • the output layer should be the total number of pixels and then converge to 1
    • single kernel and sincel convolutional layer

Update on Fwumious Wabbit from Andraz

  • continung development
  • two new features
    • from java to rust code directly JNI bindings
    • rework that will enable hogwild model loading
      • hogwild loading of models
        • training for deep models in parallel

Enzyme automatic differenation from Billy

  • https://github.com/wsmoses/Enzyme

  • lorenz

    • integrate that into the Rust ecosystem
    • creating bindings to enzyme
      • in a build.rs file
    • still creating the bindings and reading the llvm code
    • type trees problem
      • special case of -1
        • enzyme wants to know the types of data (e.g. float vs double)
        • type trees are
          • type prop rules, for any given instruction in llvm (adding to types together means the resulting type is of the same time)
          • from the Rust types, what are the corresponding type tree equivilant
        • this is a repeating type
        • positive offset is a subtree is exactly what the types are
        • -1 means for example double array and every x bytes is a double

  • Billy

    • working on parallelism
      • GPU and CPU parallelism
      • has working parallelism for CUDA mostly working

  • Hijack the linker from LTO -> LLVM

    • ideal
      • dynamically load into rustc (or custom codegen pass)
      • modify or fork the codegen pass that allows lowering of the automactially diff tree function
    • if you dynmically load enzyme, it will autmocally insert enzyme into the pass pipeline
    • co-author on eyzme is one of the main contributors of julia and julia supports the codegen pass

linfa update from lorenz

  • not a lot since last meeting
  • updated decision tree
  • moved to new CI infrastructure
    • added code coverage tool

Chris asked about how linfa datasets are handled with test data

  • How do other ecosystems package their testing data?
    • linfa is using tarballs
  • use include macro compile who dataset into project
    • then publish project
  • git submodules possibly
    • don't forget to do git submodules --recursive
      • or else you'll clone the repo but not the included modules
  • any interest in using a pytorch model where you download and cache the data locally?
    • need hasn't really shown up to put together

Billy asked if the WG is Looking into any Bayesian models at all?

  • probabilitic programming 
  • not really yet ecosystem is fairly young

Chrism talking about St. Claire license

Actions

  • Add https://github.com/wsmoses to tsuga
  • feedback about the St. Clair license
  • next meeting tenatively 20210105 possibly 20200112 due to Holidays