v5.8.1

Bug fixes:

• fixed a memory leak in the random number generator module, which meant that the amount of memory used was excessive, and that using random translations, random patches, or dropout used up all available memory within seconds
• fixed a bug in the forward1 propagate kernel, which meant that it crashed for certain geometries

v5.8.0

New:

• Josef Moudrik has started work on deepclexec, to run prediction, on pre-trained weights, for new data samples
• Added jpeg loader, so can load imagenet data now

Changes

• Added dependency on libjpeg-turbo, for the jpeg loader. This can be turned off in cmake options, if you want

Changes under the covers

• factorized applying bias into separate class for forward3 (it was already in a separate OpenCLKernel), and away from the convolutional forward opencl in forward4, and forward1
• fixed a bug in forward4, where images larger than the square root of the maximum gpu workgroupsize sometimes has incorrect values for the last few pixels
• migrated to latest version of EasyCL, which handles storing the device dirty flag for us, rather than having lots of flags in our code like weightsCopiedToHost, and so on
• GenericLoaderv2 is now stateful, rather than using static methods as per original GenericLoader
  • new NetLearnerOnDemandv2 uses GenericLoaderv2, as does new OnDemandBatcherv2
  • deepclrun migrated to use GenericLoaderv2
  • GenericLoaderv1Wrapper wraps existing GenericLoader implementations, so no need to re-write those in any way for now, and any new GenericLoader implementations can continue to be v1, via the wrapper, if they dont need state
  • making GenericLoaderv2 stateful means we can read a jpeg manifest, eg for imagenet et al, once, and then hold it in memory

v5.7.0

New:

• added Adadelta
  • in commandline, 'trainer=adadelta rho=0.9'
  • in C++, trainer = new Adadelta( cl, 0.9f );
  • In Python and Lua, as in C++

v5.6.0

New

• added WeightInitializer abstraction, so can customizer how weights are initialized
• two implementations:
  • OriginalInitializer: default, corresponds to initialization method up to now
  • UniformInitializer: samples uniformly, from a range parameterized by passed in initialWeights parameter

v5.5.0

New

• added Rmsprop
  • available in commandline, using trainer=rmsprop
  • available in c++, using Rmsprop *trainer = new Rmsprop( cl );, and using it to do training (trainer->train(...) etc)
  • available in lua wrappers, by creating an Rmsprop object, and using for training
  • available in python wrappers, by creating an Rmsprop object, and using for training

Note: builds ok on Windows.

v5.4.0

New:

• added Adagrad
  • in commandline, use trainer=adagrad
  • In C++, use Adagrad *adagrad = new Adagrad( cl ); adagrad->setLearningRate( 0.002f );
  • In Python and Lua, create an Adagrad instance, as per C++

Under the hood

• a bunch of functions added to CLMathWrapper, like: per element inverse, per element add scalar, per element multiply, per element squared, per element square root
• simplified SGD etc, by moving most stuff from bindState into the Trainer base class

v5.3.0

New

• added Nesterov trainer to python wrappers
• added Nesterov trainer to lua wrappers

v5.2.0

New:

• added Nesterov trainer
  • available from C++ api, and from commandline

v5.1.0

Under the hood:

• added new CLMathWrapper class, to make per-element gpu array arithmetic easy
  • migrated SGD and Annealer to use CLMathWrapper

Bug fixes:

• fixed crash in unittests, caused by attempting to reuse kernels across multiple EasyCL instances
• added appropriate exports, so builds on Windows again now

v5.0.0

New:

• Added Annealer trainer, and 'anneal=' commandline option
  • python and lua wrappers can also create Annealer trainer, as well as the existing SGD trainer

Changes:

• bunch of changes to many of the classes now in src/batch directory, ie xxxBatcher classes, xxxNetAction type classes, and NetLearnerxxx classes, hence bumped major version to 5, eg
  • XXXBatcher.tick: added parameter 'epoch' to 'tick' and 'run' methods
  • OnDemandBatcher constructor takes in a Batcher* object
  • created new Batcher2, and NetAction2 classes
  • removed BatchLearner class

Bug fixes

• bunch of bugfixes to the python and lua wrappers