- add CUDA_MODULE_LOADING=EAGER to force CUDA kernel preloading
- drop the no-longer working pytest-ipynb, and switch to nbmake for testing
- fix implicit imports
- fix
ipyexperiments.utils.ipython.ipython_tb_clear_frames
- skipped
- get rid of the attempt to use the backend methods in the cell logger, too many deadlock issues - just use pytorch for now if available and remove the pointless abstraction (needs more cleanup)
- completely rewrite general RAM reporting as I discovered tracemalloc can't track non-python application, so general RAM reporting was completely invalid at times. Switch to the same approach as pynvml tracking, with a thread to monitor peak memory usage.
- simplify the logic of how gpu peak memory is calculated
- fix broken tests now that RAM is reported correctly
- improve logging format
- fix preload_pytorch to load the 0th device by default
- fix the test suite with CUDA_VISIBLE_DEVICES=0 CUDA_DEVICE_ORDER=PCI_BUS_ID to that cuda and pynvml report the same gpus
preload_pytorch
is no longer automatically run in ipyexperiments.utils.mem - as it may interfere with other systems. it's an explicit call now.
- bug fix
- require pytest-ipynb for the test suite
- add a workaround for pynvml not respecting CUDA_VISIBLE_DEVICES settings
- adjust ipyexperiments.utils.mem getter functions to accept an optional torch gpu id
- adjust Makefile to test for conda install/uninstall first, otherwise it breaks pip
-
fix conda channels
-
for consistency renamed helper functions:
gpu_mem_get_free_no_cache_mbs
gpu_mem_get_used_no_cache_mbs
(added_mbs
like all the other functions that return rounded mbs)
- fix requirements - needed pynvml instead of the nvidia-ml-py3 that it relied on earlier
- skipped
- build fix
- require ipython>=6.0.0 (colab still uses ipython==5.5.0, which breaks events)
- py38 is supported (doc)
- fix bug in ipython callback functions definition
- add cl_set_seed=SEED to automatically reset the seed to a fixed value in each cell
- workaround for unreliable peak thread (avoid negative numbers)
- add locking to avoid thread race-condition
- dynamically format column widths in reports
- add , in large numbers (1,000)
- Delta Peaked calculation fixed in case of consumed memory being negative (released)
- switch to pynvx's pynvml wrapper
- fix MANIFEST.in to include project sub-dirs
- use pynvx as pynvml replacement on mac OS (thanks @phenomax)
- switched GPUMemory tuple order to be consistent with nvidia meminfo
- handle the situation where a mem peak thread gets delayed to finish
- some self.namespace.shell.user_ns vars disappear, so do a safe delete
- expose/document more of
ipyexperiments.utils.mem
helper functions
- add
ipyexperiments.utils.ipython
module with workaround for ipython memory leak on exception problem. - add
ipyexperiments.utils.mem
module with memory debug helper utils.
- exp.data and exp.cl.data now both return namedtuples for the top-level entries, so it'll be easier to extend features in the future - documenting only the dictionary access, instead of raw tuples.
- cl: time delta is now printed in hh:mm:ss.msec
- integrate IPyGPULogger as a subsystem (renamed to CellLogger), so now there is no need to install and manage two modules that share a lot of similarities.
- fix a bug where ipyexperiment obj was getting deleted when a cell was rerun
- fix bug in memory Utilization calculation
- on GPU backend loading report the ID, Name and Total RAM of the selected GPU
- print_state now gives an easier to read report
- made the module into proper subclasses, no more global function aliases. So now use directly the desired backend:
IPyExperimentsCPU
,IPyExperimentsPytorch
as an experiments module. It should be trivial now to add other backends. - and
get_stats
method has been replaced withdata
property method, which now returns one or moreIPyExperimentMemory
named tuple(s) depending on the used subclass.
- revert the attempt to account for modified variables - it doesn't work if some function during the experiment modified a variable introduced before the experiment - we surely must not delete it.
- fixed the method for detecting which variables to delete. Moved from comparing a list of variables in the ipython user namespace before and after the experiment (which was missing on variables that were previously defined) via
ipython
'snamespace.who_ls()
to saving the objects for each variable and then comparing whether they have changed at the end of the experiment. So, now, all variables defined during the experiment get correctly deleted (unless explicitly told otherwise).
- replace human size function to do 1024, instead of 1000-computations
- create a test suite - cover the main cpu tests
- add conda/pypi releases
- the core of the project implemented