syncedmem.hpp:31] Check failed: error == cudaSuccess (29 vs. 0) driver shutting down

[Eleanor-H]

*** Check failure stack trace: ***
@ 0x7fa08bf23daa (unknown)
@ 0x7fa08bf23ce4 (unknown)
@ 0x7fa08bf236e6 (unknown)
@ 0x7fa08bf26687 (unknown)
@ 0x7fa08c5791e1 caffe::SyncedMemory::~SyncedMemory()
@ 0x7fa08c5c6fb2 boost::detail::sp_counted_impl_p<>::dispose()
@ 0x40a52e boost::detail::sp_counted_base::release()
@ 0x7fa08c5df1e5 caffe::Blob<>::~Blob()
@ 0x7fa08aa8753a (unknown)
@ 0x7fa08c50ac43 (unknown)
Aborted (core dumped)

I met this issue each time at the end of training or testing. Any idea?