-
Notifications
You must be signed in to change notification settings - Fork 4.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Disable ROOT automatic class parsing during finishSchedule() #46858
base: master
Are you sure you want to change the base?
Disable ROOT automatic class parsing during finishSchedule() #46858
Conversation
cms-bot internal usage |
please test |
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46858/42887 |
Could you add e.g. "... during ScheduleItems::finishSchedule()` to the PR title? (to make it clear it the automatic parsing is still generally enabled) |
A new Pull Request was created by @fwyzard for master. It involves the following packages:
@Dr15Jones, @makortel, @smuzaffar can you please review it and eventually sign? Thanks. cms-bot commands are listed here |
@wddgit @Dr15Jones Can you think of any drawbacks? (this part of the code should be fully serial) |
finishSchedule()
finishSchedule()
Done, with only only finishSchedule() to stay on one line. |
I cannot think of any problems this would cause. +1 from me. |
+1 Size: This PR adds an extra 40KB to repository Comparison SummarySummary:
|
I'm concerned about placing the guard at that point. This decouples the setting of the behavior far from where the behavior needs to be changed. My suggestion is to
|
Hi @Dr15Jones, sure, I can make the changes you suggest. However, note that there is a single place in the code that calls void setFrozen(std::set<TypeID> const& productTypesConsumed,
std::set<TypeID> const& elementTypesConsumed,
std::string const& processName); which is the one where I would add the call to All other code paths call void setFrozen(bool initializeLookupInfo = true); Do you still prefer to add the extra flag, or should I make the change unconditionally ? |
054f633
to
5d0a065
Compare
If I disable class parsing just around constructing the diff --git a/FWCore/Framework/src/Schedule.cc b/FWCore/Framework/src/Schedule.cc
index 8796f935138..4bede1c5cb6 100644
--- a/FWCore/Framework/src/Schedule.cc
+++ b/FWCore/Framework/src/Schedule.cc
@@ -527,6 +528,8 @@ namespace edm {
assert(0 < prealloc.numberOfStreams());
streamSchedules_.reserve(prealloc.numberOfStreams());
for (unsigned int i = 0; i < prealloc.numberOfStreams(); ++i) {
+ // disable ROOT automatic class parsing while constructing the modules
+ edm::SetClassParsing guard(false);
streamSchedules_.emplace_back(make_shared_noexcept_false<StreamSchedule>(resultsInserter(),
pathStatusInserters_,
endPathStatusInserters_, I get the expected behaviour, that is, a nice error message about the missing dictionary instead of the ROOT message:
Is that a good place for it, or would it cause problems with some other features that actually need the class parsing ? |
I would think it would break any code using the |
How about disabling the class parsing only inside |
Indeed, this seems to work as well: diff --git a/FWCore/Framework/src/ProducerBase.cc b/FWCore/Framework/src/ProducerBase.cc
index 2b28284a83d..0efd433dc87 100644
--- a/FWCore/Framework/src/ProducerBase.cc
+++ b/FWCore/Framework/src/ProducerBase.cc
@@ -69,6 +69,9 @@ namespace edm {
}
TypeLabelList const& plist = typeLabelList();
+ // disable ROOT automatic class parsing while registering the products
+ edm::SetClassParsing guard(false);
+
ProductRegistryHelper::addToRegistry(plist.begin(), plist.end(), md, *(iReg), this, isListener);
if (registrationCallback()) {
iReg->watchProductAdditions(CallbackWrapper(producer, registrationCallback(), iReg, md)); |
Maybe even better inside |
Yes, that also works, and it seems as close to the point of failure as it makes sense to go: diff --git a/FWCore/Framework/src/ProductRegistryHelper.cc b/FWCore/Framework/src/ProductRegistryHelper.cc
index 290cdccaac1..94df82ae265 100644
--- a/FWCore/Framework/src/ProductRegistryHelper.cc
+++ b/FWCore/Framework/src/ProductRegistryHelper.cc
@@ -41,6 +42,9 @@ namespace edm {
std::vector<std::string> producedTypes;
std::set<std::tuple<BranchType, std::type_index, std::string>> registeredProducts;
+ // disable ROOT automatic class parsing while registering the products
+ edm::SetClassParsing guard(false);
+
for (TypeLabelList::const_iterator p = iBegin; p != iEnd; ++p) {
if (p->transition_ == Transition::BeginRun && not iProd->hasAbilityToProduceInBeginRuns()) {
throwProducesWithoutAbility("BeginRun", p->typeID_.userClassName()); |
See #47397 for an alternative approach. |
Add a test to check that the framework handles correctly the case of edm::Wrapper<T> where: - T has a dictionary - edm::Wrapper<T> does not have a dictionary - the corresponding classes.h file includes CUDA headers
ac20e4b
to
568c8b1
Compare
Rebased, and addressed the comments about the new test. |
+code-checks Logs: https://cmssdt.cern.ch/SDT/code-checks/cms-sw-PR-46858/43781
|
Pull request #46858 was updated. @Dr15Jones, @cmsbuild, @fwyzard, @makortel, @smuzaffar can you please check and sign again. |
After more digging I think I found what is the underlying issue: #47400 . |
@cmsbuild, please test |
@fwyzard Would you still happen to have the stack trace available between frames 6 and 25? |
-1 Failed Tests: UnitTests Unit TestsI found 1 errors in the following unit tests: ---> test testMissingDictionaryCUDA had ERRORS Comparison SummarySummary:
|
Here is the full stack trace from a fresh run: Thread 1 "cmsRun" hit Catchpoint 1 (exception thrown), 0x00007ffff712c2f1 in __cxxabiv1::__cxa_throw (obj=0x7fffcc1b5780, tinfo=0x7ffff7ea9690 <typeinfo for edm::Exception>, dest=0x7ffff7e6f4a0 <edm::Exception::~Exception()>) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:81
81 ../../../../libstdc++-v3/libsupc++/eh_throw.cc: No such file or directory.
#0 0x00007ffff712c2f1 in __cxxabiv1::__cxa_throw (obj=0x7fffcc1b5780, tinfo=0x7ffff7ea9690 <typeinfo for edm::Exception>, dest=0x7ffff7e6f4a0 <edm::Exception::~Exception()>) at ../../../../libstdc++-v3/libsupc++/eh_throw.cc:81
#1 0x00007ffff1c63b39 in (anonymous namespace)::RootErrorHandlerImpl(int, char const*, char const*) [clone .cold] () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/pluginFWCoreServicesPlugins.so
#2 0x00007ffff784e374 in ErrorHandler () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCore.so
#3 0x00007ffff779b5e4 in TObject::Error(char const*, char const*, ...) const () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCore.so
#4 0x00007fffe9c9405c in TCling::AutoParseImplRecurse(char const*, bool) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCling.so
#5 0x00007fffe9c9e8ec in TCling::AutoParse(char const*) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCling.so
#6 0x00007fffe9c7148a in TClingLookupHelper__AutoParse(char const*) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCling.so
#7 0x00007fffe9bcae37 in ROOT::TMetaUtils::TClingLookupHelper::GetPartiallyDesugaredNameWithScopeHandling(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, bool) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCling.so
#8 0x00007ffff7849fed in TClassEdit::GetNormalizedName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >&, std::basic_string_view<char, std::char_traits<char> >) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCore.so
#9 0x00007ffff7868160 in TClass::GetClass(char const*, bool, bool, unsigned long, unsigned long) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/external/el8_amd64_gcc12/lib/libCore.so
#10 0x00007ffff6b29101 in edm::TypeWithDict::byName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, long) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreReflection.so
#11 0x00007ffff6b2991f in edm::TypeWithDict::byName(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreReflection.so
#12 0x00007ffff6b7a2d0 in edm::BranchDescription::initFromDictionary() () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libDataFormatsProvenance.so
#13 0x00007ffff6b7a5a3 in edm::BranchDescription::BranchDescription(edm::BranchType const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::TypeWithDict const&, bool, bool, std::set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::less<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&) () from /cvmfs/cms.cern.ch/el8_amd64_gcc12/cms/cmssw/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libDataFormatsProvenance.so
#14 0x00007ffff7c5caae in edm::ProductRegistryHelper::addToRegistry(__gnu_cxx::__normal_iterator<edm::ProductRegistryHelper::TypeLabelItem const*, std::vector<edm::ProductRegistryHelper::TypeLabelItem, std::allocator<edm::ProductRegistryHelper::TypeLabelItem> > > const&, __gnu_cxx::__normal_iterator<edm::ProductRegistryHelper::TypeLabelItem const*, std::vector<edm::ProductRegistryHelper::TypeLabelItem, std::allocator<edm::ProductRegistryHelper::TypeLabelItem> > > const&, edm::ModuleDescription const&, edm::ProductRegistry&, edm::ProductRegistryHelper*, bool) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#15 0x00007ffff7c5fafe in edm::ProducerBase::registerProducts(edm::ProducerBase*, edm::SignallingProductRegistry*, edm::ModuleDescription const&) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#16 0x00007fffcea68cb8 in edm::maker::ModuleHolderT<edm::global::EDProducerBase>::registerProductsAndCallbacks(edm::SignallingProductRegistry*) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/pluginHeterogeneousCoreCUDATestPlugins.so
#17 0x00007ffff7cc0624 in edm::Maker::makeModule(edm::MakeModuleParams const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#18 0x00007ffff7c2a7d8 in edm::Factory::makeModule(edm::MakeModuleParams const&, edm::ModuleTypeResolverMaker const*, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) const () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#19 0x00007ffff7c3b23f in edm::ModuleRegistry::getModule(edm::MakeModuleParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&, edm::signalslot::Signal<void (edm::ModuleDescription const&)>&) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#20 0x00007ffff7cc0dca in edm::WorkerRegistry::getWorker(edm::WorkerParams const&, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#21 0x00007ffff7cc10a5 in edm::WorkerManager::getWorker(edm::ParameterSet&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#22 0x00007ffff7c9a5f0 in edm::(anonymous namespace)::getWorker(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, edm::ParameterSet&, edm::WorkerManager&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#23 0x00007ffff7ca313f in edm::StreamSchedule::fillWorkers(edm::ParameterSet&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, bool, std::vector<edm::WorkerInPath, std::allocator<edm::WorkerInPath> >&, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#24 0x00007ffff7ca9c3f in edm::StreamSchedule::fillTrigPath(edm::ParameterSet&, edm::SignallingProductRegistry&, edm::PreallocationConfiguration const*, std::shared_ptr<edm::ProcessConfiguration const>, int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&, std::shared_ptr<edm::HLTGlobalStatus>, std::vector<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > > const&, edm::ConditionalTaskHelper const&, std::unordered_set<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::hash<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::equal_to<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > >, std::allocator<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > > >&) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#25 0x00007ffff7c9e578 in edm::StreamSchedule::StreamSchedule(std::shared_ptr<edm::TriggerResultInserter>, std::vector<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::PathStatusInserter> > > >&, std::vector<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> >, std::allocator<edm::propagate_const<std::shared_ptr<edm::EndPathStatusInserter> > > >&, std::shared_ptr<edm::ModuleRegistry>, edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::SignallingProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::StreamID, edm::ProcessContext const*) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#26 0x00007ffff7c835da in edm::Schedule::Schedule(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::SignallingProductRegistry&, edm::ExceptionToActionTable const&, std::shared_ptr<edm::ActivityRegistry>, std::shared_ptr<edm::ProcessConfiguration const>, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#27 0x00007ffff7c97991 in edm::ScheduleItems::initModules(edm::ParameterSet&, edm::service::TriggerNamesService const&, edm::PreallocationConfiguration const&, edm::ProcessContext const*, edm::ModuleTypeResolverMaker const*) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#28 0x00007ffff7c012a7 in tbb::detail::d1::function_task<edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy)::{lambda()#1}>::execute(tbb::detail::d1::execution_data&) [clone .lto_priv.0] () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#29 0x00007ffff7e313e1 in tbb::detail::r1::task_dispatcher::local_wait_for_all<false, tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x7ffff40c3e00) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-9b0f135342cfe979b6500c59f501774e/tbb-v2021.9.0/src/tbb/task_dispatcher.h:322
#30 tbb::detail::r1::task_dispatcher::local_wait_for_all<tbb::detail::r1::external_waiter> (waiter=..., t=<optimized out>, this=0x7ffff40c3e00) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-9b0f135342cfe979b6500c59f501774e/tbb-v2021.9.0/src/tbb/task_dispatcher.h:458
#31 tbb::detail::r1::task_dispatcher::execute_and_wait (t=<optimized out>, wait_ctx=..., w_ctx=...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-9b0f135342cfe979b6500c59f501774e/tbb-v2021.9.0/src/tbb/task_dispatcher.cpp:168
#32 0x00007ffff7bcd21f in edm::EventProcessor::init(std::shared_ptr<edm::ProcessDesc>&, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#33 0x00007ffff7bd0718 in edm::EventProcessor::EventProcessor(std::shared_ptr<edm::ProcessDesc>, edm::ServiceToken const&, edm::serviceregistry::ServiceLegacy) () from /data/user/fwyzard/disable_ROOT_automatic_class_parsing/CMSSW_15_0_0_pre3/lib/el8_amd64_gcc12/libFWCoreFramework.so
#34 0x0000000000408369 in tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>::operator()() const ()
#35 0x00007ffff7e1d9ad in tbb::detail::r1::task_arena_impl::execute (ta=..., d=warning: RTTI symbol not found for class 'tbb::detail::d1::task_arena_function<main::{lambda()#1}::operator()() const::{lambda()#1}, void>'
...) at /data/cmsbld/jenkins/workspace/ib-run-pr-tests/testBuildDir/BUILD/el8_amd64_gcc12/external/tbb/v2021.9.0-9b0f135342cfe979b6500c59f501774e/tbb-v2021.9.0/src/tbb/arena.cpp:688
#36 0x000000000040a283 in main::{lambda()#1}::operator()() const ()
#37 0x00000000004051b8 in main () |
PR description:
Disable ROOT automatic class parsing during
ScheduleItems::finishSchedule()
, while checking for missing dictionaries.Add a test to check that the framework handles correctly the case of
edm::Wrapper<T>
where:T
has a dictionary;edm::Wrapper<T>
does not have a dictionary;classes.h
file includes CUDA headers.PR validation:
Unit tests and HLT pass.