Rapidjson crash when attempting to allocate memory

[FlorentP42]

Since PR #1488 it appears that any plugin using Rapidjson will crash immediately when attempting to open any document with a similar stack trace :

Example from fledgepower-south-iec104 :

Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (14) 14 0x55930b8ba4fe _start + 46---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (13) 13 0x7fb266812083 __libc_start_main + 243---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (12) 12 0x55930b8b99b7 main + 631---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (11) 11 0x55930b8d18c6 SouthService::start(std::__cxx11::basic_string, std::allocator >&, unsigned short) + 1014---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (10) 10 0x55930b8d1071 SouthService::loadPlugin() + 961---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (9) 9 0x55930b913e56 SouthPlugin::SouthPlugin(void*, ConfigCategory const&) + 150---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (8) 8 0x7fb2640d7c54 plugin_init + 948---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (7) 7 0x7fb2640b1953 IEC104::setJsonConfig(std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&) + 419---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (6) 6 0x7fb2640c5a55 IEC104ClientConfig::importProtocolConfig(std::__cxx11::basic_string, std::allocator > const&) + 357---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (5) 5 0x55930b8f56ac rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator>& rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator>::ParseStream<0u, rapidjson::UTF8, rapidjson::GenericStringStream > >(rapidjson::GenericStringStream >&) + 172---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (4) 4 0x55930b8e2565 void rapidjson::GenericReader, rapidjson::UTF8, rapidjson::CrtAllocator>::ParseObject<0u, rapidjson::GenericStringStream >, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator> >(rapidjson::GenericStringStream >&, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator>&) + 261---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (3) 3 0x55930b8dfb6d void rapidjson::GenericReader, rapidjson::UTF8, rapidjson::CrtAllocator>::ParseString<0u, rapidjson::GenericStringStream >, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator> >(rapidjson::GenericStringStream >&, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator>&, bool) + 509---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (2) 2 0x55930b8da974 rapidjson::MemoryPoolAllocator::Malloc(unsigned long) + 36---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (1) 1 0x7fb266831090 /lib/x86_64-linux-gnu/libc.so.6(+0x43090) [0x7fb266831090]---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: (0) 0 0x55930b8ca9f2 handler(int) + 82---------
Nov 5 12:28:08 7ffe61e270f3 Fledge iec104south_s1[701]: FATAL: Signal 11 (Segmentation fault) trapped: 

Example from fledgepower-south-hnz :

Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (14) 14 0x56398a2ef4fe _start + 46---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (13) 13 0x7fe19999b083 __libc_start_main + 243---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (12) 12 0x56398a2ee9b7 main + 631---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (11) 11 0x56398a3068c6 SouthService::start(std::__cxx11::basic_string, std::allocator >&, unsigned short) + 1014---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (10) 10 0x56398a306071 SouthService::loadPlugin() + 961---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (9) 9 0x56398a348e56 SouthPlugin::SouthPlugin(void*, ConfigCategory const&) + 150---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (8) 8 0x7fe198266fd1 plugin_init + 481---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (7) 7 0x7fe19824911c HNZ::reconfigure(ConfigCategory const&) + 412---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (6) 6 0x7fe198248e5b HNZ::setJsonConfig(std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&, std::__cxx11::basic_string, std::allocator > const&) + 1291---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (5) 5 0x7fe198254ee3 HNZConf::importConfigJson(std::__cxx11::basic_string, std::allocator > const&) + 643---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (4) 4 0x56398a317565 void rapidjson::GenericReader, rapidjson::UTF8, rapidjson::CrtAllocator>::ParseObject<0u, rapidjson::GenericStringStream >, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator> >(rapidjson::GenericStringStream >&, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator>&) + 261---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (3) 3 0x56398a314b6d void rapidjson::GenericReader, rapidjson::UTF8, rapidjson::CrtAllocator>::ParseString<0u, rapidjson::GenericStringStream >, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator> >(rapidjson::GenericStringStream >&, rapidjson::GenericDocument, rapidjson::MemoryPoolAllocator, rapidjson::CrtAllocator>&, bool) + 509---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (2) 2 0x56398a30f974 rapidjson::MemoryPoolAllocator::Malloc(unsigned long) + 36---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (1) 1 0x7fe1999ba090 /lib/x86_64-linux-gnu/libc.so.6(+0x43090) [0x7fe1999ba090]---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: (0) 0 0x56398a2ff9f2 handler(int) + 82---------
Nov 5 12:31:47 7ffe61e270f3 Fledge hnzsouth_s1[2391]: FATAL: Signal 11 (Segmentation fault) trapped: 

Could you please rollback to a version that is not broken?

[MarkRiddoch]

Are you still seeing this issue? With the latest updates I am not seeign it for other plugins using JSON config items. If so let me know and I will pull the plugins from FledgePower and see if I can reproduce it that way.

[FlorentP42]

@MarkRiddoch I have not made a new attempt with the develop version recently as I was testing on the tagged 2.5.0 version of Fledge and no longer on develop. But if there was no change on the rapidjson library of develop since two weeks ago, I doubt this may produce different results now...
Also I remember the FledgePower container produced by this pipeline was pointing to develop when getting the rapidjson sources and definitely contained the issue: https://github.com/fledge-power/fledgepower-deployment/actions/runs/11707131150/job/32605786603
Later generations are pointing to the 2.5.0 tag so we no longer experience it.

[kwitekrac]

I've also seen this error. I usually build Fledge using an older Dockerfile version (PMC-v1.2), but my target version is Fledge 2.5.0 release. To fix this issue, I need to copy the changes made to fledge-install-include.sh from this commit: fledge-power/fledgepower-deployment@d1ca352#diff-f5c59ff9b5f3845793ff2d22bfcc793d206cbcef489f42db4cdfd2633221bf49R6

[MarkRiddoch]

I suspect this is an issue caused when plugins and the core services are compiled against different versions of RapidJSON. Building everything with the same version we do not see the problem.

[FlorentP42]

This could make sense, we will see if this is still happening when integrating the next Fledge version then.