You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If you're using simdjson to parse multiple documents, or in a loop, you should make a parser once and reuse it. The simdjson library will allocate and retain internal buffers between parses, keeping buffers hot in cache and keeping memory allocation and initialization to a minimum. In this manner, you can parse terabytes of JSON data without doing any new allocation.
class simdjson::dom::parser only provides set_max_depth(), allocate(), but not set_capacity(). So to set just the max depth, only call allocate() if the depth actually changed, which should be infrequent
parser::parse_into_document calls ensure_capacity already, and ensure_capacity calls allocate if needed
Closescrazyxman#80 - simdjson_is_valid() and other PHP functions would
previously return false when out of memory
- Related to crazyxman#60 - other php apis (using emalloc instead) will also emit
fatal errors when out of memory and end the process.
Closescrazyxman#79 - reuse buffers for strings less than 1000000 bytes and
100000 depth. (Assumes the depth rarely changes in callers)
https://github.com/simdjson/simdjson/blob/master/doc/dom.md#reusing-the-parser-for-maximum-efficiency
class simdjson::dom::parser
only provides set_max_depth(), allocate(), but not set_capacity(). So to set just the max depth, only call allocate() if the depth actually changed, which should be infrequentparser::parse_into_document
calls ensure_capacity already, and ensure_capacity calls allocate if neededRelated to #73
Note that simdjson will not need capacities beyond the range of a uint32, and will reject requests for larger capacities
The text was updated successfully, but these errors were encountered: