JSON library options

[trws]

This is the promised issue for json library discussion, copying in @milroy and @garlick from the qmanager stats PR. I have data in case anyone is interested. I took a copy of the nativejson-benchmark by miloyip and made it work with gcc 14 and current versions of the various libraries that make sense (minus folly, not sure why but that test just wont run) and ran up the full suite. I also created a protobuf file and a structure to represent a JGF R to use the Protobuf3 canonical json support and glaze automatic structure (de)serialization for parse and stringify, the other benchmarks don't really apply to those because they only do a complete object rather than streaming parsing. If you want to look at everything, the whole website generated by the benchmark, including tables of precise numbers, is here:
result.zip

Here are the major results, largely influenced by a 400mb json file courtesy of @jameshcorbett. The 800mb one @grondo recommended ran my VM out of memory (it's only provisioned 14GB).

Updated to add results for parse and serialize to binary formats for glaze, proto3, and nlohmann. Why anyone uses protobufs is now beyond me...

In terms of performance, if we can use the structured parsing, then glaze wins by a good bit compared to the other ones that we could reasonably use. Sadly nlohmann/json, which is the nicest to use with unknown values and supports the most formats, is pretty slow for parsing. That might be improved by making it use a flat_map, but that proved difficult. Surprising to me is that the protobuf json support is so slow. I suppose if we used the actual protobuf binary format it would be much faster, but did not test that. I think I'm leaning toward either Boost::Json or Glaze, the main tradeoff between these is that glaze's current versions require C++23, and thus g++>=12 clang>=15. It looks like all our supported images have at least that, so maybe that's ok, but it's worth considering. The compile time and memory use of Boost::Json is better than that of glaze without having compile-time structure information as well, so there's that. The jansson stringify performance and memory overhead with large json were big surprises as well.

Just so you can get an idea, here's the actual structured glaze code I used for testing:

Structured glaze code

#include <fstream>
#include <iostream>
#include <map>
#include <string>
#include <vector>
#include <stdint.h>
#include <optional>

#include <glaze/glaze.hpp>

using namespace std;
std::string file(std::string path)
{
  std::ifstream fp(path, std::ios::in | std::ios::binary | std::ios::ate);
  const auto size = fp.tellg();
  std::string s;
  s.resize(size);
  fp.seekg(0u);
  fp.read(&s[0], size);
  return s;
}
struct Rlite {
  string rank;
  map<string,string> children;
};
struct Rexecution {
  std::vector<Rlite> R_lite;
  uint64_t starttime;
  uint64_t expiration;
  std::vector<string> nodelist;
  map<string, string> properties;
};
struct Rscheduling {
  struct Node {
    struct Metadata {
      string type;
      string basename;
      string name;
      int64_t id;
      int64_t uniq_id;
      int64_t rank;
      bool exclusive;
      string unit;
      int64_t size;
      std::optional<map<string,string>> properties;
      map<string,string> paths;
      std::optional<int32_t> status;
    };
    string id;
    Metadata metadata;
  };
  struct Edge {
    struct Metadata {
      map<string,string> name;
    };
    string source;
    string target;
    bool directed;
    Metadata metadata;
  };
  struct RschedulingGraph {
    bool directed;
    std::vector<Node> nodes;
    std::vector<Edge> edges;
  };
  RschedulingGraph graph;
};
struct RJGF {
  int32_t version;
  Rexecution execution;
  Rscheduling scheduling;
};

#include <sys/time.h>
#include <sys/types.h>

class Timer {
  static u_int64_t count;
  u_int64_t id_;
  struct timeval start_time_;
  string name_;

  public:
  Timer(const string& name)
      : id_(count)
      , name_(name)
  {
    ++count;
    cout << "start timer:" << id_ << " " << name_ << endl;
    gettimeofday(&start_time_, NULL);
  }

  ~Timer()
  {
    struct timeval t;
    gettimeofday(&t, NULL);
    u_int64_t time = (u_int64_t)(t.tv_sec - start_time_.tv_sec) * 1000000
        + (t.tv_usec - start_time_.tv_usec);
    cout << "end timer:" << id_ << " " << name_ << " took:" << time/1000.0 << endl;
  }
};
u_int64_t Timer::count;

int main(int argc, char* argv[])
{
  RJGF r;
  std::string s;
  {
    auto t = Timer("read file");
    s = file(argv[1]);
  }
  {
    auto t = Timer("parse json to message");
    auto status = glz::read_json(r, s);
    if (status)
      std::cout << glz::format_error(status, s) << std::endl;
  }
  {
    auto t = Timer("serialize");
    std::string res_str = glz::write_json(r).value_or("error");
  }
  {
    auto t = Timer("prettify");
    std::string res_str;
    auto status = glz::write<glz::opts{.prettify = true}>(r, res_str);
    if (status)
      std::cout << glz::format_error(status, res_str) << std::endl;
  }
}

[trws]

Hm, actually RapidJSON might be more of an option than I initially thought too. I used it once long ago, and it was fast but had a really rough interface. Now the AutoUTF setup and some added-on C++11 is surprisingly ok. It's not quite as nice as others but it's fast, has built-in jsonschema support, and a third-party built an initializer-list builder for it. That might hit the performance/functionality sweet spot, especially since it's pretty ubiquitous. If we want a dependency that we can easily get on all platforms, that one might beat out boost, where boost::json we could vendor pretty easily.