Benchmarks

[ZJONSSON]

Perhaps this is not the right place for this, but I think centralized benchmarks (using the promises-tests adaptors) would be a great addition under promises-aplus, given general lack of proper benchmarks in the wild (see for example thanpolas/perf-promises#2)

[stefanpenner]

+1

[johnyanarella]

👍

[briancavalier]

I do think a generalized, and properly reviewed, benchmarking tool for promises would be a good thing. As we've seen, if you create a benchmark, people will try to win at it, which means that people will innovate to make promises faster.

That said, I'm not sure that a standards organization such as Promises/A+ is the right place to house such a tool. I bet members would be willing to contribute, review, etc., tho. I'd certainly be willing, but I can't dedicate a large amount of time to it.

[domenic]

Yeah, I agree with @briancavalier I think. Promises/A+ community members will almost certainly be involved in this effort, which I am probably going to start soon, but it's not something we'd like to necessarily "endorse" as an organization.

[ForbesLindesay]

I actually started something today but realized so far it's no better than anything that already exists. I think the difficulty is that we keep creating really tiny micro benchmarks. What we need is to come up with something that's a genuinely realistic approximation of how promises are used.

I'm sure lots of us would be happy to collaborate on something like this. Personally I'd be happy to see it as part of the Promises/A+ organisation.

[ForbesLindesay]

I'd really love to see a benchmark created by someone who doesn't also maintain a promise library. People have a tenancy to "win" their own benchmarks. Alternatively someone like @kriskowal could make one since Q is unlikely to win on speed as that's not its priority.

[briancavalier]

I agree with @ForbesLindesay that a benchmark tool shouldn't be driven primarily by one library's owner(s), and ultimately, it needs to be developed in the open.

[gobwas]

I think we still need this benchmarks... Any news? 😎