-
Notifications
You must be signed in to change notification settings - Fork 9
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
- Loading branch information
Greg Zoller
committed
Nov 5, 2023
1 parent
826a30d
commit ee2cd18
Showing
28 changed files
with
1,915 additions
and
955 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
# Performance | ||
|
||
JSON serialization enchmarks I found in github often measured (IMO) silly things like how fast a parser | ||
could handle a small list of Int. For this benchmark I used a more substantial model + JSON. It's still | ||
not large by any measure, but it does have some nested objects and collections that make it a more | ||
realistic test. | ||
|
||
The test is run via jmh, a popular benchmarking tool. The JVM is stock--not tuned to within an inch | ||
of its life, again to be a more realistic use case. | ||
|
||
Run benchmark from the ScalaJack/benchmark directory (not the main ScalaJack project directory): | ||
``` | ||
sbt "jmh:run -i 10 -wi 10 -f 2 -t 1 co.blocke.WritingBenchmark" | ||
``` | ||
|
||
## Writing Performance: | ||
|
||
| Benchmark | Mode | Cnt | Score | Error | Units | | ||
|------------------|-------|-----|-------------|--------------|-------| | ||
| Hand-Tooled | thrpt | 20 | 2575393.513 | ± 178731.952 | ops/s | | ||
| Circe | thrpt | 20 | 1939339.085 | ± 6279.547 | ops/s | | ||
| ScalaJack 8 | thrpt | 20 | 1703256.521 | ± 12260.518 | ops/s | | ||
| ZIO JSON | thrpt | 20 | 818228.736 | ± 3070.298 | ops/s | | ||
| Argonaut | thrpt | 20 | 716228.404 | ± 6241.145 | ops/s | | ||
| Play JSON | thrpt | 20 | 438538.475 | ± 16319.198 | ops/s | | ||
| ScalaJack 7 | thrpt | 20 | 106292.338 | ± 330.111 | ops/s | | ||
|
||
### Writing Interpretation | ||
|
||
The Hand-Tooled case is straight, manual JSON creation in code. I included it to show a likely | ||
upper-bound on achievable performance. No parser, with whatever logic it must do, can be faster | ||
than hand-tooled code that is hard-wired in its output and requires zero logic. | ||
|
||
We see that both Circe and ScalaJack are very close in performance--close to each other and | ||
shockingly close to hand-tooled code. | ||
|
||
Circe is the gold standard for JSON serializers due to its many features, excellent performance, and | ||
widespread adoption. The one cost Circe imposes is the same one virtually all other serializers | ||
require: that boilerplate be provided to define encoders/decoders to aid the serializion. Circe's | ||
boilerplate is actually not terrible. Others require a fair bit of extra code per class serialized. | ||
|
||
ScalaJack's focus is first and foremost to be frictionless--no drama to the user. The very slight | ||
difference in maximal performance is a worthy expense--its still blazing fast, and positioned well | ||
vs the pack. ScalaJack requires zero boilerplate--you can throw any Scala object at it with no | ||
pre-preparation and it will serialize it. You'll notice the order-of-magnitude improvement ScalaJack 8 | ||
has over ScalaJack 7, due to moving everything possible into compile-time macros for speed. |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,59 @@ | ||
ThisBuild / organization := "co.blocke" | ||
|
||
val compilerOptions = Seq( | ||
"-deprecation", | ||
"-encoding", | ||
"UTF-8", | ||
"-feature", | ||
"-language:existentials", | ||
"-language:higherKinds", | ||
"-unchecked", | ||
"-Ywarn-dead-code", | ||
"-Ywarn-numeric-widen", | ||
"-Xfuture" | ||
) | ||
|
||
val circeVersion = "0.15.0-M1" | ||
val scalaTestVersion = "3.2.11" | ||
ThisBuild / scalaVersion := "3.3.0" | ||
|
||
def priorTo2_13(scalaVersion: String): Boolean = | ||
CrossVersion.partialVersion(scalaVersion) match { | ||
case Some((2, minor)) if minor < 13 => true | ||
case _ => false | ||
} | ||
|
||
val baseSettings = Seq( | ||
scalacOptions ++= compilerOptions | ||
) | ||
|
||
lazy val benchmark = project | ||
.in(file(".")) | ||
.settings(baseSettings ++ noPublishSettings) | ||
.settings( | ||
libraryDependencies ++= Seq( | ||
"io.circe" %% "circe-core", | ||
"io.circe" %% "circe-generic", | ||
"io.circe" %% "circe-parser" | ||
).map(_ % circeVersion), | ||
libraryDependencies ++= Seq( | ||
"org.playframework" %% "play-json" % "3.0.1", | ||
"io.argonaut" %% "argonaut" % "6.3.9", | ||
"co.blocke" %% "scalajack" % "826a30_unknown", | ||
"co.blocke" %% "scala-reflection" % "sj_fixes_edbef8", | ||
"dev.zio" %% "zio-json" % "0.6.1", | ||
// "io.circe" %% "circe-derivation" % "0.15.0-M1", | ||
// "io.circe" %% "circe-jackson29" % "0.14.0", | ||
// "org.json4s" %% "json4s-jackson" % "4.0.4", | ||
// "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-core" % "2.13.17", | ||
// "com.github.plokhotnyuk.jsoniter-scala" %% "jsoniter-scala-macros" % "2.13.17", | ||
"org.scalatest" %% "scalatest" % scalaTestVersion % Test | ||
) | ||
) | ||
.enablePlugins(JmhPlugin) | ||
|
||
lazy val noPublishSettings = Seq( | ||
publish := {}, | ||
publishLocal := {}, | ||
publishArtifact := false | ||
) |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
sbt.version=1.9.6 |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1 @@ | ||
addSbtPlugin("pl.project13.scala" % "sbt-jmh" % "0.4.6") |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,26 @@ | ||
package co.blocke | ||
|
||
import argonaut._, Argonaut._ | ||
import org.openjdk.jmh.annotations._ | ||
|
||
|
||
implicit val CodecPet: CodecJson[Pet] = | ||
casecodec3(Pet.apply, (a: Pet) => Option((a.name, a.species, a.age)))("name","species","age") | ||
|
||
implicit val CodecFriend: CodecJson[Friend] = | ||
casecodec3(Friend.apply, (a: Friend) => Option((a.name, a.age, a.email)))("name","age","email") | ||
|
||
implicit val CodecAddress: CodecJson[Address] = | ||
casecodec4(Address.apply, (a: Address) => Option((a.street, a.city, a.state, a.postal_code)))("street","city","state","postal_code") | ||
|
||
implicit val CodecPerson: CodecJson[Person] = | ||
casecodec6(Person.apply, (a: Person) => Option((a.name, a.age, a.address, a.email, a.phone_numbers, a.is_employed)))("name", "age","address","email","phone_numbers","is_employed") | ||
|
||
implicit val CodecRecord: CodecJson[Record] = | ||
casecodec4(Record.apply, (a: Record) => Option((a.person, a.hobbies, a.friends, a.pets)))("person", "hobbies", "friends", "pets") | ||
|
||
|
||
trait ArgonautWritingBenchmark { | ||
@Benchmark | ||
def writeRecordArgonaut = record.asJson | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,96 @@ | ||
package co.blocke | ||
|
||
import org.openjdk.jmh.annotations._ | ||
|
||
import java.util.concurrent.TimeUnit | ||
|
||
import co.blocke.scalajack.* | ||
|
||
import io.circe.syntax.* | ||
import io.circe.* | ||
import io.circe.generic.semiauto.* | ||
|
||
val record = ScalaJack.read[Record](jsData) | ||
|
||
implicit val recordDecoder: Decoder[Record] = deriveDecoder[Record] | ||
implicit val recordEncoder: Encoder[Record] = deriveEncoder[Record] | ||
|
||
implicit val personDecoder: Decoder[Person] = deriveDecoder[Person] | ||
implicit val personEncoder: Encoder[Person] = deriveEncoder[Person] | ||
|
||
implicit val addressDecoder: Decoder[Address] = deriveDecoder[Address] | ||
implicit val addressEncoder: Encoder[Address] = deriveEncoder[Address] | ||
|
||
implicit val friendDecoder: Decoder[Friend] = deriveDecoder[Friend] | ||
implicit val friendEncoder: Encoder[Friend] = deriveEncoder[Friend] | ||
|
||
implicit val petDecoder: Decoder[Pet] = deriveDecoder[Pet] | ||
implicit val petEncoder: Encoder[Pet] = deriveEncoder[Pet] | ||
|
||
|
||
|
||
trait CirceReadingBenchmark{ | ||
val circeJson = record.asJson | ||
def readRecordCirce = circeJson.as[Record] | ||
} | ||
|
||
// trait ScalaJackReadingBenchmark{ | ||
// def readRecordScalaJack = ScalaJack.read[Record](jsData) | ||
// } | ||
|
||
trait CirceWritingBenchmark { | ||
@Benchmark | ||
def writeRecordCirce = record.asJson | ||
} | ||
|
||
trait ScalaJackWritingBenchmark { | ||
@Benchmark | ||
def writeRecordScalaJack = ScalaJack.write(record) | ||
} | ||
|
||
trait HandTooledWritingBenchmark { | ||
@Benchmark | ||
def writeRecordHandTooled = | ||
val sb = new StringBuilder() | ||
sb.append("{") | ||
sb.append("\"person\":{") | ||
sb.append("\"name:\":\""+record.person.name+"\",") | ||
sb.append("\"age:\":"+record.person.age+",") | ||
sb.append("\"address:\":{\"street\":"+record.person.address.street+"\",") | ||
sb.append("\"city\":\""+record.person.address.city+"\",") | ||
sb.append("\"state\":\""+record.person.address.state+"\",") | ||
sb.append("\"postal_code\":\""+record.person.address.postal_code+"\"},") | ||
sb.append("\"email:\":\""+record.person.email+"\",") | ||
sb.append("\"phone_numbers:\":[") | ||
record.person.phone_numbers.map(p => sb.append("\""+p+"\",")) | ||
sb.append("],") | ||
sb.append("\"is_employed:\":"+record.person.is_employed+"},") | ||
sb.append("\"hobbies:\":[") | ||
record.hobbies.map(p => sb.append("\""+p+"\",")) | ||
sb.append("],") | ||
sb.append("\"friends:\":[") | ||
record.friends.map(f=>sb.append(s"""{"name":"${f.name},"age":${f.age},"email":"${f.email}"},""")) | ||
sb.append("],") | ||
sb.append("\"pets:\":[") | ||
record.pets.map(f=>sb.append(s"""{"name":"${f.name},"species":"${f.species}"","age":${f.age}},""")) | ||
sb.append("]}") | ||
sb.toString | ||
} | ||
|
||
// @State(Scope.Thread) | ||
// @BenchmarkMode(Array(Mode.Throughput)) | ||
// @OutputTimeUnit(TimeUnit.SECONDS) | ||
// class ReadingBenchmark | ||
// extends CirceReadingBenchmark | ||
// with ScalaJackReadingBenchmark | ||
|
||
@State(Scope.Thread) | ||
@BenchmarkMode(Array(Mode.Throughput)) | ||
@OutputTimeUnit(TimeUnit.SECONDS) | ||
class WritingBenchmark | ||
extends CirceWritingBenchmark | ||
with ScalaJackWritingBenchmark | ||
with HandTooledWritingBenchmark | ||
with ArgonautWritingBenchmark | ||
with PlayWritingBenchmark | ||
with ZIOJsonWritingBenchmark |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
package co.blocke | ||
|
||
import play.api.libs.json._ | ||
import play.api.libs.json.Reads._ | ||
import play.api.libs.functional.syntax._ | ||
import org.openjdk.jmh.annotations._ | ||
|
||
implicit val friendWrites: Writes[Friend] = ( | ||
(JsPath \ "name").write[String] and | ||
(JsPath \ "age").write[Int] and | ||
(JsPath \ "email").write[String] | ||
)(unlift((a: Friend) => Option((a.name, a.age, a.email)))) | ||
|
||
implicit val petWrites: Writes[Pet] = ( | ||
(JsPath \ "name").write[String] and | ||
(JsPath \ "species").write[String] and | ||
(JsPath \ "age").write[Int] | ||
)(unlift((a: Pet) => Option((a.name, a.species, a.age)))) | ||
|
||
implicit val addressWrites: Writes[Address] = ( | ||
(JsPath \ "street").write[String] and | ||
(JsPath \ "city").write[String] and | ||
(JsPath \ "state").write[String] and | ||
(JsPath \ "postal_code").write[String] | ||
)(unlift((a: Address) => Option((a.street, a.city, a.state, a.postal_code)))) | ||
|
||
implicit val personWrites: Writes[Person] = ( | ||
(JsPath \ "namet").write[String] and | ||
(JsPath \ "age").write[Int] and | ||
(JsPath \ "address").write[Address] and | ||
(JsPath \ "email").write[String] and | ||
(JsPath \ "phone_numbers").write[List[String]] and | ||
(JsPath \ "is_employed").write[Boolean] | ||
)(unlift((a: Person) => Option((a.name, a.age, a.address, a.email, a.phone_numbers, a.is_employed)))) | ||
|
||
implicit val recordWrites: Writes[Record] = ( | ||
(JsPath \ "person").write[Person] and | ||
(JsPath \ "hobbies").write[List[String]] and | ||
(JsPath \ "friends").write[List[Friend]] and | ||
(JsPath \ "pets").write[List[Pet]] | ||
)(unlift((a: Record) => Option((a.person, a.hobbies, a.friends, a.pets)))) | ||
|
||
trait PlayWritingBenchmark { | ||
@Benchmark | ||
def writeRecordPlay = Json.toJson(record) | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,86 @@ | ||
package co.blocke | ||
|
||
case class Person( | ||
name: String, | ||
age: Int, | ||
address: Address, | ||
email: String, | ||
phone_numbers: List[String], | ||
is_employed: Boolean | ||
) | ||
|
||
case class Address( | ||
street: String, | ||
city: String, | ||
state: String, | ||
postal_code: String | ||
) | ||
|
||
case class Friend( | ||
name: String, | ||
age: Int, | ||
email: String | ||
) | ||
|
||
case class Pet( | ||
name: String, | ||
species: String, | ||
age: Int | ||
) | ||
|
||
case class Record( | ||
person: Person, | ||
hobbies: List[String], | ||
friends: List[Friend], | ||
pets: List[Pet] | ||
) | ||
|
||
|
||
val jsData = | ||
"""{ | ||
"person": { | ||
"name": "John Doe", | ||
"age": 30, | ||
"address": { | ||
"street": "123 Main Street", | ||
"city": "Anytown", | ||
"state": "CA", | ||
"postal_code": "12345" | ||
}, | ||
"email": "[email protected]", | ||
"phone_numbers": [ | ||
"555-555-5555", | ||
"555-123-4567" | ||
], | ||
"is_employed": true | ||
}, | ||
"hobbies": [ | ||
"reading", | ||
"swimming", | ||
"traveling" | ||
], | ||
"friends": [ | ||
{ | ||
"name": "Jane Smith", | ||
"age": 28, | ||
"email": "[email protected]" | ||
}, | ||
{ | ||
"name": "Bob Johnson", | ||
"age": 32, | ||
"email": "[email protected]" | ||
} | ||
], | ||
"pets": [ | ||
{ | ||
"name": "Fido", | ||
"species": "Dog", | ||
"age": 5 | ||
}, | ||
{ | ||
"name": "Whiskers", | ||
"species": "Cat", | ||
"age": 3 | ||
} | ||
] | ||
}""" |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,8 @@ | ||
package co.blocke | ||
|
||
case class Foo() extends ZIOJsonWritingBenchmark | ||
|
||
object RunMe extends App: | ||
|
||
val f = Foo() | ||
println(f.writeRecordZIOJson) |
Oops, something went wrong.