Newworld (#234)

* first success

* config ability added

* core json write working

* ugly but inTermsOf works

* Added caching for trait writers

* writing json is working

* Rudimentary read working

* fixed seq parsing

* enum parse works

* Classes with default values work

* SelfRefs working

* Maps and more work

* modularize reader

* tests started

* New faster writer

* fix benchmark table

* more fixes

* more fixes

* more write types

* More write progress

* sealed traits and more

* most of write finished

* ZIO based parser experiments

* checkpoint

* checkpoint

* performance jump on writes

* big perf bump

* another big perf bump

* final write optimization

* Update README.md

* Update README.md

* readme fidgeting

* JsonConfig initial working

* about a third thru write tests

* more tests and fixes

* basic write tests complete

* NeoType writes work

* writes done

* First tentative steps into reading

* Got rudimentary class parsing working

* reworked read codegen

* test of Jsoniter parse string

* Massive performance boost

* Another massive performance improvement

* checkpoint

* more perf tweaks

* lastest tweaks

* More write performance improvements

* implement read primitives

* Continue read implementations

* more read cases implement

* fine tuning either and option

* checkpoint-lots of fixes and problems solved

* Java collections and maps

* Many fixes for classes

* enum tests done

* almost code complete

* docs and fixes

* lots of docs and a refactor of SJConfig

* doc tweak

* initial msgpack support

* remove msgpack

* wrapping up

* coveralls tweak

* tweak readme

---------

Co-authored-by: Greg Zoller <[email protected]>

.github/workflows/ci.yml

@@ -1,41 +1,119 @@
-name: Package Build
-# This workflow is triggered on pushes to the repository.
-on: [pull_request]
+# This file was automatically generated by sbt-github-actions using the
+# githubWorkflowGenerate task. You should add and commit this file to
+# your git repository. It goes without saying that you shouldn't edit
+# this file by hand! Instead, if you wish to make changes, you should
+# change your sbt build configuration to revise the workflow description
+# to meet your needs, then regenerate this file.
+
+name: Continuous Integration
+
+on:
+  pull_request:
+    branches: ['**']
+  push:
+    branches: ['**']
+    tags: [v*]
+
+env:
+  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+
+concurrency:
+  group: ${{ github.workflow }} @ ${{ github.ref }}
+  cancel-in-progress: true
 
 jobs:
-  build-and-test:
-    runs-on: ubuntu-latest
+  build:
     name: Build and Test
-
+    strategy:
+      matrix:
+        os: [ubuntu-latest]
+        scala: [3.4.2]
+        java: [zulu@21]
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 60
     steps:
-      - name: Ignore line ending differences in git
-        if: contains(runner.os, 'windows')
-        shell: bash
-        run: git config --global core.autocrlf false
+      - name: Checkout current branch (full)
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Setup Java (zulu@21)
+        id: setup-java-zulu-21
+        if: matrix.java == 'zulu@21'
+        uses: actions/setup-java@v4
+        with:
+          distribution: zulu
+          java-version: 21
+          cache: sbt
 
+      - name: sbt update
+        if: matrix.java == 'zulu@21' && steps.setup-java-zulu-21.outputs.cache-hit == 'false'
+        run: sbt +update
+
+      - name: Check that workflows are up to date
+        run: sbt githubWorkflowCheck
+
+      - name: Build project
+        run: sbt '++ ${{ matrix.scala }}' test
+
+      - name: Make target directories
+        if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
+        run: mkdir -p target project/target
+
+      - name: Compress target directories
+        if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
+        run: tar cf targets.tar target project/target
+
+      - name: Upload target directories
+        if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
+        uses: actions/upload-artifact@v4
+        with:
+          name: target-${{ matrix.os }}-${{ matrix.java }}-${{ matrix.scala }}
+          path: targets.tar
+
+  publish:
+    name: Publish Artifacts
+    needs: [build]
+    if: github.event_name != 'pull_request' && (github.ref == 'refs/heads/main' || startsWith(github.ref, 'refs/tags/v'))
+    strategy:
+      matrix:
+        os: [ubuntu-latest]
+        java: [zulu@21]
+    runs-on: ${{ matrix.os }}
+    steps:
       - name: Checkout current branch (full)
-        uses: actions/checkout@v3
+        uses: actions/checkout@v4
         with:
           fetch-depth: 0
 
-      - name: Setup Java (zulu@13)
-        uses: actions/setup-java@v3
+      - name: Setup Java (zulu@21)
+        id: setup-java-zulu-21
+        if: matrix.java == 'zulu@21'
+        uses: actions/setup-java@v4
         with:
           distribution: zulu
-          java-version: 13
+          java-version: 21
+          cache: sbt
 
-      - name: Cache sbt
-        uses: actions/cache@v3
+      - name: sbt update
+        if: matrix.java == 'zulu@21' && steps.setup-java-zulu-21.outputs.cache-hit == 'false'
+        run: sbt +update
+
+      - name: Download target directories (3.4.2)
+        uses: actions/download-artifact@v4
         with:
-          path: |
-            ~/.sbt
-            ~/.ivy2/cache
-            ~/.coursier/cache/v1
-            ~/.cache/coursier/v1
-            ~/AppData/Local/Coursier/Cache/v1
-            ~/Library/Caches/Coursier/v1
-          key: ${{ runner.os }}-sbt-cache-v2-${{ hashFiles('**/*.sbt') }}-${{ hashFiles('project/build.properties') }}
+          name: target-${{ matrix.os }}-${{ matrix.java }}-3.4.2
 
-      - name: Build project
-        shell: bash
-        run: sbt test
+      - name: Inflate target directories (3.4.2)
+        run: |
+          tar xf targets.tar
+          rm targets.tar
+
+      - env:
+          CI_SNAPSHOT_RELEASE: +publishSigned
+          PGP_PASSPHRASE: ${{ secrets.PGP_PASSPHRASE }}
+          SONATYPE_PASSWORD: ${{ secrets.SONATYPE_PASSWORD }}
+          SONATYPE_USERNAME: ${{ secrets.SONATYPE_USERNAME }}
+          PGP_SECRET: ${{ secrets.PGP_SECRET }}
+        run: sbt ci-release

.github/workflows/clean.yml

@@ -0,0 +1,59 @@
+# This file was automatically generated by sbt-github-actions using the
+# githubWorkflowGenerate task. You should add and commit this file to
+# your git repository. It goes without saying that you shouldn't edit
+# this file by hand! Instead, if you wish to make changes, you should
+# change your sbt build configuration to revise the workflow description
+# to meet your needs, then regenerate this file.
+
+name: Clean
+
+on: push
+
+jobs:
+  delete-artifacts:
+    name: Delete Artifacts
+    runs-on: ubuntu-latest
+    env:
+      GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+    steps:
+      - name: Delete artifacts
+        run: |
+          # Customize those three lines with your repository and credentials:
+          REPO=${GITHUB_API_URL}/repos/${{ github.repository }}
+
+          # A shortcut to call GitHub API.
+          ghapi() { curl --silent --location --user _:$GITHUB_TOKEN "$@"; }
+
+          # A temporary file which receives HTTP response headers.
+          TMPFILE=/tmp/tmp.$$
+
+          # An associative array, key: artifact name, value: number of artifacts of that name.
+          declare -A ARTCOUNT
+
+          # Process all artifacts on this repository, loop on returned "pages".
+          URL=$REPO/actions/artifacts
+          while [[ -n "$URL" ]]; do
+
+            # Get current page, get response headers in a temporary file.
+            JSON=$(ghapi --dump-header $TMPFILE "$URL")
+
+            # Get URL of next page. Will be empty if we are at the last page.
+            URL=$(grep '^Link:' "$TMPFILE" | tr ',' '\n' | grep 'rel="next"' | head -1 | sed -e 's/.*<//' -e 's/>.*//')
+            rm -f $TMPFILE
+
+            # Number of artifacts on this page:
+            COUNT=$(( $(jq <<<$JSON -r '.artifacts | length') ))
+
+            # Loop on all artifacts on this page.
+            for ((i=0; $i < $COUNT; i++)); do
+
+              # Get name of artifact and count instances of this name.
+              name=$(jq <<<$JSON -r ".artifacts[$i].name?")
+              ARTCOUNT[$name]=$(( $(( ${ARTCOUNT[$name]} )) + 1))
+
+              id=$(jq <<<$JSON -r ".artifacts[$i].id?")
+              size=$(( $(jq <<<$JSON -r ".artifacts[$i].size_in_bytes?") ))
+              printf "Deleting '%s' #%d, %'d bytes\n" $name ${ARTCOUNT[$name]} $size
+              ghapi -X DELETE $REPO/actions/artifacts/$id
+            done
+          done

.github/workflows/coveralls.yml

@@ -0,0 +1,59 @@
+name: Coveralls Publish
+
+on:
+  push:
+    branches: [master]
+    tags: ["v*"]
+
+env:
+  GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
+
+
+concurrency:
+  group: ${{ github.workflow }} @ ${{ github.ref }}
+  cancel-in-progress: true
+
+jobs:
+  build:
+    name: Build and Test
+    strategy:
+      matrix:
+        os: [ubuntu-latest, windows-latest]
+        scala: [3.4.2]
+        java: [zulu@21]
+    runs-on: ${{ matrix.os }}
+    timeout-minutes: 60
+    steps:
+      - name: Ignore line ending differences in git
+        if: contains(runner.os, 'windows')
+        shell: bash
+        run: git config --global core.autocrlf false
+
+      - name: Checkout current branch (full)
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+
+      - name: Setup Java (zulu@21)
+        id: setup-java-zulu-21
+        if: matrix.java == 'zulu@21'
+        uses: actions/setup-java@v4
+        with:
+          distribution: zulu
+          java-version: 21
+          cache: sbt
+
+      - name: sbt update
+        if: matrix.java == 'zulu@21' && steps.setup-java-zulu-21.outputs.cache-hit == 'false'
+        shell: bash
+        run: sbt +update
+
+      - name: Build project
+        run: sbt '++ ${{ matrix.scala }}' coverage test
+
+      - run: sbt '++ ${{ matrix.scala }}' coverageReport
+
+      - name: Coveralls
+        uses: coverallsapp/github-action@v2
+        with:
+          git-branch: main

.gitignore

@@ -1,12 +1,17 @@
+.DS_Store
 out/
-.bsp/
+.vscode/
+.metals/
+.bloop/
 .idea/
 .idea_modules/
 target/
 reports/
 lib_managed/
 src_managed/
 project/boot/
+project/project/
+project/project/target/
 project/plugins/project
 java_pid*.hprof
 *.swp
@@ -16,3 +21,6 @@ tags
 .history
 *~
 wip/
+.dotty-ide*
+_site/
+.bsp/

.scalafmt.conf

@@ -1,3 +1,20 @@
-align = more    // For pretty alignment.
-maxColumn = 175 // For my wide 30" display.
-assumeStandardLibraryStripMargin = true
+version = 3.7.14
+project.git = true
+maxColumn = 256
+runner.dialect = Scala3
+
+align.preset = some
+
+rewrite.rules = [Imports, RedundantBraces, SortModifiers]
+rewrite.imports.sort = scalastyle
+rewrite.redundantBraces.stringInterpolation = true
+
+rewrite.scala3.convertToNewSyntax = true
+rewrite.scala3.removeOptionalBraces = false
+
+docstrings.blankFirstLine = no
+docstrings.style = SpaceAsterisk
+docstrings.wrap = no
+
+newlines.sometimesBeforeColonInMethodReturnType = true
+lineEndings=unix

LICENSE

@@ -1,4 +1,4 @@
-MIT License
+## MIT License
 
 Copyright (c) 2020 Greg Zoller
 
@@ -19,3 +19,25 @@ AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
 LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
 OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
 SOFTWARE.
+
+## ZIO-Json Attribution
+
+Parts of JSON reading software were used either directly or derived from the 
+[ZIO-Json project](https://github.com/zio/zio-json)
+licensed under the [Apache License, Version 2.0](http://www.apache.org/licenses/LICENSE-2.0).
+In most cases, "derived" means removal of features not needed by ScalaJack or
+other changes needed to adapt the code to the ScalaJack ecosystem. Regardless,
+they are credited here as materially the same as those in the ZIO-Json project.
+
+The files used directly or derived are:
+* FastStringBuilder.scala
+* FieldKeyDecoder.scala
+* JsonDecoder.scala
+* JsonParser.scala
+* JsonReader.scala
+* Numbers.scala
+* StringMatrix.scala
+
+The terms, privileges, and restrictions provided by the Apache License 2.0 
+fully apply to these files, where these differ from the MIT license, which
+applies to the rest of ScalaJack's code.