Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

3.4+ #218

Draft
wants to merge 40 commits into
base: main
Choose a base branch
from
Draft

3.4+ #218

wants to merge 40 commits into from

Commits on Mar 17, 2024

  1. updating version range to 3.4.2 and 3.5.1

    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    cda9548 View commit details
    Browse the repository at this point in the history

  2. (re)moving some stuff from the decoupled :core module

    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    42ba339 View commit details
    Browse the repository at this point in the history

  3. adding initial new KotlinTypeInference Encoder generator, this time p… 

    …urely in Kotlin. Not complete yet
    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    4896354 View commit details
    Browse the repository at this point in the history

  4. updating all references to the old encoder<>() function in favor of t… 

    …he new kotlinEncoderFor<>()
    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    e234f40 View commit details
    Browse the repository at this point in the history

  5. updating tests tests

    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    66a42ac View commit details
    Browse the repository at this point in the history

  6. made name hack optional as it is not waterproof

    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    3e9261f View commit details
    Browse the repository at this point in the history

  7. added tests and fixed name hack

    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    0d061b6 View commit details
    Browse the repository at this point in the history

  8. Map support

    @Jolanrensen
    Jolanrensen committed Mar 17, 2024
    Configuration menu
    Copy the full SHA
    0c8f4b1 View commit details
    Browse the repository at this point in the history

Commits on Mar 18, 2024

  1. disable name hack by default again, added JCP case for auto-applying … 

    …the expression encoder without spark-connect
    @Jolanrensen
    Jolanrensen committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    9149607 View commit details
    Browse the repository at this point in the history

  2. udt fix

    @Jolanrensen
    Jolanrensen committed Mar 18, 2024
    Configuration menu
    Copy the full SHA
    4364022 View commit details
    Browse the repository at this point in the history

Commits on Mar 20, 2024

  1. enabled core as scala-helpers with VarargUnwrapper. Removed name hack… 

    … in favor of upcoming IR compiler plugin
    @Jolanrensen
    Jolanrensen committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    2c875ff View commit details
    Browse the repository at this point in the history

  2. enabled core as scala-helpers with VarargUnwrapper. Removed name hack… 

    … in favor of upcoming IR compiler plugin. Removed spark dependency in scala-helpers
    @Jolanrensen
    Jolanrensen committed Mar 20, 2024
    Configuration menu
    Copy the full SHA
    d60e4dc View commit details
    Browse the repository at this point in the history

Commits on Mar 21, 2024

  1. updating some build parameters

    @Jolanrensen
    Jolanrensen committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    4f8d874 View commit details
    Browse the repository at this point in the history

  2. bumped to kotlin 2.0.0-Beta5, included the compiler plugin with tests… 

    … as separate module
    @Jolanrensen
    Jolanrensen committed Mar 21, 2024
    Configuration menu
    Copy the full SHA
    2a878f1 View commit details
    Browse the repository at this point in the history

Commits on Mar 22, 2024

  1. Added gradle plugin and fixed package naming errors

    @Jolanrensen
    Jolanrensen committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    c0a3140 View commit details
    Browse the repository at this point in the history

  2. enabling the compiler plugin on modules, sparkifying data classes

    @Jolanrensen
    Jolanrensen committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    1b0b316 View commit details
    Browse the repository at this point in the history

  3. bumping down to kotlin 1.9.23 to make lambda's serializable again

    @Jolanrensen
    Jolanrensen committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    b7c1711 View commit details
    Browse the repository at this point in the history

  4. fixing tests

    @Jolanrensen
    Jolanrensen committed Mar 22, 2024
    Configuration menu
    Copy the full SHA
    7069a9a View commit details
    Browse the repository at this point in the history

Commits on Mar 23, 2024

  1. fixing more tests. Can now remain at Kotlin 2.0 if we set -Xlambdas=c… 

    …lass, which can be done with gradle plugin
    @Jolanrensen
    Jolanrensen committed Mar 23, 2024
    Configuration menu
    Copy the full SHA
    df021c0 View commit details
    Browse the repository at this point in the history

Commits on Mar 24, 2024

  1. added conversion for @sparkify'ed classes to scala.Product with tests.

    @Jolanrensen
    Jolanrensen committed Mar 24, 2024
    Configuration menu
    Copy the full SHA
    4c17859 View commit details
    Browse the repository at this point in the history

  2. started WIP fir plugin, disabled for now

    @Jolanrensen
    Jolanrensen committed Mar 24, 2024
    Configuration menu
    Copy the full SHA
    2f62d07 View commit details
    Browse the repository at this point in the history

  3. disabled RddTest "Work with any number" for now: Caused by: java.lang… 

    ….NoSuchMethodException: org.jetbrains.kotlinx.spark.api.RddTest$1$1$1$2$2$1.$deserializeLambda$(java.lang.invoke.SerializedLambda)
    @Jolanrensen
    Jolanrensen committed Mar 24, 2024
    Configuration menu
    Copy the full SHA
    770698b View commit details
    Browse the repository at this point in the history

Commits on Mar 25, 2024

  1. Merge branch 'main' into 3.4+

    @Jolanrensen
    Jolanrensen authored Mar 25, 2024
    Configuration menu
    Copy the full SHA
    5d5fe84 View commit details
    Browse the repository at this point in the history

  2. added publishToMavenLocal for the plugins in gh actions build.yml

    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    24952b7 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/3.4+' into 3.4+

    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    ffedb29 View commit details
    Browse the repository at this point in the history

  4. editing build so that 

    a) it can build gradle-plugin and compiler-plugin with bootstrap jars without mavenLocal.
b) bootstrap jars are updated before the actual build
    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    6b63c96 View commit details
    Browse the repository at this point in the history

  5. fixing compiler-plugin tests

    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    4e5438b View commit details
    Browse the repository at this point in the history

  6. fixed conversions for scala 2.12

    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    723152b View commit details
    Browse the repository at this point in the history

  7. small fixes, disabled jupyter module for now

    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    0db290a View commit details
    Browse the repository at this point in the history

  8. spark 3.4/3.5 compatibility issue with ProductEncoder

    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    37eec64 View commit details
    Browse the repository at this point in the history

  9. fixed streaming test scala 2.12

    @Jolanrensen
    Jolanrensen committed Mar 25, 2024
    Configuration menu
    Copy the full SHA
    0ab212b View commit details
    Browse the repository at this point in the history

Commits on Mar 27, 2024

  1. added checkIsSparkified warnings when building an encoder

    @Jolanrensen
    Jolanrensen committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    1c34429 View commit details
    Browse the repository at this point in the history

  2. updated compiler plugin to make @sparkify classes Serializable too, p… 

    …lus added test to see if the plugin is enabled in the project
    @Jolanrensen
    Jolanrensen committed Mar 27, 2024
    Configuration menu
    Copy the full SHA
    68e830a View commit details
    Browse the repository at this point in the history

Commits on Mar 30, 2024

  1. added java bean class fallback support

    @Jolanrensen
    Jolanrensen committed Mar 30, 2024
    Configuration menu
    Copy the full SHA
    48db819 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2024

  1. added encoding for DatePeriod, DateTimePeriod, Instant, LocalDateTime… 

    …, and LocalDate, Duration not working
    @Jolanrensen
    Jolanrensen committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    ab4c455 View commit details
    Browse the repository at this point in the history

Commits on Apr 10, 2024

  1. kotlin 2.0.0-RC1, enabled jupyter module. Added basic Sparkify conver… 

    …sion for jupyter. Disabled html renderes in favor of just outputting them as text. Notebooks can render them however they like. RDDs are converted to ds before rendering
    @Jolanrensen
    Jolanrensen committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    e05feac View commit details
    Browse the repository at this point in the history

  2. make render in jupyter use show instead

    @Jolanrensen
    Jolanrensen committed Apr 10, 2024
    Configuration menu
    Copy the full SHA
    7fd77be View commit details
    Browse the repository at this point in the history

Commits on May 18, 2024

  1. disabled jupyter tests relying on DISPLAY(), update jupyter to latest… 

    … java 8 version
    @Jolanrensen
    Jolanrensen committed May 18, 2024
    Configuration menu
    Copy the full SHA
    92b699d View commit details
    Browse the repository at this point in the history

  2. disabled unsupported rddtests

    @Jolanrensen
    Jolanrensen committed May 18, 2024
    Configuration menu
    Copy the full SHA
    eae0196 View commit details
    Browse the repository at this point in the history

Commits on Jun 15, 2024

  1. added spark connect example which does not yet work with kotlin-spark… 

    …-api
    @Jolanrensen
    Jolanrensen committed Jun 15, 2024
    Configuration menu
    Copy the full SHA
    22fa5ae View commit details
    Browse the repository at this point in the history