Skip to content

Commit

Permalink
Add caching for build step.
Browse files Browse the repository at this point in the history
  • Loading branch information
Christian Melchior committed Aug 18, 2022
1 parent 8515c85 commit 80bc206
Show file tree
Hide file tree
Showing 21 changed files with 306 additions and 137 deletions.
96 changes: 87 additions & 9 deletions .github/workflows/pr.yml
Original file line number Diff line number Diff line change
Expand Up @@ -8,18 +8,53 @@ env:
jobs:
# TODO Should be working, disable while iterating on further steps to increase turn-around time.
# static-analysis:
# uses: ./.github/workflows/include_static_analysis.yml
# uses: ./.github/workflows/include-static-analysis.yml

# Check if we actually need to build any of the packages. This is done by hashing all
# source files and use that as part of the version name, i.e. `1.0.0-fbc7df86ef5a8694873c863f9e30fb1e147efa54`.
check-cache:
runs-on: ubuntu-latest
name: Check cache
env:
CACHE_SKIP_SAVE: true
outputs:
packages-linux-cache-hit: ${{ steps.calculate-cache-exists.outputs.cache-hit }}
packages-linux-sha: ${{ steps.calculate-cache-key.outputs.packages-sha }}

steps:
- name: Checkout code
uses: actions/checkout@v3
with:
submodules: "recursive"

- name: Calculate source SHAs
id: calculate-cache-key
run: echo "::set-output name=packages-sha::${{ hashFiles('./packages/**') }}"

# TODO There doesn't seem to be a good way to check if a cache key exists without download it.
# https://github.com/actions/cache/issues/321
# TODO Create a custom action for this until we have a work-around?
# Name of key must match output of `Store build artifacts`.
- name: Calculate cache exists
id: calculate-cache-exists
uses: nirinchev/cache@d7c96a77c26ab70dd32b202c885cb4b34d95d8a8
with:
path: ./packages/build/m2-buildrepo
key: packages-m2-${{ runner.os }}-sync-${{ steps.calculate-cache-key.outputs.packages-sha }}

build-packages:
runs-on: ubuntu-latest
needs: check-cache
# needs: static-analysis
if: needs.check-cache.outputs.packages-linux-cache-hit != 'true'

steps:
- name: Checkout code
uses: actions/checkout@v3
with:
submodules: "recursive"

- name: Validate Gradle Wrapper
- name: Validate Gradle wrapper
uses: gradle/[email protected]

# TODO I'm not sure this catches changes to our Config.kt, what is the impact?
Expand All @@ -37,29 +72,72 @@ jobs:
with:
cache-read-only: false

# TODO This cmake version is not being used by the Android builds. Figure out why.
- name: Setup cmake
uses: jwlawson/[email protected]
with:
cmake-version: '3.21.4'
cmake-version: '3.22.1'

- name: Setup Ninja
# TODO This Ninja version is not being used by the Android builds. Figure out why.
- name: Setup ninja
uses: ashutoshvarma/setup-ninja@master

# TODO How to do ccache caching? It is unclear what the tradeoffs are?
with:
version: '1.11.0'

- name: Install ccache
uses: hendrikmuhs/[email protected]
with:
key: realm-kotlin-${{ matrix.os }}
key: ${{ github.job }}-${{ matrix.os }}
max-size: '2.0G'

- name: Prepend ccache executables to the PATH
run: echo PATH="/usr/lib/ccache:/usr/local/opt/ccache/libexec:$PATH" >> $GITHUB_ENV
run: echo "/usr/lib/ccache:/usr/local/opt/ccache/libexec" >> $GITHUB_PATH

# TOOD This matches 23.2.8568313, but what happens if we a define specific version in our build?
# TODO See https://github.com/hendrikmuhs/ccache-action/issues/94
- name: Configure ccache
run: |
ccache --set-config="compiler_check=content"
ccache --show-config
# TODO This matches 23.2.8568313, but what happens if we a define specific ndk version in our build?
- name: Setup NDK
uses: nttld/setup-ndk@v1
with:
ndk-version: r23c

- name: Debug environment
run: |
env
type cmake
cmake --version
type ninja
ninja --version
- name: Build packages
working-directory: packages
run: ./gradlew publishCIPackages --info

# TODO Figure out naming schema and retention policy
# We cannot use artifacts as they cannot be shared between workflows, so use cache instead.
- name: Store build artifacts
uses: actions/cache@v3
with:
path: ./packages/build/m2-buildrepo
key: packages-m2-${{ runner.os }}-sync-${{ needs.check-cache.outputs.packages-linux-sha }}

test-packages:
runs-on: ubuntu-latest
needs: [check-cache, build-packages]
if: |
always() &&
needs.build-packages != 'failure'
steps:
- name: Store build artifacts
uses: actions/cache@v3
with:
path: ./packages/build/m2-buildrepo
key: packages-m2-${{ runner.os }}-sync-${{ needs.check-cache.outputs.packages-linux-sha }}

- name: Run tests
run: echo "Run test for ${{ needs.check-cache.outputs.packages-linux-sha }}"
28 changes: 28 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,3 +1,31 @@
## 1.2.0-GHA (YYYY-MM-DD)

### Breaking Changes
* None.

### Enhancements
* None

### Fixed
* None.

### Compatibility
* This release is compatible with:
* Kotlin 1.6.10 and above.
* Coroutines 1.6.0-native-mt. Also compatible with Coroutines 1.6.0 but requires enabling of the new memory model and disabling of freezing, see https://github.com/realm/realm-kotlin#kotlin-memory-model-and-coroutine-compatibility for details on that.
* AtomicFu 0.17.0.
* Minimum Gradle version: 6.1.1.
* Minimum Android Gradle Plugin version: 4.0.0.
* Minimum Android SDK: 16.

### Internal
* Updated to Android Gradle Plugin 7.2.2.
* Updated to Gradle 7.5.1.
* CI jobs are now running on Github Actions instead of Jenkins.




## 1.1.0 (YYYY-MM-DD)

### Breaking Changes
Expand Down
Binary file modified benchmarks/gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
2 changes: 1 addition & 1 deletion benchmarks/gradle/wrapper/gradle-wrapper.properties
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.2-all.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-all.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
6 changes: 6 additions & 0 deletions benchmarks/gradlew
Original file line number Diff line number Diff line change
Expand Up @@ -205,6 +205,12 @@ set -- \
org.gradle.wrapper.GradleWrapperMain \
"$@"

# Stop when "xargs" is not available.
if ! command -v xargs >/dev/null 2>&1
then
die "xargs is not available"
fi

# Use "xargs" to parse quoted args.
#
# With -n1 it outputs one arg per line, with the quotes and backslashes removed.
Expand Down
14 changes: 8 additions & 6 deletions benchmarks/gradlew.bat
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,7 @@
@rem limitations under the License.
@rem

@if "%DEBUG%" == "" @echo off
@if "%DEBUG%"=="" @echo off
@rem ##########################################################################
@rem
@rem Gradle startup script for Windows
Expand All @@ -25,7 +25,7 @@
if "%OS%"=="Windows_NT" setlocal

set DIRNAME=%~dp0
if "%DIRNAME%" == "" set DIRNAME=.
if "%DIRNAME%"=="" set DIRNAME=.
set APP_BASE_NAME=%~n0
set APP_HOME=%DIRNAME%

Expand All @@ -40,7 +40,7 @@ if defined JAVA_HOME goto findJavaFromJavaHome

set JAVA_EXE=java.exe
%JAVA_EXE% -version >NUL 2>&1
if "%ERRORLEVEL%" == "0" goto execute
if %ERRORLEVEL% equ 0 goto execute

echo.
echo ERROR: JAVA_HOME is not set and no 'java' command could be found in your PATH.
Expand Down Expand Up @@ -75,13 +75,15 @@ set CLASSPATH=%APP_HOME%\gradle\wrapper\gradle-wrapper.jar

:end
@rem End local scope for the variables with windows NT shell
if "%ERRORLEVEL%"=="0" goto mainEnd
if %ERRORLEVEL% equ 0 goto mainEnd

:fail
rem Set variable GRADLE_EXIT_CONSOLE if you need the _script_ return code instead of
rem the _cmd.exe /c_ return code!
if not "" == "%GRADLE_EXIT_CONSOLE%" exit 1
exit /b 1
set EXIT_CODE=%ERRORLEVEL%
if %EXIT_CODE% equ 0 set EXIT_CODE=1
if not ""=="%GRADLE_EXIT_CONSOLE%" exit %EXIT_CODE%
exit /b %EXIT_CODE%

:mainEnd
if "%OS%"=="Windows_NT" endlocal
Expand Down
7 changes: 3 additions & 4 deletions buildSrc/src/main/kotlin/Config.kt
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ object Versions {
const val targetSdk = 31
const val compileSdkVersion = 31
const val buildToolsVersion = "31.0.0"
const val buildTools = "7.1.0" // https://maven.google.com/web/index.html?q=gradle#com.android.tools.build:gradle
const val buildTools = "7.2.2" // https://maven.google.com/web/index.html?q=gradle#com.android.tools.build:gradle
const val ndkVersion = "23.2.8568313"
}
const val androidxBenchmarkPlugin = "1.1.0-beta04" // https://maven.google.com/web/index.html#androidx.benchmark:androidx.benchmark.gradle.plugin
Expand All @@ -66,9 +66,8 @@ object Versions {
// Must be built with same (major.minor!?) kotlin version as 'kotlin' variable below, to be binary compatible with kotlin
const val atomicfu = "0.17.0" // https://github.com/Kotlin/kotlinx.atomicfu
const val autoService = "1.0" // https://mvnrepository.com/artifact/com.google.auto.service/auto-service
// Not currently used, so mostly here for documentation. Core requires minimum 3.15, but 3.18.1 is available through the Android SDK.
// Build also tested successfully with 3.21.4 (latest release).
const val cmake = "3.18.1"
// Not currently used, so mostly here for documentation. Core requires minimum 3.15, but 3.22.1 is available through the Android SDK.
const val cmake = "3.22.1"
const val coroutines = "1.6.0-native-mt" // https://mvnrepository.com/artifact/org.jetbrains.kotlinx/kotlinx-coroutines-core
const val datetime = "0.3.2" // https://github.com/Kotlin/kotlinx-datetime
const val detektPlugin = "1.19.0-RC1" // https://github.com/detekt/detekt
Expand Down
Binary file modified examples/kmm-sample/gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -1,5 +1,5 @@
distributionBase=GRADLE_USER_HOME
distributionPath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.2-all.zip
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-all.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Binary file modified examples/realm-java-compatibility/gradle/wrapper/gradle-wrapper.jar
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -1,6 +1,5 @@
#Tue May 24 14:11:29 CEST 2022
distributionBase=GRADLE_USER_HOME
distributionUrl=https\://services.gradle.org/distributions/gradle-7.2-bin.zip
distributionPath=wrapper/dists
zipStorePath=wrapper/dists
distributionUrl=https\://services.gradle.org/distributions/gradle-7.5.1-all.zip
zipStoreBase=GRADLE_USER_HOME
zipStorePath=wrapper/dists
Loading

0 comments on commit 80bc206

Please sign in to comment.