merge master

.Rbuildignore

@@ -12,6 +12,7 @@
 
 ^\.ci$
 ^\.dev$
+^\.devcontainer$
 ^\.graphics$
 ^\.github$
 
@@ -20,8 +21,10 @@
 
 ^Makefile$
 ^NEWS\.0\.md$
+^NEWS\.1\.md$
 ^_pkgdown\.yml$
 ^src/Makevars$
+^CODEOWNERS$
 
 ^\.RData$
 ^\.Rhistory$
@@ -37,5 +40,7 @@
 
 ^bus$
 ^pkgdown$
+^docs$
 ^lib$
 ^library$
+^devwd$

.appveyor.yml

@@ -16,18 +16,14 @@ environment:
   global:
     CRAN: http://cloud.r-project.org
     WARNINGS_ARE_ERRORS: 1
-    R_CHECK_ARGS: --no-manual --no-multiarch
-    R_ARCH: i386
-# R_CHECK_ARGS specified in order to turn off --as-cran (on by default) as that can be slow
-# multiarch is on by default which (when R_ARCH: x64) compiles and tests both 32bit and 64bit in one x64 job
-# --no-multiarch so as to not run both 32bit and 64bit on every commit in PRs to save dev cycle time; GLCI after merge is full-strength
-# GHA has MacOS 64bit (test-coverage) and Ubuntu 64bit, therefore picked 32bit for Windows
-    GCC_PATH: mingw_64
-# Default GCC_PATH appears to be gcc-4.6.3 which is now unsupported as from Rtools.exe v3.4.
+    R_CHECK_ARGS: --as-cran --no-manual
+# --no-manual to avoid error 'pdflatex is not available'
+# --as-cran no longer a lot slower (now takes under 6 mins with and without); logs show _R_CHECK_CRAN_INCOMING_=FALSE which could take 5+ mins
     _R_CHECK_NO_STOP_ON_TEST_ERROR_: true
 # continue tests even if some script failed
     _R_CHECK_TESTS_NLINES_: 0
 # Block truncation of any error messages in R CMD check
+# R is 64-bit only on Windows from 4.2.0 (prior default was build and test both 32bit and 64bit) so we no longer use R_ARCH to pick one to reduce CI time in PRs
 
   matrix:
 

.ci/README.md

@@ -1,72 +1,50 @@
 # data.table continuous integration and deployment
 
-On each Pull Request opened in GitHub we run Travis CI and Appveyor to provide prompt feedback about the status of PR. Our main CI pipeline runs on GitLab CI. GitLab repository automatically mirrors our GitHub repository and runs pipeline on `master` branch. It tests more environments and different configurations. It publish variety of artifacts.
+On each Pull Request opened in GitHub we run GitHub Actions test jobs to provide prompt feedback about the status of PR. Our main CI pipeline runs on GitLab CI nightly. GitLab repository automatically mirrors our GitHub repository and runs pipeline on `master` branch every night. It tests more environments and different configurations. It publish variety of artifacts.
 
 ## Environments
 
 ### [GitLab CI](./../.gitlab-ci.yml)
 
 Test jobs:
-- `test-rel-lin` - `r-release` on Linux, most comprehensive test environment, `-O3 -flto -fno-common -Wunused-result`, extra check for no compilation warnings, includes testing [_with other packages_](./../inst/tests/other.Rraw)
-- `test-rel-cran-lin` - `--as-cran` on Linux, `-g0`, extra check for final status of `R CMD check` where we allow one NOTE (_size of tarball_).
-- `test-dev-cran-lin` - `r-devel` and `--as-cran` on Linux, `--with-recommended-packages --enable-strict-barrier --disable-long-double`, tests for compilation warnings in pkg install and new NOTEs/Warnings in pkg check, and because it is R-devel it is marked as allow_failure
-- `test-rel-vanilla-lin` - `r-release` on Linux, no suggested deps, no OpenMP, `-O0`, tracks memory usage during tests
-- `test-310-cran-lin` - R 3.1.0 on Linux
-- `test-344-cran-lin` - R 3.4.4 on Linux
-- `test-350-cran-lin` - R 3.5.0 on Linux, no `r-recommended`
-- `test-rel-win` - `r-release` on Windows
-- `test-dev-win` - `r-devel` on Windows
-- `test-old-win` - `r-oldrel` on Windows
-- `test-rel-osx` - MacOSX build not yet deployed, see [#3326](https://github.com/Rdatatable/data.table/issues/3326) for status
+- `test-lin-rel` - `r-release` on Linux, most comprehensive test environment, force all suggests, `-O3 -flto=auto -fno-common -Wunused-result`, test for no compilation warnings.
+- `test-lin-rel-vanilla` - `r-release` on Linux, no suggested deps, no zlib, no OpenMP, flags `-g -O0 -fno-openmp`, skip manual and vignettes.
+- `test-lin-rel-cran` - `--as-cran` on Linux, strict test for final status of `R CMD check`.
+- `test-lin-dev-gcc-strict-cran` - `--as-cran` on Linux, `r-devel` built with `-enable-strict-barrier --disable-long-double`, test for compilation warnings, test for new NOTEs/WARNINGs from `R CMD check`.
+- `test-lin-dev-clang-cran` - same as `gcc-strict` job but R built with `clang` and  no `--enable-strict-barrier --disable-long-double` flags.
+- `test-lin-310-cran` - R 3.1.0 on Linux, stated R dependency version.
+- `test-win-rel` - `r-release` on Windows.
+- `test-win-dev` - `r-devel` on Windows.
+- `test-win-old` - `r-oldrel` on Windows.
+- `test-mac-rel` - macOS build not yet available, see [#3326](https://github.com/Rdatatable/data.table/issues/3326) for status
+
+Tests jobs are allowed to fail, summary and logs of test jobs are later published at _CRAN-like checks_ page, see artifacts below.
 
 Artifacts:
 - [homepage](https://rdatatable.gitlab.io/data.table) - made with [pkgdown](https://github.com/r-lib/pkgdown)
 - [html manual](https://rdatatable.gitlab.io/data.table/library/data.table/html/00Index.html)
 - [pdf manual](https://rdatatable.gitlab.io/data.table/web/packages/data.table/data.table.pdf)
 - [html vignettes](https://rdatatable.gitlab.io/data.table/library/data.table/doc/index.html)
-- R packages repository for `data.table` and all _Suggests_ dependencies, url: `https://Rdatatable.gitlab.io/data.table`
+- R packages repository for `data.table` and all _Suggests_ dependencies, url: `https://rdatatable.gitlab.io/data.table`
   - sources
   - Windows binaries for `r-release`, `r-devel` and `r-oldrel`
 - [CRAN-like homepage](https://rdatatable.gitlab.io/data.table/web/packages/data.table/index.html)
-- [CRAN-like checks results](https://rdatatable.gitlab.io/data.table/web/checks/check_results_data.table.html) - note that all artifacts, including check results page, are being published only when all test jobs successfully pass, thus one will not see an _ERROR_ status there (unless error happened on a job marked as `allow_failure`).
-- [docker images](https://gitlab.com/Rdatatable/data.table/container_registry) - copy/paste-able `docker pull` commands can be found at the bottom of our [CRAN-like homepage](https://rdatatable.gitlab.io/data.table/web/packages/data.table/index.html)
+- [CRAN-like checks results](https://rdatatable.gitlab.io/data.table/web/checks/check_results_data.table.html)
 
-### [Travis CI](./../.travis.yml)
+### [GitHub Actions](./../.github/workflows)
 
-Test jobs:
-- `r-release` on Linux, includes code coverage check
-- _(might be disabled)_ `r-release` on OSX
-
-Artifacts:
-- R packages repository having `data.table` sources only, url: `https://Rdatatable.github.io/data.table`
-- code coverage stats pushed to [codecov.io/gh/Rdatatable/data.table](https://codecov.io/gh/Rdatatable/data.table)
+TODO document
 
 ### [Appveyor](./../.appveyor.yml)
 
-Test jobs:
-- Windows `r-release`
-- _(might be disabled)_ Windows `r-devel`
-
-Artifacts:
-- Windows `r-release` binaries accessed only via web UI
+TODO document
 
-## Tools
+## CI tools
 
 ### [`ci.R`](./ci.R)
 
-Base R implemented helper script, [originally proposed to R](https://svn.r-project.org/R/branches/tools4pkgs/src/library/tools/R/packages.R), that ease the process of extracting dependency information from description files, also to mirror packages and their recursive dependencies from CRAN to local CRAN-like directory. It is widely used in our [GitLab CI pipeline](./../.gitlab-ci.yml).
+Base R implemented helper script, [originally proposed to base R](https://svn.r-project.org/R/branches/tools4pkgs/src/library/tools/R/packages.R), that ease the process of extracting dependency information from description files, and to mirror packages and their recursive dependencies from CRAN to local CRAN-like directory. It is used in [GitLab CI pipeline](./../.gitlab-ci.yml).
 
 ### [`publish.R`](./publish.R)
 
-Base R implemented helper script to orchestrate generation of most artifacts. It is being used only in [_integration_ stage in GitLab CI pipeline](./../.gitlab-ci.yml).
-
-### [`Dockerfile.in`](./Dockerfile.in)
-
-Template file to produce `Dockerfile` for, as of now, three docker images. Docker images are being built and published in [_deploy_ stage in GitLab CI pipeline](./../.gitlab-ci.yml).
-- `r-base-dev` using `r-release`: publish docker image of `data.table` on R-release
-- `r-builder` using `r-release`: publish on R-release and OS dependencies for building Rmarkdown vignettes
-- `r-devel`: publish docker image of `data.table` on R-devel built with `--with-recommended-packages --enable-strict-barrier --disable-long-double`
-
-### [`deploy.sh`](./deploy.sh)
-
-Script used on Travis CI to publish CRAN-like repository of `data.table` sources. It publishes to `gh-pages` branch in GitHub repository. It depends on a token, which is provided based on `secure` environment variable in [.travis.yml](./../.travis.yml). It has been generated by @jangorecki.
+Base R implemented helper script to orchestrate generation of most artifacts and to arrange them nicely. It is being used only in [_integration_ stage in GitLab CI pipeline](./../.gitlab-ci.yml).

.ci/ci.R

@@ -185,7 +185,3 @@ function(pkgs,
   dp
 }
 
-## set repositories for CI tests
-if (as.logical(Sys.getenv("GITLAB_CI","false")) && identical(Sys.getenv("CI_PROJECT_NAME"), "data.table")) {
-  options("repos" = if (.Platform$OS.type == "windows") file.path("file://",getwd(),"bus/mirror-packages/cran") else file.path("file:", normalizePath("bus/mirror-packages/cran", mustWork=FALSE)))
-}