From 426400c1dcdf163f9c855d25cae8e903b101bb2b Mon Sep 17 00:00:00 2001 From: Josiah Parry Date: Fri, 15 Nov 2024 15:26:05 -0800 Subject: [PATCH] update tests --- NEWS.md | 1 + R/cran-compliance.R | 32 ++++++++++- inst/templates/configure | 2 +- inst/templates/configure.win | 2 +- man/cran.Rd | 51 +++++++++--------- man/vendor_pkgs.Rd | 37 +++++++++++++ tests/testthat/_snaps/use_extendr.md | 33 ++++-------- tests/testthat/test-cran-compliance.R | 14 ++++- tests/testthat/test-use_extendr.R | 13 ++--- vignettes/articles/cran-compliance.Rmd | 74 -------------------------- 10 files changed, 125 insertions(+), 134 deletions(-) create mode 100644 man/vendor_pkgs.Rd delete mode 100644 vignettes/articles/cran-compliance.Rmd diff --git a/NEWS.md b/NEWS.md index 075c7108..ebe8e1c2 100644 --- a/NEWS.md +++ b/NEWS.md @@ -1,5 +1,6 @@ # rextendr (development version) +* `use_cran_default()` has been removed as the default package template is CRAN compatible * `use_extendr()` now creates `tools/msrv.R`, `configure` and `configure.win`. These have been moved out of `use_cran_defaults()` * `Makevars` now prints linked static libraries at compile time by adding `--print=native-static-libs` to `RUSTFLAGS` * `use_extendr()` sets the `DESCRIPTION`'s `SystemRequirements` field according to CRAN policy to `Cargo (Rust's package manager), rustc` (#329) diff --git a/R/cran-compliance.R b/R/cran-compliance.R index 154fa1ef..1af01806 100644 --- a/R/cran-compliance.R +++ b/R/cran-compliance.R @@ -16,7 +16,6 @@ #' \dontrun{ #' vendor_pkgs() #' } -#' @name cran #' @export vendor_pkgs <- function(path = ".", quiet = FALSE, overwrite = NULL) { stderr_line_callback <- function(x, proc) { @@ -123,3 +122,34 @@ vendor_pkgs <- function(path = ".", quiet = FALSE, overwrite = NULL) { # return packages and versions invisibly invisible(res) } + + +#' CRAN compliant extendr packages +#' +#' R packages developed using extendr are not immediately ready to +#' be published to CRAN. The extendr package template ensures that +#' CRAN publication is (farily) painless. +#' +#' @section CRAN requirements: +#' +#' In order to publish a Rust based package on CRAN it must meet certain +#' requirements. These are: +#' +#' - Rust dependencies are vendored +#' - The package is compiled offline +#' - the `DESCRIPTION` file's `SystemRequirements` field contains `Cargo (Rust's package manager), rustc` +#' +#' The extendr templates handle all of this _except_ vendoring dependencies. +#' This must be done prior to publication using [`vendor_pkgs()`]. +#' +#' In addition, it is important to make sure that CRAN maintainers +#' are aware that the package they are checking contains Rust code. +#' Depending on which and how many crates are used as a dependencies +#' the `vendor.tar.xz` will be larger than a few megabytes. If a +#' built package is larger than 5mbs CRAN may reject the submission. +#' +#' To prevent rejection make a note in your `cran-comments.md` file +#' (create one using [`usethis::use_cran_comments()`]) along the lines of +#' "The package tarball is 6mb because Rust dependencies are vendored within src/rust/vendor.tar.xz which is 5.9mb." +#' @name cran +NULL \ No newline at end of file diff --git a/inst/templates/configure b/inst/templates/configure index ba384901..822ea414 100644 --- a/inst/templates/configure +++ b/inst/templates/configure @@ -3,7 +3,7 @@ "${R_HOME}/bin/Rscript" tools/msrv.R # Set CRAN_FLAGS based on the NOT_CRAN value -if [ "${NOT_CRAN}" = "false" ]; then +if [ "${NOT_CRAN}" != "true" ]; then export CRAN_FLAGS="-j 2 --offline" else export CRAN_FLAGS="" diff --git a/inst/templates/configure.win b/inst/templates/configure.win index f3867c45..bb18d407 100644 --- a/inst/templates/configure.win +++ b/inst/templates/configure.win @@ -2,7 +2,7 @@ "${R_HOME}/bin${R_ARCH_BIN}/Rscript.exe" tools/msrv.R # Set CRAN_FLAGS based on the NOT_CRAN value -if [ "${NOT_CRAN}" = "false" ]; then +if [ "${NOT_CRAN}" != "true" ]; then export CRAN_FLAGS="-j 2 --offline" else export CRAN_FLAGS="" diff --git a/man/cran.Rd b/man/cran.Rd index 8c410182..05a481fe 100644 --- a/man/cran.Rd +++ b/man/cran.Rd @@ -2,37 +2,34 @@ % Please edit documentation in R/cran-compliance.R \name{cran} \alias{cran} -\alias{vendor_pkgs} -\title{Vendor Rust dependencies} -\usage{ -vendor_pkgs(path = ".", quiet = FALSE, overwrite = NULL) +\title{CRAN compliant extendr packages} +\description{ +R packages developed using extendr are not immediately ready to +be published to CRAN. The extendr package template ensures that +CRAN publication is (farily) painless. } -\arguments{ -\item{path}{File path to the package for which to generate wrapper code.} +\section{CRAN requirements}{ -\item{quiet}{Logical indicating whether any progress messages should be -generated or not.} -\item{overwrite}{Logical scalar or \code{NULL} indicating whether the files in the \code{path} should be overwritten. -If \code{NULL} (default), the function will ask the user whether each file should -be overwritten in an interactive session or do nothing in a non-interactive session. -If \code{FALSE} and each file already exists, the function will do nothing. -If \code{TRUE}, all files will be overwritten.} -} -\value{ +In order to publish a Rust based package on CRAN it must meet certain +requirements. These are: \itemize{ -\item \code{vendor_pkgs()} returns a data.frame with two columns \code{crate} and \code{version} -} -} -\description{ -\code{vendor_pkgs()} is used to package the dependencies as required by CRAN. -It executes \verb{cargo vendor} on your behalf creating a \verb{vendor/} directory and a -compressed \code{vendor.tar.xz} which will be shipped with package itself. -If you have modified your dependencies, you will need need to repackage +\item Rust dependencies are vendored +\item The package is compiled offline +\item the \code{DESCRIPTION} file's \code{SystemRequirements} field contains \verb{Cargo (Rust's package manager), rustc} } -\examples{ -\dontrun{ - vendor_pkgs() -} +The extendr templates handle all of this \emph{except} vendoring dependencies. +This must be done prior to publication using \code{\link[=vendor_pkgs]{vendor_pkgs()}}. + +In addition, it is important to make sure that CRAN maintainers +are aware that the package they are checking contains Rust code. +Depending on which and how many crates are used as a dependencies +the \code{vendor.tar.xz} will be larger than a few megabytes. If a +built package is larger than 5mbs CRAN may reject the submission. + +To prevent rejection make a note in your \code{cran-comments.md} file +(create one using \code{\link[usethis:use_cran_comments]{usethis::use_cran_comments()}}) along the lines of +"The package tarball is 6mb because Rust dependencies are vendored within src/rust/vendor.tar.xz which is 5.9mb." } + diff --git a/man/vendor_pkgs.Rd b/man/vendor_pkgs.Rd new file mode 100644 index 00000000..98e75d57 --- /dev/null +++ b/man/vendor_pkgs.Rd @@ -0,0 +1,37 @@ +% Generated by roxygen2: do not edit by hand +% Please edit documentation in R/cran-compliance.R +\name{vendor_pkgs} +\alias{vendor_pkgs} +\title{Vendor Rust dependencies} +\usage{ +vendor_pkgs(path = ".", quiet = FALSE, overwrite = NULL) +} +\arguments{ +\item{path}{File path to the package for which to generate wrapper code.} + +\item{quiet}{Logical indicating whether any progress messages should be +generated or not.} + +\item{overwrite}{Logical scalar or \code{NULL} indicating whether the files in the \code{path} should be overwritten. +If \code{NULL} (default), the function will ask the user whether each file should +be overwritten in an interactive session or do nothing in a non-interactive session. +If \code{FALSE} and each file already exists, the function will do nothing. +If \code{TRUE}, all files will be overwritten.} +} +\value{ +\itemize{ +\item \code{vendor_pkgs()} returns a data.frame with two columns \code{crate} and \code{version} +} +} +\description{ +\code{vendor_pkgs()} is used to package the dependencies as required by CRAN. +It executes \verb{cargo vendor} on your behalf creating a \verb{vendor/} directory and a +compressed \code{vendor.tar.xz} which will be shipped with package itself. +If you have modified your dependencies, you will need need to repackage +} +\examples{ + +\dontrun{ + vendor_pkgs() +} +} diff --git a/tests/testthat/_snaps/use_extendr.md b/tests/testthat/_snaps/use_extendr.md index 2fbd3645..168f3319 100644 --- a/tests/testthat/_snaps/use_extendr.md +++ b/tests/testthat/_snaps/use_extendr.md @@ -7,18 +7,18 @@ i Setting `Config/rextendr/version` to "*.*.*" in the 'DESCRIPTION' file. i Setting `SystemRequirements` to "Cargo (Rust's package manager), rustc" in the 'DESCRIPTION' file. v Creating 'src/rust/src'. - v Writing 'src/entrypoint.c' - v Writing 'src/Makevars.in' - v Writing 'src/Makevars.win.in' - v Writing 'src/Makevars.ucrt' - v Writing 'src/.gitignore' - v Writing 'src/rust/Cargo.toml' - v Writing 'src/rust/src/lib.rs' - v Writing 'src/testpkg-win.def' + v Writing 'src/entrypoint.c'. + v Writing 'src/Makevars.in'. + v Writing 'src/Makevars.win.in'. + v Writing 'src/Makevars.ucrt'. + v Writing 'src/.gitignore'. + v Writing 'src/rust/Cargo.toml'. + v Writing 'src/rust/src/lib.rs'. + v Writing 'src/testpkg-win.def'. v Writing 'R/extendr-wrappers.R' - v Writing 'tools/msrv.R' - v Writing 'configure' - v Writing 'configure.win' + v Writing 'tools/msrv.R'. + v Writing 'configure'. + v Writing 'configure.win'. v Adding "^src/\\.cargo$" to '.Rbuildignore'. v Adding "^src/rust/vendor$" to '.Rbuildignore'. v Adding "src/rust/vendor" to '.gitignore'. @@ -234,18 +234,7 @@ Code use_extendr() Message - > File 'src/entrypoint.c' already exists. Skip writing the file. - > File 'src/Makevars.in' already exists. Skip writing the file. - > File 'src/Makevars.win.in' already exists. Skip writing the file. - > File 'src/Makevars.ucrt' already exists. Skip writing the file. - > File 'src/.gitignore' already exists. Skip writing the file. - > File 'src/rust/Cargo.toml' already exists. Skip writing the file. - > File 'src/rust/src/lib.rs' already exists. Skip writing the file. - > File 'src/testpkg.wrap-win.def' already exists. Skip writing the file. > File 'R/extendr-wrappers.R' already exists. Skip writing the file. - > File 'tools/msrv.R' already exists. Skip writing the file. - > File 'configure' already exists. Skip writing the file. - > File 'configure.win' already exists. Skip writing the file. v Finished configuring extendr for package testpkg.wrap. * Please run `rextendr::document()` for changes to take effect. diff --git a/tests/testthat/test-cran-compliance.R b/tests/testthat/test-cran-compliance.R index c250347f..ecb8a52f 100644 --- a/tests/testthat/test-cran-compliance.R +++ b/tests/testthat/test-cran-compliance.R @@ -13,7 +13,7 @@ test_that("vendor_pkgs() vendors dependencies", { }) -test_that("rextendr passes NOT_CRAN=false checks", { +test_that("rextendr passes CRAN checks", { skip_if_not_installed("usethis") skip_if_not_installed("rcmdcheck") @@ -23,7 +23,17 @@ test_that("rextendr passes NOT_CRAN=false checks", { use_extendr() document() vendor_pkgs() - res <- rcmdcheck::rcmdcheck(env = c("NOT_CRAN" = "false")) + res <- rcmdcheck::rcmdcheck(env = c("NOT_CRAN" = "")) + + # --offline flag should be set + expect_true(grepl("--offline", res$install_out)) + # -j 2 flag should be set + expect_true(grepl("-j 2", res$install_out)) + + # "Downloading" should not be present + expect_false(grepl("Downloading", res$install_out)) + + expect_true( rlang::is_empty(res$errors) && rlang::is_empty(res$warnings) ) diff --git a/tests/testthat/test-use_extendr.R b/tests/testthat/test-use_extendr.R index f8e19296..a8a42123 100644 --- a/tests/testthat/test-use_extendr.R +++ b/tests/testthat/test-use_extendr.R @@ -9,8 +9,8 @@ test_that("use_extendr() sets up extendr files correctly", { # DESCRITION file version_in_desc <- stringi::stri_trim_both(desc::desc_get("Config/rextendr/version", path)[[1]]) sysreq_in_desc <- stringi::stri_trim_both(desc::desc_get("SystemRequirements", path)[[1]]) - expect_equal(version_in_desc, as.character(packageVersion("rextendr"))) - expect_equal(sysreq_in_desc, "Cargo (Rust's package manager), rustc") + expect_identical(version_in_desc, as.character(packageVersion("rextendr"))) + expect_identical(sysreq_in_desc, "Cargo (Rust's package manager), rustc") # directory structure expect_true(dir.exists("src")) @@ -119,7 +119,7 @@ test_that("use_extendr() handles R packages with dots in the name", { use_extendr() document() devtools::load_all() - expect_equal(hello_world(), "Hello world!") + expect_identical(hello_world(), "Hello world!") }) # Specify crate name and library names explicitly @@ -133,7 +133,7 @@ test_that("use_extendr() handles R package name, crate name and library name sep use_extendr(crate_name = "crate_name", lib_name = "lib_name") document() devtools::load_all() - expect_equal(hello_world(), "Hello world!") + expect_identical(hello_world(), "Hello world!") }) # Pass unsupported values to `crate_name` and `lib_name` and expect errors. @@ -173,7 +173,7 @@ test_that("Message if the SystemRequirements field is already set.", { ) expect_true(created) - expect_equal(desc::desc_get("SystemRequirements")[[1]], sys_req) + expect_identical(desc::desc_get("SystemRequirements")[[1]], sys_req) }) test_that("`use_extendr()` works correctly when path is specified explicitly", { @@ -195,8 +195,9 @@ test_that("`use_extendr()` passes R CMD check", { usethis::use_mit_license() use_extendr() document() + # store results - res <- rcmdcheck::rcmdcheck() + res <- rcmdcheck::rcmdcheck(env = c("NOT_CRAN"="true")) # check the output expect_true( diff --git a/vignettes/articles/cran-compliance.Rmd b/vignettes/articles/cran-compliance.Rmd deleted file mode 100644 index ba43272b..00000000 --- a/vignettes/articles/cran-compliance.Rmd +++ /dev/null @@ -1,74 +0,0 @@ ---- -title: "CRAN compliant extendr packages" -author: "Josiah Parry" ---- - -```{r, include = FALSE} -knitr::opts_chunk$set( - collapse = TRUE, - comment = "#>" -) -``` - -In order for Rust-based packages to exist on CRAN, there are a number of -fairly stringent requirements that must be adhered to. CRAN published [Using Rust in CRAN packages](https://cran.r-project.org/web/packages/using_rust.html) in mid-2023, outlining their requirements for building and hosting Rust-based packages. - -This article describes CRAN requirements as of the day of writing and illustrates how `{rextendr}` can be used to adhere to them. - -## `SystemRequirements` - -Building Rust-backed packages from source requires the system dependencies `cargo` and `rustc`. CRAN has stipulated their preferred way of tracking this is using the following line in a packages `DESCRIPTION` file. - -``` -SystemRequirements: Cargo (Rust's package manager), rustc -``` -Even though this is a free-form field, having consistency can help the whole ecosystem keep track of Rust-based R packages. - -## `cargo` and `rustc` availability - -In order for an R package to be built from source, `cargo` and `rustc` need to be available to the machine compiling the package. The expectation for R packages using external dependencies is to have a `configure` and `configure.win` files that check if the dependencies are available before attempting to compile the package. If the checks fail, the build process will be stopped prematurely. - -CRAN expects that if `cargo` is not on the `PATH`, the user's home directory is checked at `~/.cargo/bin`. The configuration files must perform these checks. - -## `cargo build` settings - -CRAN also imposes restrictions on how `cargo` builds crates. CRAN has requested that no more than two logical CPUs be used in the build process. By default, `cargo` uses multiple threads to speed up the compilation process. CRAN policy allows for a maximum of two. This is set using the `-j 2` option, which is passed to `cargo build`. - -Additionally, to minimize security risks and ensure package stability, CRAN requires that packages be built completely offline. This prevents external dependencies from being downloaded at compile time. Because of this requirement, vendored dependencies must be used. - -## Vendored dependencies - -Vendoring dependencies is the act of including the dependency itself in a package source code. In the case of Rust, dependencies are fetched only at compile time. To enable compilation in an offline environment, dependencies must be vendored, which is accomplished using the `cargo vendor` command. - -`cargo vendor` creates a local directory with the default name `vendor`, which contains the source code for each of the recursive dependencies of the crate that is being built. For CRAN compatibility, the `vendor` directory must be compressed using tar xz compression and included in the source of the package. - -During the build time, the dependencies are extracted, compiled, and then discarded. This process is controlled by the `Makevars` and `Makevars.win` files. - -## Package compilation - -All of this comes together during package compilation time, providing all of the following requirements are met: - -- cargo must be able to be called from a user's home directory -- the user's home directory must not be modified or written to -- the package must be compiled offline -- no more than two logical CPUs are used -- the versions of `cargo` and `rustc` are printed - - -## Using CRAN defaults - -rextendr provides default CRAN compliant scaffolding via the `use_cran_defaults()` function and appropriate vendoring with `vendor_pkgs()`. - -### Making a package CRAN compliant - -To create a CRAN compliant R package begin by creating a new R package. Do so by calling `usethis::create_package()`. In the new R project, run `rextendr::use_extendr()` to create the minimal scaffolding necessary for a Rust-powered R package. Once you have done this, you can now run `rextendr::use_cran_defaults()`. - -`use_cran_defaults()` will create the `configure` and `configure.win` files. Additionally, it will create new `Makevars` and `Makevars.win` that print the versions of `cargo` and `rustc` as well as use the `cargo build` argument `-j 2 --offline`. - -### Vendoring packages - -After having configured your R package to use CRAN defaults, you will need to vendor your dependencies. - -`vendor_pkgs()` runs `cargo vendor` on your behalf, compresses the `vendor/` directory, and updates the `vendor-config.toml` file accordingly. - -When you have added new dependencies, changed the version or source of the crates, you should use `vendor_pkgs()` again. Doing so ensures that the compressed `vendor.tar.xz` contains the updates too. This is very important for CI and publishing to CRAN.