Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Miscellaneous test fixes #418

Merged
merged 5 commits into from
Aug 13, 2024
Merged

Miscellaneous test fixes #418

merged 5 commits into from
Aug 13, 2024

Conversation

kyleam
Copy link
Collaborator

@kyleam kyleam commented Aug 12, 2024

This series resolves three test failures that occur on the latest Metworx under R 4.3.

The large number of changed files is coming from the deletion of cmd/load_test.go and associated files.

$ git diff --name-only af56210c9^-1 | wc -l
    3350

$ git diff --name-only af56210c9^-1 | cut -d/ -f1-4 | sort | uniq -c
   1 cmd/load_test.go
1716 cmd/testsite/golden/load
1633 cmd/testsite/golden/load-fail

$ git diff --name-only af56210c9..5401687
integration_tests/baseline/install_test.go
integration_tests/mixed-source/.gitignore
integration_tests/mixed-source/mixed_source_test.go
integration_tests/mixed-source/pkgr-linux.yml.template
integration_tests/mixed-source/setup-binary-repo.sh
localrepos/bad-xml2/src/contrib/PACKAGES
localrepos/bad-xml2/src/contrib/PACKAGES.gz
localrepos/bad-xml2/src/contrib/PACKAGES.rds
localrepos/bad-xml2/src/contrib/RCurl_1.95-4.12.tar.gz
localrepos/bad-xml2/src/contrib/RCurl_1.98-1.16.tar.gz

diff excluding first commit: af56210...5401687

kyleam added 5 commits August 12, 2024 10:10
On the latest Metworx, the load test fail with a bunch of dyn.load()
error like

  package or namespace load failed for ‘pillar’ in
  dyn.load(file, DLLpath = DLLpath, ...): unable to load shared object
  '[...]/cmd/testsite/golden/load/test-library-4.0/rlang/libs/rlang.so':
  [.../cmd/testsite/golden/load/test-library-4.0/rlang/libs/rlang.so:
  invalid ELF header Execution halted

These cases are already covered by the load integration test, so
remove the load tests under cmd.
With R 4.3.1 on the latest Metworx, TestInstallWithoutRollback fails
to install RCurl

  using C compiler: ‘gcc (Ubuntu 9.4.0-1ubuntu1~20.04.1) 9.4.0’
  base64.c: In function ‘R_base64_decode’:
  base64.c:35:6: error: ‘PROBLEM’ undeclared (first use in this function)
     35 |      PROBLEM \"decoding from base64 failed\"
        |      ^~~~~~~
  base64.c:35:6: note: each undeclared identifier is reported only once for each function it appears in
  base64.c:35:13: error: expected ‘;’ before string constant
     35 |      PROBLEM \"decoding from base64 failed\"
        |             ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
        |             ;
  base64.c: In function ‘R_base64_encode’:
  base64.c:73:5: error: ‘PROBLEM’ undeclared (first use in this function)
     73 |     PROBLEM \"failed to encode the data\"
        |     ^~~~~~~
  base64.c:73:12: error: expected ‘;’ before string constant
     73 |     PROBLEM \"failed to encode the data\"
        |            ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
        |            ;

Based on targeted tests:

 * RCurl installation fails with the RCurl version used by this test
   (1.95-4.12) under R 4.3

 * RCurl installation succeeds with RCurl 1.95-4.12 under R 4.1

 * RCurl installation succeeds with the latest RCurl version
   (1.98-1.16) under both R 4.1 and R 4.3

Update the RCurl version to 1.98-1.16 with these steps:

  $ cd localrepos/bad-xml2/src/contrib
  $ curl -fSsLO https://cran.r-project.org/src/contrib/RCurl_1.98-1.16.tar.gz
  $ curl -fSsLO https://cran.r-project.org/src/contrib/Archive/xml2/xml2_1.2.2.tar.gz
  $ cd ../..
  $ Rscript -e 'tools::write_PACKAGES("src/contrib/", type = "source")'

Then revert the xml2 change because it's intentionally corrupted for
the rollback tests.
TestInstall2  invokes Rscript with

  install.packages(c("ellipsis", "digest"), lib="lib", ...)

and expects ellipsis, digest, _and_ rlang (and ellipsis dependency) to
be installed.  But, with that approach, the dependency won't be pulled
in if it's already available on the system.

Set R_LIBS to isolate the call.
TestMixedSource expects the binary packages to be available at
<https://mpn.metworx.com/snapshots/stable/2020-07-19>.  That has
binary packages only for R 3.6 and 4.0 on bionic.

Remove assumptions about the R and Ubuntu versions by teaching the
test to create its own binary repo locally.
@kyleam
Copy link
Collaborator Author

kyleam commented Aug 12, 2024

@kyleam kyleam requested a review from seth127 August 12, 2024 17:26
@kyleam
Copy link
Collaborator Author

kyleam commented Aug 12, 2024

Metworx results for tests of packages touched by this series:

$ ./t.sh
ok  	github.com/metrumresearchgroup/pkgr/cmd	119.169s
ok  	github.com/metrumresearchgroup/pkgr/integration_tests/mixed-source	20.395s
ok  	github.com/metrumresearchgroup/pkgr/integration_tests/baseline	248.520s
test script
#!/bin/sh

set -eu

tdir=$(mktemp -d "${TMPDIR:-/tmp}"/pkgr-test-XXXXX)
trap 'rm -rf "$tdir"' 0

cache=$tdir/cache
mkdir "$cache"
# Prevent tests from touching user's real cache.
export XDG_CACHE_HOME="$cache"

go test -count 1 ./cmd

bin=$tdir/bin
mkdir "$bin"

go build -o "$bin/pkgr" cmd/pkgr/pkgr.go

export PATH="$bin:$PATH"

cd integration_tests/mixed-source
go test -count 1 ./...
cd ../../integration_tests/baseline
go test -count 1 ./...

Copy link
Collaborator

@seth127 seth127 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This all makes sense to me. Thanks for documenting the changes and rationale in the commit messages. That made this much easier to review, especially since so many (mostly incidental) files changed. I also ran the test script you included and see the same passing results.

One comment for posterity: in the commit message for af56210 you say

These cases are already covered by the load integration test, so
remove the load tests under cmd.

I agree with that rationale. When I use the machinery on the valtools branch (in #419), which includes these changes, I can see that the test coverage for cmd/load.go is still above 90%, even with these tests removed.

$ make vt-all
$ cat internal/valtools/output/pkgr_3.1.1-46-g35eb4df/pkgr_3.1.1-46-g35eb4df.coverage.json
...
    {
      "file": "cmd/load.go",
      "coverage": 90.9090909090909
    }
...

@kyleam
Copy link
Collaborator Author

kyleam commented Aug 13, 2024

When I use the machinery on the valtools branch (in #419), which includes these changes, I can see that the test coverage for cmd/load.go is still above 90%, even with these tests removed

Thanks for adding that.

@kyleam kyleam merged commit 9e62693 into develop Aug 13, 2024
@kyleam kyleam deleted the test-fixes branch August 13, 2024 19:26
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants