Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add user-friendly functions to add locations #184

Merged
merged 5 commits into from
Oct 18, 2024
Merged
Show file tree
Hide file tree
Changes from 3 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
Package: orderly2
Title: Orderly Next Generation
Version: 1.99.41
Version: 1.99.42
Authors@R: c(person("Rich", "FitzJohn", role = c("aut", "cre"),
email = "[email protected]"),
person("Robert", "Ashton", role = "aut"),
Expand Down
92 changes: 57 additions & 35 deletions R/location.R
Original file line number Diff line number Diff line change
Expand Up @@ -3,9 +3,12 @@
##' based locations are supported, with limited support for custom
##' locations. Note that adding a location does *not* pull metadata
##' from it, you need to call
##' [orderly2::orderly_location_pull_metadata] first.
##' [orderly2::orderly_location_pull_metadata] first. The function
##' `orderly_location_add` can add any sort of location, but the other
##' functions documented here (`orderly_location_add_path`, etc) will
##' typically be much easier to use in practice.
##'
##' We currently support two types of locations - `path`, which points
##' We currently support three types of locations - `path`, which points
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

two types of locations and off by one errors 😁

##' to an outpack archive accessible by path (e.g., on the same
##' computer or on a mounted network share), `http`, which requires
##' that an outpack server is running at some url and uses an HTTP API
Expand All @@ -14,13 +17,12 @@
##' options to these location types will definitely be needed in
##' future.
##'
##' Configuration options for different location types:
##' Configuration options for different location types are described
##' in the arguments to their higher-level functions.
##'
##' **Path locations**:
##'
##' * `path`: The path to the other archive root. This should
##' generally be an absolute path, or the behaviour of outpack will
##' be unreliable.
##' Use `orderly_location_add_path`, which accepts a `path` argument.
##'
##' **HTTP locations**:
##'
Expand All @@ -29,24 +31,16 @@
##' the API, but also as we move to support things like TLS and
##' authentication.
##'
##' * `url`: The location of the server, including protocol, for
##' example `http://example.com:8080`
##' Use `orderly_location_add_http`, which accepts a `url` argument.
##'
##' **Packit locations**:
##'
##' Packit locations work over HTTPS, and include everything in an
##' outpack location but also provide authentication and later will
##' have more capabilities we think.
##'
##' * `url`: The location of the server
##'
##' * `token`: The value for your your login token (currently this is
##' a GitHub token with `read:org` scope). If missing or NULL, orderly2 will
##' perform an interactive authentication against GitHub to obtain one.
##'
##' * `save_token`: If no token is provided and interactive authentication is
##' used, this controls whether the GitHub token should be saved to disk.
##' Defaults to TRUE if missing.
##' Use `orderly_location_add_packit`, which accepts `url`, `token`
##' and `save_token` arguments.
##'
##' **Custom locations**:
##'
Expand Down Expand Up @@ -100,26 +94,16 @@ orderly_location_add <- function(name, type, args, root = NULL, locate = TRUE) {

loc <- new_location_entry(name, type, args, call = environment())
if (type == "path") {
## We won't be necessarily be able to do this _generally_ but
## here, let's confirm that we can read from the outpack archive
## at the requested path; this will just fail but without
## providing the user with anything actionable yet.
assert_scalar_character(loc$args[[1]]$path, name = "args$path",
call = environment())
root_open(loc$args[[1]]$path, locate = FALSE, require_orderly = FALSE)
assert_scalar_character(args$path, name = "path")
root_open(args$path, locate = FALSE, require_orderly = FALSE)
} else if (type == "http") {
assert_scalar_character(loc$args[[1]]$url, name = "args$url",
call = environment())
assert_scalar_character(args$url, name = "url")
} else if (type == "packit") {
assert_scalar_character(loc$args[[1]]$url, name = "args$url",
call = environment())
assert_scalar_character(loc$args[[1]]$token, name = "args$token",
allow_null = TRUE,
call = environment())
assert_scalar_logical(loc$args[[1]]$save_token, name = "args$save_token",
allow_null = TRUE,
call = environment())
if (!is.null(loc$args[[1]]$token) && !is.null(loc$args[[1]]$save_token)) {
assert_scalar_character(args$url, name = "url")
assert_scalar_character(args$token, name = "token", allow_null = TRUE)
assert_scalar_logical(args$save_token, name = "save_token",
allow_null = TRUE)
if (!is.null(args$token) && !is.null(args$save_token)) {
cli::cli_abort("Cannot specify both 'token' and 'save_token'")
}
}
Expand All @@ -132,6 +116,44 @@ orderly_location_add <- function(name, type, args, root = NULL, locate = TRUE) {
}


##' @rdname orderly_location_add
##'
##' @param path The path to the other archive root. This should
##' generally be an absolute path, or the behaviour of outpack will
##' be unreliable.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Might be worth cleaning up and defining the semantics of a relative path at some point (not in this PR). IMO it should be relative to the orderly root, and be resolved each time we use the location, not when it is first added.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are you thinking here again of people on the cluster where directories "move"?

orderly_location_add_path <- function(name, path, root = NULL) {
args <- list(path = path)
orderly_location_add(name, "path", args, root = root)
}


##' @rdname orderly_location_add
##'
##' @param url The location of the server, including protocol, for
##' example `http://example.com:8080`
orderly_location_add_http <- function(name, url, root = NULL) {
args <- list(url = url)
orderly_location_add(name, "http", args, root = root)
}


##' @rdname orderly_location_add
##'
##' @param token The value for your your login token (currently this
##' is a GitHub token with `read:org` scope). If `NULL`, orderly2
##' will perform an interactive authentication against GitHub to
##' obtain one.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Technically this can be a Packit JWT instead of a GitHub PAT, but we don't really publicize any way of obtaining those so I think it is fine if you don't want to mention it here.

This is the only place a JWT would be used today: https://github.com/mrc-ide/orderly-action/blob/main/packit-login.R#L65-L69

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I just copied this in from the string that was present in the Details for now

##'
##' @param save_token If no token is provided and interactive
##' authentication is used, this controls whether the GitHub token
##' should be saved to disk. Defaults to `TRUE` if `NULL`.
orderly_location_add_packit <- function(name, url, token = NULL,
save_token = NULL, root = NULL) {
args <- list(url = url, token = token, save_token = save_token)
orderly_location_add(name, "packit", args, root = root)
}


##' Rename an existing location
##'
##' @title Rename a location
Expand Down
3 changes: 3 additions & 0 deletions _pkgdown.yml
Original file line number Diff line number Diff line change
Expand Up @@ -63,6 +63,9 @@ reference:
- subtitle: Manage locations
contents:
- orderly_location_add
- orderly_location_add_path
- orderly_location_add_http
- orderly_location_add_packit
- orderly_location_list
- orderly_location_remove
- orderly_location_rename
Expand Down
66 changes: 45 additions & 21 deletions man/orderly_location_add.Rd

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

27 changes: 9 additions & 18 deletions tests/testthat/test-location-path.R
Original file line number Diff line number Diff line change
Expand Up @@ -151,8 +151,7 @@ test_that("can detect differences between locations when destination empty", {
ids <- create_random_packet_chain(client, 4)

server <- create_temporary_root(use_file_store = TRUE, path_archive = NULL)
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)

files <- lapply(ids, function(id) client$index$metadata(id)$files$hash)

Expand Down Expand Up @@ -183,8 +182,7 @@ test_that("Import complete tree via push into server", {
ids <- create_random_packet_chain(client, 4)

server <- create_temporary_root(use_file_store = TRUE, path_archive = NULL)
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)

plan <- orderly_location_push(ids[[4]], "server", client)

Expand All @@ -208,8 +206,7 @@ test_that("Import packets into root with archive as well as store", {

server <- create_temporary_root(use_file_store = TRUE,
path_archive = "archive")
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)

plan <- orderly_location_push(ids[[4]], "server", client)

Expand All @@ -228,8 +225,7 @@ test_that("Prevent pushing things that would corrupt the store", {
ids <- create_random_packet_chain(client, 4)

server <- create_temporary_root(use_file_store = TRUE, path_archive = NULL)
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)

id <- ids[[3]]
str <- read_string(file.path(client$path, ".outpack", "metadata", id))
Expand Down Expand Up @@ -259,8 +255,7 @@ test_that("Can only push into a root with a file store", {
client <- create_temporary_root()
ids <- create_random_packet_chain(client, 2)
server <- create_temporary_root()
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)
expect_error(
orderly_location_push(ids[[2]], "server", client),
"Can't push files into this server, as it does not have a file store")
Expand All @@ -271,8 +266,7 @@ test_that("pushing twice does nothing", {
client <- create_temporary_root()
ids <- create_random_packet_chain(client, 4)
server <- create_temporary_root(use_file_store = TRUE, path_archive = NULL)
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)
plan1 <- orderly_location_push(ids[[4]], "server", client)
plan2 <- orderly_location_push(ids[[4]], "server", client)
expect_equal(plan2, list(packet_id = character(), files = character()))
Expand All @@ -282,8 +276,7 @@ test_that("pushing twice does nothing", {
test_that("push overlapping tree", {
client <- create_temporary_root()
server <- create_temporary_root(use_file_store = TRUE, path_archive = NULL)
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)

id_base <- create_random_packet(server)
orderly_location_pull_metadata(root = client)
Expand All @@ -302,8 +295,7 @@ test_that("Push single packet", {
id <- create_random_packet(client)

server <- create_temporary_root(use_file_store = TRUE, path_archive = NULL)
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)

plan <- orderly_location_push(id, "server", client)

Expand Down Expand Up @@ -357,8 +349,7 @@ test_that("Fail to push sensibly if files have been changed", {
ids <- create_random_packet_chain(client, 4)

server <- create_temporary_root(use_file_store = TRUE, path_archive = NULL)
orderly_location_add("server", "path", list(path = server$path),
root = client)
orderly_location_add_path("server", path = server$path, root = client)

## Corrupt one file:
path <- file.path(client$path, "archive", "b", ids[["b"]], "script.R")
Expand Down
Loading
Loading