Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support quarto vignettes #2656

Merged
merged 51 commits into from
Jun 13, 2024
Merged
Show file tree
Hide file tree
Changes from 30 commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
565726d
WIP
hadley Jun 10, 2024
c04dba3
Complete end-to-end prototype
hadley Jun 11, 2024
af13a4d
Fix encoding issue
hadley Jun 11, 2024
89c225c
Code tidying
hadley Jun 11, 2024
a396af6
Start working through formatting
hadley Jun 11, 2024
1646d05
Figures, equations, and cross-references
hadley Jun 11, 2024
2e257de
Features that need extra deps in head
hadley Jun 11, 2024
2c62d51
Temporarily create project file if needed
hadley Jun 11, 2024
926c25d
Manully drop jquery deps
hadley Jun 11, 2024
914d759
Split out build_rmarkdown_article() once more
hadley Jun 11, 2024
4ada20e
Fix mermaid styling
hadley Jun 11, 2024
90b8e62
Plumb up to build_articles()
hadley Jun 11, 2024
2fd27b6
Update call
hadley Jun 11, 2024
3681ef9
Fixes
hadley Jun 11, 2024
afcc9a0
Merged origin/main into quarto-vignettes
hadley Jun 11, 2024
9dd3ac3
Revert accidental remotes change
hadley Jun 11, 2024
cb7ec92
Organise pkgdown.scss a bit better
hadley Jun 11, 2024
68f1286
Add support for more features
hadley Jun 11, 2024
fa30ca7
Thinking about tomorrow's todos
hadley Jun 11, 2024
658f471
Install quarto
hadley Jun 12, 2024
64cfe4c
Install quarto in more places
hadley Jun 12, 2024
3831ccf
Ignore quarto working directory
hadley Jun 12, 2024
1f68636
Separate out heavy test features to keep vignettes light
hadley Jun 12, 2024
1bfa1c0
Fix space
hadley Jun 12, 2024
9ee23a0
Mention GHA setup
hadley Jun 12, 2024
7d01800
Support non-html vignettes
hadley Jun 12, 2024
e251b87
Tweak copying messages
hadley Jun 12, 2024
0fa7c9f
Don't actually need pkgdown
hadley Jun 12, 2024
1c16400
Add news bullet
hadley Jun 12, 2024
8b39778
Make build_article() work with quarto
hadley Jun 12, 2024
d6338e8
Don't need tinytex from quarto
hadley Jun 12, 2024
731a480
Update vignettes/quarto.qmd
hadley Jun 12, 2024
f09ec87
Merged origin/main into quarto-vignettes
hadley Jun 12, 2024
9a65509
Simplify/standardise quarto setup
hadley Jun 12, 2024
a275962
Start adding tests
hadley Jun 12, 2024
d66fa7c
Capture quarto styles
hadley Jun 12, 2024
eb952ef
Minor tweaks
hadley Jun 13, 2024
674d2b8
Test included in index
hadley Jun 13, 2024
b8fc296
Check anchors are tweaked correctly
hadley Jun 13, 2024
5c3fc0e
Fix silly mistake
hadley Jun 13, 2024
c7c694a
Need to skip tests on no-pandoc runner
hadley Jun 13, 2024
99add62
Get more debugging info
hadley Jun 13, 2024
f208a77
Actually pass quiet along
hadley Jun 13, 2024
8a0b2f8
Need tinytex for pdf
hadley Jun 13, 2024
060c759
Actually install tinytex
hadley Jun 13, 2024
6acbda3
More debugging
hadley Jun 13, 2024
4ffd69c
Try disabling parallelism
hadley Jun 13, 2024
48b3842
More debugging
hadley Jun 13, 2024
22698a1
Do need GITHUB_TOKEN
hadley Jun 13, 2024
f920131
Need pre-release quarto?
hadley Jun 13, 2024
e6e5222
Revert various debugging changes
hadley Jun 13, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
1 change: 1 addition & 0 deletions .Rbuildignore
Original file line number Diff line number Diff line change
Expand Up @@ -20,3 +20,4 @@
^CRAN-SUBMISSION$
^tools$
^\.lintr.R$
^vignettes/\.quarto$
7 changes: 7 additions & 0 deletions .github/workflows/R-CMD-check.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -44,6 +44,13 @@ jobs:

- uses: r-lib/actions/setup-pandoc@v2

- name: Set up Quarto
uses: quarto-dev/quarto-actions/setup@v2
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tinytex: true

- uses: r-lib/actions/setup-r@v2
with:
r-version: ${{ matrix.config.r }}
Expand Down
7 changes: 7 additions & 0 deletions .github/workflows/netlify.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,13 @@ jobs:

- uses: r-lib/actions/setup-pandoc@v2

- name: Set up Quarto
uses: quarto-dev/quarto-actions/setup@v2
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tinytex: true

- uses: r-lib/actions/setup-r@v2
with:
use-public-rspm: true
Expand Down
9 changes: 7 additions & 2 deletions .github/workflows/pkgdown.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -24,9 +24,14 @@ jobs:
steps:
- uses: actions/checkout@v4

- uses: r-lib/actions/setup-pandoc@v2
- name: Set up Quarto
uses: quarto-dev/quarto-actions/setup@v2
env:
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
tinytex: true

- uses: r-lib/actions/setup-tinytex@v2
- uses: r-lib/actions/setup-pandoc@v2

- uses: r-lib/actions/setup-r@v2
with:
Expand Down
4 changes: 3 additions & 1 deletion DESCRIPTION
Original file line number Diff line number Diff line change
Expand Up @@ -54,14 +54,16 @@ Suggests:
magick,
methods,
pkgload (>= 1.0.2),
quarto,
rsconnect,
rstudioapi,
rticles,
sass,
testthat (>= 3.1.3),
tools
VignetteBuilder:
knitr
knitr,
quarto
Config/Needs/website: usethis, servr
Config/potools/style: explicit
Config/testthat/edition: 3
Expand Down
1 change: 1 addition & 0 deletions NEWS.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
# pkgdown (development version)

* `build_articles()` and `build_article()` now support articles/vignettes written with quarto. Combining the disparate quarto and pkgdown templating systems is a delicate art, so while I've done my best to make it work, there may be some rough edges. So please file an issue you encounter quarto features that don't work quite right. Learn more in `vignette("quarto")`(#2210).
* `preview_page()` has been deprecated (#2650).
* `build_article()` now translates the "Abstract" title if it's used.
* `build_*()` (apart from `build_site()`) functions no longer default to previewing in interactive sessions since they now all emit specific links to newly generated files.
Expand Down
38 changes: 34 additions & 4 deletions R/build-article.R
Original file line number Diff line number Diff line change
Expand Up @@ -28,6 +28,7 @@ build_article <- function(name,
input <- pkg$vignettes$file_in[vig]
output_file <- pkg$vignettes$file_out[vig]
depth <- pkg$vignettes$depth[vig]
type <- pkg$vignettes$type[vig]

input_path <- path_abs(input, pkg$src_path)
output_path <- path_abs(output_file, pkg$dst_path)
Expand All @@ -36,10 +37,39 @@ build_article <- function(name,
return(invisible())
}

cli::cli_inform("Reading {src_path(input)}")
if (type == "rmd") {
build_rmarkdown_article(
pkg = pkg,
input_file = input,
input_path = input_path,
output_file = output_file,
output_path = output_path,
depth = depth,
seed = seed,
new_process = new_process,
pandoc_args = pandoc_args,
quiet = quiet
)
} else {
build_quarto_articles(pkg = pkg, article = name)
}
}

build_rmarkdown_article <- function(pkg,
input_file,
input_path,
output_file,
output_path,
depth,
seed = NULL,
new_process = TRUE,
pandoc_args = character(),
quiet = TRUE,
call = caller_env() ) {
cli::cli_inform("Reading {src_path(input_file)}")
digest <- file_digest(output_path)

data <- data_article(pkg, input)
data <- data_article(pkg, input_file, call = call)
if (data$as_is) {
if (identical(data$ext, "html")) {
setup <- rmarkdown_setup_custom(pkg, input_path, depth = depth, data = data)
Expand Down Expand Up @@ -68,7 +98,7 @@ build_article <- function(name,
if (new_process) {
path <- withCallingHandlers(
callr::r_safe(rmarkdown_render_with_seed, args = args, show = !quiet),
error = function(cnd) wrap_rmarkdown_error(cnd, input)
error = function(cnd) wrap_rmarkdown_error(cnd, input_file, call)
)
} else {
path <- inject(rmarkdown_render_with_seed(!!!args))
Expand All @@ -93,9 +123,9 @@ build_article <- function(name,
}

invisible(path)

}


data_article <- function(pkg, input, call = caller_env()) {
yaml <- rmarkdown::yaml_front_matter(path_abs(input, pkg$src_path))

Expand Down
3 changes: 2 additions & 1 deletion R/build-articles.R
Original file line number Diff line number Diff line change
Expand Up @@ -212,13 +212,14 @@ build_articles <- function(pkg = ".",

build_articles_index(pkg)
unwrap_purrr_error(purrr::walk(
pkg$vignettes$name,
pkg$vignettes$name[pkg$vignettes$type == "rmd"],
build_article,
pkg = pkg,
lazy = lazy,
seed = seed,
quiet = quiet
))
build_quarto_articles(pkg, quiet = quiet)

preview_site(pkg, "articles", preview = preview)
}
Expand Down
141 changes: 141 additions & 0 deletions R/build-quarto-articles.R
Original file line number Diff line number Diff line change
@@ -0,0 +1,141 @@
build_quarto_articles <- function(pkg = ".", article = NULL, quiet = TRUE) {
check_required("quarto")
pkg <- as_pkgdown(pkg)

qmds <- pkg$vignettes[pkg$vignettes$type == "qmd", ]
if (!is.null(article)) {
qmds <- qmds[qmds$name == article, ]
}
if (nrow(qmds) == 0) {
return()
}

for (file in qmds$file_in) {
cli::cli_inform("Reading {src_path(file)}")
}
cli::cli_inform("Running {.code quarto render}")

old_digest <- purrr::map_chr(path(pkg$dst_path, qmds$file_out), file_digest)

# Override default quarto format
metadata_path <- withr::local_tempfile(
fileext = ".yml",
pattern = "pkgdown-quarto-metadata-"
)
write_yaml(quarto_format(pkg), metadata_path)

# If needed, temporarily make a quarto project so we can build entire dir
if (is.null(article)) {
project_path <- path(pkg$src_path, "vignettes", "_quarto.yaml")
if (!file_exists(project_path)) {
yaml::write_yaml(list(project = list(render = list("*.qmd"))), project_path)
withr::defer(file_delete(project_path))
}
}

if (is.null(article)) {
src_path <- path(pkg$src_path, "vignettes")
} else {
src_path <- path(pkg$src_path, qmds$file_in)
}

output_dir <- withr::local_tempdir("pkgdown-quarto-")
quarto::quarto_render(
src_path,
metadata_file = metadata_path,
execute_dir = output_dir,
quarto_args = c("--output-dir", output_dir),
quiet = quiet,
as_job = FALSE
)

# Read generated data from quarto template and render into pkgdown template
purrr::walk2(qmds$file_in, qmds$file_out, function(input_file, output_file) {
built_path <- path(output_dir, path_rel(output_file, "articles"))
if (path_ext(output_file) == "html") {
data <- data_quarto_article(pkg, built_path, input_file)
render_page(pkg, "quarto", data, output_file, quiet = TRUE)

update_html(path(pkg$dst_path, output_file), tweak_quarto_html)
} else {
file_copy(built_path, path(pkg$dst_path, output_file), overwrite = TRUE)
}
})

# Report on which files have changed
new_digest <- purrr::map_chr(path(pkg$dst_path, qmds$file_out), file_digest)
changed <- new_digest != old_digest
for (file in qmds$file_out[changed]) {
writing_file(path(pkg$dst_path, file), file)
}

# Copy resources
resources <- setdiff(
dir_ls(output_dir, recurse = TRUE, type = "file"),
path(output_dir, path_rel(qmds$file_out, "articles"))
)
file_copy_to(
src_paths = resources,
dst_paths = path(pkg$dst_path, "articles", path_rel(resources, output_dir)),
src_root = output_dir,
dst_root = pkg$dst_path,
src_label = NULL
)

invisible()
}

quarto_format <- function(pkg) {
list(
lang = pkg$lang,
format = list(
html = list(
template = system_file("quarto", "template.html", package = "pkgdown"),
minimal = TRUE,
theme = "none",
`html-math-method` = config_math_rendering(pkg),
`embed-resources` = FALSE,
`citations-hover` = TRUE,
`link-citations` = TRUE,
toc = FALSE # pkgdown generates with js
)
)
)
}

data_quarto_article <- function(pkg, path, input_path) {
html <- xml2::read_html(path, encoding = "UTF-8")
meta_div <- xml2::xml_find_first(html, "//body/div[@class='meta']")

# Manually drop any jquery deps
head <- xpath_xml(html, "//head/script|//head/link")
head <- head[!grepl("jquery", xml2::xml_attr(head, "src"))]

list(
pagetitle = escape_html(xpath_text(html, "//head/title")),
toc = TRUE,
source = repo_source(pkg, input_path),
includes = list(
head = xml2str(head),
before = xpath_contents(html, "//body/div[@class='includes-before']"),
after = xpath_contents(html, "//body/div[@class='includes-after']"),
style = xpath_text(html, "//head/style")
),
meta = list(
title = xpath_contents(meta_div, "./h1"),
subtitle = xpath_contents(meta_div, "./p[@class='subtitle']"),
author = xpath_contents(meta_div, "./p[@class='author']"),
date = xpath_contents(meta_div, "./p[@class='date']"),
abstract = xpath_contents(meta_div, "./div[@class='abstract']")
),
body = xpath_contents(html, "//main")
)
}

tweak_quarto_html <- function(html) {
# If top-level headings use h1, move everything down one level
h1 <- xml2::xml_find_all(html, "//h1")
if (length(h1) > 1) {
tweak_section_levels(html)
}
}
38 changes: 33 additions & 5 deletions R/package.R
Original file line number Diff line number Diff line change
Expand Up @@ -319,11 +319,12 @@ package_vignettes <- function(path = ".") {
vig_path <- vig_path[!grepl("^_", path_file(vig_path))]
vig_path <- vig_path[!grepl("^tutorials", path_dir(vig_path))]

yaml <- purrr::map(path(base, vig_path), rmarkdown::yaml_front_matter)
title <- purrr::map_chr(yaml, list("title", 1), .default = "UNKNOWN TITLE")
desc <- purrr::map_chr(yaml, list("description", 1), .default = NA_character_)
ext <- purrr::map_chr(yaml, c("pkgdown", "extension"), .default = "html")
title[ext == "pdf"] <- paste0(title[ext == "pdf"], " (PDF)")
type <- tolower(path_ext(vig_path))

meta <- purrr::map(path(base, vig_path), article_metadata)
title <- purrr::map_chr(meta, "title")
desc <- purrr::map_chr(meta, "desc")
ext <- purrr::map_chr(meta, "ext")

# Vignettes will be written to /articles/ with path relative to vignettes/
# *except* for vignettes in vignettes/articles, which are moved up a level
Expand All @@ -336,6 +337,7 @@ package_vignettes <- function(path = ".") {

out <- tibble::tibble(
name = as.character(path_ext_remove(vig_path)),
type = type,
file_in = as.character(file_in),
file_out = as.character(file_out),
title = title,
Expand All @@ -345,6 +347,32 @@ package_vignettes <- function(path = ".") {
out[order(path_file(out$file_out)), ]
}

article_metadata <- function(path) {
if (path_ext(path) == "qmd") {
inspect <- quarto::quarto_inspect(path)
meta <- inspect$formats[[1]]$metadata

out <- list(
title = meta$title %||% "UNKNOWN TITLE",
desc = meta$description %||% NA_character_,
ext = path_ext(inspect$formats[[1]]$pandoc$`output-file`) %||% "html"
)
} else {
yaml <- rmarkdown::yaml_front_matter(path)
out <- list(
title = yaml$title[[1]] %||% "UNKNOWN TITLE",
desc = yaml$description[[1]] %||% NA_character_,
ext = yaml$pkgdown$extension %||% "html"
)
}

if (out$ext == "pdf") {
out$title <- paste0(out$title, " (PDF)")
}

out
}

find_template_config <- function(package,
bs_version = NULL,
error_call = caller_env()) {
Expand Down
8 changes: 3 additions & 5 deletions R/render.R
Original file line number Diff line number Diff line change
Expand Up @@ -23,7 +23,7 @@ render_page <- function(pkg = ".", name, data, path, depth = NULL, quiet = FALSE
pkg <- as_pkgdown(pkg)

if (is.null(depth)) {
depth <- length(strsplit(path, "/")[[1]]) - 1L
depth <- dir_depth(path)
}

html <- render_page_html(pkg, name = name, data = data, depth = depth)
Expand Down Expand Up @@ -64,7 +64,7 @@ render_page_html <- function(pkg, name, data = list(), depth = 0L) {
rendered <- render_template(template, components)

# Strip trailing whitespace
rendered <- gsub("\\s+\n", "\n", rendered, perl = TRUE)
rendered <- gsub(" +\n", "\n", rendered, perl = TRUE)

xml2::read_html(rendered, encoding = "UTF-8")
}
Expand Down Expand Up @@ -234,9 +234,7 @@ write_if_different <- function(pkg, contents, path, quiet = FALSE, check = TRUE)
full_path <- path_abs(path, start = pkg$dst_path)

if (check && !made_by_pkgdown(full_path)) {
if (!quiet) {
cli::cli_inform("Skipping {.file {path}}: not generated by pkgdown")
}
cli::cli_inform("Skipping {.file {path}}: not generated by pkgdown")
return(FALSE)
}

Expand Down
2 changes: 1 addition & 1 deletion R/tweak-tags.R
Original file line number Diff line number Diff line change
Expand Up @@ -156,7 +156,7 @@ tweak_tables <- function(html) {

# from https://github.com/rstudio/bookdown/blob/ed31991df3bb826b453f9f50fb43c66508822a2d/R/bs4_book.R#L307
tweak_footnotes <- function(html) {
container <- xml2::xml_find_all(html, ".//div[contains(@class, 'footnotes')]")
container <- xml2::xml_find_all(html, ".//div[contains(@class, 'footnotes')]|.//section[contains(@class, 'footnotes')]")
if (length(container) != 1) {
return()
}
Expand Down
Loading
Loading