rstan/develop doesn't work with stan/develop

[rgiordan]

Summary:

The develop branch of rstan does not work with the develop branch of stan.

Description:

The developer process overview says that submitted changes should keep all develop branches in working order. I am trying to make a change to stan in order to incorporate it into rstan. (stan-dev/stan#2692 and #587). However, it seems that two two develop branches do not work together.

Reproducible Steps:

Update the install_StanHeaders.R to check out the devlop branches.

path_rstan <- tempfile(pattern = "git2r-")
print(path_rstan)
# Here, put whatever local directory has up-to-date versions of the git repo.
path_stan_dev <- "/home/rgiordan/Documents/git_repos/stan-dev/"

 git2r::clone(file.path(path_stan_dev, "rstan"),
              path_rstan, branch = "develop")

 git2r::clone(file.path(path_stan_dev, "stan"),
              file.path(path_rstan, "StanHeaders", "inst", "include", "upstream"),
              branch = "develop")

# The master branch of math seems to be required for cvodes.
 git2r::clone(file.path(path_stan_dev, "math"),
              file.path(path_rstan, "StanHeaders", "inst", "include", "mathlib"),
              branch = "master")

devtools::install(file.path(path_rstan, "StanHeaders"), args = "--preclean")

After running the above install_StanHeaders.R script, run make clean, make build, and make install in stan-dev/rstan/rstan.

Current Output:

... lots of warnings, and the error:

g++ -std=gnu++14 -I/usr/share/R/include -DNDEBUG -I"../inst/include" -I"`"/usr/lib/R/bin/Rscript" --vanilla -e "cat(system.file('include', 'src', package = 'StanHeaders'))"`" -DBOOST_DISABLE_ASSERTS -DBOOST_PHOENIX_NO_VARIADIC_EXPRESSION -I"/usr/local/lib/R/site-library/Rcpp/include" -I"/usr/local/lib/R/site-library/RcppEigen/include" -I"/usr/local/lib/R/site-library/BH/include" -I"/usr/local/lib/R/site-library/StanHeaders/include"    -fpic  -g -O2 -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -g -c lang__grammars__var_deccls_grammar_inst.cpp -o lang__grammars__var_deccls_grammar_inst.o
lang__grammars__var_deccls_grammar_inst.cpp:19:57: fatal error: stan/lang/grammars/var_decls_grammar_inst.cpp: No such file or directory
compilation terminated.
/usr/lib/R/etc/Makeconf:168: recipe for target 'lang__grammars__var_deccls_grammar_inst.o' failed

Expected Output:

A successful install of rstan.

RStan Version:

N/A

R Version:

N/A

Operating System:

Ubuntu 16.04

[bgoodri]

It seems that var_decls_grammar_inst.cpp got renamed / refactored into something else. So, we can't have a rstan develop branch that is compatible with both the master and develop branches of Stan simultaneously. Once the hessian and hessian_times_vector methods get exposed in Stan, I will make a branch of 2.19 branch of rstan that works with them.

[HeNine]

I have been hunting down a bug (and mostly trying to grok how r/stan even works internally) where variables get misnamed in the hessian that is output by optimizing and ran into this bug. Does this work on the "native" hessian mean I shouldn't bother any more and just wait for that?

[bob-carpenter]

Once the hessian and hessian_times_vector methods get exposed in Stan

I still don't know what that means and commented as much on @rgiordan's issue to expose them.

The developer process overview says that submitted changes should keep all develop branches in working order.

That's definitely true of CmdStan. We should clarify that RStan plays by its own rules.

[bgoodri]

RStan tries to play by the same rules, but that doesn't seem to stop people from refactoring things that cause RStan not to build (in this case the parser)

…

On Fri, Nov 16, 2018 at 11:09 AM Bob Carpenter ***@***.***> wrote: Once the hessian and hessian_times_vector methods get exposed in Stan I still don't know what that means and commented as much on @rgiordan <https://github.com/rgiordan>'s issue to expose them. The developer process overview says that submitted changes should keep all develop branches in working order. That's definitely true of CmdStan. We should clarify that RStan plays by its own rules. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#588 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADOrqnLHL7PMGc9ipQp1iUbfMqwFVe0Bks5uvuNSgaJpZM4YjOlV> .

[bob-carpenter]

@bgoodri --- what I meant is that I didn't think there was even an attempt right now to keep RStan in a deployable state w.r.t. the current develop branches of stan and math. I'd like to move in that direction to cut down on time between releasing CmdStan and releasing RStan.

@seantalts or @syclik or @mitzimorris : how much upstream testing of RStan is being done in the Stan CI? I'd like to make sure this kind of thing doesn't happen by having enough tests in RStan and enough calls to those tests from Stan to avoid it.

If everyone's on board with this, let's try to strengthen our CI testing of RStan upstream.

@bgoodri and @mitzimorris : we seem to be missing top-level headers for the language, which would explain why low level files are being called directly. This kind of change should not interrupt RStan's ability to build in general, as there's probably not much reason to be directly including a low-level grammar file (they're mostly mutually recursive, so if you include one, you usually need them all).

[bgoodri]

There is a pipeline for it but it errors out for various reasons not all having to do with renaming grammar header files.

…

On Fri, Nov 16, 2018 at 1:44 PM Bob Carpenter ***@***.***> wrote: @bgoodri <https://github.com/bgoodri> --- what I meant is that I didn't think there was even an attempt right now to keep RStan in a deployable state w.r.t. the current develop branches of stan and math. I'd like to move in that direction to cut down on time between releasing CmdStan and releasing RStan. @seantalts <https://github.com/seantalts> or @syclik <https://github.com/syclik> or @mitzimorris <https://github.com/mitzimorris> : how much upstream testing of RStan is being done in the Stan CI? I'd like to make sure this kind of thing doesn't happen by having enough tests in RStan and enough calls to those tests from Stan to avoid it. If everyone's on board with this, let's try to strengthen our CI testing of RStan upstream. @bgoodri <https://github.com/bgoodri> and @mitzimorris <https://github.com/mitzimorris> : we seem to be missing top-level headers for the language, which would explain why low level files are being called directly. This kind of change should not interrupt RStan's ability to build in general, as there's probably not much reason to be directly including a low-level grammar file (they're mostly mutually recursive, so if you include one, you usually need them all). — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#588 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADOrqhGxrrCBPt5Cd0subH_Um6Km-tfHks5uvweVgaJpZM4YjOlV> .

[bgoodri]

@bgoodri and @mitzimorris : we seem to be missing top-level headers for the language, which would explain why low level files are being called directly. This kind of change should not interrupt RStan's ability to build in general, as there's probably not much reason to be directly including a low-level grammar file (they're mostly mutually recursive, so if you include one, you usually need them all).

Can we add top-level headers for the language before 2.19? RStan is building again with develop but I have to copy all the .cpp files from lang/ .

[mitzimorris]

we seem to be missing top-level headers for the language

was there a particular header file somewhere?

[bgoodri]

I don't think it ever existed

…

On Sun, Mar 10, 2019, 4:02 PM Mitzi Morris ***@***.***> wrote: we seem to be missing top-level headers for the language was there a particular header file somewhere? — You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub <#588 (comment)>, or mute the thread <https://github.com/notifications/unsubscribe-auth/ADOrqiWbR6VsPe3OZlh4j5piSzjrnUu0ks5vVWTBgaJpZM4YjOlV> .