Setup CI infrastructure for maintanance

[rok-cesnovar]

Actions that could be done automatically:

• adding/updating a package
• building Win/Mac binaries
• pruning git history weekly

[rok-cesnovar]

I am going to add @serban-nicusor-toptal to this repository if that is fine with you @jgabry

He is going to help with setting up some of this and review CI stuff. We are debating whether to use Jenkins or Github Actions. In general, it should not be that difficult to move from one to the other as all of these are going to be R scripts and this should all be fast, outside of maybe building rstanarm-like packages. For posterior/cmdstanr and such these are all jobs that take a minute or so.

I don't have a strong opinion on which of these to use, maybe use Jenkins for building packages and G.A. for the rest of the maintenance. @serban-nicusor-toptal would you be cool with that. I will upload the scripts and we can then start doing this whenever you have time.

[serban-nicusor-toptal]

Sounds good to me, Jenkins can do the heavy lifting as we have very strong machines.
Let me know when to get started!

[rok-cesnovar]

So anyone with write permissions to this repository should be able to run a custom job on Jenkins.
Its input would be the repository to an R package Github repository (not this one) and a git hash or branch name.

Repo: https://github.com/stan-dev/posterior
branch: master

Jenkins then clones that repository and runs an R script that does all the rest and then just pushes the result to this repository. Apart from the R script you did this kind of stuff a bunch of times so that should be doable I guess. This will run on Windows and Mac to build the binaries for those OS.

[jgabry]

I am going to add @serban-nicusor-toptal to this repository if that is fine with you @jgabry

Sounds good, and thanks @serban-nicusor-toptal for helping out with this!

[serban-nicusor-toptal]

Hey @rok-cesnovar that sounds fairly simple, may I ask about the runs an R script that does all the rest part, do we have that or should I write it ? If so what should it do ?

[rok-cesnovar]

I will prepare that, should be simple but never ran it on a mac so who knows.

[rok-cesnovar]

To build the binary packages on Windows/Mac you need to run:

install.packages("devtools")
remotes::install_deps(dependencies = TRUE)
devtools::build(pkg = ".", binary = TRUE, vignettes = FALSE, manual = FALSE)

Set pkg argument to the folder where the repository is clone. This build a .zip file on Windows and a .tgz file on Mac. They are built in the parent folder.

In addition to Windows/Mac binaries, this should build a source package (

install.packages("devtools")
remotes::install_deps(dependencies = TRUE)
devtools::build(pkg = ".", binary = FALSE)

The source package can be run on any machine and it should build a .tar.gz file. You can try with the stan-dev/cmdstanr that should build quickly. A more demanding package that requires a lot of resources is stan-dev/rstanarm. I was unable to build the latter on a Github Actions Windows machine so I guess it requires more than 7GB of RAM.

After you have the archives built you can push them to this repository with

install.packages("drat")
drat::insertPackage(file = "archiveFilepath.extension", repodir = "path/to/the/cloned/r-packages/repo", pullfirst = TRUE, commit = TRUE)

This should also commit and push the changes. Otherwise you can set commit = FALSE and commit & push outside of R.

[serban-nicusor-toptal]

Hey @rok-cesnovar
Here you can find the Jenkins job: https://jenkins.mc-stan.org/job/R-Packages/

Ran with:

Repo: https://github.com/stan-dev/posterior
branch: master

https://jenkins.mc-stan.org/blue/organizations/jenkins/R-Packages/detail/R-Packages/43/pipeline

Please give it a try and let me know if it's all functioning properly, thanks!

[rok-cesnovar]

Works great, just needs a minor change that is on me.
Change the second line to:

remotes::install_deps(dependencies = TRUE, repos = c(getOption("repos"), "https://mc-stan.org/r-packages/"))

That enables us to also build packages that have some dependency on this repo only.

[serban-nicusor-toptal]

Did the change and tried with https://github.com/stan-dev/rstanarm branch feature/survival seems it doesn't work
https://jenkins.mc-stan.org/blue/organizations/jenkins/R-Packages/detail/R-Packages/48/pipeline

Looks like an R error, can you please take a look? Let me know if I need to change anything in Jenkins, thanks!

[rok-cesnovar]

https://github.com/stan-dev/rstanarm branch feature/survival is going to be a tough one. If we can build that branch, then we can build anything :) It seems to run out of RAM.

I have to figure out if we can add any compiler flags to decrease the ram usage there.

To test remotes::install_deps(dependencies = TRUE, repos = c(getOption("repos"), "https://mc-stan.org/r-packages/"))
just try building

https://github.com/stan-dev/cmdstanr on master

[serban-nicusor-toptal]

Hmh the errors says 18762128 bytes which would go around 18 MB for sure we have more RAM than that.
Will try on cmdstanr

[jgabry]

Thanks for trying that rstanarm branch. I think it needs a lot more than 18MB of RAM, so I'm not sure why it says that.

https://github.com/stan-dev/rstanarm branch feature/survival is going to be a tough one. If we can build that branch, then we can build anything :) It seems to run out of RAM.

I think that's why it's not on CRAN yet. @bgoodri is it definitely the RAM issue that is preventing us from merging @sambrilleman's branch and submitting to CRAN? We really need to figure out a way for people to use that branch. It's been sitting around way too long and people are having trouble installing it from github.

[serban-nicusor-toptal]

cmdstanr is building fine on windows but not on mac for some reason: https://jenkins.mc-stan.org/blue/organizations/jenkins/R-Packages/detail/R-Packages/50/pipeline/33

[rok-cesnovar]

Have you tried: https://stackoverflow.com/questions/9689104/installing-r-on-mac-warning-messages-setting-lc-ctype-failed-using-c

[serban-nicusor-toptal]

Yea, that fixed it at first. Gonna look into it to see why we're getting that again.

[serban-nicusor-toptal]

cmdstanr is building fine now on mac https://jenkins.mc-stan.org/blue/organizations/jenkins/R-Packages/detail/R-Packages/54/pipeline
Gonna look into rstanarm

[jgabry]

Awesome, thank you!

[serban-nicusor-toptal]

rstanarm builds fine on mac now: https://jenkins.mc-stan.org/blue/organizations/jenkins/R-Packages/detail/R-Packages/61/pipeline/8
But on windows it just gets stuck, gonna try today on a beefier machine from AWS.

[serban-nicusor-toptal]

Seems to be failing on windows with the following:

E> C:/Program Files/R/R-4.0.2/library/RcppEigen/include/Eigen/src/Core/DenseCoeffsBase.h:55:30: warning: ignoring attributes on template argument 'Eigen::internal::packet_traits<double>::type' {aka '__vector(4) double'} [-Wignored-attributes]
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s: Assembler messages:
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78354: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78356: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78358: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78360: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78362: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78364: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78366: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:78368: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:80564: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:80566: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:80568: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:143883: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:143885: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:143887: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:143889: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:143891: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:218649: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:238928: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:271722: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:271724: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:281744: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:281746: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:291870: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:310102: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:310104: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:310106: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:310108: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:323981: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:323983: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:323985: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:323987: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:363627: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:363629: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:363631: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:363633: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:363635: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:363637: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:366155: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:366157: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:366159: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:366161: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:366163: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:372208: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:372210: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:372212: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:374276: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377165: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377167: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377169: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377171: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377173: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377175: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377177: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377179: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377181: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377183: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:377185: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:378971: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381635: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381637: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381639: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381641: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381643: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381645: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381647: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381649: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381651: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381653: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381655: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381657: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381659: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381661: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381663: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:381665: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387385: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387387: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387389: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387391: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387393: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387395: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387397: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387399: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387401: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387403: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387405: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387407: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387409: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387411: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387413: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:387415: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:411979: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:411981: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:489893: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:489895: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:489897: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:489899: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:489901: Error: invalid register for .seh_savexmm
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:386095: Error: value of 285 too large for field of 1 byte at 0000000000000001
E> C:\Users\ADMINI~1\AppData\Local\Temp\2\cc5h8Cwe.s:391847: Error: value of 285 too large for field of 1 byte at 0000000000000001
E> make: *** [C:/PROGRA~1/R/R-40~1.2/etc/i386/Makeconf:227: stan_files/surv.o] Error 1
E> rm stan_files/surv.cc
E> ERROR: compilation failed for package 'rstanarm'
E> * removing 'C:/Users/Administrator/AppData/Local/Temp/2/Rtmpi0TqJv/temp_libpath1ee458e4101c/rstanarm'

Execution halted

Will run now through Jenkins so I can give you guys the full output.
Any idea why an asm error here ? This can be from the code or an OS misconfiguration?

PS: While building it tops at around 10 GB RAM! <- Bad numbers, it actually hits around 3~3.5GB(edit)
I've installed R version R-4.0.2 !

[jgabry]

Thanks for working on this! Glad it's working on mac.

Any idea why an asm error here ? This can be from the code or an OS misconfiguration?

Hmm, I'm not sure. Unfortunately I'm pretty useless with Windows-specific issues. @bgoodri, @sambrilleman or @rok-cesnovar any ideas?

PS: While building it tops at around 10 GB RAM!

Wow, I bet that's why many people failed to install from GitHub. I think a lot of people still have laptops with 8GB or less.

[rok-cesnovar]

Thanks @serban-nicusor-toptal for this and also fixing the instances with Rtools!

[sambrilleman]

Hey guys, huge thanks for trying to find a solution to this rstanarm issue. I guess a few things I am curious about:

• do we know if it is specifically surv.stan that has the huge memory requirements? Or was surv.stan just the unlucky one that pushed total memory usage over the brink? I can't think of how it would use more memory compared with the other .stan files, except perhaps the fact it is one .stan file rather than "chunks", but I assume this has no impact because I would have thought that the text chunks would be bound together before any compilation takes place.

• the feature/survival branch is now probably way out of sync with the master branch. It has been idle for 9 months or more. So could the additional battle with feature/survival compared to master be something related to either: i) changes made to the master branch of rstanarm in that time, and/or ii) changes that were made in R4.x (because I'm guessing there hasn't been any changes to the feature/survival branch since the days of R3.6 or thereabouts).

Beyond that I probably have very little useful input. Although I use Windows, all the stuff around compilers and their issues is quite a foreign language to me 🤦‍♂️

[serban-nicusor-toptal]

Just to clarify, my first numbers were wrong.
It actually uses around 3~3.5 GB.

I couldn't get it to work on the windows server.

Locally I'm getting (master branch):

g++.exe: error: Files/R/R-4.0.2/library/RcppParallel/include": No such file or directory

Locally I'm getting (feature/survivalbranch):

cc1plus.exe: out of memory allocating 14907328 bytes

While running:

install.packages("devtools", repos = "http://cran.us.r-project.org")
remotes::install_deps(dependencies = TRUE, repos = c(getOption("repos"), "https://mc-stan.org/r-packages/"))
devtools::build(pkg = ".", binary = TRUE, vignettes = FALSE, manual = FALSE)

Using:
rtools40
R-4.0.2

[serban-nicusor-toptal]

Hey, did anyone managed to get rstanarm feature/survival branch to work on windows ? I'm stuck in getting it to work locally/on the windows server and I'm not sure what I'm doing wrong here.

[rok-cesnovar]

I tried a few times but was unable to. Does the master branch build?

[serban-nicusor-toptal]

It doesn't for me locally, getting:

g++.exe: error: Files/R/R-4.0.2/library/RcppParallel/include": No such file or directory

[jgabry]

Thanks for trying again.

g++.exe: error: Files/R/R-4.0.2/library/RcppParallel/include": No such file or directory

Hmm, I'm not sure what's causing this since RcppParallel is included in the dependencies for rstanarm, so it should be there. I don't have a Windows machine so I'm not much help here, sorry.

I think we really need @bgoodri to chime in here.

Also, @sambrilleman can you confirm that you are still able to build your feature/survival branch successfully on your windows machine?

[rok-cesnovar]

@serban-nicusor-toptal Can you try

devtools::build(pkg = ".", binary = TRUE, vignettes = FALSE, manual = FALSE, args = c("--no-multiarch"))

[serban-nicusor-toptal]

That gives me locally

E> C:/rtools40/mingw64/bin/../lib/gcc/x86_64-w64-mingw32/8.3.0/../../../../x86_64-w64-mingw32/bin/ld.exe: stan_files/binomial.o:binomial.cc:(.text$_ZNK24model_binomial_namespace14model_binomial8log_probILb1ELb1EdEET1_RSt6vectorIS2_SaIS2_EERS3_IiSaIiEEPSo[_ZNK24model_binomial_namespace14model_binomial8log_probILb1ELb1EdEET1_RSt6vectorIS2_SaIS2_EERS3_IiSaIiEEPSo]+0x2c7a): undefined reference to `Eigen::Matrix<boost::math::tools::promote_args<double, double, float, float, float, float>::type, -1, 1, 0, -1, 1> model_binomial_namespace::csr_matrix_times_vector2<double, double>(int const&, int const&, Eigen::Matrix<double, -1, 1, 0, -1, 1> const&, std::vector<int, std::allocator<int> > const&, std::vector<int, std::allocator<int> > const&, Eigen::Matrix<double, -1, 1, 0, -1, 1> const&, std::ostream*)'
E> collect2.exe: error: ld returned 1 exit status
E> rm stan_files/jm.cc stan_files/binomial.cc stan_files/lm.cc stan_files/continuous.cc stan_files/bernoulli.cc stan_files/polr.cc stan_files/surv.cc stan_files/count.cc stan_files/mvmer.cc
E> no DLL was created
E> ERROR: compilation failed for package 'rstanarm'
E> * removing 'C:/Users/Nick/AppData/Local/Temp/RtmpMhKITc/temp_libpath22e830b55e54/rstanarm'

Execution halted

Windows 10
Version 2004 ( OS Build 19041.330 )

[sambrilleman]

@jgabry Nah, unfortunately I am having just as much trouble as everyone else. I was even struggling to build the master branch locally when I tried earlier in the week (or at least it built, but then running the rstanarm::example_model made my R session crash every time). I haven't been actively working on this for 6 months or more, so I'm not sure at what point things started to fall over! 😞

[sambrilleman]

fwiw I get the same error as @serban-nicusor-toptal, i.e.

g++.exe: error: Files/R/R-4.0.2/library/RcppParallel/include": No such file or directory

when trying to build the master branch, as follows:

> remotes::install_github(
+     "stan-dev/rstanarm", 
+     build_vignettes = FALSE,
+     build_opts = c("--no-multiarch"), 
+     upgrade = "never" # this should ignore packages that need updates like @ermeel86 suggested (not sure if necessary though) 
+ )
Downloading GitHub repo stan-dev/rstanarm@master
stan-dev-rstanarm-14ad3ec/tests/testthat/include: Can't create '\\\\?\\C:\\Users\\SBRILL~1\\AppData\\Local\\Temp\\RtmpU1BusS\\remotes2ff0b1127db\\stan-dev-rstanarm-14ad3ec\\tests\\testthat\\include'
stan-dev-rstanarm-14ad3ec/tests/testthat/stan_files: Can't create '\\\\?\\C:\\Users\\SBRILL~1\\AppData\\Local\\Temp\\RtmpU1BusS\\remotes2ff0b1127db\\stan-dev-rstanarm-14ad3ec\\tests\\testthat\\stan_files'
tar.exe: Error exit delayed from previous errors.
   Warning: unknown option '--no-multiarch'
√  checking for file 'C:\Users\sbrilleman\AppData\Local\Temp\RtmpU1BusS\remotes2ff0b1127db\stan-dev-rstanarm-14ad3ec/DESCRIPTION' ... 
-  preparing 'rstanarm': (554ms)
√  checking DESCRIPTION meta-information ... 
-  cleaning src
-  running 'cleanup.win' (397ms)
-  checking for LF line-endings in source and make files and shell scripts (447ms)
-  checking for empty or unneeded directories
-  building 'rstanarm_2.19.3.tar.gz'
   Warning: file 'rstanarm/cleanup' did not have execute permissions: corrected
   
* installing *source* package 'rstanarm' ...
** using staged installation
** libs

*** arch - i386
"C:/PROGRA~1/R/R-40~1.2/bin/i386/Rscript" -e "source(file.path('..', 'tools', 'make_cc.R')); make_cc(commandArgs(TRUE))" stan_files/continuous.stan
Wrote C++ file "stan_files/continuous.cc"


C:/Rtools/mingw_32/bin/g++ -m32 -std=c++1y -std=gnu++14 -I"C:/PROGRA~1/R/R-40~1.2/include" -DNDEBUG -I"../inst/include" -I"C:/Program Files/R/R-4.0.2/library/StanHeaders/include/src" -DBOOST_DISABLE_ASSERTS -DEIGEN_NO_DEBUG `"C:/PROGRA~1/R/R-40~1.2/bin/Rscript" -e "RcppParallel::CxxFlags()"` `"C:/PROGRA~1/R/R-40~1.2/bin/Rscript" -e "StanHeaders:::CxxFlags()"`	 -I'C:/Program Files/R/R-4.0.2/library/StanHeaders/include' -I'C:/Program Files/R/R-4.0.2/library/rstan/include' -I'C:/Program Files/R/R-4.0.2/library/BH/include' -I'C:/Program Files/R/R-4.0.2/library/Rcpp/include' -I'C:/Program Files/R/R-4.0.2/library/RcppEigen/include' -I'C:/Program Files/R/R-4.0.2/library/RcppParallel/include'        -O3 -march=native -c stan_files/continuous.cc -o stan_files/continuous.o
g++.exe: error: Files/R/R-4.0.2/library/RcppParallel/include": No such file or directory
make: *** [C:/PROGRA~1/R/R-40~1.2/etc/i386/Makeconf:227: stan_files/continuous.o] Error 1
rm stan_files/continuous.cc
ERROR: compilation failed for package 'rstanarm'
* removing 'C:/Program Files/R/R-4.0.2/library/rstanarm'
Error: Failed to install 'rstanarm' from GitHub:
  (converted from warning) installation of package ‘C:/Users/SBRILL~1/AppData/Local/Temp/RtmpU1BusS/file2ff0206528e8/rstanarm_2.19.3.tar.gz’ had non-zero exit status
In addition: Warning message:
In utils::untar(tarfile, ...) :
  ‘tar.exe -xf "C:\Users\SBRILL~1\AppData\Local\Temp\RtmpU1BusS\file2ff07f7a2f82.tar.gz" -C "C:/Users/SBRILL~1/AppData/Local/Temp/RtmpU1BusS/remotes2ff0b1127db"’ returned error code 1

[serban-nicusor-toptal]

Looks like rstanarm is not compatible with newer versions of R as it didn't work for me on 4.0.2 on windows and same reported here on mac

[jgabry]

I think (hope) this might be easier to get working now with rstanarm and rstan >= 2.21. At least it might work with the latest changes on the rstanarm master branch. @sambrilleman It may also work with the survival branch but that will need to get updated from master first.

At least I can now get it working on Mac with R 4.0.2. It might also be ok on Windows without -march=native in the ~/.R/Makevars.win file, which apparently causes problems for the latest RStan. Apparently CXX14FLAGS=-O3 -mtune=native -mmmx -msse -msse2 -msse3 -mssse3 -msse4.1 -msse4.2 is fine but not -march=native:

https://discourse.mc-stan.org/t/rstan-on-windows/16673