[ci] automate building CRAN package

[jameslamb]

Opening this PR, from a LightGBM branch, to test a GitHub Actions job that can be used to automatically build the CRAN package.

A step towards #3283

I'm not planning to do the package binaries...that can come in a separate PR.

If this works, it would allow any maintainer to build the CRAN package from a pull request, without needing their own local R environment.

[jameslamb]

/gha build r-artifacts

[jameslamb]

ok this created this job: https://github.com/microsoft/LightGBM/actions/runs/332889387

which failed like this:

No files were found with the provided path: /__w/LightGBM/LightGBM/lightgbm-3.0.0.99-r-cran.tar.gz\n. No artifacts will be uploaded.

[jameslamb]

/gha build-r-artifacts

[jameslamb]

/gha build r-artifacts

[jameslamb]

this kicked off https://github.com/microsoft/LightGBM/actions/runs/332893223

which produced one artifact

this creates a zip of the file, which is a little annoying, and it looks like you can't get around that (https://github.com/actions/upload-artifact#zipped-artifact-downloads):

when a workflow run finishes and an artifact is downloaded from either the UI or through the download api, a zip is dynamically created with all the file contents that were uploaded. There is currently no way to download artifacts after a workflow run finishes in a format other than a zip or to download artifact contents individually

But I don't think it's too bad. So for releases, the workflow would be:

1. comment with /gha build r-artifacts
2. go to https://github.com/microsoft/LightGBM/actions, then to the artifacts action
3. once it's done running, download the .zip
4. unzip it, to get the artifact
5. manually upload artifacts to the release

[jameslamb]

ok I think this is ready for review!

[StrikerRUS]

I think this can be documented in https://github.com/microsoft/LightGBM/blob/master/R-package/README.md#preparing-a-cran-package.

[jameslamb]

oh good idea. Added in bd895bf

[StrikerRUS]

@jameslamb

Awesome!

I'm not planning to do the package binaries...that can come in a separate PR.

If I'm not mistaken, we discussed that binaries should be taken from CRAN release and added to GitHub release, right? Pre-built binaries are needed only temporary for the time we are not accepted at CRAN.

[jameslamb]

@jameslamb

Awesome!

I'm not planning to do the package binaries...that can come in a separate PR.

If I'm not mistaken, we discussed that binaries should be taken from CRAN release and added to GitHub release, right? Pre-built binaries are needed only temporary for the time we are not accepted at CRAN.

oh right! Yes you are totally correct, I forgot about that. Excellent 😎

[jameslamb]

I think the check-docs test failure is real and that the paper linked there has moved. I'll submit a separate PR to fix it

URL        `https://www.mcs.anl.gov/~thakur/papers/ijhpca-coll.pdf'
Name       `Optimization of collective communication operations in MPICH'
Parent URL file:///home/travis/build/microsoft/LightGBM/docs/_build/html/Features.html, line 408, col 63
Real URL   https://www.mcs.anl.gov/~thakur/papers/ijhpca-coll.pdf
Check time 60.892 seconds
Size       160B
Result     Error: 504 Gateway Time-out

[StrikerRUS]

@jameslamb Looks like the whole authors' section on that site (https://www.mcs.anl.gov/~thakur/) is down. So I think we can wait a little bit.

[StrikerRUS]

LGTM for now, thanks! But I think that in the (hopefully near) future we will need R package build jobs only for new releases and it will be possible to prepare artifacts at Azure Pipelines without any manual triggers but on v* tag pushes. Just like NuGet artifact in a separate step and as GitHubRelease@0 task with certain conditions:

LightGBM/.vsts-ci.yml

Lines 159 to 171 in 5cc9e67

	- task: NuGetCommand@2
	inputs:
	command: pack
	packagesToPack: '$(Build.SourcesDirectory)/.nuget/*.nuspec'
	packDestination: '$(Build.ArtifactStagingDirectory)'
	- task: PublishBuildArtifacts@1
	inputs:
	pathtoPublish: '$(Build.ArtifactStagingDirectory)'
	artifactName: NuGet
	artifactType: container
	- task: GitHubRelease@0
	displayName: 'Create GitHub Release'
	condition: and(succeeded(), startsWith(variables['Build.SourceBranch'], 'refs/tags/v'))

[jameslamb]

@jameslamb Looks like the whole authors' section on that site (https://www.mcs.anl.gov/~thakur/) is down. So I think we can wait a little bit.

this is back now! Glad it was just something temporary.

[jameslamb]

in the (hopefully near) future we will need R package build jobs only for new releases and it will be possible to prepare artifacts at Azure Pipelines without any manual triggers but on v* tag pushes

Sure! I think that for now the comment-triggered thing is better, since we might have to iterate multiple times on the same release to get to CRAN (like we did with #3338 ).

Since this job is a small handful of shell commands in a container, I think it will be easy enough to run in Azure DevOps if/when we decide that that artifacts job is a better place for this to run.

[github-actions]

This pull request has been automatically locked since there has not been any recent activity since it was closed. To start a new related discussion, open a new issue at https://github.com/microsoft/LightGBM/issues including a reference to this.