[discussion] Publish packages to Conda Forge

[kinow]

Posted to riot by @oliver-sanders , we will need to consider what to do, assuming #1 proves it works. One possibility is to upload the packages to conda forge - https://conda-forge.org/#add_recipe

Instead of using a personal space, as we are doing now for testing.

(cylc-uiserver depends on graphene-tornado, graphql-ws, and cylc-flow; cylc depends on all the packages being released; the review of cylc recipe may take longer as it's not a conventional conda python package)

• metomi-isodatetime Add metomi-isodatetime recipe for conda-forge conda-forge/staged-recipes#9610
• graphene-tornado Publish graphene-tornado to Conda Forge graphql-python/graphene-tornado#35 & Add graphene-tornado conda-forge/staged-recipes#9763
• graphql-ws (for subscriptions; need to check if it has transitive dependencies that also need publishing) Publish graphql-ws to Conda Forge graphql-python/graphql-ws#36 & Add graphql-ws conda-forge/staged-recipes#9765
• cylc-flow Add cylc-flow recipe conda-forge/staged-recipes#9661
• cylc-uiserver Add Cylc UI Server conda-forge/staged-recipes#9786
• cylc

[matthewrmshin]

I vote to use Conda Forge.

[oliver-sanders]

Can't think why we would not publish to Forge.

[kinow]

Twitter thread about conda forge and with some details about PYPI synchronization: https://twitter.com/xhochy/status/1173182116689059840

[kinow]

I think I can give it a try. Will try to deploy graphene-tornado and metomi-isodatetime first, as they are requirements for other projects.

This will be a good way to give my brain some time to rest before jumping on the next issues for Cylc 8.0a2 in Cylc UI.

[kinow]

Done for metomi-isodatetime. Depending how it goes, will start working on graphene-tornado and other modules (will check first on Riot before doing that to see if another person would be interested in doing it, so that I'm not the only know that knows how to do it).

conda-forge/staged-recipes#9610

[kinow]

Graphene Tornado recipe submitted: conda-forge/staged-recipes#9763

[kinow]

GraphQL recipe submitted: conda-forge/staged-recipes#9765

[kinow]

Struggled a bit finding what was wrong with the graphql-ws recipe, until I found out that had copied by accident the pip requirement version without modifying it to the conda equivalent syntax (which I have no idea if that's in some PEP, or was created by them).

Added an issue to include a bit more of information in the log message, though not sure if the best option conda/conda-build#3761

[marcelotrevisani]

Graphene Tornado recipe submitted: conda-forge/staged-recipes#9763

I merged it already! :)

[kinow]

Hooray! Still have a couple more recipes to bother you guys with later 🙏 Really impressed with how quick and well the recipes PR's are reviewed on Conda Forge. Thanks / Obrigado @marcelotrevisani !

[marcelotrevisani]

Hooray! Still have a couple more recipes to bother you guys with later pray Really impressed with how quick and well the recipes PR's are reviewed on Conda Forge. Thanks / Obrigado @marcelotrevisani !

😃 Sometimes a PR may take a while to review if it happens just mark the conda-forge/staged-recipes and we will try our best to review it asap! :)

[marcelotrevisani]

Hooray! Still have a couple more recipes to bother you guys with later pray Really impressed with how quick and well the recipes PR's are reviewed on Conda Forge. Thanks / Obrigado @marcelotrevisani !

Tamo junto! hahaha 😆

[marcelotrevisani]

GraphQL recipe submitted: conda-forge/staged-recipes#9765

Merged

Struggled a bit finding what was wrong with the graphql-ws recipe, until I found out that had copied by accident the pip requirement version without modifying it to the conda equivalent syntax (which I have no idea if that's in some PEP, or was created by them).

Added an issue to include a bit more of information in the log message, though not sure if the best option conda/conda-build#3761

Merged as well

[kinow]

And I thinking that I would need to wait a few days before working on the next cylc-uiserver recipe. It will be a simple one I believe.

I hope it's fine if I bother you here very quickly @marcelotrevisani. I was going ask on Gitter (I am lurking there already), but just in case it's something quick/easy to answer.

Q/ we have a package that is not a conventional Python package, https://github.com/cylc/cylc-conda/blob/master/recipes/cylc/meta.yaml

It uses url + sha256 + folder to download & unzip the GitHub release of our Vue.js frontend application, and also installs all the required Conda packages for the Cylc system to work.

Can we still submit a PR for Conda Forge for packages like this one?

Thanks!
Bruno

[oliver-sanders]

Not a solution but I guess we should package the Cylc UI separately?

[kinow]

Not a solution but I guess we should package the Cylc UI separately?

I think it is already being packaged separately. It is packaged as a ZIP and uploaded to a GitHub release/tag. We could use NPM (though not recommended as it is tailored for libraries, not complete web apps), or one of the new GitHub repositories.

But I think it would not change our Conda recipe for cylc. If we want to have one conda install command that provisions everything, then this command will have to download the Cylc UI package from somewhere and wouldn't be very different from what we have I guess.

Or are you suggesting that conda install cylc installs the Python dependencies, and we then ask the user to download Cylc UI, unpack it somewhere, and update jupyterhub_config.py accordingly?

[marcelotrevisani]

And I thinking that I would need to wait a few days before working on the next cylc-uiserver recipe. It will be a simple one I believe.

I hope it's fine if I bother you here very quickly @marcelotrevisani. I was going ask on Gitter (I am lurking there already), but just in case it's something quick/easy to answer.

Q/ we have a package that is not a conventional Python package, https://github.com/cylc/cylc-conda/blob/master/recipes/cylc/meta.yaml

It uses url + sha256 + folder to download & unzip the GitHub release of our Vue.js frontend application, and also installs all the required Conda packages for the Cylc system to work.

Can we still submit a PR for Conda Forge for packages like this one?

Thanks!
Bruno

Yes sure, you can submit a PR for that as well! you can create conda package for any language and platform

You can also create a conda package for the Cylc UI which will rely on cylc as a dependency

[kinow]

Thanks @marcelotrevisani !!!

[marcelotrevisani]

@kinow feel free to send messages to me on gitter as well, that is not a problem :)

[kinow]

Had some hiccups with cylc-uiserver as it had some different requirements for pytest-runner, to be fixed in cylc/cylc-uiserver#91

[kinow]

99% done. Just waiting for cylc-ui to show in conda forge. Then will prepare Cylc metapackage. Closing this ticket as README info will be moved to cylc-doc, and this repo can then be archived.