Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Consider multiple size-based releases 🔬🔍🔭📡 #42

Open
bollwyvl opened this issue Sep 18, 2019 · 16 comments
Open

Consider multiple size-based releases 🔬🔍🔭📡 #42

bollwyvl opened this issue Sep 18, 2019 · 16 comments
Labels
🚧 build Pertains to CI and build

Comments

@bollwyvl
Copy link
Collaborator

RobotLab is big and likely to get bigger as more cool features need more (exotic) dependencies.

We could just as easily spin a couple different intstallers a la Miniconda/Anaconda in CI.

MicrobotLab 🔬

  • for CI environments
    • you'll already have browsers, etc. hopefully
  • don't even ship conda (pip usually works, right? and conda is showing up on more CIs)

RobotLabLite 🔍

  • still a full working environment
  • cut it down to the bone (opencv, nodejs, pandoc), and even repackage some things (jupyterlab assets)
  • still ship conda so you can upgrade stuff

RobotLab 🔭

  • as it is now!
    • even shipping Firefox is great!
  • this is ideal for the original intent of training on non-trivial things, where the setup of some of the things have proven (really hard for people to install](https://stackoverflow.com/search?q=opencv+install).

RobotHub 📡
@bollwyvl bollwyvl added the 🚧 build Pertains to CI and build label Sep 19, 2019
@bollwyvl bollwyvl mentioned this issue Sep 19, 2019
Closed
@datakurre
Copy link
Contributor

My workshop for the next robot conference was accepted, which confirms that I will be attending the sprints there. Apparently there is still no installer available for Robot Framework, so I've been thinking, if I should try to fork this project and try this "multiple size-based releases" for that. For plain robot it would make sense to have minimal version with just robot framework, but also a version for getting started with selenium testing (the real reason for the installer).

@bollwyvl
Copy link
Collaborator Author

workshop for the next robot conference was accepted

Hooray!

fork this project

You know I'm glad to help, and a fork is just more repos to watch: I don't see any reason why we wouldn't start it here, and why we wouldn't want synchronized, automated releases of the whole shooting match. If somebody wants to then fork that, at some point, that's great.

We can even stop doing the kinda gross templating, and just manage the yaml generation more simply in python directly, if we start feeling confident. The CI build would take longer, but i think there's value in managing a single pipeline for all the stuff together.

minimal version with just robot framework

yup, as a CI thing, that would be great. MiniRobotConda. Even if it doesn't ship conda (which it probably should, if only for activation and administration chores) it can't get much smaller than about 70mb, but that's not so bad!

version for getting started with selenium testing

so even there i would imagine two flavors:

  • WebRobotConda (just rf, rfsl, sl and maybe rfstl, if that plays out, and geckodriver)
  • FireWebRobotConda (as above, but also firefox)

i'm really liking having firefox in our installer, and if we can do our little part to help more people test on firefox, i'm all for it. The only gotcha is on Linux, you do still have to have gtk3 installed, even when running --headless, but i'll still take it.

Unless we ship all the chromedrivers, we can't really ship any chromedrivers, but we could ship webdrivermanager, and indeed ship all of them pre-cached.

@datakurre
Copy link
Contributor

Thanks. I can agree with your reasoning. I also prefer Firefox being bundled because now things just work. It also makes sense that whatever version or browser you use for surfing in the web should not interfere with learning Selenium testing. Finally, promoting Firefox would not hurt the diversity in browser ecosystem...

That said I cannot really focus on this before November, but will then.

@bollwyvl
Copy link
Collaborator Author

That said I cannot really focus on this before November, but will then.

I hear you, my friend. Who knows, maybe i'll be able to throw some stuff at it. What is your priority for getting something out? Maybe hack on the summary of this issue to make it clear. I guess I would lean towards the smallest functioning robot environment, just so we have a size/complexity baseline, but if the web one is more useful we could start there.

@datakurre
Copy link
Contributor

I’d vote for the baseline. We already know that we can make a GB sized installer with everything :)

After the baseline, another baseline with SL, gecko and Firefox, and then documented approach to fork and extend those baselines.

Finally, at the conf it makes sense to discuss what libraries should be bundled with the default starter (for learning and getting started purposes):

@bollwyvl
Copy link
Collaborator Author

bollwyvl commented Oct 1, 2019

i did a little playing with this last night:

609M Sep 30 23:58 RobotLab-2019.9.1-Linux-x86_64.sh
 84M Oct  1 00:05 MiniRobot-2019.9.1-Linux-x86_64.sh
161M Oct  1 00:35 ExoRobot-2019.9.1-Linux-x86_64.sh

Mini is rf + conda. not shipping conda only saves about 2mb.
Exo is Mini, plus every optional dependency i could find in the rf codebase. The heavy hitter is definitely wx, which screenshots uses. Might as well throw in RIDE at that point...

@datakurre
Copy link
Contributor

Very cool! Which screenshots feature relies on wx? Does dialogs library work? The point for a installer is to give environment that just works, so from that point of view 200M is not much nowadays.

And now that you mentioned RIDE... does it already support Python 3? Sure many would like it to be included and it’s been hard to install on many systems because of wx. (Although that probably was because for long it required very old version of that.)

@bollwyvl
Copy link
Collaborator Author

bollwyvl commented Oct 2, 2019

here's the full package list of `Exo` (on linux) 
asn1crypto-0.24.0-py36_1003.tar.bz2
atk-2.32.0-haf93ef1_0.tar.bz2
ca-certificates-2019.9.11-hecc5488_0.tar.bz2
cairo-1.16.0-hfb77d84_1002.tar.bz2
certifi-2019.9.11-py36_0.tar.bz2
cffi-1.12.3-py36h8022711_0.tar.bz2
chardet-3.0.4-py36_1003.tar.bz2
conda-4.7.12-py36_0.tar.bz2
conda-package-handling-1.6.0-py36h516909a_0.tar.bz2
cryptography-2.7-py36h72c5cf5_0.tar.bz2
docutils-0.15.2-py36_0.tar.bz2
expat-2.2.5-he1b5a44_1003.tar.bz2
fontconfig-2.13.1-h86ecdb6_1001.tar.bz2
freetype-2.10.0-he983fc9_1.tar.bz2
fribidi-1.0.5-h516909a_1002.tar.bz2
gdk-pixbuf-2.36.12-h7a26e22_1003.tar.bz2
gettext-0.19.8.1-hc5be6a0_1002.tar.bz2
glib-2.58.3-h6f030ca_1002.tar.bz2
gobject-introspection-1.58.2-py36h5503ade_1002.tar.bz2
graphite2-1.3.13-hf484d3e_1000.tar.bz2
gst-plugins-base-1.14.5-h0935bb2_0.tar.bz2
gstreamer-1.14.5-h36ae1b5_0.tar.bz2
gtk2-2.24.32-h586f36d_1.tar.bz2
harfbuzz-2.4.0-h9f30f68_3.tar.bz2
icu-64.2-he1b5a44_1.tar.bz2
idna-2.8-py36_1000.tar.bz2
jpeg-9c-h14c3975_1001.tar.bz2
libffi-3.2.1-he1b5a44_1006.tar.bz2
libgcc-ng-9.1.0-hdf63c60_0.tar.bz2
libglu-9.0.0-hf484d3e_1000.tar.bz2
libiconv-1.15-h516909a_1005.tar.bz2
libpng-1.6.37-hed695b0_0.tar.bz2
libstdcxx-ng-9.1.0-hdf63c60_0.tar.bz2
libtiff-4.0.10-h57b8799_1003.tar.bz2
libuuid-2.32.1-h14c3975_1000.tar.bz2
libxcb-1.13-h14c3975_1002.tar.bz2
libxml2-2.9.9-hee79883_5.tar.bz2
libxslt-1.1.33-h31b3aaa_0.tar.bz2
lxml-4.4.1-py36h7ec2d77_0.tar.bz2
lz4-c-1.8.3-he1b5a44_1001.tar.bz2
ncurses-6.1-hf484d3e_1002.tar.bz2
openssl-1.1.1c-h516909a_0.tar.bz2
pango-1.42.4-ha030887_1.tar.bz2
pathlib2-2.3.5-py36_0.tar.bz2
pcre-8.41-hf484d3e_1003.tar.bz2
pip-19.2.3-py36_0.tar.bz2
pixman-0.38.0-h516909a_1003.tar.bz2
pthread-stubs-0.4-h14c3975_1001.tar.bz2
pycosat-0.6.3-py36h14c3975_1001.tar.bz2
pycparser-2.19-py36_1.tar.bz2
pyopenssl-19.0.0-py36_0.tar.bz2
pypubsub-4.0.3-py_0.tar.bz2
pysocks-1.7.1-py36_0.tar.bz2
pyte-0.8.0-py_0.tar.bz2
python-3.6.7-h357f687_1005.tar.bz2
pyyaml-5.1.2-py36h516909a_0.tar.bz2
readline-8.0-hf8c457e_0.tar.bz2
requests-2.22.0-py36_1.tar.bz2
robotframework-3.1.2-py_0.tar.bz2
ruamel_yaml-0.15.71-py36h14c3975_1000.tar.bz2
setuptools-41.2.0-py36_0.tar.bz2
six-1.12.0-py36_1000.tar.bz2
sqlite-3.29.0-hcee41ef_1.tar.bz2
tk-8.6.9-hed695b0_1003.tar.bz2
tqdm-4.36.1-py_0.tar.bz2
urllib3-1.25.6-py36_0.tar.bz2
wcwidth-0.1.7-py_1.tar.bz2
wheel-0.33.6-py36_0.tar.bz2
wxpython-4.0.6-py36h8f66242_1.tar.bz2
xorg-kbproto-1.0.7-h14c3975_1002.tar.bz2
xorg-libice-1.0.10-h516909a_0.tar.bz2
xorg-libsm-1.2.3-h84519dc_1000.tar.bz2
xorg-libx11-1.6.8-h516909a_0.tar.bz2
xorg-libxau-1.0.9-h14c3975_0.tar.bz2
xorg-libxdmcp-1.1.3-h516909a_0.tar.bz2
xorg-libxext-1.3.4-h516909a_0.tar.bz2
xorg-libxrender-0.9.10-h516909a_1002.tar.bz2
xorg-renderproto-0.11.1-h14c3975_1002.tar.bz2
xorg-xextproto-7.3.0-h14c3975_1002.tar.bz2
xorg-xproto-7.0.31-h14c3975_1007.tar.bz2
xz-5.2.4-h14c3975_1001.tar.bz2
yaml-0.1.7-h14c3975_1001.tar.bz2
zlib-1.2.11-h516909a_1006.tar.bz2
zstd-1.4.0-h3b9ef0a_0.tar.bz2

Which screenshots feature relies on wx?

Looks like wx, gtk or pillow would work. I don't think gtk is supported on windows, but pillow is handy, anyhow...

Does dialogs library work?

if all it takes is tk! haven't tested.

RIDE... does it already support Python 3?

looks like they test with 3.6. and don't pin wx. I haven't tried packaging it in a while... couldn't get hidpi to work correctly..

@bollwyvl bollwyvl mentioned this issue Oct 3, 2019
Merged
@bollwyvl
Copy link
Collaborator Author

bollwyvl commented Oct 3, 2019 via email

@datakurre
Copy link
Contributor

datakurre commented Oct 3, 2019 via email

@bollwyvl
Copy link
Collaborator Author

bollwyvl commented Oct 3, 2019 via email

@datakurre
Copy link
Contributor

@bollwyvl To avoid misunderstandings, don’t waste your time to rework anything yet. As you said, all the pieces are already there and probably even I could put them together if I really need to. Let’s continue to focus on what this should mean for the “Lab” and only return to the generic installers when there is real need (and a possibly a maintainer).

Disclaimer about the CI-version: I don’t have personal use for it, because we have Nix on our CI machines :/

@bollwyvl
Copy link
Collaborator Author

bollwyvl commented Oct 4, 2019 via email

@bollwyvl
Copy link
Collaborator Author

Sry, have been deep in the language server stuff. I'll try to get back to this after a bit of a holiday this weekend: basically, I intend to keep everything in #52, but not actually build any of the general purpose stuff, land that, and then have a look at some of the other concepts we're developing here. Maybe do some design sketches on the road :)

Back on the LSP stuff: it's taken some really nasty stuff down in python/r/node land, and am only now getting to where I can help out with the browser testing. Been stealing a lot from this repo :P The end game there is to get all of that into JupyterLab itself, but there are a lot of nuts and bolts to figure out before then.

When that's good to go (on pip and npm) some time next month, I think we'll want the UI things it already offers (jump to definition, highlight references, etc) for .robot, .robot.ipynb, output.xml and more possible stuff down the pipe (outline view, tidy). Shipping the existing robot language server implementation in node is no problem for robotlab, but there are things it can't do that kernels can, e.g. you have to libdoc your 3rd party stuff ahead of time. We can at least check it out! It should just be:

  • install the node thing (conda package it, i suppose... the vscode ecosystem has its own challenges)
  • add a small entry_point wrapper in python (or do some jupyter_notebook_config.d stuff)
  • profit

I can also imagine a future where kernels become implicit language servers and clients: with what we ended up with, it's a many-to-many relationship between documents and language servers, everything goes through the notebook server, what's a few more? Heck, there are even nix language servers (though can't tell which of the rust, go, or haskell implementations will win). Additionally, to that end, here's a cool thread about a guix kernel: https://groups.google.com/forum/#!topic/jupyter/jUfLBPSlbXY

@bollwyvl
Copy link
Collaborator Author

And 'cuz I know you love looking at robot reports...

@bollwyvl
Copy link
Collaborator Author

Whoops, meant to post this here, not there:

I did do some drawing: nothing super solid to show for it. There's not much you can do with a feature table, and there's certainly bounded complexity to how much you can stuff in there. There are a few hierarchies that came to mind, but i think one of the more logical ones was...

  • get: download buttons labelled with file size (maybe make it sticky as you scroll)
  • setup: estimate of install time (it's pretty real on windows) and space on disk
  • learn: what docs you'll get to help you (man/libdoc/inspector/lsp/...)
  • system-under-test: what you can test (py/cli/web/wx/white/vms)
  • use: what UIs you can use to build stuff (tty/ipython/lab/ride)
  • automate: how you can use it in a pipeline (azure ain't easy, travis is dying... maybe actions once out of beta?)
  • analyze: how to use many test reports for trends (html/xml/pandas/tcms)
  • upgrade: how you will be able to get new stuff (pip/jlpm/conda)
  • remove: rm -rf vs click-and-uninstall

A funny thing that came to mind, sort of related to the Hub concept, is a "Pro" spin (think a second, full day workshop), which includes the things you would need to extract, build, test, and package new robot libraries (or extensions to kernel) as part/in support of a team. On the backend, you'd have your tooling like git, while on the front-end, I'd stuff in jupyterlab-git. Ship pytest, robotframework-statuschecker, robotframework-lint, black, etc. Ship conda-build, and maybe even constructor (since we can't ship nix 😜 ). Anyhow 🍕 for thought.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
🚧 build Pertains to CI and build
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@bollwyvl @datakurre