Re-enable some browser-testing CI

[drzraf]

• https://www.browserstack.com ($30/mo) karma-plugin
• https://testingbot.com/ (free for FLOSS) karma-plugin
• https://saucelabs.com/pricing ($20/mo) karma-plugin
• https://www.lambdatest.com/ (free-tiers 60 mins/mo, then $15/mo)

One issue with SauceLabs is that the karma plugin apparently got outdated (because protocol evolves, etc...)

Now there is also this karma-webdriver-launcher plugins which aims to be more standard via webdriver (which wap them all).
It's said to support SauceLabs and even advised by lambdatest.
Even more interesting (from the above pricing), TestingBot says it's supported by webdrive.io.

@AidasK ?

[AidasK]

Great research, any of these is ok for me as long as it works. 👍

[drzraf]

• I shared the access to testingBot.
• I could not find "public test page" so far
• Travis variable may need to be https://docs.travis-ci.com/user/environment-variables/#defining-variables-in-repository-settings

By the way, it seems that you didn't actually removed sauceLabs : I see it running on v3 commits.
But then, do you know why it stopped being shown (eg: at #321) ?
I (maybe wrongly) thought you purposefully disabled it.

[AidasK]

You can write to: [email protected]

I have tried adding these to travis, though travis is currently crashing. It says I don't have enough permissions on https://travis-ci.org/ and once I have moved to travis-ci.com, it does not work at all.

[AidasK]

I have given you admin access to this repo, maybe this repo will appear on your travis-ci account

[evilaliv3]

Thank you for supporting this @drzraf !

[evilaliv3]

* https://www.browserstack.com ($30/mo) [karma-plugin](https://github.com/karma-runner/karma-browserstack-launcher)

* https://testingbot.com/ (free for FLOSS) [karma-plugin](https://github.com/testingbot/karma-testingbot-example)

* https://saucelabs.com/pricing ($20/mo) [karma-plugin](https://github.com/karma-runner/karma-sauce-launcher)

* https://www.lambdatest.com/ (free-tiers 60 mins/mo, then $15/mo)

One issue with SauceLabs is that the karma plugin apparently got outdated (because protocol evolves, etc...)

Now there is also this karma-webdriver-launcher plugins which aims to be more standard via webdriver (which wap them all).
It's said to support SauceLabs and even advised by lambdatest.
Even more interesting (from the above pricing), TestingBot says it's supported by webdrive.io.

@AidasK ?

Actually Saucelabs is for free for opensource projects (https://saucelabs.com/open-sauce); If you are looking for alternatives let's try to see if they offer similar plans

[AidasK]

@evilaliv3 good to see you back. @drzraf is migrating flow.js to v3 version with some breaking changes. He is mostly redoing each callback and event in promise based approach

[drzraf]

@evilaliv3 TestingLabs is free for FLOSS too (I added you there too btw).
I didn't intended at all to change something that worked (SauceLabs) but wondered why I was not seeing the testsuite being run for each PR anymore.

If it's "just" that CI wouldn't run for PR done against the v3 branch then we may not need to switch (unless you still find other compelling argument in favor). If we decide to switch, the PR is almost ready.

@AidasK : I'm not used to GitHub CI settings/travis integration but I'll have a look.

[evilaliv3]

I'm on it, let me check and i will be back to you

[evilaliv3]

@drzraf : Actually saucelabs accepts all the builds we make and we don't have conditional code in relation to the branch name.

I suspect the bug could lie in the addition of these lines:

  if (config.sauceLabs && (!config.sauceLabs.username || !config.sauceLabs.accessKey)) {
    console.log('Undefined sauce username/accessKey.');
    process.exit(1)
  }

I cannot retest now but i would suggest to remove those and give it an other try

[drzraf]

@evilaliv3 : You're right.

• Github handover to Travis.
• .travis-ci.yml defines SAUCE_*
• Gruntfile.js takes from process.env.SAUCE_* into sauceLabs (grunt.initConfig)
• normally passed over to karma.conf.js
  But karma.conf.js (karma-sauce-launcher) works without having to define this (native support for environment variables) and the change you mentioned kept it from running.

The two possible path are:

• Reenable SauceLabs CI in v3 #327
• Testing bot #325

If SauceLabs is simply fixed (#327): Do you still think OP (karma-plugin /support) is a compelling arguments for switching or not? Are there others?

[drzraf]

@AidasK, could you have a look at travis-ci in order to enable it for v3?
Here is what I see (in my case, flow.js isn't available to integrate with)

[AidasK]

I couldn’t activate this. I tried reinstalling the app, but it does not work anyway. I have made you an owner of flow.js, so you should have an access now to try this out yourself. Hope this helps

[drzraf]

I sent their support a message. BTW when enabling Travis-CI for "All repositories", I only see "fusty-flow.js" and "ngx-flow".
What could be particular to these two? And what is "fusty-flow.js"?

[AidasK]

Great! It must be an issue at Travis side. fusty-flow.js is a fallback for flow.js for old browsers such as IE8. It’s no longer relevant.

[drzraf]

I was able to do without Travis at all (Only saucelabs and codeclimate). Will do a PR soon.
But codeclimate experiences the same problem as Travis regarding detecting available organizations (according to Travis support, it's an ongoing Github API problem)