Run Travis-CI on all branches,

[tomalec]

Changes proposed in this Pull Request:

Run Travis-CI on all branches,
Remove safelist, to go with the default allow all, except gh-pages.

I believe this or a similar change would be useful as we started working on a feature/* branch for contact info.
With the current setup, none of our Travis & bundlewatch checks are being run there.

I would say we run quite a limited number of branches and we are now public repo, so hopefully hitting Travis minutes won't be an issue. The benefit we will gain:

1. Checks on the contact info feature branch.
2. Checks for PRs to PRs (which we use from time to time, even for trunk)
3. Check for experiment/proposal branches without a PR yet.

Screenshots:

PRs to feature/contact-info branch

like #914

PRs to PRs

like #918

Detailed test instructions:

1. Merge this branch 😉
2. Check the Travis checks for them all

Changelog entry

Additional notes:

• Alternatively, we could just safelist feature/contact-info or /^feature/.*$/
• @jconroy, was there a specific reason I missed, for safe listing only trunk and develop initially?

[layoutd]

Was there a specific reason I missed, for safe listing only trunk and develop initially?

I believe this was copied over from other repos, where time-/resource-heavy checks were being run twice on pushes to branches that had open PRs. In fact, you can see it on the checks for this PR, too:

I think /^feature/.*$/ would at least reduce double-testing on tweak/, fix/, add/ etc. (which should all have PRs sooner or later, in theory).

[tomalec]

To avoid "double builds" we should control build triggers, and builds being run, not the branches being used.

https://docs.travis-ci.com/user/pull-requests/#double-builds-on-pull-requests

If you see two build status icons on your GitHub pull request, it means there is one build for the branch, and one build for the pull request itself (actually the build for the merge of the head branch with the base branch specified in the pull request).

Build pushed branches and Build pushed pull requests control this behaviour.

So, if we are not extremely under the pressure of too many Travis builds, I'd still build it for all branches. Simply disable PR builds, or skip it for other jobs than bundlewatch (which may need PR build to have the target to compere)

Because with allowlist: trunk, we disable all CI for feature branches, and PR-to-PRs.

---

As all our branches have the code that we would rather test.

I think the Travis-overload strategy should go in levels like:

1. Allow all branches, run for brunches and PRs -
   YOLO, I don't care strategy
2. Allow all branches, run for PRs, and only basic jobs for branches (like, only linters, build for one combination of envs, still run all for trunk&develop - that would require .travis.yml tweak) -
   runs for everything, but without duplicates
3. 2 + "Building Only the Latest Commit" setting -
   runs for everything, without duplicates, plus stop the outdated
4. 3 only for PRs
   runs for everything, without duplicates, plus stop the outdated
5. 4 only for trunk|develop & (feature|tweak|fix|add)/ -
   as above, but do not run for personal branches, experiments, POCs, etc.
6. 4 only for trunk|develop & feature/ -
   very limiting, but tests the "thickest" branches

Currently, we are

• only trunk|develop, for branches & PRs, with duplicates, with outdated -
  very limiting and YOLO 😉 (edited per comment)
• only trunk|develop, for PRs & branches (but only 2), stop the outdated -
  safe but limiting

---

In this PR I suggest starting with 1. as we are YOLO already 😎
or at least with 2 or 3, as I think we do not experience much load.
WDYT?

[tomalec]

After giving it a second thought, I realized, we'd rather need more detailed checks for PRs, especially from forks. So I updated level 2 in my strategies list.

[layoutd]

I don't think we're YOLO yet - only this PR has branch+PR checks because of the .travis.yml changes (remove safelist). If you look at other PRs, they only have one Travis check + bundlewatch. And we are skipping outdated.

That said, I don't really have a horse in this race but I would probably lean towards option 4. It can be frustrating to have duplicate checks if you just want to make a small change before merging a PR (PHPCS fix, for example), and you have to wait for all the checks to pass twice. And, I think we're on a shared Travis account, so running double tests this repo means taking processing time away from other teams/repos, right?

[tomalec]

I don't think we're YOLO yet - only this PR has branch+PR checks because of the .travis.yml changes (remove safelist). If you look at other PRs, they only have one Travis check + bundlewatch. And we are skipping outdated.

Thanks, you're right. I didn't notice that we have to skip outdated disabled, and that's true that with branches limited do trunk|develop we are effectively not running Build pushed branches for anything else than trunk where we are not running PR build.

[…] lean towards option 4. It can be frustrating to have duplicate checks if you just want to make a small change before merging a PR (PHPCS fix, for example), and you have to wait for all the checks to pass twice.

If we do the 3, so it contains 2 the nice way, we would not run all check twice, only a few we would run for branches.

I think we're on a shared Travis account, so running double tests this repo means taking processing time away from other teams/repos, right?

I'm not sure. It's strange, but Travis plan usage shows 0 credits used over the last 2 months:confused:

So I'm now voting for 3 or 4.

3. As I personally like to push a branch, and check on Travis that it passes all the checks on all env combinations, before submitting a PR or proposing something. But I can live without that.
4. As it's easier to set up, and it could save more Travis-time, especially if it affects the other teams.

[tomalec]

FWIW, 4. requires pushing this branch as is, and changing one switch in Travis settings.

[jconroy]

Just bombing in 💣 to point out that we can actually use github actions now the repo is public - nice list of some possibilities here: https://github.com/woocommerce/woocommerce-gutenberg-products-block/actions

[tomalec]

we can actually use github actions now the repo is public

I tried it last time when I was adding bundlewatch, and it turned out that it's hard to get just a base branch name in case of a PR to send for comparison.

But for the other jobs, I guess we can try moving them to github actions.

[tomalec]

Closed in favor of moving to GitHub actions #1039 #1040

[eason9487]

Ahh, I just closed the testing PR #1039. 🙏
Please refer to the formal one #1040.