Limitation "No extensibility / inheritance of tests"

[sprankhub]

Can you share some information why this limitation exists? Is it a design decision, are there technical issues with it or what is the main reason? I mean, in the end, we all love Magento so much due to its extensibility, right? Inheriting core behaviour and only extending needed parts is a nice thing, so I would love to see this here as well. Currently, I do not have any concrete ideas yet as the code is not even there, but at least I wanted to get this discussion going :)

[peterjaap]

The only reason; we haven't spent time on it yet.

[tmotyl]

Useful link to read get a common understanding on the possible patterns: https://applitools.com/blog/page-objects-app-actions-cypress/ https://www.cypress.io/blog/2019/01/03/stop-using-page-objects-and-start-using-app-actions/

I would imagine this repo to become a cypress library/plugin providing:

1. building blocks for writing custom Magento tests e.g.:

• cypress actions like cy.myCustomAction()
• page objects
• app actions

2. default test suite which utilizes the building blocks, and follows the test scenarios

For custom projects you have 3 possible situations:
a) you keep the default test flow, so you don't modify test scenarios, but you modify the selector of the building blocks to your markup
b) you change/rewrite the test flow, but keep the building blocks
c) you do change both, test flow and building blocks

From my experience, in one project you encounter all three scenarios.
Some tests will be good out of the box as they are present in the library (both flow and selectors are the same), for some tests you need to change the flow as you have customized flow in the shop, for some test you just need to change few selectors in the building blocks, and you're good to go.

[peterjaap]

We added a package.json and a package-lock.json to the repo recently, which makes it one step closer to making this repo an npm package.

One of the thoughts I had is that you could install this suite through npm by running npm install @elgentos/magento2-cypress-testing-suite. It will then of course be installed in node_modules. I'm not a Javascript pro but I'm guessing we can run some after-install hooks like Composer has, so we can copy configuration files into the projects' root directory.

From there, we should be then able to start the default testing suite by running npx cypress run from the project root.

Now here's where the extensibility comes in. It would be great if we could then put tests in the same location relative from this project into the project repo, so;

cp node_modules/@elgentos/magento2-cypress-testing-suite/cypress/integration/hyva/catalog/product.spec.js cypress/integration/hyva/catalog/product.spec.js

You could then alter/remove tests in your local copy as you wish.

There are in my opinion two ways this could work out;

1. When a local file (not in node_modules) is present, Cypress uses only that file. That means deleting tests from the local file will not run those tests.
2. When a local file is present, it will be merged with the file in node_modules. Tests in the local file with the same name will take precedence. That means deleting tests from the local file will run the original test in the node_modules file. If you want a test to be skipped, you could add .skip so the test starts with it.skip().

To make option 2 a reality, we'd probably need some preprocessor to generate a Javascript file that combines the local and node_modules files. This would apply to all types of files - spec files, fixtures and page objects.

Any thoughts on this @tmotyl @sprankhub @Vinai @lfolco @nisrineMach ?

[sprankhub]

Thanks for starting this, @peterjaap!

So given that spec files can be quite "big" like https://github.com/elgentos/magento2-cypress-testing-suite/blob/7438e23c1201b43114693f69b6d712901f91676c/cypress/integration/hyva/catalog/product.spec.js, I would really prefer the merging approach (option 2). Because I do not want to override a 180 lines file just to adapt two lines of one specific spec from it.

However, I could imagine that merging the JS files is not trivial. If it is indeed hard to do, I could imagine a combined approach: For spec files and page objects, use option 1. For fixtures, use option 2. The reason is that fixtures are JSON files and these should be fairly simple to merge.

[peterjaap]

Maybe someone with more JS experience can pitch in here on how hard it is to merge the specfiles. Maybe @jesse-deboer?

[Vinai]

I see two approaches without having investigated anything.

1. Use babel to merge the ast of the two files, maybe based on test names.

2. Dynamically set filters for cypress. This would definitely be much easier to implement

[peterjaap]

How would dynamically setting filters work? Do something like this and create a wrapper function that checks on existing tests in another location?

[Vinai]

Interesting post, bit I was thinking of looking at the tagify implementation first, and then implementing the filter as a small node module, that makes the changes to the test configuration so the right tests are executed.
The function probably would be called in the setupNodeEvents callback.