Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Execute Behat tests only for a single content graph adapter at a time #4179

Closed
bwaidelich opened this issue Apr 7, 2023 · 3 comments
Closed

Comments

@bwaidelich
Copy link
Member

bwaidelich commented Apr 7, 2023

With the introduction of the Neos.ContentGraph.PostgreSQLAdapter the behat tests have been adjusted to run against both implementations.

This was done for performance reasons mainly. But it has some drawbacks:

  • Tests are far more complex because they have to keep state of multiple adapters in memory and compare them individually
  • Failure resolution is more difficult because one has to determine the failing implementation from the (sometimes misleading) error message
  • Currently the way to skip tests for certain adapters is to use the @adapters tag in the feature files – this concept can't be used for future 3rd party implementations
  • Tests for multiple adapters can't be executed in parallel

I would suggest to simplify the implementation again and allow to specify the adapter via behat.yml configuration.
To allow certain tests to be skipped during development, we could either use an allow (or deny) list of tests in the configuration or introduce feature specific annotations (like @features=contentSubgraph.findDescendantNodes, contentSubgraph.findChildNodes or similar).

Note: Previously our CI took ~1.25 hrs to execute behat tests (for PHP 8.1 and 8.2 each). With the sync mode (activated with #4110) they take less than 15 minutes. Besides we could probably execute them in parallel

@bwaidelich bwaidelich converted this from a draft issue Apr 7, 2023
@nezaniel nezaniel self-assigned this Aug 1, 2023
@mhsdesign
Copy link
Member

mhsdesign commented Aug 1, 2023

Current workaround for only executing maria is to add

        if ($excludePostgres = true) {
            $adapterKeys = array_filter($adapterKeys, fn ($key) => $key !== 'Postgres');
        }

to \Neos\ContentRepository\Core\Tests\Behavior\Features\Bootstrap\EventSourcedTrait::beforeEventSourcedScenarioDispatcher

@nezaniel
Copy link
Member

Has been adjusted in #4455; Which adapter to use can now be set via configuration

@nezaniel nezaniel moved this from Todo to Under Review 👀 in Neos 9.0 Release Board Aug 29, 2023
@nezaniel
Copy link
Member

nezaniel commented Sep 1, 2023

has been merged and thus is resolved

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
Status: Done
Development

No branches or pull requests

3 participants