Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Behat tests in theme_adaptable will not run #30

Open
dualcode opened this issue Aug 8, 2024 · 12 comments
Open

Behat tests in theme_adaptable will not run #30

dualcode opened this issue Aug 8, 2024 · 12 comments
Assignees
Labels
Bug Something isn't working Duplicate This issue or pull request already exists

Comments

@dualcode
Copy link

dualcode commented Aug 8, 2024

Describe the bug
If you were to write a Behat test for theme_adatpable, they'll fail due to the presense of contending step definitions in theme_adaptable.

To Reproduce
Execute the feature file I'll provide in a branch and observe the failures. Note that all this test does it log in.

vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml /var/www/html/theme/adaptable/tests/behat/navigation.feature

Here is an example of such a failure:

`root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml /var/www/html/theme/adaptable/tests/behat/navigation.feature
Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070
Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64
Run optional tests:

  • Accessibility: Yes
  • SCSS deprecations: No
    Server OS "Linux", Browser: "firefox"
    Started at 09-08-2024, 06:24
    FFFF

--- Failed steps:

001 Scenario: Basic navigation without JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:13
And I log in as "admin" # /var/www/html/theme/adaptable/tests/behat/navigation.feature:7
Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()

    behat_theme_adaptable_behat_auth::i_log_out()
    behat_theme_adaptable_behat_auth::i_log_out()

002 Scenario: Basic navigation without JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:13
Then I log out # /var/www/html/theme/adaptable/tests/behat/navigation.feature:11
Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()

    behat_theme_adaptable_behat_auth::i_log_out()
    behat_theme_adaptable_behat_auth::i_log_out()

003 Scenario: Basic navigation with JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:16
And I log in as "admin" # /var/www/html/theme/adaptable/tests/behat/navigation.feature:7
Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()

    behat_theme_adaptable_behat_auth::i_log_out()
    behat_theme_adaptable_behat_auth::i_log_out()

004 Scenario: Basic navigation with JavaScript # /var/www/html/theme/adaptable/tests/behat/navigation.feature:16
Then I log out # /var/www/html/theme/adaptable/tests/behat/navigation.feature:11
Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()

    behat_theme_adaptable_behat_auth::i_log_out()
    behat_theme_adaptable_behat_auth::i_log_out()

2 scenarios (2 failed)
4 steps (4 failed)
0m7.32s (52.12Mb)`

Note that as you fix the root cause of one failure, you'll run into another. I dont' think it's neccessary to demonstrate every single failure I encountered, but let me know if you really need to see it.

Expected behavior
A successful Behat run as follows:
`root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml /var/www/html/theme/adaptable/tests/behat/navigation.feature
Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070
Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64
Run optional tests:

  • Accessibility: Yes
  • SCSS deprecations: No
    Server OS "Linux", Browser: "firefox"
    Started at 09-08-2024, 06:32
    ....

2 scenarios (2 passed)
4 steps (4 passed)
0m8.84s (52.56Mb)

Versions (please complete the following information):

  • Moodle: 4.4.1
  • Theme: 404.1.0

Additional context
I'll link a branch with a fix for all of these in which the commit wil further explain the resolution.

@dualcode dualcode added Bug Something isn't working Pending replication Pending the issue being replicated to demonstrate that it is indeed an issue. labels Aug 8, 2024
@gjb2048
Copy link
Collaborator

gjb2048 commented Aug 8, 2024 via email

@eric-s-richer
Copy link

I've created a pull request #31 to address this issue.

@eric-s-richer
Copy link

eric-s-richer commented Aug 8, 2024

But you don't write Behat tests for Adaptable because it's a theme. The overrides are there for when you run the core tests and set Adaptable as the theme, and they allow them to succeed due to the differences between it and Boost.

You could absolutely write Behat tests to validate your theme's behaviour though. There's nothing preventing this from being done.

@eric-s-richer
Copy link

@gjb2048 as a follow up to your comment, consider the following in which I contrast a test run against the adaptable and default themes:

root@0991e34a996f:/var/www/html# php admin/tool/behat/cli/init.php --add-core-features-to-theme
You are already using the latest available Composer version 2.7.7 (stable channel).
Installing dependencies from lock file (including require-dev)
Verifying lock file contents can be installed on current platform.
Nothing to install, update or remove
Generating autoload files
53 packages you are using are looking for funding.
Use the composer fund command to find out more!
Behat test environment already installed
Creating Behat configuration ... done in 0.38 seconds.
Building theme CSS for adaptable [ltr] ... done in 1.03 seconds.
Building theme CSS for boost [ltr] ... done in 0.91 seconds.
Building theme CSS for classic [ltr] ... done in 0.93 seconds.
Testing environment themes built
Acceptance tests environment enabled on http://webserver, to run the tests use:
vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml
root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --suite adaptable /var/www/html/login/tests/behat/navigation.feature
Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070
Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64
Run optional tests:
- Accessibility: Yes
- SCSS deprecations: No
Server OS "Linux", Browser: "firefox"
Started at 09-08-2024, 07:17
FFFF

--- Failed steps:

001 Scenario: Basic navigation without JavaScript # /var/www/html/login/tests/behat/navigation.feature:7
And I log in as "admin" # /var/www/html/login/tests/behat/navigation.feature:4
Step "/^I log out$/" is already defined in behat_theme_adaptable_behat_auth::i_log_out()
\
behat_theme_adaptable_behat_auth::i_log_out()
behat_theme_adaptable_behat_auth::i_log_out()

root@0991e34a996f:/var/www/html# vendor/bin/behat --config /var/www/behatdata/behatrun/behat/behat.yml --suite default /var/www/html/login/tests/behat/navigation.feature
Moodle 4.4.1+ (Build: 20240802), e0381615aa0e12a9e2df7713aee749062ec18070
Php: 8.1.29, mysqli: 8.0.36, OS: Linux 6.5.0-45-generic x86_64
Run optional tests:
- Accessibility: Yes
- SCSS deprecations: No
Server OS "Linux", Browser: "firefox"
Started at 09-08-2024, 07:18
....

2 scenarios (2 passed)
4 steps (4 passed)
0m7.04s (52.40Mb)

In both instances, I am executing the login/tests/behat/navigation.feature Behat test whose contents are the exact same as the file in my pull request. Note that the tests fail against the adaptable suite; which is the theme.

@gjb2048
Copy link
Collaborator

gjb2048 commented Aug 9, 2024

Revisited, all very odd. Need to put on the backlog as not urgent.

@gjb2048 gjb2048 added the Backlog To to but not urgent label Aug 9, 2024
@eric-s-richer
Copy link

@gjb2048 what's odd? If there's anything I can do to clarify or explain please let me know. I've literally written 100s of Behat tests for Moodle and various 3rd party plugins.

It would be really good to be able to write Behat tests which validate theme behaviour.

@gjb2048
Copy link
Collaborator

gjb2048 commented Aug 9, 2024

Odd in a way that's odd in relation to what has transpired. Will get back to this when can.

@gjb2048
Copy link
Collaborator

gjb2048 commented Aug 9, 2024

@eric-s-richer Note that your account for this issue has access to the development repository where you can see the latest code and will be a better place for the issue and any subsequent pull request(s).

@dualcode
Copy link
Author

@gjb2048 I can now see your gjbarnard/moodle-theme_adaptable_dev repository but I am unable to fork it or submit pull requests. Am I not understanding the process here or am I maybe missing some permissions?

@gjb2048
Copy link
Collaborator

gjb2048 commented Aug 13, 2024

@gjb2048 I can now see your gjbarnard/moodle-theme_adaptable_dev repository but I am unable to fork it or submit pull requests. Am I not understanding the process here or am I maybe missing some permissions?

Try now please.

@dualcode
Copy link
Author

It all seems to work now. As requested, I've opened a pull request.

@gjb2048 gjb2048 added Duplicate This issue or pull request already exists and removed Pending replication Pending the issue being replicated to demonstrate that it is indeed an issue. Backlog To to but not urgent labels Aug 20, 2024
@gjb2048
Copy link
Collaborator

gjb2048 commented Aug 20, 2024

Fix in dev, will close when release here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working Duplicate This issue or pull request already exists
Projects
None yet
Development

No branches or pull requests

3 participants