Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Develop to master #255

Merged
merged 51 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
51 commits
Select commit Hold shift + click to select a range
9a2bd47
fix: dev-requirements-2.9-py2.txt to reduce vulnerabilities
snyk-bot May 21, 2024
5aa8fed
fix: dev-requirements.txt to reduce vulnerabilities
snyk-bot May 21, 2024
42e6f5b
fix: dev-requirements-2.9.txt to reduce vulnerabilities
snyk-bot May 22, 2024
2e87423
fix: dev-requirements-2.9-py2.txt to reduce vulnerabilities
snyk-bot Jun 18, 2024
1aba72e
fix: dev-requirements-2.9.txt to reduce vulnerabilities
snyk-bot Jun 19, 2024
1401973
fix: dev-requirements.txt to reduce vulnerabilities
snyk-bot Jun 20, 2024
695136a
fix: dev-requirements-2.9.txt to reduce vulnerabilities
snyk-bot Jul 10, 2024
59f08b0
fix: dev-requirements.txt to reduce vulnerabilities
snyk-bot Jul 10, 2024
abb8959
fix: dev-requirements-2.9-py2.txt to reduce vulnerabilities
snyk-bot Jul 10, 2024
cfa5ff7
fix: dev-requirements.txt to reduce vulnerabilities
snyk-bot Jul 14, 2024
c7f4221
[QOLSVC-7731] Replace 'News and events' and 'Case studies' with 'Visu…
ienxckl Sep 30, 2024
fab1625
[QOLSVC-7731] Updated container and added patch, and added test for n…
ienxckl Sep 30, 2024
44d314a
Merge pull request #254 from qld-gov-au/QOLSVC-7731-new-pages-nav
ienxckl Sep 30, 2024
9fd94c9
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
176d73c
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
78a6906
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
6c94ff5
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
e1c7cc8
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
f617478
Merge branch 'snyk-fix-eeede7ceb82e758c961ced3e17183989' of https://g…
ThrawnCA Oct 3, 2024
f428ac3
Merge branch 'snyk-fix-c2aac598af38da65255e5215557d0ce5' of https://g…
ThrawnCA Oct 3, 2024
e150924
Merge branch 'snyk-fix-484e2e3a5e3b80ad2ae74ed5b0c5b117' of https://g…
ThrawnCA Oct 3, 2024
f6c0401
Merge branch 'snyk-fix-dc80555c607873abdd75bc134d4fedbb' of https://g…
ThrawnCA Oct 3, 2024
41e4db2
Merge branch 'snyk-fix-0995087860516ab0b701f45f410d88bb' of https://g…
ThrawnCA Oct 3, 2024
ad3eb18
Merge branch 'snyk-fix-50acbde32c6492a0166f4236e54eb172' of https://g…
ThrawnCA Oct 3, 2024
f1485ea
Merge pull request #247 from qld-gov-au/snyk-fix-484e2e3a5e3b80ad2ae7…
ThrawnCA Oct 3, 2024
52d497b
Merge pull request #248 from qld-gov-au/snyk-fix-eeede7ceb82e758c961c…
ThrawnCA Oct 3, 2024
cbf57ca
Merge pull request #246 from qld-gov-au/snyk-fix-0995087860516ab0b701…
ThrawnCA Oct 3, 2024
b58ff87
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
7e019ee
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
6b6d093
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
6dbedbe
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
6b767df
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
cf02277
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
25907f2
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 3, 2024
abf9cff
Merge pull request #242 from qld-gov-au/snyk-fix-e7c24e696656966bd38b…
ThrawnCA Oct 4, 2024
6b00433
Merge pull request #245 from qld-gov-au/snyk-fix-c2aac598af38da65255e…
ThrawnCA Oct 4, 2024
1665523
split tests to improve build speed
ThrawnCA Oct 4, 2024
3cb8953
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 4, 2024
821e991
Merge branch 'snyk-fix-df3e1843278b946b44f03fa23bd02bce' of https://g…
ThrawnCA Oct 4, 2024
7a01408
Merge branch 'snyk-fix-df3e1843278b946b44f03fa23bd02bce' of https://g…
ThrawnCA Oct 4, 2024
b3a1541
Merge branch 'snyk-fix-df3e1843278b946b44f03fa23bd02bce' of https://g…
ThrawnCA Oct 4, 2024
48e6db2
Merge branch 'snyk-fix-df3e1843278b946b44f03fa23bd02bce' of https://g…
ThrawnCA Oct 4, 2024
1093eba
Merge pull request #244 from qld-gov-au/snyk-fix-dc80555c607873abdd75…
ThrawnCA Oct 4, 2024
ff7db18
Merge pull request #243 from qld-gov-au/snyk-fix-50acbde32c6492a0166f…
ThrawnCA Oct 4, 2024
1c428af
add debugging to problematic scenario test
ThrawnCA Oct 4, 2024
e8d19a4
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 4, 2024
0c2ea73
fix test expectations to wait for usability upgrade
ThrawnCA Oct 4, 2024
2ff5bf7
Merge pull request #240 from qld-gov-au/snyk-fix-0cbb4fac9a523cd415b6…
ThrawnCA Oct 4, 2024
0e17708
Merge branch 'develop' of https://github.com/qld-gov-au/ckanext-data-…
ThrawnCA Oct 4, 2024
605676f
disable debugging
ThrawnCA Oct 4, 2024
f01e611
Merge pull request #241 from qld-gov-au/snyk-fix-a6ab60e923f3d263c827…
ThrawnCA Oct 4, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 4 additions & 1 deletion .ahoy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -159,9 +159,12 @@ commands:
) || [ "${ALLOW_BDD_FAIL:-0}" -eq 1 ]
elif [ "$BEHAVE_TAG" = "authenticated" ]; then
# run any tests that don't have a specific tag
ahoy cli "behave -k ${*:-test/features} --tags=-unauthenticated --tags=-smoke --tags=-OpenData" \
ahoy cli "behave -k ${*:-test/features} --tags=-unauthenticated --tags=-smoke --tags=-OpenData --tags=-multi_plugin" \
Copy link
Member

@duttonw duttonw Oct 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why remove multi plugins? or you wanted them run separately?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

They're run separately. The idea is to split the tests in a somewhat logical and even fashion, to speed up the build.

|| [ "${ALLOW_BDD_FAIL:-0}" -eq 1 ]
else
if [ "$BEHAVE_TAG" != "multi_plugin" ]; then
BEHAVE_TAG="$BEHAVE_TAG --tags=-multi_plugin"
fi
# run tests with the specified tag
ahoy cli "behave -k ${*:-test/features} --tags=$BEHAVE_TAG" \
|| [ "${ALLOW_BDD_FAIL:-0}" -eq 1 ]
Expand Down
8 changes: 6 additions & 2 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -27,16 +27,20 @@ jobs:
fail-fast: true
matrix:
ckan-version: ["2.10", 2.9]
behave-tag: [smoke, unauthenticated, OpenData, authenticated]
behave-tag: [smoke, unauthenticated, multi_plugin, OpenData, authenticated]

name: Run ${{ matrix.behave-tag }} tests on CKAN ${{ matrix.ckan-version }}
runs-on: ubuntu-latest
container: drevops/ci-builder:23.7.0
container: drevops/ci-runner:23.12.0
env:
CKAN_VERSION: ${{ matrix.ckan-version }}
BEHAVE_TAG: ${{ matrix.behave-tag }}

steps:
# Patch https://github.com/actions/runner/issues/863
- name: Preserve $HOME set in the container
run: echo HOME=/root >> "$GITHUB_ENV"

- uses: actions/checkout@v4
timeout-minutes: 2

Expand Down
4 changes: 2 additions & 2 deletions ckanext/data_qld/templates/header.html
Original file line number Diff line number Diff line change
Expand Up @@ -109,10 +109,10 @@
{% set dataset_search_route = 'dataset.search' %}
<li {% if h.current_url() != '/' %}class="active"{% endif %}><a href="{{ h.url_for(dataset_search_route) }}">Data</a></li>
<li>
<a href="/article/news-and-events">News and events</a>
<a href="/visualisations">Visualisations</a>
</li>
<li>
<a href="/article/case-studies">Case studies</a>
<a href="/news-and-case-studies">News and Case Studies</a>
</li>
<li>
<a href="/article/standards-and-guidance">Standards and guidance</a>
Expand Down
4 changes: 3 additions & 1 deletion dev-requirements-2.9-py2.txt
Original file line number Diff line number Diff line change
Expand Up @@ -14,5 +14,7 @@ splinter>=0.13.0,<0.17
xlrd==1.2.0
python-magic==0.4.18
progressbar==2.5

pika>=1.1.0,<1.3.0 # resolve harvester conflict
requests>=2.32.0 # not directly required, pinned by Snyk to avoid a vulnerability
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability
4 changes: 3 additions & 1 deletion dev-requirements-2.9.txt
Original file line number Diff line number Diff line change
Expand Up @@ -12,5 +12,7 @@ splinter>=0.13.0,<0.17
xlrd==1.2.0
python-magic==0.4.18
progressbar==2.5

pika>=1.1.0,<1.3.0 # resolve harvester conflict
requests>=2.32.0 # not directly required, pinned by Snyk to avoid a vulnerability
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability
14 changes: 9 additions & 5 deletions dev-requirements.txt
Original file line number Diff line number Diff line change
@@ -1,17 +1,21 @@
behaving==3.1.5
Appium-Python-Client==2.10.1
certifi>=2024.7.4 # not directly required, pinned by Snyk to avoid a vulnerability
ckanapi==4.3
ckantoolkit>=0.0.4
factory-boy
Faker
flake8==6.0.0
idna>=3.7 # not directly required, pinned by Snyk to avoid a vulnerability
mock
pika>=1.1.0,<1.3.0 # resolve harvester conflict
progressbar==2.5
pytest-ckan
python-magic==0.4.18
requests>=2.32.0 # not directly required, pinned by Snyk to avoid a vulnerability
selenium<4.10
six>=1.13.0
xlrd==1.2.0
python-magic==0.4.18
progressbar==2.5

typer<0.11 # We still need Click 7 for now
pika>=1.1.0,<1.3.0 # resolve harvester conflict
urllib3>=2.2.2 # not directly required, pinned by Snyk to avoid a vulnerability
xlrd==1.2.0
zipp>=3.19.1 # not directly required, pinned by Snyk to avoid a vulnerability
1 change: 1 addition & 0 deletions test/features/admin_reporting.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@reporting
@OpenData
@multi_plugin
Feature: AdminReporting

Scenario: As an admin user of my organisation, I can view 'My Reports' tab in the dashboard and show the 'Admin Report' with filters and table columns
Expand Down
1 change: 1 addition & 0 deletions test/features/comments.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@comments
@OpenData
@multi_plugin
Feature: Comments

@unauthenticated
Expand Down
7 changes: 7 additions & 0 deletions test/features/data_qld_theme.feature
Original file line number Diff line number Diff line change
Expand Up @@ -129,3 +129,10 @@ Feature: Theme customisations
When I go to "/robots.txt"
Then I should see "Disallow: /"
And I should not see "Allow:"

@unauthenticated
Scenario: When I go to the home page, I can see Visualisations and News and Case Studies in the navbar
Given "Unauthenticated" as the persona
When I go to homepage
Then I should see an element with xpath "//a[string()='Visualisations']"
And I should see an element with xpath "//a[string()='News and Case Studies']"
5 changes: 3 additions & 2 deletions test/features/data_usability_rating.feature
Original file line number Diff line number Diff line change
Expand Up @@ -26,8 +26,9 @@ Feature: Data usability rating
When I log in
And I create a dataset and resource with key-value parameters "license=other-open" and "format=CSV::upload=test_game_data.csv::schema=default"
And I press the element with xpath "//ol[contains(@class, 'breadcrumb')]//a[starts-with(@href, '/dataset/')]"
And I reload page every 3 seconds until I see an element with xpath "//div[contains(@class, 'qa') and contains(@class, 'openness-')]" but not more than 10 times
And I reload page every 3 seconds until I see an element with xpath "//div[contains(@class, 'qa') and contains(@class, 'openness-4')]" but not more than 10 times
Then I should not see an element with xpath "//div[contains(@class, 'openness-3')]"
And I should see an element with xpath "//div[contains(@class, 'openness-4')]"
When I press "Test Resource"
When I take a debugging screenshot
And I press "Test Resource"
Then I should see an element with xpath "//div[contains(@class, 'qa openness-4')]"
1 change: 1 addition & 0 deletions test/features/data_validation.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@config
@OpenData
@multi_plugin
Feature: Data Validation

Scenario Outline: As a sysadmin, admin and editor user of the dataset organisation I can see the 'JSON' button
Expand Down
1 change: 1 addition & 0 deletions test/features/datarequest.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@datarequest
@OpenData
@multi_plugin
Feature: Data Request

@unauthenticated
Expand Down
1 change: 1 addition & 0 deletions test/features/datarequest_circumstances.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@data-requests-circumstances
@OpenData
@multi_plugin
Feature: Datarequest-circumstances

Scenario: As a sysadmin user when I go to the admin config page I can view the data requests closing circumstances textarea
Expand Down
1 change: 1 addition & 0 deletions test/features/dataset_schema.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@dataset_schema
@OpenData
@multi_plugin
Feature: Dataset Schema

Scenario Outline: Add new dataset metadata fields for default data schema validation
Expand Down
1 change: 1 addition & 0 deletions test/features/engagement_reporting.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@reporting
@OpenData
@multi_plugin
Feature: Engagement Reporting

Scenario Outline: As a user with admin or editor role capacity of an organisation, I can view 'My Reports' tab in the dashboard and show the engagement report with filters
Expand Down
1 change: 1 addition & 0 deletions test/features/reporting.feature
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
@reporting
@OpenData
@multi_plugin
Feature: Reporting

@unauthenticated
Expand Down
1 change: 1 addition & 0 deletions test/features/schema_generation.feature
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
@OpenData
@multi_plugin
Feature: Schema Generation
Enable worker with `ckan jobs clear && ckan jobs worker`, since these tests rely on background tasks

Expand Down