Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Spaces] Moved tests to agnostic setup #200606

Open
wants to merge 38 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from 28 commits
Commits
Show all changes
38 commits
Select commit Hold shift + click to select a range
5f94ea3
Moved tests to agnostic setup
elena-shostak Nov 13, 2024
6b93044
wip
elena-shostak Nov 18, 2024
3ec74cd
Fixes
elena-shostak Nov 18, 2024
2cec1ea
Fixes
elena-shostak Nov 18, 2024
1231e09
fixes
elena-shostak Nov 18, 2024
3d9b71a
Fixes
elena-shostak Nov 18, 2024
20f2a1a
fixes
elena-shostak Nov 19, 2024
25619ec
cleaned up apm_user
elena-shostak Nov 19, 2024
c9efafc
basic license - discuss with the team
elena-shostak Nov 19, 2024
ffa165a
SO types change
elena-shostak Nov 21, 2024
99e4827
wip
elena-shostak Nov 22, 2024
a719a87
Tests
elena-shostak Dec 2, 2024
1e5ec58
wip
elena-shostak Dec 23, 2024
ffb4dfe
wip
elena-shostak Dec 27, 2024
132189d
Test fixes
elena-shostak Dec 27, 2024
3b2cad8
Fixes
elena-shostak Dec 27, 2024
85a080b
Fixes
elena-shostak Dec 30, 2024
1a54a19
get_shareable_references test fix
elena-shostak Dec 30, 2024
0e4c336
wip
elena-shostak Dec 30, 2024
50cbe35
update_objects_spaces and disable_legacy_url_aliases tests
elena-shostak Dec 31, 2024
e82dcfe
Fixes
elena-shostak Jan 3, 2025
f292abc
Added waitFor for delete test
elena-shostak Jan 3, 2025
ef9730d
Fixes
elena-shostak Jan 3, 2025
78a31e8
Fixes
elena-shostak Jan 3, 2025
2fdb153
Fixup
elena-shostak Jan 3, 2025
e85a5ca
[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…
kibanamachine Jan 3, 2025
912fe47
Lint fix
elena-shostak Jan 3, 2025
424d9d3
Merge branch 'space-deployment-agnostic' of github.com:elena-shostak/…
elena-shostak Jan 3, 2025
c91bfc0
Fixup
elena-shostak Jan 3, 2025
1d1e6b8
Serverless config
elena-shostak Jan 3, 2025
7d68cd6
Fixes
elena-shostak Jan 6, 2025
f4b09c7
Fixes
elena-shostak Jan 6, 2025
6f5f03c
Fixes
elena-shostak Jan 6, 2025
89482c1
Service fix
elena-shostak Jan 6, 2025
4f21b9e
Fixes
elena-shostak Jan 7, 2025
e8c8bfc
Merge branch 'main' into space-deployment-agnostic
elasticmachine Jan 7, 2025
ea7beae
Fixes
elena-shostak Jan 17, 2025
5e7de03
Merge branch 'main' into space-deployment-agnostic
elasticmachine Jan 17, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 0 additions & 2 deletions .buildkite/ftr_platform_stateful_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -341,9 +341,7 @@ enabled:
- x-pack/test/security_functional/expired_session.config.ts
- x-pack/test/session_view/basic/config.ts
- x-pack/test/spaces_api_integration/security_and_spaces/config_basic.ts
- x-pack/test/spaces_api_integration/security_and_spaces/copy_to_space_config_basic.ts
- x-pack/test/spaces_api_integration/security_and_spaces/config_trial.ts
- x-pack/test/spaces_api_integration/security_and_spaces/copy_to_space_config_trial.ts
- x-pack/test/spaces_api_integration/spaces_only/config.ts
- x-pack/test/task_manager_claimer_update_by_query/config.ts
- x-pack/test/ui_capabilities/security_and_spaces/config.ts
Expand Down
5 changes: 5 additions & 0 deletions .buildkite/ftr_security_stateful_configs.yml
Original file line number Diff line number Diff line change
Expand Up @@ -104,3 +104,8 @@ enabled:
- x-pack/test/cloud_security_posture_functional/data_views/config.ts
- x-pack/test/automatic_import_api_integration/apis/config_basic.ts
- x-pack/test/automatic_import_api_integration/apis/config_graphs.ts
- x-pack/test/spaces_api_integration/deployment_agnostic/spaces_only/config.ts
- x-pack/test/spaces_api_integration/deployment_agnostic/security_and_spaces/stateful.config_basic.ts
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we discussed it with @azasypkin and @jeramysoucy , that while basic license is stateful-only, these tests cannot be considered deployment-agnostic. wdyt?

Copy link
Contributor Author

@elena-shostak elena-shostak Jan 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can move them out of the deployment_agnostic folder to eliminate confusion, but all those tests still use the same test suite the serverless/trial config does

- x-pack/test/spaces_api_integration/deployment_agnostic/security_and_spaces/stateful.config_trial.ts
- x-pack/test/spaces_api_integration/deployment_agnostic/security_and_spaces/stateful.copy_to_space.config_trial.ts
- x-pack/test/spaces_api_integration/deployment_agnostic/security_and_spaces/stateful.copy_to_space.config_basic.ts
62 changes: 62 additions & 0 deletions packages/kbn-es/src/stateful_resources/roles.yml
Original file line number Diff line number Diff line change
Expand Up @@ -128,3 +128,65 @@ system_indices_superuser:
privileges: ['*']
resources: ['*']
run_as: ['*']

machine_learning_admin:
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

roles.yml is expected to list only default Cloud roles, we keep consistency with Cloud (both MKI and ESS) because we can't provision roles from this file to Cloud deployment. In order to make things work, we introduced custom roles support in FTR:

// First set privileges for custom role
await samlAuth.setCustomRole({
        elasticsearch: {
          indices: [{ names: ['logstash-*'], privileges: ['read', 'view_index_metadata'] }],
        },
        kibana: [
          {
            feature: {
              discover: ['read'],
            },
            spaces: ['*'],
          },
        ],
      });
    });

// Then you can login in browser as a user with newly defined privileges
await pageObjects.svlCommonPage.loginWithCustomRole()

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, but in stateful they are built in already, isn't it? (screenshots are from cloud deployment)
Am I missing smth?

Screenshot 2025-01-10 at 11 01 58 Screenshot 2025-01-10 at 11 01 49

Copy link
Contributor

@jeramysoucy jeramysoucy Jan 14, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These roles are built-in to stateful, but not serverless. The predefined serverless roles can be found in the roles.yml files in packages/kbn-es/src/serverless_resources/project_roles, nested under solution-specific folders.

Any tests that need a role other than these built-in serveless roles, will require a custom role, and will not be able to run on MKI from the FTR (at least for the time being, as it is not yet supported).

The stateful roles file is meant to mirror the serverless roles to keep the testing methodology the same.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ok, I will move it to custom role

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

UPD: we cannot use our api for custom roles or at least I don't see how. All those 3 roles rely on reserved built in privileges: ml_admin, ml_user, reserved_monitoring . I cannot pass those in, since we accept only read, all, none. Going one layer down to use es api to create those roles and adding other if/else statements seems not feasible.

I've moved those 3 tests to only stateful for now.

cluster: ['manage_ml']
indices:
- names: ['.ml-anomalies*','.ml-notifications*','.ml-state*','.ml-meta*','.ml-stats-*']
privileges: ['view_index_metadata', 'read']
allow_restricted_indices: true
- names: ['.ml-annotations*']
privileges: ['write', 'read', 'view_index_metadata']
allow_restricted_indices: true
applications:
- application: 'kibana-*'
privileges: ['reserved_ml_admin']
resources: ['*']
run_as: []

machine_learning_user:
cluster: ['monitor_ml']
indices:
- names: ['.ml-anomalies*','.ml-notifications*',]
privileges: ['view_index_metadata', 'read']
allow_restricted_indices: false
- names: ['.ml-annotations*']
privileges: ['write', 'read', 'view_index_metadata']
allow_restricted_indices: false
applications:
- application: 'kibana-*'
privileges: ['reserved_ml_user']
resources: ['*']
run_as: []

monitoring_user:
cluster:
- "cluster:monitor/main"
- "cluster:monitor/xpack/info"
- "cluster:monitor/remote/info"
indices:
- names:
- ".monitoring-*"
privileges:
- "read"
- "read_cross_cluster"
allow_restricted_indices: false
- names:
- "/metrics-(beats|elasticsearch|enterprisesearch|kibana|logstash).*/"
privileges:
- "read"
- "read_cross_cluster"
allow_restricted_indices: false
- names:
- "metricbeat-*"
privileges:
- "read"
- "read_cross_cluster"
allow_restricted_indices: false
applications:
- application: "kibana-*"
privileges:
- "reserved_monitoring"
resources:
- "*"
run_as: []
6 changes: 6 additions & 0 deletions src/dev/eslint/types.eslint.config.template.cjs
Original file line number Diff line number Diff line change
Expand Up @@ -34,5 +34,11 @@ module.exports = {
'@typescript-eslint/no-floating-promises': 'error',
},
},
{
files: ['*spaces_api_integration/common/services/basic_auth_supertest.ts'],
Copy link
Contributor Author

@elena-shostak elena-shostak Jan 3, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

agent.auth(this.user.username!, this.user.password!);

Needed to suppress the eslint error for this line in our custom test service, the promise is awaited in the tests itself and adding a catch block breaks some of the test cases

eslint-disable line gets removed because this rule runs dynamically

rules: {
'@typescript-eslint/no-floating-promises': 'off',
},
},
],
};
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ export function createStatefulTestConfig<T extends DeploymentAgnosticCommonServi
if (options.esServerArgs || options.kbnServerArgs) {
throw new Error(
`FTR doesn't provision custom ES/Kibana server arguments into the ESS deployment.
It may lead to unexpected test failures on Cloud. Please contact #appex-qa.`
It may lead to unexpected test failures on Cloud. Please contact #appex-qa.`
);
}

Expand Down
4 changes: 4 additions & 0 deletions x-pack/test/spaces_api_integration/common/config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,8 @@ import path from 'path';
import { REPO_ROOT } from '@kbn/repo-info';
import type { FtrConfigProviderContext } from '@kbn/test';

import { RoleScopedSupertestProvider } from '../deployment_agnostic/services/role_scoped_supertest';

interface CreateTestConfigOptions {
license: string;
disabledPlugins?: string[];
Expand Down Expand Up @@ -45,6 +47,8 @@ export function createTestConfig(name: string, options: CreateTestConfigOptions)
kibanaServer: config.kibana.functional.get('services.kibanaServer'),
spaces: config.xpack.api.get('services.spaces'),
usageAPI: config.xpack.api.get('services.usageAPI'),
roleScopedSupertest: RoleScopedSupertestProvider,
samlAuth: () => {},
},
junit: {
reportName: 'X-Pack Spaces API Integration Tests -- ' + name,
Expand Down
Loading