[scout] adding unit tests #204567

dmlemeshko · 2024-12-17T12:48:24Z

Summary

Adding tests and making adjustments/fixes based on the findings.

Note: no integration tests were added to verify servers start as it is mostly equal to @kbn-test functionality that has jest integration tests. We can add it later, when Scout has specific logic.

How to run: node scripts/jest --config packages/kbn-scout/jest.config.js

Scope:

 PASS  packages/kbn-scout/src/config/config.test.ts
 PASS  packages/kbn-scout/src/config/loader/read_config_file.test.ts
 PASS  packages/kbn-scout/src/config/utils/get_config_file.test.ts
 PASS  packages/kbn-scout/src/config/utils/load_servers_config.test.ts
 PASS  packages/kbn-scout/src/config/utils/save_scout_test_config.test.ts
 PASS  packages/kbn-scout/src/playwright/config/create_config.test.ts
 PASS  packages/kbn-scout/src/playwright/runner/config_validator.test.ts
 PASS  packages/kbn-scout/src/playwright/runner/flags.test.ts
 PASS  packages/kbn-scout/src/playwright/utils/runner_utils.test.ts
 PASS  packages/kbn-scout/src/servers/flags.test.ts

dmlemeshko · 2024-12-17T12:49:43Z

packages/kbn-scout/src/common/services/clients.ts

@@ -29,7 +29,7 @@ function createClientUrlWithAuth({ serviceName, url, username, password, log }:
  return clientUrl.toString();
 }

-export function createEsClient(config: ScoutServerConfig, log: ToolingLog) {
+export function createEsClient(config: ScoutTestConfig, log: ToolingLog) {


Renamed to differentiate servers configuration and Scout test configuration, that used by Playwright fixtures/tests.

dmlemeshko · 2024-12-17T12:51:34Z

packages/kbn-scout/src/config/config.ts

@@ -13,15 +13,15 @@ import Path from 'path';
 import { cloneDeepWith, get, has, toPath } from 'lodash';
 import { REPO_ROOT } from '@kbn/repo-info';
 import { schema } from './schema';
-import { ScoutServerConfig } from '../types';
-import { formatCurrentDate, getProjectType } from './utils';
+import { ScoutServerConfig, ScoutTestConfig } from '../types';


ScoutServerConfig is compatible with kbn-test functionality and used to start the servers, while ScoutTestConfig is for Playwright tests and exposes hosts, credentials, licence and other test-required data.

dmlemeshko · 2024-12-17T12:51:49Z

packages/kbn-scout/src/config/config.ts

    return {
      serverless: this.get('serverless'),
      projectType: this.get('serverless')
        ? getProjectType(this.get('kbnTestServer.serverArgs'))
        : undefined,
      isCloud: false,
+      license: this.get('esTestCluster.license'),


might be useful for some tests

dmlemeshko · 2024-12-17T12:52:39Z

packages/kbn-scout/src/config/schema/schema.ts

@@ -75,7 +75,7 @@ export const schema = Joi.object()

    esTestCluster: Joi.object()
      .keys({
-        license: Joi.valid('basic', 'trial', 'gold').default('basic'),
+        license: Joi.valid('basic', 'trial', 'gold').default('trial'),


default trial to match default deployment-agnostic test design

elasticmachine · 2024-12-18T14:48:32Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 392b139

Failed CI Steps

Test Failures

[job] [logs] FTR Configs #70 / serverless search UI Serverless Inference Management UI delete action "before each" hook for "deletes modal successfully with usage"
[job] [logs] FTR Configs #2 / Spaces app space solution tour solution tour does not show the solution tour after deleting spaces and leave only the default

Metrics [docs]

✅ unchanged

History

💛 Build #261455 was flaky 91f53ff
💔 Build #261381 failed 4b15eff

cc @dmlemeshko

pheyos

LGTM
Left a few questions and suggestions, but approved in order to not block.

pheyos · 2024-12-19T14:08:47Z

packages/kbn-scout/src/config/utils/get_config_file.test.ts

+
+    expect(result).toBe(expectedPath);
+  });
+});


What do you think about adding two negative test cases?

completely unknown config (e.g. config = 'unsupportedConfig')

a serverless config with invalid type (e.g. config = 'serverless=unsupportedType')

good ideas, I actually covered the 2nd one in other area, but missed here. I will add it in the follow-up, as auto-merge was activated

config = 'unsupportedConfig'

I just realised why I skipped it: it is handled by Type check, basically https://github.com/dmlemeshko/kibana/blob/392b1394dea417b40bc32accfc88ab3ec494b4da/packages/kbn-scout/src/config/utils/get_config_file.ts#L13 restricts the argument passed to the function.

Same is actually valid for config = 'serverless=unsupportedType'

packages/kbn-scout/src/playwright/utils/runner_utils.test.ts

kibanamachine · 2024-12-19T16:36:30Z

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/12416890020

## Summary Adding tests and making adjustments/fixes based on the findings. Note: no integration tests were added to verify servers start as it is mostly equal to `@kbn-test` functionality that has jest integration tests. We can add it later, when Scout has specific logic. How to run: `node scripts/jest --config packages/kbn-scout/jest.config.js` Scope: ``` PASS packages/kbn-scout/src/config/config.test.ts PASS packages/kbn-scout/src/config/loader/read_config_file.test.ts PASS packages/kbn-scout/src/config/utils/get_config_file.test.ts PASS packages/kbn-scout/src/config/utils/load_servers_config.test.ts PASS packages/kbn-scout/src/config/utils/save_scout_test_config.test.ts PASS packages/kbn-scout/src/playwright/config/create_config.test.ts PASS packages/kbn-scout/src/playwright/runner/config_validator.test.ts PASS packages/kbn-scout/src/playwright/runner/flags.test.ts PASS packages/kbn-scout/src/playwright/utils/runner_utils.test.ts PASS packages/kbn-scout/src/servers/flags.test.ts ``` (cherry picked from commit 2ba3247)

kibanamachine · 2024-12-19T16:41:51Z

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation

## Summary Adding tests and making adjustments/fixes based on the findings. Note: no integration tests were added to verify servers start as it is mostly equal to `@kbn-test` functionality that has jest integration tests. We can add it later, when Scout has specific logic. How to run: `node scripts/jest --config packages/kbn-scout/jest.config.js` Scope: ``` PASS packages/kbn-scout/src/config/config.test.ts PASS packages/kbn-scout/src/config/loader/read_config_file.test.ts PASS packages/kbn-scout/src/config/utils/get_config_file.test.ts PASS packages/kbn-scout/src/config/utils/load_servers_config.test.ts PASS packages/kbn-scout/src/config/utils/save_scout_test_config.test.ts PASS packages/kbn-scout/src/playwright/config/create_config.test.ts PASS packages/kbn-scout/src/playwright/runner/config_validator.test.ts PASS packages/kbn-scout/src/playwright/runner/flags.test.ts PASS packages/kbn-scout/src/playwright/utils/runner_utils.test.ts PASS packages/kbn-scout/src/servers/flags.test.ts ```

# Backport This will backport the following commits from `main` to `8.x`: - [[scout] adding unit tests (#204567)](#204567)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Dzmitry Lemechko <[email protected]>

## Summary Adding tests and making adjustments/fixes based on the findings. Note: no integration tests were added to verify servers start as it is mostly equal to `@kbn-test` functionality that has jest integration tests. We can add it later, when Scout has specific logic. How to run: `node scripts/jest --config packages/kbn-scout/jest.config.js` Scope: ``` PASS packages/kbn-scout/src/config/config.test.ts PASS packages/kbn-scout/src/config/loader/read_config_file.test.ts PASS packages/kbn-scout/src/config/utils/get_config_file.test.ts PASS packages/kbn-scout/src/config/utils/load_servers_config.test.ts PASS packages/kbn-scout/src/config/utils/save_scout_test_config.test.ts PASS packages/kbn-scout/src/playwright/config/create_config.test.ts PASS packages/kbn-scout/src/playwright/runner/config_validator.test.ts PASS packages/kbn-scout/src/playwright/runner/flags.test.ts PASS packages/kbn-scout/src/playwright/utils/runner_utils.test.ts PASS packages/kbn-scout/src/servers/flags.test.ts ```

dmlemeshko added 4 commits December 16, 2024 18:56

add tests

689acbb

refactor: ScoutServerConfig / ScoutTestConfig

d12a830

refactor + more tests

72cb4e8

fix getConfigFilePath

7d58ec4

dmlemeshko commented Dec 17, 2024

View reviewed changes

Merge branch 'main' into scout/add-unit-tests

449be88

dmlemeshko self-assigned this Dec 17, 2024

dmlemeshko added release_note:skip Skip the PR/issue when compiling release notes backport:skip This commit does not require backporting v9.0.0 test:scout labels Dec 17, 2024

dmlemeshko marked this pull request as ready for review December 17, 2024 12:55

dmlemeshko requested a review from a team as a code owner December 17, 2024 12:55

dmlemeshko added 3 commits December 17, 2024 14:06

update docs

4b15eff

remove unused logger

73a73e0

Merge branch 'main' into scout/add-unit-tests

91f53ff

dmlemeshko enabled auto-merge (squash) December 17, 2024 17:14

Merge branch 'main' into scout/add-unit-tests

392b139

dmlemeshko added backport:prev-minor Backport to (8.x) the previous minor version (i.e. one version back from main) and removed backport:skip This commit does not require backporting labels Dec 18, 2024

pheyos approved these changes Dec 19, 2024

View reviewed changes

dmlemeshko merged commit 2ba3247 into elastic:main Dec 19, 2024
12 checks passed

kibanamachine mentioned this pull request Dec 19, 2024

[8.x] [scout] adding unit tests (#204567) #204980

Merged

kibanamachine added the v8.18.0 label Dec 19, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[scout] adding unit tests #204567

[scout] adding unit tests #204567

dmlemeshko commented Dec 17, 2024 •

edited by kibanamachine

Loading

dmlemeshko Dec 17, 2024

dmlemeshko Dec 17, 2024

dmlemeshko Dec 17, 2024

dmlemeshko Dec 17, 2024

elasticmachine commented Dec 18, 2024

pheyos left a comment

pheyos Dec 19, 2024

dmlemeshko Dec 19, 2024

dmlemeshko Dec 19, 2024 •

edited

Loading

kibanamachine commented Dec 19, 2024

kibanamachine commented Dec 19, 2024

[scout] adding unit tests #204567

[scout] adding unit tests #204567

Conversation

dmlemeshko commented Dec 17, 2024 • edited by kibanamachine Loading

Summary

dmlemeshko Dec 17, 2024

Choose a reason for hiding this comment

dmlemeshko Dec 17, 2024

Choose a reason for hiding this comment

dmlemeshko Dec 17, 2024

Choose a reason for hiding this comment

dmlemeshko Dec 17, 2024

Choose a reason for hiding this comment

elasticmachine commented Dec 18, 2024

💛 Build succeeded, but was flaky

Failed CI Steps

Test Failures

Metrics [docs]

History

pheyos left a comment

Choose a reason for hiding this comment

pheyos Dec 19, 2024

Choose a reason for hiding this comment

dmlemeshko Dec 19, 2024

Choose a reason for hiding this comment

dmlemeshko Dec 19, 2024 • edited Loading

Choose a reason for hiding this comment

kibanamachine commented Dec 19, 2024

kibanamachine commented Dec 19, 2024

💚 All backports created successfully

Questions ?

dmlemeshko commented Dec 17, 2024 •

edited by kibanamachine

Loading

dmlemeshko Dec 19, 2024 •

edited

Loading