Why is it necessary to specify test level when passing tests in a deployment?

[mnunezdm]

Is your feature request related to a problem? Please describe.

There ire 4 possible ways to perform a deployment regarding tests:

• Without running tests
• Running specified tests
• Running all local tests
• Running all tests

For deploying without tests there is not necessary to pass anything
For deploying with local / and all tests you need to pass the test-level flag
For running specified tests, you need to pass the test-level flag, and also a test argument for every test

What are you trying to do

n/a

Describe the solution you'd like

For me, it would be more natural to just pass the test list without needing to specify the test level flag

Describe alternatives you've considered

n/a

Additional context
Add any other context or screenshots about the feature request here.

[git2gus]

This issue has been linked to a new work item: W-13897188

[github-actions]

Thank you for filing this feature request. We appreciate your feedback and will review the feature at our next grooming or sprint planning session. We prioritize feature requests with more upvotes and comments.

[iowillhoit]

Hey @mnunezdm, I just want to clarify what you are suggesting.

Currently if you want to run specific tests, you would need to run:

sf project deploy start --test-level RunSpecifiedTests --tests Test1 --tests Test2

Are you suggesting that if we are passing --tests that --test-level RunSpecifiedTests should be assumed? So the new command would just be:

sf project deploy start --tests Test1 --tests Test2

[mnunezdm]

That's it!

[cristiand391]

Hey @mnunezdm 👋🏼 , sorry for the late reply

I think that's possible (sf apex run test --tests already does that). I'll check with my team and PM next week.

[mnunezdm]

hello @cristiand391 in deployments I think it still is necessary

sf project deploy start --source-dir delta --tests test1 --target-org alias --dry-run                          
Error (1): The following error occurred:
  All of the following must be provided when using --tests: --test-level

[cristiand391]

sorry, I meant to say it would be possible to do this without any breaking change, so far only apex run test seems to allow this.

[cristiand391]

We are good making this change in the non-deprecated commands, marking it as help wanted in case someone wants to add this, some tips:

oclif (the CLI framework we use) allows to set a default value for a flag, here we just want to set --test-level to RunSpecifiedTests only when --tests was specified.

first, you would have to delete the dependsOn prop from --tests from here
--tests flag definition:
https://github.com/salesforcecli/plugin-deploy-retrieve/blob/85eacd3e3c58748325859d0d19012fff16ea8404/src/utils/flags.ts#L91-L103

after that, running something like sf project deploy start --ignore-conflicts --tests GeocodingServiceTest -o dreamhouse will prob throw an API error saying the test-level is missing, ⬇️

flag.default can be an function that takes a "context" obj with references to the specified flags, use it to check if --tests was specified and if so set the right test level

--test-level flag definition:
https://github.com/salesforcecli/plugin-deploy-retrieve/blob/85eacd3e3c58748325859d0d19012fff16ea8404/src/utils/flags.ts#L71-L75

something like this should be a good start:

  default: async (context) => {
    if (context.flags.tests) return TestLevel.RunSpecifiedTests
  },

oclif docs: https://oclif.io/docs/flags/#custom-flags

[cbohlman]

Ran with the above guidance from @cristiand391 and added a NUT to cover the test case.

Apologies if I missed anything, this is my first OSS contribution. Also if there's any tips for setup to run SourceTestkit or cli-plugins-testkit efficiently it would be appreciated. I was able to get the tests to run but did find I would pretty quickly run into my dev hubs daily provisioning limit for scratch orgs.

[shetzel]

@cbohlman - merged into plugin-deploy-retrieve and will be picked up by the next RC. Or you can get it in the just published v2.1.0 of the plugin. Thanks for the contribution!