sf force mdapi listmetadata --metadata-type WaveDataflow does not retrieve recipe dataflows

[mnunezdm]

Summary

When trying to look for WaveDataflows related to WaveRecipe using the org browser, which I assume relies on the list metadata command these dataflows do not exist and need to be manually downloaded from the terminal with the same API name as the recipe

sf force source retrieve -m WaveRecipe:Recipe1 -m WaveDataflow:Recipe1

Steps To Reproduce

1. Create an scratch org with the config in the additional infomration section
2. Assign CRM Analytics Platform Admin permission set
3. Go to Analytics Studio app
   
4. Go to Data Manager
5. Go to Recipes > New Recipe
6. Save the recipe with name Test
7. Run sf force mdapi listmetadata --metadata-type WaveDataflow
8. Run sf force mdapi listmetadata --metadata-type WaveRecipe
9. Run sf force source retrieve -m WaveDataflow:Test

Expected result

Both list metadata for WaveDataflow and WaveRecipe should retrieve the Test Recipe / Dataflow

Actual result

The dataflow is not listed

System Information

zsh

{
  "architecture": "darwin-arm64",
  "cliVersion": "@salesforce/cli/2.36.8",
  "nodeVersion": "node-v20.11.1",
  "osVersion": "Darwin 23.4.0",
  "rootPath": "/Users/miguelnunezdiaz-montes/.local/share/sf/client/2.36.8-fa01960",
  "shell": "zsh",
  "pluginVersions": [
    "@oclif/plugin-autocomplete 3.0.13 (core)",
    "@oclif/plugin-commands 3.2.2 (core)",
    "@oclif/plugin-help 6.0.20 (core)",
    "@oclif/plugin-not-found 3.1.1 (core)",
    "@oclif/plugin-plugins 5.0.5 (core)",
    "@oclif/plugin-search 1.0.20 (core)",
    "@oclif/plugin-update 4.2.3 (core)",
    "@oclif/plugin-version 2.0.16 (core)",
    "@oclif/plugin-warn-if-update-available 3.0.15 (core)",
    "@oclif/plugin-which 3.1.7 (core)",
    "@salesforce/cli 2.36.8 (core)",
    "apex 3.1.3 (core)",
    "auth 3.5.1 (core)",
    "data 3.2.4 (core)",
    "deploy-retrieve 3.5.3 (core)",
    "info 3.1.2 (core)",
    "limits 3.2.1 (core)",
    "marketplace 1.1.0 (core)",
    "org 3.6.3 (core)",
    "packaging 2.3.0 (core)",
    "schema 3.2.0 (core)",
    "settings 2.1.2 (core)",
    "sobject 1.2.1 (core)",
    "source 3.2.2 (core)",
    "telemetry 3.1.17 (core)",
    "templates 56.1.0 (core)",
    "trust 3.5.1 (core)",
    "user 3.4.3 (core)"
  ]
}

Additional information

{
	"orgName": "Demo company",
	"edition": "Developer",
	"features": ["AnalyticsAdminPerms", "AnalyticsAppEmbedded", "WavePlatform"],
	"settings": {
		"analyticsSettings": {}
	},
	"language": "en_US"
}

$ sf force mdapi listmetadata --metadata-type WaveDataflow --target-org [email protected]                 
Warning: No metadata found for type: WaveDataflow in org: [email protected].
$ sf force mdapi listmetadata --metadata-type WaveRecipe --target-org [email protected]
[
  {
    "createdById": "0051w00000Ag7o4AAB",
    "createdByName": "User User",
    "createdDate": "2024-04-06T09:31:31.000Z",
    "fileName": "wave/Test.wdpr",
    "fullName": "Test",
    "id": "05v1w00000002iAAAQ",
    "lastModifiedById": "0051w00000Ag7o4AAB",
    "lastModifiedByName": "User User",
    "lastModifiedDate": "2024-04-06T09:31:31.000Z",
    "manageableState": "unmanaged",
    "type": "WaveRecipe"
  }
]
$ sf force source retrieve -m WaveDataflow:Test --target-org [email protected]      
Retrieving v59.0 metadata from [email protected] using the v60.0 SOAP API
Preparing retrieve request... done
=== Retrieved Source

 FULL NAME TYPE         PROJECT PATH                                  STATE   
 ───────── ──────────── ───────────────────────────────────────────── ─────── 
 Test      WaveDataflow force-app/main/default/wave/Test.wdf          Created 
 Test      WaveDataflow force-app/main/default/wave/Test.wdf-meta.xml Created

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[WillieRuemmele]

Hi @mnunezdm could you please elaborate more on
"create a recipe"

what do I need to enable in a scratch org, what buttons to click, what steps to follow etc

[mnunezdm]

Hello!

Updated issue description! Hope that is enough

[WillieRuemmele]

Hi @mnunezdm - you're steps look really good now, but I'm still getting stuck. I create a new scratch org, with the additional settings provided, but then don't see a CRM Analytics Platform Admin permission set in the org. Is there any source that needs to be deployed? If so can you provide a link to a github project, or zip folder, with the source project

you might try adding the --folder wave flag and value to your listmetadata command too

I've been able to reproduce this, but if I query the SourceMember object (which tracks remote changes) I don't see any WaveDataFlow metadata in the org, so the command is printing the correct result

 ➜  sf data query -q "SELECT MemberName, MemberType FROM SourceMember" -t
 MEMBERNAME MEMBERTYPE 
 ────────── ────────── 
 test       WaveRecipe 
Total number of records retrieved: 1.
Querying Data... done

If you can create a WaveDataflow, and have it show up in this query, it should appear in the listmetadata command

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.

[mnunezdm]

Hi @mnunezdm - you're steps look really good now, but I'm still getting stuck. I create a new scratch org, with the additional settings provided, but then don't see a CRM Analytics Platform Admin permission set in the org. Is there any source that needs to be deployed? If so can you provide a link to a github project, or zip folder, with the source project

you might try adding the --folder wave flag and value to your listmetadata command too

I've been able to reproduce this, but if I query the SourceMember object (which tracks remote changes) I don't see any WaveDataFlow metadata in the org, so the command is printing the correct result

 ➜  sf data query -q "SELECT MemberName, MemberType FROM SourceMember" -t
 MEMBERNAME MEMBERTYPE 
 ────────── ────────── 
 test       WaveRecipe 
Total number of records retrieved: 1.
Querying Data... done

If you can create a WaveDataflow, and have it show up in this query, it should appear in the listmetadata command

Hello @WillieRuemmele, but the thing is that, if you try to retrieve it, you are going to be able to retrieve that metadata

Also, if you try to deploy the WaveRecipe without the WaveDataflow, its going to fail

[mshanemc]

Hello @WillieRuemmele, but the thing is that, if you try to retrieve it, you are going to be able to retrieve that metadata

What exactly is the problem with retrieving it and then deploying it?

[mnunezdm]

I think I'm not explaining correctly, the problem is not that you need to retrieve it and deploy it

The problem is that, when you list the WaveDataflow type, the WaveDataflow related to WaveRecipe don't appear in the list

For me, that is an issue

[mshanemc]

Maybe it's just the list command not returning something you expect? And all the deploy/retrieve are downstream effects of the bug?

Or is the issue that deploy requires the metadata for recipe but you don't think it should?

Could you rewrite your issue, as simply as possible?

[mnunezdm]

The issue is that list metadata on Dataflows does not retrieve the whole list of wavedataflows in the org as the dataflow related to waverecipes can be downloaded but not listed

[github-actions]

We have determined that the issue you reported exists in code owned by another team that uses only the official support channels. To ensure that your issue is addressed, open an official Salesforce customer support ticket with a link to this issue. We encourage anyone experiencing this issue to do the same to increase the priority. We will keep this issue open for the community to collaborate on.

[github-actions]

This issue has not received a response in 7 days. It will auto-close in 7 days unless a response is posted.