Duplicate name '<Artifact name>' specified in ci/cd pipeline using sf version 2.9.1

[surbhigarg11]

Note
Before you submit your issue, make sure that:

• You're using the latest version of Salesforce CLI.
• You've searched both open and closed issues for related posts.
• You've used the doctor command to diagnose common issues.
• You understand that GitHub Issues don't adhere to any agreement or SLA.
  • If you require immediate assistance, use official channels such as Salesforce Customer Support.

Summary

Started using export CLIURL=https://developer.salesforce.com/media/salesforce-cli/sf/channels/stable/sf-linux-x64.tar.xz for our ci/cd gitlab pipeline and now the deployment fails with below erros:

Case.General | Record Type Duplicate name 'Case.General' specified
Opportunity.Fundraising | Record Type | 0 | 0 | Duplicate name 'Opportunity.Fundraising' specified
Opportunity.Sales | Record Type | 0 | 0 | Duplicate name 'Opportunity.Sales' specified
Opportunity.Fundraising | Record Type | 219 | 18 | Picklist value: Elevator Management in picklist: Opportunity_Type__c not found
Opportunity.Sales | Record Type | 799 | 18 | Picklist value: Elevator Management in picklist: Opportunity_Type__c not found
Case.General | Compact Layout | 0 | 0 | Duplicate name 'Case.General' specified

Steps To Reproduce

1. Use the latest CLI download url https://developer.salesforce.com/media/salesforce-cli/sf/channels/stable/sf-linux-x64.tar.xz
2. Run the piepline on gitlab

Expected result

Actual result

System Information

gitlab yaml file

PASTE_VERSION_OUTPUT_HERE
```@salesforce/cli/2.6.6 linux-x64 node-v18.15.0
### Additional information
I was initially using https://developer.salesforce.com/media/salesforce-cli/sfdx-linux-amd64.tar.xz url to install cli for the gitlab but since this version is not stable it was giving me cannot read file erros, so I moved to the stable url whihc is not working too. 
<img width="754" alt="image" src="https://github.com/forcedotcom/cli/assets/99746101/27a2564b-9ccd-4742-beba-9cb29adc8705">

[github-actions]

Hello @surbhigarg11 👋 It looks like you didn't include the full Salesforce CLI version information in your issue.
Please provide the output of version --verbose --json for the CLI you're using (sf or sfdx).

A few more things to check:

• Make sure you've provided detailed steps to reproduce your issue.
  • A repository that clearly demonstrates the bug is ideal.
• Make sure you've installed the latest version of Salesforce CLI. (docs)
  • Better yet, try the rc or nightly versions. (docs)
• Try running the doctor command to diagnose common issues.
• Search GitHub for existing related issues.

Thank you!

[github-actions]

Thank you for filing this issue. We appreciate your feedback and will review the issue as soon as possible. Remember, however, that GitHub isn't a mechanism for receiving support under any agreement or SLA. If you require immediate assistance, contact Salesforce Customer Support.

[surbhigarg11]

This is how my yaml looks:

variables:
SONAR_USER_HOME: "${CI_PROJECT_DIR}/.sonar" # Defines the location of the analysis task cache
GIT_DEPTH: "0" # Tells git to fetch all the branches of the project, required by the analysis task

default:
image: ruby:2.5
default:
image: ruby:2.5
before_script:
# Decrypt server key
- openssl aes-256-cbc -d -salt -pbkdf2 -in assets/server.key.aes -out assets/server.key -k $SERVER_KEY_PASSWORD
# Install jq
- apt update && apt -y install jq
# Setup SFDX environment variables
- export CLIURL=https://developer.salesforce.com/media/salesforce-cli/sf/channels/stable/sf-linux-x64.tar.xz
- export SF_AUTOUPDATE_DISABLE=false
- export SF_USE_GENERIC_UNIX_KEYCHAIN=true
- export SF_DOMAIN_RETRY=300
- export SF_DISABLE_APP_HUB=true
- export SF_LOG_LEVEL=debug
- export ROOTDIR=force-app/main/default/
- export TESTLEVEL=RunLocalTests
- export SCRATCH_ORG_ALIAS=test
- export PACKAGEVERSION=""
- export MIN_TEST_COVERAGE=75
# Setting up Salesforce CLI
- mkdir sfdx
- wget -qO- $CLIURL | tar xJ -C sfdx --strip-components 1
- export PATH=$(pwd)/sfdx/bin/:$PATH
# Output CLI version and plug-in information
- sfdx --version

[AllanOricil]

Use a docker image in your builds instead of installing sfdx on every run.

[mshanemc]

Please see the installation instructions for sf here
https://developer.salesforce.com/docs/atlas.en-us.sfdx_setup.meta/sfdx_setup/sfdx_setup_install_cli.htm

The section you have in # Setting up Salesforce CLI has some updates.

[surbhigarg11]

Use a docker image in your builds instead of installing sfdx on every run.

Yes, Thanks for the suggestion. We switched to a new docker image now to use npm to install CLI

[surbhigarg11]

Please see the installation instructions for sf here https://developer.salesforce.com/docs/atlas.en-us.sfdx_setup.meta/sfdx_setup/sfdx_setup_install_cli.htm

The section you have in # Setting up Salesforce CLI has some updates.

So I followed the instructions already through this link, used a new docker image too, still I get the below errors:

This is how my before script looks like now:

[AllanOricil]

I suggested you to use a docker image based on the ones salesforce maintain in docker hub. It has the cli already.

[mshanemc]

@surbhigarg11 any chance that your metadata has the problems that the error messages are saying they do? ex: if you search for General do you find it in duplicate places? Are the missing picklist values really there?

[surbhigarg11]

@surbhigarg11 any chance that your metadata has the problems that the error messages are saying they do? ex: if you search for General do you find it in duplicate places? Are the missing picklist values really there?

Yup, it is all there. There are no duplicates too. I checked. And the weird thing is, the errors started coming when we changed the cli version. I have found a workaround or more like a temporary fix, the cli version we are using now does not cause any issue during org deployment, but it still fails in scratch org deployment. So for now, we are not testing our feature branch on scratch org and directly deploying to qa org.

This is the error we are getting in our scratch org:

[surbhigarg11]

I was able to resolve the issue. Tried a newer SF V2 version and suddenly everything works. Thanks all for the help.

[AllanOricil]

you're welcome

[mshanemc]

You were using 7.65.4 ?!

Without looking at your metadata, I know there were issues where old CLI versions would let you put multiple versions of the same filepath into the zip file that gets sent to the metadata api, and that no longer is allowed.

The other clue I see is this:

the cli version we are using now does not cause any issue during org deployment, but it still fails in scratch org deployment. So for now, we are not testing our feature branch on scratch org and directly deploying to qa org.

That, and the errors around Standard Objects suggests that there is metadata in your qa/prod org that isn't in your scratch org, and what you're deploying depends on that.

---

next issue is that scratch org create error. It looks like you've got a wrong instanceUrl in there, somehow, with dev4s.salesforce.com. Usually urls should be some myDomain