Skip to content

Commit

Permalink
Adding ADO template and sample default app settings
Browse files Browse the repository at this point in the history
  • Loading branch information
adityanarayanp authored Dec 16, 2024
1 parent 89529ce commit 265cdca
Show file tree
Hide file tree
Showing 4 changed files with 509 additions and 0 deletions.
78 changes: 78 additions & 0 deletions doc/UsageDoc/CA_UsageDocument.md
Original file line number Diff line number Diff line change
Expand Up @@ -508,7 +508,85 @@ Incase your project has both NPM/Nuget components it can be handled by merely ru

4. Once this is done after the dll run you can find that the components from the first run for "**NPM**" and the components from second run for "**NUGET**" will be merged into one BOM file

# Templates

## Azure DevOps

Sample templates for integrating the Continuous Clearing Tool (CCTool) workflow in Azure Pipelines can be found at `templates\azureDevops`.
For more details on Azure DevOps templates, refer to the official [Microsoft Documentation](https://learn.microsoft.com/en-us/azure/devops/pipelines/process/templates?view=azure-devops&pivots=templates-includes).

### **Advantages of Running CA Tool via Templates**
- **Simplified Setup:** Avoids adding manual steps for different CCTool stages.
- **Consistency and Standardization:** Ensures uniform execution across the organization.
- **Automated File Uploads:** Handles uploading of logs and BOM files after execution.

---

## Integration

1. **Check-in Templates:** Commit the templates into an Azure DevOps repository.
2. **Reference the Repository:** Include the repository in a new pipeline as shown below:

```yaml
resources:
repositories:
- repository: Templates_Pipeline
type: git
name: YourProject/Templates_Pipeline
```
:point_right: Note: If the Appsettingsfilepath parameter is not passed, the sample default app settings file is used by the template.
The sample default app settings file is located at `templates\sample-default-app-settings.json` and can be customized as needed.

### Add a New Template Calling Step

#### NuGet Template Example

```yaml
- template: pipeline-template-step-install-run-cctool-nuget.yml@Templates_Pipeline
parameters:
sw360url: '$(sw360url)'
sw360token: '$(sw360token)'
sw360projectID: '$(sw360projectID)'
sw360projectName: '$(sw360projectName)'
Sw360AuthTokenType: '$(Sw360AuthTokenType)'
projecttype: '$(projecttype)'
Artifactorytoken: '$(ARTIFACTORYAPIKEY)'
packageFilePath: '$(packageFilePath)'
BomFolderPath: '$(BomFolderPath)'
bomFilePath: '$(BomFolderPath)/$(sw360projectName)_Bom.cdx.json'
```

#### Debian Template Example

```yaml
- template: pipeline-template-step-install-run-cctool-docker.yml@Templates_Pipeline
parameters:
sw360url: '$(sw360url)'
sw360token: '$(sw360token)'
sw360projectID: '$(sw360projectID)'
sw360projectName: '$(sw360projectName)'
Sw360AuthTokenType: '$(Sw360AuthTokenType)'
projecttype: '$(projecttype)'
Artifactorytoken: '$(ARTIFACTORYAPIKEY)'
packageFilePath: '$(packageFilePath)'
BomFolderPath: '$(BomFolderPath)'
bomFilePath: '$(BomFolderPath)/$(sw360projectName)_Bom.cdx.json'
```

### Paramters
| Name | Description |
|--|--|
| `Appsettingsfilepath`| Add the appSetting.json file path |
| `sw360url`| Provide the Sw360 URL |
| `sw360token`| Provide the Sw360 Token|
| `sw360projectID` | Provide the Sw360 ProjectID which you're going to make an entry in Sw360|
|`sw360projectName`|Provide the Sw360 Project Name|
|`Sw360AuthTokenType`|Add the Sw360 tokentype either **Token or Bearer**|
|`projecttype`|Package type NPM/NUGET/DEBIAN/MAVEN/PYTHON|
|`Artifactorytoken`|JFrog Artifatory token|
|`packageFilePath`|Path where the input files resides|
|`BomFolderPath`|Path for creating BOM's files after the run of CCTool|

# Troubleshoot
1. In case your pipeline takes a lot of time to run(more than 1 hour) when there are many components. It is advisable to increase the pipeline timeout and set it to a minimum of 1 hr.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,155 @@
parameters:
- name: sw360projectID
default: ''

- name: sw360projectName
default: ''

- name: projecttype
default: ''

- name: projecttype2
default: ''

- name: IdentifierBomFilePath
default: ''

- name: CAtoolversion
default: ''

- name: Appsettingsfilepath
default: ''

- name: fossologyurl
default: ''

- name: sw360url
default: ''

- name: sw360token
default: ''

- name: sW360AuthTokenType
default: ''

- name: Artifactorytoken
default: ''

- name: ArtifactoryUploadUser
default: ''

- name: JfrogApi
default: ''

- name: packageFilePath
default: ''

- name: packageFilePath2
default: ''

- name: BomFolderPath
default: ''

- name: bomFilePath
default: ''

- name: Release
default: 'false'

- name: Timeout
default: '300'

steps:
##### CA Tool install #####

- task: PowerShell@2
displayName: CA Tool Install ${{ parameters.CAtoolversion }}
inputs:
targetType: 'inline'
script: |
$Value = "${{ parameters.CAtoolversion }}"
if (!($Value)) {
Write-Host "install ca tool latest"
docker pull ghcr.io/siemens/continuous-clearing
}
else {
Write-Host "install ca tool ${{ parameters.CAtoolversion }}"
docker pull ghcr.io/siemens/continuous-clearing:v${{ parameters.CAtoolversion }}
}
##### Prerequisites for CATool Running.. #####
- task: PowerShell@2
continueOnError: false
displayName: CATool Prerequisites
inputs:
targetType: 'inline'
script: |
## Creating required directories ##
mkdir "$(Build.SourcesDirectory)\Logs_CAtool"
mkdir "$(Build.SourcesDirectory)\CAConfig"
mkdir "$(Build.SourcesDirectory)\Output"
## Creating required directories ##
## Selecting the AppSetting file ##
$appsettings = "${{ parameters.Appsettingsfilepath }}"
if (!($appsettings)) {
Write-Host "Package Identifier run with Default appsettings.json"
$appsettings = "$(Build.SourcesDirectory)\templates\sample-default-app-settings.json"
}
Write-Host "Selected settings file : $appsettings"
## Selecting the AppSetting file ##
## Coping to local directories ##
Copy-Item "$appsettings" -Destination "$(Build.SourcesDirectory)\CAConfig"
## Coping to local directories ##
##### PacakgeIdentifier #####
- task: PowerShell@2
displayName: PacakgeIdentifier for ${{ parameters.projecttype }}
inputs:
targetType: 'inline'
script: |
$appSettingappSettingFileName = (Get-ChildItem -Path $(Build.SourcesDirectory)\CAConfig | Select-Object -First 1).Name
Write-Host "AppSetting used : $appSettingappSettingFileName"
docker run --rm --network="host" -v ${{parameters.packageFilePath}}:/mnt/Input -v ${{parameters.BomFolderPath}}:/mnt/Output -v $(Build.SourcesDirectory)\CAConfig:/etc/CATool -v $(Build.SourcesDirectory)\Logs_CAtool:/var/log ghcr.io/siemens/continuous-clearing dotnet PackageIdentifier.dll --packageFilePath "/mnt/Input" --bomFolderPath "/mnt/Output" --sW360URL "${{ parameters.sw360url }}" --sw360Token "${{ parameters.sw360token }}" --sw360ProjectID "${{ parameters.sw360projectID }}" --sW360ProjectName "{{ parameters.sw360projectName }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --ArtifactoryUploadApiKey "${{ parameters.Artifactorytoken }}" --projectType "${{ parameters.projecttype }}" --fossologyurl "${{ parameters.fossologyurl }}" --JFrogApi "${{ parameters.JFrogApi }}" --settingsfilepath /etc/CATool/$appSettingappSettingFileName --timeOut "${{ parameters.Timeout }}"
##### PacakgeIdentifier For Multiple package type #####
- ${{ if parameters.IdentifierBomFilePath }}:
- task: PowerShell@2
displayName: PacakgeIdentifier for ${{ parameters.projecttype2 }}
inputs:
targetType: 'inline'
script: |
$appSettingappSettingFileName = (Get-ChildItem -Path $(Build.SourcesDirectory)\CAConfig | Select-Object -First 1).Name
Write-Host "AppSetting used : $appSettingappSettingFileName"
$IdentifierBomFilePath = Split-Path -Path "${{parameters.IdentifierBomFilePath}}" -Leaf
Write-Host "Identifier Bom FilePath : $IdentifierBomFilePath"
docker run --rm --network="host" -v ${{parameters.packageFilePath2}}:/mnt/Input -v ${{parameters.BomFolderPath}}:/mnt/Output -v $(Build.SourcesDirectory)\CAConfig:/etc/CATool -v $(Build.SourcesDirectory)\Logs_CAtool:/var/log ghcr.io/siemens/continuous-clearing dotnet PackageIdentifier.dll --packageFilePath "/mnt/Input" --bomFolderPath "/mnt/Output" --identifierBomFilePath "/mnt/Output/$IdentifierBomFilePath" --sW360URL "${{ parameters.sw360url }}" --sw360Token "${{ parameters.sw360token }}" --sw360ProjectID "${{ parameters.sw360projectID }}" --sW360ProjectName "{{ parameters.sw360projectName }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --projectType "${{ parameters.projecttype2 }}" --ArtifactoryUploadApiKey "${{ parameters.Artifactorytoken }}" --fossologyurl "${{ parameters.fossologyurl }}" --JFrogApi "${{ parameters.JFrogApi }}" /etc/CATool/$appSettingappSettingFileName --timeOut "${{ parameters.Timeout }}"
##### PacakgeCreator #####
- task: PowerShell@2
continueOnError: true
displayName: PacakgeCreator
inputs:
targetType: 'inline'
script: |
$appSettingappSettingFileName = (Get-ChildItem -Path $(Build.SourcesDirectory)\CAConfig | Select-Object -First 1).Name
$BomFileName = "${{ parameters.sw360projectName }}_Bom.cdx.json"
Write-Host "AppSetting used : $appSettingappSettingFileName"
Write-Host "Bom FileName : $BomFileName"
docker run --rm --network="host" -v ${{parameters.packageFilePath}}:/mnt/Input -v ${{parameters.BomFolderPath}}:/mnt/Output -v $(Build.SourcesDirectory)\CAConfig:/etc/CATool -v $(Build.SourcesDirectory)\Logs_CAtool:/var/log ghcr.io/siemens/continuous-clearing dotnet SW360PackageCreator.dll --bomfilePath "/mnt/Output/$BomFileName" --sW360URL "${{ parameters.sw360url }}" --sw360Token "${{ parameters.sw360token }}" --sw360ProjectID "${{ parameters.sw360projectID }}" --sW360ProjectName "{{ parameters.sw360projectName }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --projectType "${{ parameters.projecttype }}" --ArtifactoryUploadApiKey "${{ parameters.Artifactorytoken }}" --fossologyurl "${{ parameters.fossologyurl }}" --settingsfilepath /etc/CATool/$appSettingappSettingFileName --timeOut "${{ parameters.Timeout }}"
##### ArtifactoryUploader #####
- task: PowerShell@2
continueOnError: true
displayName: ArtifactoryUploader
inputs:
targetType: 'inline'
script: |
$appSettingappSettingFileName = (Get-ChildItem -Path $(Build.SourcesDirectory)\CAConfig | Select-Object -First 1).Name
$BomFileName = "${{ parameters.sw360projectName }}_Bom.cdx.json"
Write-Host "AppSetting used : $appSettingappSettingFileName"
Write-Host "Bom FileName : $BomFileName"
docker run --rm --network="host" -v ${{parameters.packageFilePath}}:/mnt/Input -v ${{parameters.BomFolderPath}}:/mnt/Output -v $(Build.SourcesDirectory)\CAConfig:/etc/CATool -v $(Build.SourcesDirectory)\Logs_CAtool:/var/log ghcr.io/siemens/continuous-clearing dotnet ArtifactoryUploader.dll --bomfilePath "/mnt/Output/$BomFileName" --sW360URL "${{ parameters.sw360url }}" --sw360Token "${{ parameters.sw360token }}" --sw360ProjectID "${{ parameters.sw360projectID }}" --sW360ProjectName "{{ parameters.sw360projectName }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --projectType "${{ parameters.projecttype }}" --ArtifactoryUploadApiKey "${{ parameters.Artifactorytoken }}" --JFrogApi "${{ parameters.JFrogApi }}" --settingsfilepath /etc/CATool/$appSettingappSettingFileName --timeOut "${{ parameters.Timeout }}"
Original file line number Diff line number Diff line change
@@ -0,0 +1,150 @@
parameters:
- name: sw360projectID
default: ''

- name: sw360projectName
default: ''

- name: projecttype
default: ''

- name: projecttype2
default: ''

- name: IdentifierBomFilePath
default: ''

- name: CAtoolversion
default: ''

- name: Appsettingsfilepath
default: ''

- name: fossologyurl
default: ''

- name: sw360url
default: ''

- name: sw360token
default: ''

- name: sW360AuthTokenType
default: ''

- name: Artifactorytoken
default: ''

- name: ArtifactoryUploadUser
default: ''

- name: JfrogApi
default: ''

- name: ProjectID
default: ''

- name: packageFilePath
default: ''

- name: packageFilePath2
default: ''

- name: BomFolderPath
default: ''

- name: bomFilePath
default: ''

- name: Release
default: 'false'

- name: Timeout
default: '300'

- name: PackageCreatorEnabled
type: boolean
default: true

- name: ArtifactoryUploaderEnabled
type: boolean
default: true

steps:
##### CA Tool install #####

- task: PowerShell@2
displayName: CA Tool Install ${{ parameters.CAtoolversion }}
inputs:
targetType: 'inline'
script: |
$Value = "${{ parameters.CAtoolversion }}"
if (!($Value )) {
Write-Host $Value
Write-Host "Installing latest CATool.."
nuget install continuous-clearing -x
}
else {
Write-Host "Installing latest CATool with version ${{ parameters.CAtoolversion }}"
nuget install continuous-clearing -Version ${{ parameters.CAtoolversion }}
}
##### PacakgeIdentifier #####
- task: PowerShell@2
displayName: PacakgeIdentifier for ${{ parameters.projecttype }}
inputs:
targetType: 'inline'
script: |
$appsettings = "${{ parameters.Appsettingsfilepath }}"
if (!($appsettings)) {
Write-Host "Package Identifier without appsettings.json"
$appsettings = "$(Build.SourcesDirectory)\templates\sample-default-app-settings.json"
ECHO $appsettings
}
$(Build.SourcesDirectory)\continuous-clearing.${{ parameters.CAtoolversion }}\tools\PackageIdentifier.exe --PackageFilePath "${{ parameters.packageFilePath }}" --BomFolderPath "${{ parameters.BomFolderPath }}" --projecttype "${{ parameters.projecttype }}" --sW360ProjectName "{{ parameters.sw360projectName }}" --sW360ProjectID "${{ parameters.sw360projectID }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --sW360URL "${{ parameters.sw360url }}" --sw360Token "${{ parameters.sw360token }}" --fossologyurl "${{ parameters.fossologyurl }}" --JFrogApi "${{ parameters.JFrogApi }}" --ArtifactoryUploadApiKey "${{ parameters.Artifactorytoken }}" --settingsfilepath "$appsettings" --timeOut "${{ parameters.Timeout }}"
##### PacakgeIdentifier For Multiple package type #####
- task: PowerShell@2
condition: ne('${{ parameters.IdentifierBomFilePath }}', '')
displayName: PacakgeIdentifier for multiple-project ${{ parameters.projecttype2 }}
inputs:
targetType: 'inline'
script: |
$appsettings = "${{ parameters.Appsettingsfilepath }}"
if (!($appsettings)) {
Write-Host "Package Identifier running with Default_appSettings.json"
$appsettings = "$(Build.SourcesDirectory)\templates\sample-default-app-settings.json"
}
$(Build.SourcesDirectory)\continuous-clearing.${{ parameters.CAtoolversion }}\tools\PackageIdentifier.exe --PackageFilePath "${{ parameters.packageFilePath2 }}" --identifierBomFilePath "${{ parameters.IdentifierBomFilePath }}" --BomFolderPath "${{ parameters.BomFolderPath }}" --projecttype "${{ parameters.projecttype2 }}" --sW360ProjectName "${{ parameters.sw360projectName }}" --sW360ProjectID "${{ parameters.sw360projectID }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --sW360URL "${{ parameters.sw360url }}" --fossologyurl "${{ parameters.fossologyurl }}" --sw360Token "${{ parameters.sw360token }}" --JFrogApi "${{ parameters.JFrogApi }}" --ArtifactoryUploadApiKey "${{ parameters.Artifactorytoken }}" --settingsfilepath "$appsettings" --timeOut "${{ parameters.Timeout }}"
##### PacakgeCreator #####
- task: PowerShell@2
condition: eq('${{ parameters.PackageCreatorEnabled }}', true)
continueOnError: true
displayName: PacakgeCreator
inputs:
targetType: 'inline'
script: |
$appsettings = "${{ parameters.Appsettingsfilepath }}"
if (!($appsettings)) {
Write-Host "Package Identifier running with Default_appSettings.json"
$appsettings = "$(Build.SourcesDirectory)\templates\sample-default-app-settings.json"
ECHO $appsettings
}
$(Build.SourcesDirectory)\continuous-clearing.${{ parameters.CAtoolversion }}\tools\SW360PackageCreator.exe --bomFilePath "${{ parameters.bomFilePath }}" --sW360ProjectName "${{ parameters.sw360projectName }}" --projecttype "${{ parameters.projecttype }}" --sW360ProjectID "${{ parameters.sw360projectID }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --sW360URL "${{ parameters.sw360url }}" --sw360Token "${{ parameters.sw360token }}" --fossologyurl "${{ parameters.fossologyurl }}" --settingsfilepath "$appsettings" --timeOut "${{ parameters.Timeout }}"
##### ArtifactoryUploader #####
- task: PowerShell@2
condition: eq('${{ parameters.ArtifactoryUploaderEnabled }}', true)
continueOnError: true
displayName: ArtifactoryUploader
inputs:
targetType: 'inline'
script: |
$appsettings = "${{ parameters.Appsettingsfilepath }}"
if (!($appsettings)) {
Write-Host "Package Identifier running with Default_appSettings.json"
$appsettings = "$(Build.SourcesDirectory)\templates\sample-default-app-settings.json"
ECHO $appsettings
}
$(Build.SourcesDirectory)\continuous-clearing.${{ parameters.CAtoolversion }}\tools\ArtifactoryUploader.exe --bomFilePath "${{ parameters.bomFilePath }}" --sW360ProjectName "${{ parameters.sw360projectName }}" --projecttype "${{ parameters.projecttype }}" --sW360ProjectID "${{ parameters.sw360projectID }}" --sW360AuthTokenType "${{ parameters.sW360AuthTokenType }}" --sW360URL "${{ parameters.sw360url }}" --sw360Token "${{ parameters.sw360token }}" --JFrogApi "${{ parameters.JFrogApi }}" --ArtifactoryUploadApiKey "${{ parameters.Artifactorytoken }}" --release "${{ parameters.Release }}" --settingsfilepath "$appsettings" --timeOut "${{ parameters.Timeout }}"
Loading

0 comments on commit 265cdca

Please sign in to comment.