Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Only needed secrets #649

Closed
wants to merge 35 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
2b1da43
Attempt 1
freddydk Aug 5, 2023
8143cd5
casing
freddydk Aug 5, 2023
0e3f73e
get and dump secrets
freddydk Aug 5, 2023
d4d5eaa
use with
freddydk Aug 5, 2023
4e13f78
str
freddydk Aug 5, 2023
b1deb36
rename to env:Secrets
freddydk Aug 5, 2023
130fb76
fix env vars
freddydk Aug 6, 2023
c8ef1c4
inherit
freddydk Aug 6, 2023
010ea9f
syntax
freddydk Aug 6, 2023
22fc940
secrets
freddydk Aug 6, 2023
ed459b1
move secrets
freddydk Aug 6, 2023
4601524
do not set empty secrets
freddydk Aug 7, 2023
7712a48
remove
freddydk Aug 7, 2023
ed11c2c
use secrets
freddydk Aug 7, 2023
9580920
secrets is set
freddydk Aug 7, 2023
bf5d470
comment
freddydk Aug 7, 2023
0a5ea37
support unread secrets
freddydk Aug 7, 2023
20f1763
do not rewrite settings in ReadSecrets
freddydk Aug 7, 2023
ec28d33
get type and init projects
freddydk Aug 7, 2023
64cfb1d
do not rewrite settings in DetermineArtifact
freddydk Aug 7, 2023
61f6397
spell
freddydk Aug 7, 2023
cf05ca7
catch
freddydk Aug 7, 2023
474cf21
fail on CR or LF
freddydk Aug 7, 2023
1416767
remove trailing spaces
freddydk Aug 7, 2023
50d9f30
add additional comment + try/catch
freddydk Aug 7, 2023
099c184
do not download probingPaths
freddydk Aug 7, 2023
946740a
test null
freddydk Aug 7, 2023
e1b3ca0
settingValue
freddydk Aug 7, 2023
1ad10db
test str
freddydk Aug 7, 2023
8e4f60c
analyzerepo
freddydk Aug 7, 2023
d429047
id
freddydk Aug 7, 2023
e902976
secrets
freddydk Aug 7, 2023
abc2a79
frombase64
freddydk Aug 7, 2023
708584b
syntax
freddydk Aug 7, 2023
cd32172
Merge branch 'main' into OnlyNeededSecrets
freddydk Aug 7, 2023
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
368 changes: 190 additions & 178 deletions Actions/AL-Go-Helper.ps1

Large diffs are not rendered by default.

5 changes: 3 additions & 2 deletions Actions/AddExistingApp/AddExistingApp.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -83,6 +83,7 @@ $telemetryScope = $null
$bcContainerHelperPath = $null

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
Expand Down Expand Up @@ -189,7 +190,7 @@ try {
try {
$settingsJsonFile = Join-Path $projectFolder $ALGoSettingsFile
$SettingsJson = Get-Content $settingsJsonFile -Encoding UTF8 | ConvertFrom-Json
if (@($settingsJson.appFolders)+@($settingsJson.testFolders)) {
if (@($settingsJson.appFolders) + @($settingsJson.testFolders)) {
if ($ttype -eq "Test App") {
if ($SettingsJson.testFolders -notcontains $foldername) {
$SettingsJson.testFolders += @($folderName)
Expand Down Expand Up @@ -219,7 +220,7 @@ try {
$workspace | Set-JsonContentLF -Path $workspaceFile
}
catch {
throw "$workspaceFileName is malformed.$([environment]::Newline) $($_.Exception.Message)"
throw "$workspaceFileName is malformed.$([environment]::Newline) $($_.Exception.Message)"
}
}
}
Expand Down
1 change: 1 addition & 0 deletions Actions/AnalyzeTests/AnalyzeTests.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ $telemetryScope = $null
$bcContainerHelperPath = $null

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
Expand Down
78 changes: 42 additions & 36 deletions Actions/CalculateArtifactNames/CalculateArtifactNames.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -17,41 +17,47 @@ function Set-EnvVariable([string] $name, [string] $value) {

$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0

$settings = $env:Settings | ConvertFrom-Json

if ($project -eq ".") {
$project = $settings.repoName
}

$branchName = $branchName.Replace('\','_').Replace('/','_')
$projectName = $project.Replace('\','_').Replace('/','_')

# If the buildmode is default, then we don't want to add it to the artifact name
if ($buildMode -eq 'Default') {
$buildMode = ''
}
Set-EnvVariable -name "BuildMode" -value $buildMode

if($suffix) {
# Add the date to the suffix
$suffix = "$suffix-$([DateTime]::UtcNow.ToString('yyyyMMdd'))"
# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
$settings = $env:Settings | ConvertFrom-Json

if ($project -eq ".") {
$project = $settings.repoName
}

$branchName = $branchName.Replace('\', '_').Replace('/', '_')
$projectName = $project.Replace('\', '_').Replace('/', '_')

# If the buildmode is default, then we don't want to add it to the artifact name
if ($buildMode -eq 'Default') {
$buildMode = ''
}
Set-EnvVariable -name "BuildMode" -value $buildMode

if ($suffix) {
# Add the date to the suffix
$suffix = "$suffix-$([DateTime]::UtcNow.ToString('yyyyMMdd'))"
}
else {
# Default suffix is the build number
$suffix = "$($settings.repoVersion).$($settings.appBuild).$($settings.appRevision)"
}

'Apps', 'Dependencies', 'TestApps', 'TestResults', 'BcptTestResults', 'BuildOutput', 'ContainerEventLog' | ForEach-Object {
$name = "$($_)ArtifactsName"
$value = "$($projectName)-$($branchName)-$buildMode$_-$suffix"
Set-EnvVariable -name $name -value $value
}

# Set this build artifacts name
'Apps', 'TestApps' | ForEach-Object {
$name = "ThisBuild$($_)ArtifactsName"
$value = "thisbuild-$($projectName)-$($buildMode)$($_)"
Set-EnvVariable -name $name -value $value
}
}
else {
# Default suffix is the build number
$suffix = "$($settings.repoVersion).$($settings.appBuild).$($settings.appRevision)"
}

'Apps','Dependencies','TestApps','TestResults','BcptTestResults','BuildOutput','ContainerEventLog' | ForEach-Object {
$name = "$($_)ArtifactsName"
$value = "$($projectName)-$($branchName)-$buildMode$_-$suffix"

Set-EnvVariable -name $name -value $value
}

# Set this build artifacts name
'Apps', 'TestApps' | ForEach-Object {
$name = "ThisBuild$($_)ArtifactsName"
$value = "thisbuild-$($projectName)-$($buildMode)$($_)"

Set-EnvVariable -name $name -value $value
catch {
OutputError -message "AnalyzeTests action failed.$([environment]::Newline)Error: $($_.Exception.Message)$([environment]::Newline)Stacktrace: $($_.scriptStackTrace)"
}
1 change: 1 addition & 0 deletions Actions/CheckForUpdates/CheckForUpdates.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ $telemetryScope = $null
$bcContainerHelperPath = $null

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
Expand Down
3 changes: 2 additions & 1 deletion Actions/CreateApp/CreateApp.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ $bcContainerHelperPath = $null
$tmpFolder = Join-Path ([System.IO.Path]::GetTempPath()) ([Guid]::NewGuid().ToString())

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
Expand Down Expand Up @@ -67,7 +68,7 @@ try {
if ($type -eq "Performance Test App") {
try {
$settings = ReadSettings -baseFolder $baseFolder -project $project
$settings = AnalyzeRepo -settings $settings -token $token -baseFolder $baseFolder -project $project -doNotIssueWarnings
$settings = AnalyzeRepo -settings $settings -token $token -baseFolder $baseFolder -project $project -doNotIssueWarnings -doNotCheckAppDependencyProbingPaths
$folders = Download-Artifacts -artifactUrl $settings.artifact -includePlatform
$sampleApp = Join-Path $folders[0] "Applications.*\Microsoft_Performance Toolkit Samples_*.app"
if (Test-Path $sampleApp) {
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ $telemetryScope = $null
$bcContainerHelperPath = $null

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
Expand Down
1 change: 1 addition & 0 deletions Actions/CreateReleaseNotes/CreateReleaseNotes.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -16,6 +16,7 @@ $telemetryScope = $null
$bcContainerHelperPath = $null

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1")
Expand Down
5 changes: 3 additions & 2 deletions Actions/Deliver/Deliver.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -47,6 +47,7 @@ function EnsureAzStorageModule() {
}

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
$baseFolder = $ENV:GITHUB_WORKSPACE
Expand Down Expand Up @@ -187,7 +188,7 @@ try {
Write-Host "Found custom script $customScript for delivery target $deliveryTarget"

$projectSettings = ReadSettings -baseFolder $baseFolder -project $thisProject
$projectSettings = AnalyzeRepo -settings $projectSettings -baseFolder $baseFolder -project $thisProject -doNotCheckArtifactSetting -doNotIssueWarnings
$projectSettings = AnalyzeRepo -settings $projectSettings -baseFolder $baseFolder -project $thisProject -doNotCheckArtifactSetting -doNotCheckAppDependencyProbingPaths -doNotIssueWarnings
$parameters = @{
"Project" = $thisProject
"ProjectName" = $projectName
Expand Down Expand Up @@ -408,7 +409,7 @@ try {
}
elseif ($deliveryTarget -eq "AppSource") {
$projectSettings = ReadSettings -baseFolder $baseFolder -project $thisProject
$projectSettings = AnalyzeRepo -settings $projectSettings -baseFolder $baseFolder -project $thisProject -doNotCheckArtifactSetting -doNotIssueWarnings
$projectSettings = AnalyzeRepo -settings $projectSettings -baseFolder $baseFolder -project $thisProject -doNotCheckArtifactSetting -doNotCheckAppDependencyProbingPaths -doNotIssueWarnings
# if type is Release, we only get here with the projects that needs to be delivered to AppSource
# if type is CD, we get here for all projects, but should only deliver to AppSource if AppSourceContinuousDelivery is set to true
if ($type -eq 'Release' -or ($projectSettings.Keys -contains 'AppSourceContinuousDelivery' -and $projectSettings.AppSourceContinuousDelivery)) {
Expand Down
4 changes: 2 additions & 2 deletions Actions/Deploy/Deploy.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -22,10 +22,11 @@ $bcContainerHelperPath = $null

if ($projects -eq '') {
Write-Host "No projects to deploy"
exit
}
else {

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
Expand Down Expand Up @@ -197,4 +198,3 @@ catch {
finally {
CleanupAfterBcContainerHelper -bcContainerHelperPath $bcContainerHelperPath
}
}
31 changes: 10 additions & 21 deletions Actions/DetermineArtifactUrl/DetermineArtifactUrl.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -2,16 +2,15 @@ Param(
[Parameter(HelpMessage = "Specifies the parent telemetry scope for the telemetry signal", Mandatory = $false)]
[string] $parentTelemetryScopeJson = '7b7d',
[Parameter(HelpMessage = "Project folder", Mandatory = $false)]
[string] $project = ".",
[Parameter(HelpMessage = "Secrets from repository in compressed Json format", Mandatory = $false)]
[string] $secretsJson = '{"insiderSasToken":""}'
[string] $project = "."
)

$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0
$telemetryScope = $null
$bcContainerHelperPath = $null

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
#region Action: Setup
Expand All @@ -22,40 +21,30 @@ try {

#region Action: Determine artifacts to use
$telemetryScope = CreateScope -eventId 'DO0084' -parentTelemetryScopeJson $parentTelemetryScopeJson
$secrets = $secretsJson | ConvertFrom-Json | ConvertTo-HashTable
if ($secrets.ContainsKey('insiderSasToken')) {
$insiderSasToken = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($secrets.insiderSasToken))
} else {
$insiderSasToken = ""
}
$projectSettings = $env:Settings | ConvertFrom-Json | ConvertTo-HashTable
$projectSettings = AnalyzeRepo -settings $projectSettings -project $project -doNotCheckArtifactSetting -doNotIssueWarnings
$artifactUrl = Determine-ArtifactUrl -projectSettings $projectSettings -insiderSasToken $insiderSasToken
$secrets = $env:Secrets | ConvertFrom-Json | ConvertTo-HashTable
$insiderSasToken = [System.Text.Encoding]::UTF8.GetString([System.Convert]::FromBase64String($Secrets.insiderSasToken))
$settings = $env:Settings | ConvertFrom-Json | ConvertTo-HashTable
$settings = AnalyzeRepo -settings $settings -project $project -doNotCheckArtifactSetting -doNotCheckAppDependencyProbingPaths -doNotIssueWarnings
$artifactUrl = Determine-ArtifactUrl -projectSettings $settings -insiderSasToken $insiderSasToken
$artifactCacheKey = ''
$projectSettings.artifact = $artifactUrl
if ($projectSettings.useCompilerFolder) {
if ($settings.useCompilerFolder) {
$artifactCacheKey = $artifactUrl.Split('?')[0]
}
#endregion

#region Action: Output
# Set output variables
Write-Host "SETTINGS:"
$projectSettings | ConvertTo-Json -Depth 99 | Out-Host
Add-Content -Encoding UTF8 -Path $env:GITHUB_ENV -Value "Settings=$($projectSettings | ConvertTo-Json -Depth 99 -Compress)"

Add-Content -Encoding UTF8 -Path $env:GITHUB_ENV -Value "artifact=$artifactUrl"
Write-Host "Artifact=$artifactUrl"
Write-Host "artifact=$artifactUrl"
Add-Content -Encoding UTF8 -Path $env:GITHUB_ENV -Value "artifactCacheKey=$artifactCacheKey"
Write-Host "ArtifactCacheKey=$artifactCacheKey"
Write-Host "artifactCacheKey=$artifactCacheKey"
#endregion

TrackTrace -telemetryScope $telemetryScope
}
catch {
OutputError -message "DetermineArtifactUrl action failed.$([environment]::Newline)Error: $($_.Exception.Message)$([environment]::Newline)Stacktrace: $($_.scriptStackTrace)"
TrackException -telemetryScope $telemetryScope -errorRecord $_
exit
}
finally {
CleanupAfterBcContainerHelper -bcContainerHelperPath $bcContainerHelperPath
Expand Down
6 changes: 3 additions & 3 deletions Actions/DetermineArtifactUrl/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@ Determines the artifactUrl to use for a given project
| Name | Description |
| :-- | :-- |
| Settings | env.Settings must be set by a prior call to the ReadSettings Action |
| Secrets | env.Secrets with insiderSasToken must be read by a prior call to the ReadSecrets Action |

### Parameters
| Name | Required | Description | Default value |
| :-- | :-: | :-- | :-- |
| shell | | The shell (powershell or pwsh) in which the PowerShell script in this action should run | powershell |
| parentTelemetryScopeJson | | Specifies the parent telemetry scope for the telemetry signal | {} |
| project | | Project folder if repository is setup for multiple projects | . |
| secretsJson | | Secrets from repository in compressed Json format | |

## OUTPUT

### ENV variables
| Name | Description |
| :-- | :-- |
| ArtifactUrl | The ArtifactUrl to use |
| ArtifactCacheKey | The Cache Key to use for caching the artifacts when using CompilerFolder |
| artifact | The ArtifactUrl to use for the build |
| artifactCacheKey | The Cache Key to use for caching the artifacts when using CompilerFolder |
7 changes: 1 addition & 6 deletions Actions/DetermineArtifactUrl/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -13,10 +13,6 @@ inputs:
description: Project folder
required: false
default: '.'
secretsJson:
description: Secrets from repository in compressed Json format
required: false
default: '{"insiderSasToken":""}'
runs:
using: composite
steps:
Expand All @@ -25,8 +21,7 @@ runs:
env:
_parentTelemetryScopeJson: ${{ inputs.parentTelemetryScopeJson }}
_project: ${{ inputs.project }}
_secretsJson: ${{ inputs.secretsJson }}
run: try { ${{ github.action_path }}/DetermineArtifactUrl.ps1 -parentTelemetryScopeJson $env:_parentTelemetryScopeJson -project $ENV:_project -secretsJson $ENV:_secretsJson } catch { Write-Host "::Error::Unexpected error when running action ($($_.Exception.Message.Replace("`r",'').Replace("`n",' ')))"; exit 1 }
run: try { ${{ github.action_path }}/DetermineArtifactUrl.ps1 -parentTelemetryScopeJson $env:_parentTelemetryScopeJson -project $ENV:_project } catch { Write-Host "::Error::Unexpected error when running action ($($_.Exception.Message.Replace("`r",'').Replace("`n",' ')))"; exit 1 }
branding:
icon: terminal
color: blue
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,7 @@ $telemetryScope = $null
$bcContainerHelperPath = $null

# IMPORTANT: No code that can fail should be outside the try/catch
# IMPORTANT: All actions needs a try/catch here and not only in the yaml file, else they can silently fail

try {
#region Action: Setup
Expand All @@ -23,11 +24,11 @@ try {
Import-Module (Join-Path -Path $PSScriptRoot -ChildPath "..\TelemetryHelper.psm1" -Resolve) -DisableNameChecking
#endregion

$telemetryScope = CreateScope -eventId 'DO0085' -parentTelemetryScopeJson $parentTelemetryScopeJson

#region Action: Determine projects to build
. (Join-Path -Path $PSScriptRoot -ChildPath "DetermineProjectsToBuild.ps1" -Resolve)
$allProjects, $projectsToBuild, $projectDependencies, $buildOrder = Get-ProjectsToBuild -baseFolder $baseFolder -modifiedFiles $modifiedFiles -maxBuildDepth $maxBuildDepth

$telemetryScope = CreateScope -eventId 'DO0085' -parentTelemetryScopeJson $parentTelemetryScopeJson
AddTelemetryProperty -telemetryScope $telemetryScope -key "projects" -value "$($allProjects -join ', ')"
#endregion

Expand All @@ -51,7 +52,6 @@ try {
catch {
OutputError -message "DetermineProjectsToBuild action failed.$([environment]::Newline)Error: $($_.Exception.Message)$([environment]::Newline)Stacktrace: $($_.scriptStackTrace)"
TrackException -telemetryScope $telemetryScope -errorRecord $_
exit
}
finally {
CleanupAfterBcContainerHelper -bcContainerHelperPath $bcContainerHelperPath
Expand Down
Loading