Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update AL-Go Telemetry Module #1049

Merged
merged 43 commits into from
Aug 6, 2024
Merged
Show file tree
Hide file tree
Changes from 11 commits
Commits
Show all changes
43 commits
Select commit Hold shift + click to select a range
ea27c60
Telemetry Module (#31)
aholstrup1 Apr 23, 2024
394912b
Telemetry updates
aholstrup1 Apr 24, 2024
2bd9877
Rename to Add-TelemetryProperty
aholstrup1 Apr 24, 2024
a721e41
Split on multiple characters
aholstrup1 Apr 25, 2024
813718f
Merge branch 'main' into aholstrup/telemetrymodule
aholstrup1 May 1, 2024
e32a67e
Casing
aholstrup1 May 2, 2024
9f76e3a
Invoke-Action wrapper (#32)
aholstrup1 May 3, 2024
902f8aa
Merge branch 'main' of https://github.com/microsoft/al-go into aholst…
aholstrup1 May 3, 2024
c9ba11c
ALGO Branch updates (#33)
aholstrup1 May 3, 2024
61d1d68
paginate
aholstrup1 May 3, 2024
d5624f8
Script analyzer
aholstrup1 May 3, 2024
aca333a
Casing
aholstrup1 May 13, 2024
f829668
Merge conflicts
aholstrup1 May 13, 2024
90b9e86
Merge branch 'main' of https://github.com/microsoft/al-go into aholst…
aholstrup1 May 16, 2024
9bc1993
Uptake in PP
aholstrup1 May 16, 2024
7bc17db
Add env variables
aholstrup1 May 16, 2024
1d31326
Update telemetry module
aholstrup1 May 16, 2024
d6a72c4
Merge branch 'main' of https://github.com/microsoft/al-go into aholst…
aholstrup1 Jun 27, 2024
499597a
Precommit
aholstrup1 Jun 27, 2024
44415e5
Fix test failures
aholstrup1 Jun 28, 2024
12fd3c0
Update microsoftTelemetryConnectionString
aholstrup1 Jun 28, 2024
b6740a9
Get-PackageVersion
aholstrup1 Jun 28, 2024
f85beb6
Log github runner info
aholstrup1 Jul 2, 2024
3cfa4b9
Merge branch 'main' of https://github.com/microsoft/al-go into aholst…
aholstrup1 Jul 12, 2024
4e86d52
Require actioname to be passed to wrapper
aholstrup1 Jul 12, 2024
4ea3187
Log different events for workflows depending on conclusion
aholstrup1 Jul 12, 2024
dc8caca
Improved docs
aholstrup1 Jul 12, 2024
706a3ac
Merge changes from main
aholstrup1 Jul 30, 2024
8bc33f4
Fix YamlTest
aholstrup1 Jul 30, 2024
28cef06
Add action read permission
aholstrup1 Jul 30, 2024
e9b012d
Add AdditionalData
aholstrup1 Jul 30, 2024
4a0560e
Give action: read permission to workflow
aholstrup1 Jul 31, 2024
d50adb0
Order permissions
aholstrup1 Jul 31, 2024
ee65cbd
Catch error so we dont fail workflow if telemetry fails
aholstrup1 Jul 31, 2024
32d3bf1
Docs updates
aholstrup1 Aug 1, 2024
4dcca5c
Handle timedout and cancelled as conclusions
aholstrup1 Aug 1, 2024
04f0492
Change names of actionnames for telemetry
aholstrup1 Aug 1, 2024
36dc7e1
Log more telemetry events
aholstrup1 Aug 5, 2024
3485bfc
Merge branch 'main' of https://github.com/microsoft/al-go into aholst…
aholstrup1 Aug 5, 2024
90d56e2
Remove timedout and cancelled events
aholstrup1 Aug 5, 2024
2348e94
Change bccontainerhelper version telemetry
aholstrup1 Aug 5, 2024
8acdbd4
Remove outer check
aholstrup1 Aug 5, 2024
f7a6fae
Merge branch 'main' into aholstrup/telemetrymodule
aholstrup1 Aug 5, 2024
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 2 additions & 3 deletions Actions/AL-Go-Helper.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -18,8 +18,7 @@ $defaultCICDPushBranches = @( 'main', 'release/*', 'feature/*' )
[Diagnostics.CodeAnalysis.SuppressMessageAttribute('PSUseDeclaredVarsMoreThanAssignments', 'defaultCICDPullRequestBranches', Justification = 'False positive.')]
$defaultCICDPullRequestBranches = @( 'main' )
$runningLocal = $local.IsPresent
$defaultBcContainerHelperVersion = "preview" # Must be double quotes. Will be replaced by BcContainerHelperVersion if necessary in the deploy step - ex. "https://github.com/organization/navcontainerhelper/archive/refs/heads/branch.zip"
$microsoftTelemetryConnectionString = "InstrumentationKey=84bd9223-67d4-4378-8590-9e4a46023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/"
$defaultBcContainerHelperVersion = "preview" # Must be double quotes. Will be replaced by BcContainerHelperVersion if necessary in the deploy step
$notSecretProperties = @("Scopes","TenantId","BlobName","ContainerName","StorageAccountName","ServerUrl")

$runAlPipelineOverrides = @(
Expand Down Expand Up @@ -630,7 +629,7 @@ function ReadSettings {
"cacheImageName" = "my"
"cacheKeepDays" = 3
"alwaysBuildAllProjects" = $false
"microsoftTelemetryConnectionString" = $microsoftTelemetryConnectionString
"microsoftTelemetryConnectionString" = "InstrumentationKey=84bd9223-67d4-4378-8590-9e4a46023be2;IngestionEndpoint=https://westeurope-1.in.applicationinsights.azure.com/"
"partnerTelemetryConnectionString" = ""
"sendExtendedTelemetryToMicrosoft" = $false
"environments" = @()
Expand Down
222 changes: 103 additions & 119 deletions Actions/AddExistingApp/AddExistingApp.ps1
Original file line number Diff line number Diff line change
Expand Up @@ -3,8 +3,6 @@
[string] $actor,
[Parameter(HelpMessage = "The GitHub token running the action", Mandatory = $false)]
[string] $token,
[Parameter(HelpMessage = "Specifies the parent telemetry scope for the telemetry signal", Mandatory = $false)]
[string] $parentTelemetryScopeJson = '7b7d',
[Parameter(HelpMessage = "Project name if the repository is setup for multiple projects", Mandatory = $false)]
[string] $project = '.',
[Parameter(HelpMessage = "Direct Download Url of .app or .zip file", Mandatory = $true)]
Expand Down Expand Up @@ -78,149 +76,135 @@ function expandfile {
}
}

$telemetryScope = $null
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
$serverUrl, $branch = CloneIntoNewFolder -actor $actor -token $token -updateBranch $updateBranch -DirectCommit $directCommit -newBranchPrefix 'add-existing-app'
$baseFolder = (Get-Location).path
DownloadAndImportBcContainerHelper -baseFolder $baseFolder

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
$serverUrl, $branch = CloneIntoNewFolder -actor $actor -token $token -updateBranch $updateBranch -DirectCommit $directCommit -newBranchPrefix 'add-existing-app'
$baseFolder = (Get-Location).path
DownloadAndImportBcContainerHelper -baseFolder $baseFolder

import-module (Join-Path -path $PSScriptRoot -ChildPath "..\TelemetryHelper.psm1" -Resolve)
$telemetryScope = CreateScope -eventId 'DO0070' -parentTelemetryScopeJson $parentTelemetryScopeJson

$type = "PTE"
Write-Host "Reading $RepoSettingsFile"
$settingsJson = Get-Content $RepoSettingsFile -Encoding UTF8 | ConvertFrom-Json
if ($settingsJson.PSObject.Properties.Name -eq "type") {
$type = $settingsJson.type
}
$type = "PTE"
Write-Host "Reading $RepoSettingsFile"
$settingsJson = Get-Content $RepoSettingsFile -Encoding UTF8 | ConvertFrom-Json
if ($settingsJson.PSObject.Properties.Name -eq "type") {
$type = $settingsJson.type
}

CheckAndCreateProjectFolder -project $project
$projectFolder = (Get-Location).path
CheckAndCreateProjectFolder -project $project
$projectFolder = (Get-Location).path

$appNames = @()
getfiles -url $url | ForEach-Object {
$appFolder = $_
"?Content_Types?.xml", "MediaIdListing.xml", "navigation.xml", "NavxManifest.xml", "DocComments.xml", "SymbolReference.json" | ForEach-Object {
Remove-Item (Join-Path $appFolder $_) -Force -ErrorAction SilentlyContinue
}
$appJson = Get-Content (Join-Path $appFolder "app.json") -Encoding UTF8 | ConvertFrom-Json
$appNames += @($appJson.Name)
$appNames = @()
getfiles -url $url | ForEach-Object {
$appFolder = $_
"?Content_Types?.xml", "MediaIdListing.xml", "navigation.xml", "NavxManifest.xml", "DocComments.xml", "SymbolReference.json" | ForEach-Object {
Remove-Item (Join-Path $appFolder $_) -Force -ErrorAction SilentlyContinue
}
$appJson = Get-Content (Join-Path $appFolder "app.json") -Encoding UTF8 | ConvertFrom-Json
$appNames += @($appJson.Name)

$ranges = @()
if ($appJson.PSObject.Properties.Name -eq "idRanges") {
$ranges += $appJson.idRanges
}
if ($appJson.PSObject.Properties.Name -eq "idRange") {
$ranges += @($appJson.idRange)
}
$ranges = @()
if ($appJson.PSObject.Properties.Name -eq "idRanges") {
$ranges += $appJson.idRanges
}
if ($appJson.PSObject.Properties.Name -eq "idRange") {
$ranges += @($appJson.idRange)
}

# Determine whether the app is PTE or AppSource App based on one of the id ranges (the first)
if ($ranges[0].from -lt 100000 -and $ranges[0].to -lt 100000) {
$ttype = "PTE"
}
else {
$ttype = "AppSource App"
}
# Determine whether the app is PTE or AppSource App based on one of the id ranges (the first)
if ($ranges[0].from -lt 100000 -and $ranges[0].to -lt 100000) {
$ttype = "PTE"
}
else {
$ttype = "AppSource App"
}

if ($appJson.PSObject.Properties.Name -eq "dependencies") {
foreach($dependency in $appJson.dependencies) {
if ($dependency.PSObject.Properties.Name -eq "AppId") {
$id = $dependency.AppId
}
else {
$id = $dependency.Id
}
if ($testRunnerApps.Contains($id)) {
$ttype = "Test App"
}
if ($appJson.PSObject.Properties.Name -eq "dependencies") {
foreach($dependency in $appJson.dependencies) {
if ($dependency.PSObject.Properties.Name -eq "AppId") {
$id = $dependency.AppId
}
else {
$id = $dependency.Id
}
if ($testRunnerApps.Contains($id)) {
$ttype = "Test App"
}
}
}

if ($ttype -ne "Test App") {
foreach($appName in (Get-ChildItem -Path $appFolder -Filter "*.al" -Recurse).FullName) {
$alContent = (Get-Content -Path $appName -Encoding UTF8) -join "`n"
if ($alContent -like "*codeunit*subtype*=*test*[test]*") {
$ttype = "Test App"
}
if ($ttype -ne "Test App") {
foreach($appName in (Get-ChildItem -Path $appFolder -Filter "*.al" -Recurse).FullName) {
$alContent = (Get-Content -Path $appName -Encoding UTF8) -join "`n"
if ($alContent -like "*codeunit*subtype*=*test*[test]*") {
$ttype = "Test App"
}
}
}

if ($ttype -ne "Test App" -and $ttype -ne $type) {
OutputWarning -message "According to settings, repository is for apps of type $type. The app you are adding seams to be of type $ttype"
}
if ($ttype -ne "Test App" -and $ttype -ne $type) {
OutputWarning -message "According to settings, repository is for apps of type $type. The app you are adding seams to be of type $ttype"
}

$appFolders = Get-ChildItem -Path $appFolder | Where-Object { $_.PSIsContainer -and (Test-Path (Join-Path $_.FullName 'app.json')) }
if (-not $appFolders) {
$appFolders = @($appFolder)
# TODO: What to do about the über app.json - another workspace? another setting?
}
$appFolders = Get-ChildItem -Path $appFolder | Where-Object { $_.PSIsContainer -and (Test-Path (Join-Path $_.FullName 'app.json')) }
if (-not $appFolders) {
$appFolders = @($appFolder)
# TODO: What to do about the über app.json - another workspace? another setting?
}

$orgfolderName = $appJson.name.Split([System.IO.Path]::getInvalidFileNameChars()) -join ""
$folderName = GetUniqueFolderName -baseFolder $projectFolder -folderName $orgfolderName
if ($folderName -ne $orgfolderName) {
OutputWarning -message "$orgFolderName already exists as a folder in the repo, using $folderName instead"
}
$orgfolderName = $appJson.name.Split([System.IO.Path]::getInvalidFileNameChars()) -join ""
$folderName = GetUniqueFolderName -baseFolder $projectFolder -folderName $orgfolderName
if ($folderName -ne $orgfolderName) {
OutputWarning -message "$orgFolderName already exists as a folder in the repo, using $folderName instead"
}

Move-Item -Path $appFolder -Destination $projectFolder -Force
Rename-Item -Path ([System.IO.Path]::GetFileName($appFolder)) -NewName $folderName
$appFolder = Join-Path $projectFolder $folderName
Move-Item -Path $appFolder -Destination $projectFolder -Force
Rename-Item -Path ([System.IO.Path]::GetFileName($appFolder)) -NewName $folderName
$appFolder = Join-Path $projectFolder $folderName

Get-ChildItem $appFolder -Filter '*.*' -Recurse | ForEach-Object {
if ($_.Name.Contains('%20')) {
Rename-Item -Path $_.FullName -NewName $_.Name.Replace('%20', ' ')
}
Get-ChildItem $appFolder -Filter '*.*' -Recurse | ForEach-Object {
if ($_.Name.Contains('%20')) {
Rename-Item -Path $_.FullName -NewName $_.Name.Replace('%20', ' ')
}
}

$appFolders | ForEach-Object {
# Modify .AL-Go\settings.json
try {
$settingsJsonFile = Join-Path $projectFolder $ALGoSettingsFile
$SettingsJson = Get-Content $settingsJsonFile -Encoding UTF8 | ConvertFrom-Json
if (@($settingsJson.appFolders) + @($settingsJson.testFolders)) {
if ($ttype -eq "Test App") {
if ($SettingsJson.testFolders -notcontains $foldername) {
$SettingsJson.testFolders += @($folderName)
}
$appFolders | ForEach-Object {
# Modify .AL-Go\settings.json
try {
$settingsJsonFile = Join-Path $projectFolder $ALGoSettingsFile
$SettingsJson = Get-Content $settingsJsonFile -Encoding UTF8 | ConvertFrom-Json
if (@($settingsJson.appFolders) + @($settingsJson.testFolders)) {
if ($ttype -eq "Test App") {
if ($SettingsJson.testFolders -notcontains $foldername) {
$SettingsJson.testFolders += @($folderName)
}
else {
if ($SettingsJson.appFolders -notcontains $foldername) {
$SettingsJson.appFolders += @($folderName)
}
}
else {
if ($SettingsJson.appFolders -notcontains $foldername) {
$SettingsJson.appFolders += @($folderName)
}
$SettingsJson | Set-JsonContentLF -Path $settingsJsonFile
}
$SettingsJson | Set-JsonContentLF -Path $settingsJsonFile
}
catch {
throw "$ALGoSettingsFile is malformed. Error: $($_.Exception.Message)"
}
}
catch {
throw "$ALGoSettingsFile is malformed. Error: $($_.Exception.Message)"
}

# Modify workspace
Get-ChildItem -Path $projectFolder -Filter "*.code-workspace" | ForEach-Object {
try {
$workspaceFileName = $_.Name
$workspaceFile = $_.FullName
$workspace = Get-Content $workspaceFile -Encoding UTF8 | ConvertFrom-Json
if (-not ($workspace.folders | Where-Object { $_.Path -eq $foldername })) {
$workspace.folders += @(@{ "path" = $foldername })
}
$workspace | Set-JsonContentLF -Path $workspaceFile
}
catch {
throw "$workspaceFileName is malformed.$([environment]::Newline) $($_.Exception.Message)"
# Modify workspace
Get-ChildItem -Path $projectFolder -Filter "*.code-workspace" | ForEach-Object {
try {
$workspaceFileName = $_.Name
$workspaceFile = $_.FullName
$workspace = Get-Content $workspaceFile -Encoding UTF8 | ConvertFrom-Json
if (-not ($workspace.folders | Where-Object { $_.Path -eq $foldername })) {
$workspace.folders += @(@{ "path" = $foldername })
}
$workspace | Set-JsonContentLF -Path $workspaceFile
}
catch {
throw "$workspaceFileName is malformed.$([environment]::Newline) $($_.Exception.Message)"
}
}
}
Set-Location $baseFolder
CommitFromNewFolder -serverUrl $serverUrl -commitMessage "Add existing apps ($($appNames -join ', '))" -branch $branch | Out-Null

TrackTrace -telemetryScope $telemetryScope
}
catch {
if (Get-Module BcContainerHelper) {
TrackException -telemetryScope $telemetryScope -errorRecord $_
}
throw
}
Set-Location $baseFolder
CommitFromNewFolder -serverUrl $serverUrl -commitMessage "Add existing apps ($($appNames -join ', '))" -branch $branch | Out-Null
1 change: 0 additions & 1 deletion Actions/AddExistingApp/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,7 +12,6 @@ none
| shell | | The shell (powershell or pwsh) in which the PowerShell script in this action should run | powershell |
| actor | | The GitHub actor running the action | github.actor |
| token | | The GitHub token running the action | github.token |
| parentTelemetryScopeJson | | Specifies the parent telemetry scope for the telemetry signal | {} |
| project | | Project name if the repository is setup for multiple projects | . |
| url | Yes | Direct Download Url of .app or .zip file to add to the repository | |
| updateBranch | | Which branch should the app be added to | github.ref_name |
Expand Down
14 changes: 2 additions & 12 deletions Actions/AddExistingApp/action.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,10 +16,6 @@ inputs:
description: The GitHub token running the action
required: false
default: ${{ github.token }}
parentTelemetryScopeJson:
description: Specifies the parent telemetry scope for the telemetry signal
required: false
default: '7b7d'
project:
description: Project name if the repository is setup for multiple projects
required: false
Expand All @@ -43,19 +39,13 @@ runs:
env:
_actor: ${{ inputs.actor }}
_token: ${{ inputs.token }}
_parentTelemetryScopeJson: ${{ inputs.parentTelemetryScopeJson }}
_project: ${{ inputs.project }}
_url: ${{ inputs.url }}
_updateBranch: ${{ inputs.updateBranch }}
_directCommit: ${{ inputs.directCommit }}
run: |
$errorActionPreference = "Stop"; $ProgressPreference = "SilentlyContinue"; Set-StrictMode -Version 2.0
try {
${{ github.action_path }}/AddExistingApp.ps1 -actor $ENV:_actor -token $ENV:_token -parentTelemetryScopeJson $ENV:_parentTelemetryScopeJson -project $ENV:_project -url $ENV:_url -updateBranch $ENV:_updateBranch -directCommit ($ENV:_directCommit -eq 'true')
}
catch {
Write-Host "::ERROR::Unexpected error when running action. Error Message: $($_.Exception.Message.Replace("`r",'').Replace("`n",' ')), StackTrace: $($_.ScriptStackTrace.Replace("`r",'').Replace("`n",' <- '))";
exit 1
${{ github.action_path }}/../Invoke-AlGoAction.ps1 -Action {
${{ github.action_path }}/AddExistingApp.ps1 -actor $ENV:_actor -token $ENV:_token -project $ENV:_project -url $ENV:_url -updateBranch $ENV:_updateBranch -directCommit ($ENV:_directCommit -eq 'true')
}
branding:
icon: terminal
Expand Down
56 changes: 18 additions & 38 deletions Actions/AnalyzeTests/AnalyzeTests.ps1
Original file line number Diff line number Diff line change
@@ -1,49 +1,29 @@
Param(
[Parameter(HelpMessage = "Specifies the parent telemetry scope for the telemetry signal", Mandatory = $false)]
[string] $parentTelemetryScopeJson = '7b7d',
[Parameter(HelpMessage = "Project to analyze", Mandatory = $false)]
[string] $project
)

$telemetryScope = $null
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
. (Join-Path -Path $PSScriptRoot 'TestResultAnalyzer.ps1')

try {
. (Join-Path -Path $PSScriptRoot -ChildPath "..\AL-Go-Helper.ps1" -Resolve)
DownloadAndImportBcContainerHelper
$testResultsFile = Join-Path $ENV:GITHUB_WORKSPACE "$project\TestResults.xml"
if (Test-Path $testResultsFile) {
$testResults = [xml](Get-Content "$project\TestResults.xml")
$testResultSummary = GetTestResultSummary -testResults $testResults -includeFailures 50

import-module (Join-Path -path $PSScriptRoot -ChildPath "..\TelemetryHelper.psm1" -Resolve)
$telemetryScope = CreateScope -eventId 'DO0082' -parentTelemetryScopeJson $parentTelemetryScopeJson
Add-Content -Encoding UTF8 -Path $env:GITHUB_OUTPUT -Value "TestResultMD=$testResultSummary"
Write-Host "TestResultMD=$testResultSummary"

. (Join-Path -Path $PSScriptRoot 'TestResultAnalyzer.ps1')

$testResultsFile = Join-Path $ENV:GITHUB_WORKSPACE "$project\TestResults.xml"
if (Test-Path $testResultsFile) {
$testResults = [xml](Get-Content "$project\TestResults.xml")
$testResultSummary = GetTestResultSummary -testResults $testResults -includeFailures 50

Add-Content -Encoding UTF8 -Path $env:GITHUB_OUTPUT -Value "TestResultMD=$testResultSummary"
Write-Host "TestResultMD=$testResultSummary"

Add-Content -path $ENV:GITHUB_STEP_SUMMARY -value "$($testResultSummary.Replace("\n","`n"))`n"
}
else {
Write-Host "Test results not found"
}

$bcptTestResultsFile = Join-Path $ENV:GITHUB_WORKSPACE "$project\BCPTTestResults.json"
if (Test-Path $bcptTestResultsFile) {
# TODO Display BCPT Test Results
}
else {
#Add-Content -path $ENV:GITHUB_STEP_SUMMARY -value "*BCPT test results not found*`n`n"
}

TrackTrace -telemetryScope $telemetryScope
Add-Content -path $ENV:GITHUB_STEP_SUMMARY -value "$($testResultSummary.Replace("\n","`n"))`n"
}
else {
Write-Host "Test results not found"
}
catch {
if (Get-Module BcContainerHelper) {
TrackException -telemetryScope $telemetryScope -errorRecord $_
}

throw
$bcptTestResultsFile = Join-Path $ENV:GITHUB_WORKSPACE "$project\BCPTTestResults.json"
if (Test-Path $bcptTestResultsFile) {
# TODO Display BCPT Test Results
}
else {
#Add-Content -path $ENV:GITHUB_STEP_SUMMARY -value "*BCPT test results not found*`n`n"
}
1 change: 0 additions & 1 deletion Actions/AnalyzeTests/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,6 @@ none
| Name | Required | Description | Default value |
| :-- | :-: | :-- | :-- |
| shell | | The shell (powershell or pwsh) in which the PowerShell script in this action should run | powershell |
| parentTelemetryScopeJson | | Specifies the parent telemetry scope for the telemetry signal | {} |
| project | Yes | Name of project to analyze or . if the repository is setup for single project | |

## OUTPUT
Expand Down
Loading
Loading