Snap Start 6/23/23 (#430)

* Create build_pullrequest.yml (#415) * Update build_pullrequest.yml * Update build_pullrequest.yml (#416) * Update build_pullrequest.yml * Rename config.json.txt to config.json * Update build_pullrequest.yml to run ps1 script tests only for Windows. * Update build_pullrequest.yml * Update build_pullrequest.yml * Create yml file for release build and updates, including unit tests for Windows (#417) * add non windows tests * include release * add display name for bash * fix extra ' * build fix * fix windows tests and take out linux * powershell 2 * add back in conditions * fix password * fix test path * spacing and test file * remove style check * fix build * Update table routes (#414) * Function order * format * updating names * update routes * spacing and order * fix system folders * spacing * remove unnecessary system folders * release trigger (#418) * Microsoft mandatory file (#419) Co-authored-by: microsoft-github-policy-service[bot] <77245923+microsoft-github-policy-service[bot]@users.noreply.github.com> Co-authored-by: nasc17 <[email protected]> * Update build pipelines for including Linux. (#420) * include linux * include style check * style fixes and build fix * styling * styling * update build file names * Fix display of partitioned tables (#422) * Include partitioned tables in catalog queries * Exclude partitions themselves from table list view * Have Insiders build publish to Pre-Release on Github (#425) * publish to github * syntax * Psycopg version bump 2 to 3 (#428) * Fix broken language service * Fixing tests * remove qt literals * Fix test codes * Fix psycopg2 code * autopep * test debugger * Fix psycopg2 references * Add support for showing stored procedures (#423) * init routing updates * procedure templates * spacing * script fixes * remove qtLiteral condition from sql files * fix quotation issue --------- Co-authored-by: Daeun Yim <[email protected]> * Use service connection for publishing insiders build (#426) * publish to github * syntax * user service connection * add userSpecifiedTag * new service connection * more permissions * add jobs * spacing * fix jobs * task name fix * update condition * lint change * spacing for powershell * powershell inlinescript * Update powershell again * continue on error for deleting pre-release * Remove deleting builds and add build number * use workspace * download build artififacts * use System.ArtifactsDirectory for release * include vmImage * download in second job * fix spacing * github for release build * changelogtype * Use insiders tag, delete old pre-release * add code sign to PR build and lint fixes * Dynamically set versioning for apple silicon artifacts (#429) * Fix broken language service * use variable for vsts artifact versioning * use variable for vsts artifact versioning * use package name as a variable too * Update common.py * trigger ci * fix (#433) --------- Co-authored-by: Pradeep Kintali <[email protected]> Co-authored-by: microsoft-github-policy-service[bot] <77245923+microsoft-github-policy-service[bot]@users.noreply.github.com> Co-authored-by: Joe Nelson <[email protected]> Co-authored-by: Daeun Yim <[email protected]> Co-authored-by: Daeun Yim <[email protected]>
microsoft · Jun 29, 2023 · e630fd5 · e630fd5
1 parent afcefe9
commit e630fd5
Show file tree

Hide file tree

Showing 309 changed files with 2,431 additions and 1,169 deletions.
diff --git a/.pipeline/build-CI.yml b/.pipeline/build-CI.yml
@@ -1,46 +1,188 @@
 trigger: none
 pr: none
-strategy:
-  matrix:
-    mac-x64:
-      platform: 'mac-x64'
-      imageName: 'macOS-latest'
-      buildScript: 'scripts/build-mac.sh'
-      archiveFileName: 'pgsqltoolsservice-osx.tar.gz'
-      archiveType: 'tar'
-    mac-arm64:
-      platform: 'mac-arm64'
-      imageName: 'macOS-latest'
-      archiveFileName: 'pgsqltoolsservice-osx-arm64.tar.gz'
-      archiveType: 'tar'
-    windows:
-      platform: 'windows'
-      imageName: 'windows-latest'
-      buildScript: 'scripts/build.sh'
-      archiveFileName: 'pgsqltoolsservice-win-x64.zip'
-      archiveType: 'zip'
-pool:
-  vmImage: $(imageName)
-steps:
-- template: templates/build.yml
-- template: templates/code-sign.yml
-- task: DeleteFiles@1
-  inputs:
-    SourceFolder: '$(Build.SourcesDirectory)/dist/pgsqltoolsservice'
-    Contents: 'CodeSignSummary*.md'
-- task: ArchiveFiles@2
-  inputs:
-    rootFolderOrFile: '$(Build.SourcesDirectory)/dist/pgsqltoolsservice'
-    includeRootFolder: true
-    archiveType: '$(archiveType)'
-    archiveFile: '$(Build.SourcesDirectory)/dist/$(archiveFileName)'
-    replaceExistingArchive: true
-- task: CopyFiles@2
-  inputs:
-    sourceFolder: '$(Build.SourcesDirectory)'
-    contents: 'dist/$(archiveFileName)'
-    targetFolder: '$(Build.ArtifactStagingDirectory)'
-- task: PublishBuildArtifacts@1
-  inputs:
-    pathToPublish: '$(Build.ArtifactStagingDirectory)'
-    artifactName: dist
+jobs:
+  - job: platform_matrix
+    displayName: Build and Publish Artifacts
+    strategy:
+      matrix:
+        linux:
+          platform: ubuntu
+          imageName: ubuntu-latest
+          buildScript: scripts/build.sh
+          archiveFileName: pgsqltoolsservice-linux-x64-insiders.tar.gz
+          archiveType: tar
+        mac-x64:
+          platform: mac-x64
+          imageName: macOS-latest
+          buildScript: scripts/build.sh
+          archiveFileName: pgsqltoolsservice-osx-insiders.tar.gz
+          archiveType: tar
+        mac-arm64:
+          platform: mac-arm64
+          imageName: macOS-latest
+          archiveFileName: pgsqltoolsservice-osx-arm64-insiders.tar.gz
+          archiveType: tar
+        windows:
+          platform: windows
+          imageName: windows-latest
+          buildScript: scripts/build-ps1.sh
+          archiveFileName: pgsqltoolsservice-win-x64-insiders.zip
+          archiveType: zip
+    pool:
+      vmImage: $(imageName)
+    steps:
+      - template: templates/build.yml
+      - template: templates/code-sign.yml
+      - task: PowerShell@1
+        displayName: Download Postgres windows
+        condition: eq(variables.platform, 'windows')
+        inputs:
+          scriptType: inlineScript
+          inlineScript: >
+            cd $env:BUILD_BINARIESDIRECTORY
+
+            (New-Object System.Net.WebClient).DownloadFile('https://get.enterprisedb.com/postgresql/postgresql-14.1-1-windows-x64-binaries.zip', $env:BUILD_BINARIESDIRECTORY + '\postgres.zip')
+      - task: ExtractFiles@1
+        displayName: Extract Postgres windows
+        condition: eq(variables.platform, 'windows')
+        inputs:
+          archiveFilePatterns: $(Build.BinariesDirectory)/*.zip
+          destinationFolder: $(Build.BinariesDirectory)/postgres
+      - powershell: >
+          Remove-Item -path
+          "$env:BUILD_BINARIESDIRECTORY\postgres\pgsql\pgAdmin 4\*" -Recurse
+          -Force -EA SilentlyContinue
+
+
+          cd "$env:BUILD_BINARIESDIRECTORY\postgres\pgsql\bin"
+
+
+          $config = @{
+                  host = "localhost"
+                  port = 5432
+                  user = "postgres"
+                  password = "$(PostgresPassword)"
+                  dbname = "postgres"
+              }
+
+          New-Item -ItemType File -Path "$env:BUILD_SOURCESDIRECTORY\tests\integration\config.json" -Force | Out-Null
+
+
+          $config | ConvertTo-Json -Depth 4 | Set-Content -Path "$env:BUILD_SOURCESDIRECTORY\tests\integration\config.json" -Encoding utf8
+
+
+          Get-Content "$env:BUILD_SOURCESDIRECTORY\tests\integration\config.json" -Raw
+
+
+          $pw = ConvertTo-SecureString "$(PostgresPassword)" -AsPlainText -Force
+
+          New-LocalUser -Name "postgres" -Password $pw
+
+          icacls "$env:BUILD_BINARIESDIRECTORY" /grant:r "users:(F)" /T /C
+
+          $cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList "postgres",$pw
+
+          Start-Process ".\initdb.exe" -ArgumentList "-U postgres -D ..\db1" -NoNewWindow -Wait -Credential $cred
+
+
+          mkdir ..\db1\pg_log
+
+          Start-Process ".\pg_ctl.exe"  -Credential $cred -NoNewWindow -ArgumentList "-D ..\db1 -l ..\db1\pg_log\postgres.log start"
+        displayName: Start Postgres windows
+        condition: eq(variables.platform, 'windows')
+      - task: PowerShell@2
+        displayName: Style Check windows
+        condition: eq(variables.platform, 'windows')
+        inputs:
+          filePath: scripts/flake8.ps1
+          script: |-
+            Get-ChildItem -Path "$env:BUILD_BINARIESDIRECTORY\postgres\pgsql"
+
+            & "$env:BUILD_SOURCESDIRECTORY\scripts\flake8.ps1"
+          warningPreference: stop
+          failOnStderr: true
+      - task: PowerShell@2
+        displayName: Run Tests windows
+        condition: eq(variables.platform, 'windows')
+        inputs:
+          targetType: filePath
+          filePath: ./scripts/test-all.ps1
+          errorActionPreference: default
+        continueOnError: true
+      - task: PublishTestResults@2
+        displayName: Publish test results windows **/nose2-junit.xml
+        condition: eq(variables.platform, 'windows')
+        inputs:
+          testResultsFormat: JUnit
+          testResultsFiles: "**/nose2-junit.xml"
+      - task: PublishCodeCoverageResults@1
+        displayName: Publish code coverage from **/htmlcov windows
+        condition: eq(variables.platform, 'windows')
+        inputs:
+          codeCoverageTool: Cobertura
+          summaryFileLocation: "**/cchtml"
+          reportDirectory: $(Build.SourcesDirectory)/htmlcov
+      - task: DeleteFiles@1
+        inputs:
+          SourceFolder: $(Build.SourcesDirectory)/dist/pgsqltoolsservice
+          Contents: CodeSignSummary*.md
+      - task: ArchiveFiles@2
+        displayName: Archive files
+        inputs:
+          rootFolderOrFile: $(Build.SourcesDirectory)/dist/pgsqltoolsservice
+          includeRootFolder: true
+          archiveType: $(archiveType)
+          archiveFile: $(Build.SourcesDirectory)/dist/$(archiveFileName)
+          replaceExistingArchive: true
+      - task: ArchiveFiles@1
+        displayName: Archive files - coverage
+        inputs:
+          rootFolder: $(Build.SourcesDirectory)/cover
+          includeRootFolder: false
+          archiveType: tar
+          archiveFile: $(Build.ArtifactStagingDirectory)/coverage.tar.gz
+        enabled: false
+      - task: CopyFiles@2
+        displayName: "Copy Files to: $(Build.ArtifactStagingDirectory)"
+        inputs:
+          sourceFolder: $(Build.SourcesDirectory)
+          contents: dist/$(archiveFileName)
+          targetFolder: $(Build.ArtifactStagingDirectory)
+      - task: PublishBuildArtifacts@1
+        inputs:
+          pathToPublish: $(Build.ArtifactStagingDirectory)
+          artifactName: dist
+  - job: github_release
+    displayName: Manage GitHub Release
+    dependsOn:
+      - platform_matrix
+    pool:
+      vmImage: windows-latest
+    steps:
+      - task: GithubRelease@1
+        displayName: 'Delete GitHub Release'
+        continueOnError: true
+        inputs:
+          gitHubConnection: azuredatastudio-postgresqlext2
+          repositoryName: 'microsoft/pgtoolsservice'
+          action: delete
+          tag: '$(ReleaseVersion)-insiders'
+      - task: DownloadBuildArtifacts@0
+        inputs:
+          buildType: current
+          downloadType: specific
+          artifactName: dist
+          downloadPath: $(System.ArtifactsDirectory)
+      - task: GithubRelease@1
+        displayName: Create GitHub Release
+        inputs:
+          gitHubConnection: azuredatastudio-postgresqlext2
+          repositoryName: microsoft/pgtoolsservice
+          tagSource: userSpecifiedTag
+          tag: $(ReleaseVersion)-insiders
+          title: $(ReleaseVersion)-insiders
+          changeLogType: issueBased
+          isPreRelease: true
+          assets: |
+            $(System.ArtifactsDirectory)/**/*.gz
+            $(System.ArtifactsDirectory)/**/*.zip
diff --git a/.pipeline/build_pullrequest.yml b/.pipeline/build_pullrequest.yml
@@ -0,0 +1,153 @@
+---
+trigger: none
+pr:
+  - master
+strategy:
+  matrix:
+    linux:
+      platform: ubuntu
+      imageName: ubuntu-latest
+      buildScript: scripts/build.sh
+      archiveFileName: pgsqltoolsservice-linux-x64.tar.gz
+      archiveType: tar
+    mac-x64:
+      platform: mac-x64
+      imageName: macOS-latest
+      buildScript: scripts/build.sh
+      archiveFileName: pgsqltoolsservice-osx.tar.gz
+      archiveType: tar
+    mac-arm64:
+      platform: mac-arm64
+      imageName: macOS-latest
+      archiveFileName: pgsqltoolsservice-osx-arm64.tar.gz
+      archiveType: tar
+    windows:
+      platform: windows
+      imageName: windows-latest
+      buildScript: scripts/build-ps1.sh
+      archiveFileName: pgsqltoolsservice-win-x64.zip
+      archiveType: zip
+pool:
+  vmImage: $(imageName)
+steps:
+  - template: templates/build.yml
+  - template: templates/code-sign.yml
+  - task: PowerShell@1
+    displayName: Download Postgres windows
+    condition: eq(variables.platform, 'windows')
+    inputs:
+      scriptType: inlineScript
+      inlineScript: >
+        cd $env:BUILD_BINARIESDIRECTORY
+
+        (New-Object System.Net.WebClient).DownloadFile('https://get.enterprisedb.com/postgresql/postgresql-14.1-1-windows-x64-binaries.zip', $env:BUILD_BINARIESDIRECTORY + '\postgres.zip')
+  - task: ExtractFiles@1
+    displayName: Extract Postgres windows
+    condition: eq(variables.platform, 'windows')
+    inputs:
+      archiveFilePatterns: $(Build.BinariesDirectory)/*.zip
+      destinationFolder: $(Build.BinariesDirectory)/postgres
+  - powershell: >
+      Remove-Item -path
+      "$env:BUILD_BINARIESDIRECTORY\postgres\pgsql\pgAdmin 4\*" -Recurse -Force
+      -EA SilentlyContinue
+
+
+      cd "$env:BUILD_BINARIESDIRECTORY\postgres\pgsql\bin"
+
+
+      $config = @{
+              host = "localhost"
+              port = 5432
+              user = "postgres"
+              password = "$(PostgresPassword)"
+              dbname = "postgres"
+          }
+
+      New-Item -ItemType File -Path "$env:BUILD_SOURCESDIRECTORY\tests\integration\config.json" -Force | Out-Null
+
+
+      $config | ConvertTo-Json -Depth 4 | Set-Content -Path "$env:BUILD_SOURCESDIRECTORY\tests\integration\config.json" -Encoding utf8
+
+
+      Get-Content "$env:BUILD_SOURCESDIRECTORY\tests\integration\config.json" -Raw
+
+
+      $pw = ConvertTo-SecureString "$(PostgresPassword)" -AsPlainText -Force
+
+      New-LocalUser -Name "postgres" -Password $pw
+
+      icacls "$env:BUILD_BINARIESDIRECTORY" /grant:r "users:(F)" /T /C
+
+      $cred = New-Object -TypeName System.Management.Automation.PSCredential -ArgumentList "postgres",$pw
+
+      Start-Process ".\initdb.exe" -ArgumentList "-U postgres -D ..\db1" -NoNewWindow -Wait -Credential $cred
+
+
+      mkdir ..\db1\pg_log
+
+      Start-Process ".\pg_ctl.exe"  -Credential $cred -NoNewWindow -ArgumentList "-D ..\db1 -l ..\db1\pg_log\postgres.log start"
+    displayName: Start Postgres windows
+    condition: eq(variables.platform, 'windows')
+  - task: PowerShell@2
+    displayName: Style Check windows
+    condition: eq(variables.platform, 'windows')
+    inputs:
+      filePath: scripts/flake8.ps1
+      script: |-
+        Get-ChildItem -Path "$env:BUILD_BINARIESDIRECTORY\postgres\pgsql"
+
+        & "$env:BUILD_SOURCESDIRECTORY\scripts\flake8.ps1"
+      warningPreference: stop
+      failOnStderr: true
+  - task: PowerShell@2
+    displayName: Run Tests windows
+    condition: eq(variables.platform, 'windows')
+    inputs:
+      targetType: filePath
+      filePath: ./scripts/test-all.ps1
+      errorActionPreference: default
+    continueOnError: true
+  - task: PublishTestResults@2
+    displayName: Publish test results windows **/nose2-junit.xml
+    condition: eq(variables.platform, 'windows')
+    inputs:
+      testResultsFormat: JUnit
+      testResultsFiles: "**/nose2-junit.xml"
+  - task: PublishCodeCoverageResults@1
+    displayName: Publish code coverage from **/htmlcov windows
+    condition: eq(variables.platform, 'windows')
+    inputs:
+      codeCoverageTool: Cobertura
+      summaryFileLocation: "**/cchtml"
+      reportDirectory: $(Build.SourcesDirectory)/htmlcov
+  - task: DeleteFiles@1
+    inputs:
+      SourceFolder: $(Build.SourcesDirectory)/dist/pgsqltoolsservice
+      Contents: CodeSignSummary*.md
+  - task: ArchiveFiles@2
+    displayName: Archive files
+    inputs:
+      rootFolderOrFile: $(Build.SourcesDirectory)/dist/pgsqltoolsservice
+      includeRootFolder: true
+      archiveType: $(archiveType)
+      archiveFile: $(Build.SourcesDirectory)/dist/$(archiveFileName)
+      replaceExistingArchive: true
+  - task: ArchiveFiles@1
+    displayName: Archive files - coverage
+    inputs:
+      rootFolder: $(Build.SourcesDirectory)/cover
+      includeRootFolder: false
+      archiveType: tar
+      archiveFile: $(Build.ArtifactStagingDirectory)/coverage.tar.gz
+    enabled: false
+  - task: CopyFiles@2
+    displayName: "Copy Files to: $(Build.ArtifactStagingDirectory)"
+    inputs:
+      sourceFolder: $(Build.SourcesDirectory)
+      contents: dist/$(archiveFileName)
+      targetFolder: $(Build.ArtifactStagingDirectory)
+  - task: PublishBuildArtifacts@1
+    inputs:
+      pathToPublish: $(Build.ArtifactStagingDirectory)
+      artifactName: dist