Support for multiple services, cluster sync no longer its own column (#…

…19) * Update to use storage, display pipelines correctly correlated together * Bump handlebars from 4.1.1 to 4.1.2 Bumps [handlebars](https://github.com/wycats/handlebars.js) from 4.1.1 to 4.1.2. - [Release notes](https://github.com/wycats/handlebars.js/releases) - [Changelog](https://github.com/wycats/handlebars.js/blob/master/release-notes.md) - [Commits](handlebars-lang/handlebars.js@v4.1.1...v4.1.2) Signed-off-by: dependabot[bot] <[email protected]> * Bump lodash from 4.17.11 to 4.17.14 (#7) Bumps [lodash](https://github.com/lodash/lodash) from 4.17.11 to 4.17.14. - [Release notes](https://github.com/lodash/lodash/releases) - [Commits](lodash/lodash@4.17.11...4.17.14) Signed-off-by: dependabot[bot] <[email protected]> * Fix accidental hardcoding of partition key * Add duration column, fix date types * Bring links back and improve duration * Introducing icons for statuses * Add progress state * Scripts to update the container journey db from pipelines (#5) * Adding function to update hld pipeline * Adding requirements.txt * Updating requirements.txt * Updating requirements.txt * Updating requirements.txt * Updating requirements.txt * Adding new script for manifest pipeline * Combining scripts into one * Handle pipelines started in between * Check for key correctly * Support for adding a new entry * Update task to entity * Minor updates to script * Attempting to reduce requirements * Fix Null bug in pipeline script * Allow p3 to be started in between * Updating README * Updating README * Adding cluster sync status * Adding support for getting author of a commit in src * Major refactor, using axios, adding dummy cli * Supporting a third key value pair in scripts * Added support for environment in prototype * Preliminary CLI commands * Refactoring to run threads in parallel and use promise.all to wait #efficiency * Adding logs command to cli * Adding very very basic tests * Disabling tslint no console rule (because CLI) * Starting CI pipeline with Azure Pipelines [skip ci] * Preliminary tests * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Loading config in tests * Updating config test * test cleanup * Improving config test * Improving deployment test * Adding test for author * Update README.md * Clean up * Unit test for author * Update azure-pipelines.yml * Update azure-pipelines.yml for Azure Pipelines * Update azure-pipelines.yml for Azure Pipelines * Hide branch column and improve build-cli script * Fix failing tests * Fix bug with CLI where config was not being initialized * Adding instructions on how to run the CLI * Adding output wide, json and normal feature where default is normal * Verify app configuration in tests * Automatically open URL on logs command * Output help on no command * Lowercase all the values in python scripts * Case insensitive querying * Preliminary support for private pipelines, private GitHub repos and AzureDevOps repos * Completing support for azure devops repos by pulling in cluster sync tag successfully * Adding tests for github private repos * Improve private github test * Improving prototype table * Optimizing columns * Using a better table format * Minor ui improvements * Add contribution instructions to README * Update from CJ to Spektate in README * Updating README.md * Adding support for service column (initial commit) * Added support for releaseIdFilter in azure pipelines for releases * Updating config * Prettify * Minor changes * Update config to use master * Updating README to add service from pipelines to storage
microsoft · Aug 29, 2019 · 1972d5c · 1972d5c
1 parent 7500934
commit 1972d5c
Show file tree

Hide file tree

Showing 5 changed files with 104 additions and 105 deletions.
diff --git a/README.md b/README.md
@@ -36,8 +36,8 @@ If you have already followed the steps [here](https://github.com/microsoft/bedro
         tag_name="$(PARTITION_KEY)-$(Build.SourceBranchName)-$(Build.BuildId)"
         commitId=$(Build.SourceVersion)
         commitId=$(echo "${commitId:0:7}")
-        echo "python update_pipeline.py $(ACCOUNT_NAME) $(ACCOUNT_KEY) $(TABLE_NAME) $(PARTITION_KEY) p1 $(Build.BuildId) imageTag $tag_name commitId $commitId"
-        python update_pipeline.py $(ACCOUNT_NAME) $(ACCOUNT_KEY) $(TABLE_NAME) $(PARTITION_KEY) p1 $(Build.BuildId) imageTag $tag_name commitId $commitId 
+        echo "python update_pipeline.py $(ACCOUNT_NAME) $(ACCOUNT_KEY) $(TABLE_NAME) $(PARTITION_KEY) p1 $(Build.BuildId) imageTag $tag_name commitId $commitId service $(Build.Repository.Name)"
+        python update_pipeline.py $(ACCOUNT_NAME) $(ACCOUNT_KEY) $(TABLE_NAME) $(PARTITION_KEY) p1 $(Build.BuildId) imageTag $tag_name commitId $commitId service $(Build.Repository.Name)
     displayName: Update source pipeline details in Spektate db
     ```
 

diff --git a/azure-pipelines.yml b/azure-pipelines.yml
@@ -35,4 +35,4 @@ steps:
 - script: |
     npm run test
   workingDirectory: "./"
-  displayName: "npm run test"
+  displayName: "npm run test"
diff --git a/src/Dashboard.tsx b/src/Dashboard.tsx
@@ -44,6 +44,7 @@ export interface IDashboardState {
 }
 export interface IDeploymentField {
   deploymentId: string;
+  service: string;
   startTime?: Date;
   imageTag?: string;
   srcCommitId?: string;
@@ -63,8 +64,8 @@ export interface IDeploymentField {
   hldPipelineResult?: string;
   duration: string;
   status: string;
-  clusterSync?: string;
-  clusterSyncDate?: string;
+  clusterSync?: boolean;
+  clusterSyncDate?: Date;
   endTime?: Date;
   authorName?: string;
   authorURL?: string;
@@ -94,26 +95,76 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
     );
   }
 
+  private getDeployments = () => {
+    const srcPipeline = new AzureDevOpsPipeline(
+      config.AZURE_ORG,
+      config.AZURE_PROJECT,
+      config.SRC_PIPELINE_ID,
+      false,
+      config.AZURE_PIPELINE_ACCESS_TOKEN
+    );
+    const hldPipeline = new AzureDevOpsPipeline(
+      config.AZURE_ORG,
+      config.AZURE_PROJECT,
+      config.DOCKER_PIPELINE_ID,
+      true,
+      config.AZURE_PIPELINE_ACCESS_TOKEN
+    );
+    const clusterPipeline = new AzureDevOpsPipeline(
+      config.AZURE_ORG,
+      config.AZURE_PROJECT,
+      config.HLD_PIPELINE_ID,
+      false,
+      config.AZURE_PIPELINE_ACCESS_TOKEN
+    );
+
+    const manifestRepo: Repository = new GitHub(
+      config.GITHUB_MANIFEST_USERNAME,
+      config.MANIFEST,
+      config.MANIFEST_ACCESS_TOKEN
+    );
+    // const manifestRepo: Repository = new AzureDevOpsRepo(config.AZURE_ORG, config.AZURE_PROJECT, config.MANIFEST, config.MANIFEST_ACCESS_TOKEN);
+    manifestRepo.getManifestSyncState(syncCommit => {
+      this.setState({ manifestSync: syncCommit });
+    });
+    Deployment.getDeployments(
+      config.STORAGE_PARTITION_KEY,
+      srcPipeline,
+      hldPipeline,
+      clusterPipeline,
+      (deployments: Deployment[]) => {
+        this.setState({ deployments });
+        this.getAuthors();
+      }
+    );
+    return <div />;
+  };
+
   private renderPrototypeTable = () => {
     const columns: Array<ITableColumn<IDeploymentField>> = [
+      {
+        id: "status",
+        name: "State",
+        renderCell: this.renderDeploymentStatus,
+        width: new ObservableValue(70)
+      },
       {
         id: "deploymentId",
         name: "Deployment ID",
         renderCell: this.renderDeploymentId,
         width: new ObservableValue(140)
       },
       {
-        id: "status",
-        name: "State",
-        renderCell: this.renderDeploymentStatus,
-        width: new ObservableValue(70)
+        id: "service",
+        name: "Service",
+        renderCell: this.renderSimpleText,
+        width: new ObservableValue(220),
       },
-      // { id: 'imageTag', name: 'Image Tag', width: new ObservableValue(220), renderCell: this.renderSimpleText},
       {
         id: "srcBranchName",
         name: "Branch",
         renderCell: this.renderSimpleText,
-        width: new ObservableValue(180)
+        width: new ObservableValue(180),
       },
       {
         id: "environment",
@@ -131,7 +182,7 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
         id: "dockerPipelineId",
         name: "ACR to HLD",
         renderCell: this.renderDockerRelease,
-        width: new ObservableValue(250)
+        width: new ObservableValue(250),
       },
       {
         id: "hldPipelineId",
@@ -145,12 +196,6 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
         renderCell: this.renderSimpleBoldText,
         width: new ObservableValue(200)
       },
-      {
-        id: "clusterSync",
-        name: "Cluster-Sync",
-        renderCell: this.renderClusterSync,
-        width: new ObservableValue(120)
-      },
       {
         id: "deployedAt",
         name: "Deployed at",
@@ -164,6 +209,7 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
       const author = this.getAuthor(deployment);
       return {
         deploymentId: deployment.deploymentId,
+        service: deployment.service,
         startTime: deployment.srcToDockerBuild
           ? deployment.srcToDockerBuild.startTime
           : new Date(),
@@ -215,15 +261,13 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
         clusterSync:
           this.state.manifestSync &&
           deployment.manifestCommitId === this.state.manifestSync.commit &&
-          this.state.manifestSync.commit !== ""
-            ? "Synced"
-            : "",
+          this.state.manifestSync.commit !== "",
         clusterSyncDate:
           this.state.manifestSync &&
           deployment.manifestCommitId === this.state.manifestSync.commit &&
           this.state.manifestSync.commit !== ""
-            ? this.state.manifestSync.date.toString()
-            : "",
+            ? this.state.manifestSync.date
+            : new Date(),
         endTime: deployment.hldToManifestBuild
           ? Number.isNaN(deployment.hldToManifestBuild!.finishTime.valueOf())
             ? new Date()
@@ -244,79 +288,6 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
     );
   };
 
-  private getDeployments = () => {
-    const srcPipeline = new AzureDevOpsPipeline(
-      config.AZURE_ORG,
-      config.AZURE_PROJECT,
-      config.SRC_PIPELINE_ID,
-      false,
-      config.AZURE_PIPELINE_ACCESS_TOKEN
-    );
-    const hldPipeline = new AzureDevOpsPipeline(
-      config.AZURE_ORG,
-      config.AZURE_PROJECT,
-      config.DOCKER_PIPELINE_ID,
-      true,
-      config.AZURE_PIPELINE_ACCESS_TOKEN
-    );
-    const clusterPipeline = new AzureDevOpsPipeline(
-      config.AZURE_ORG,
-      config.AZURE_PROJECT,
-      config.HLD_PIPELINE_ID,
-      false,
-      config.AZURE_PIPELINE_ACCESS_TOKEN
-    );
-
-    const manifestRepo: Repository = new GitHub(
-      config.GITHUB_MANIFEST_USERNAME,
-      config.MANIFEST,
-      config.MANIFEST_ACCESS_TOKEN
-    );
-    // const manifestRepo: Repository = new AzureDevOpsRepo(config.AZURE_ORG, config.AZURE_PROJECT, config.MANIFEST, config.MANIFEST_ACCESS_TOKEN);
-    manifestRepo.getManifestSyncState(syncCommit => {
-      this.setState({ manifestSync: syncCommit });
-    });
-    Deployment.getDeployments(
-      config.STORAGE_PARTITION_KEY,
-      srcPipeline,
-      hldPipeline,
-      clusterPipeline,
-      (deployments: Deployment[]) => {
-        this.setState({ deployments });
-        this.getAuthors();
-      }
-    );
-    return <div />;
-  };
-
-  private renderClusterSync = (
-    rowIndex: number,
-    columnIndex: number,
-    tableColumn: ITableColumn<IDeploymentField>,
-    tableItem: IDeploymentField
-  ): JSX.Element => {
-    if (tableItem.clusterSyncDate !== "") {
-      return (
-        <TwoLineTableCell
-          key={"col-" + columnIndex}
-          columnIndex={columnIndex}
-          tableColumn={tableColumn}
-          line1={this.WithIcon({
-            children: tableItem.clusterSync!,
-            className: "fontSize font-size",
-            iconProps: { iconName: "CloudDownload" }
-          })}
-          line2={this.WithIcon({
-            children: <Ago date={new Date(tableItem.clusterSyncDate!)} />,
-            className: "fontSize font-size",
-            iconProps: { iconName: "Calendar" }
-          })}
-        />
-      );
-    }
-    return <SimpleTableCell columnIndex={columnIndex} />;
-  };
-
   private renderSimpleText = (
     rowIndex: number,
     columnIndex: number,
@@ -555,7 +526,6 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
     tableColumn: ITableColumn<IDeploymentField>,
     tableItem: IDeploymentField
   ): JSX.Element => {
-    console.log(tableItem.status);
     if (!tableItem.status) {
       return <SimpleTableCell columnIndex={columnIndex} />;
     }
@@ -567,10 +537,24 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
         contentClassName="fontWeightSemiBold font-weight-semibold fontSizeM font-size-m scroll-hidden"
       >
         <Status
-          {...this.getStatusIndicatorData(tableItem.status).statusProps}
+          {...this.getStatusIndicatorData(
+            tableItem.status,
+            tableItem.clusterSync
+          ).statusProps}
           className="icon-large-margin"
           size={StatusSize.l}
         />
+        {tableItem.clusterSync && (
+          <Tooltip
+            overflowOnly={false}
+            text={"Synced at " + tableItem.clusterSyncDate!.toLocaleString()}
+          >
+            {this.WithIcon({
+              className: "fontSizeM font-size-m",
+              iconProps: { iconName: "CloudUpload" }
+            })}
+          </Tooltip>
+        )}
       </SimpleTableCell>
     );
   };
@@ -588,10 +572,12 @@ class Dashboard<Props> extends React.Component<Props, IDashboardState> {
     );
   };
 
-  private getStatusIndicatorData = (status: string): IStatusIndicatorData => {
+  private getStatusIndicatorData = (
+    status: string,
+    clusterSync?: boolean
+  ): IStatusIndicatorData => {
     status = status || "";
     status = status.toLowerCase();
-    console.log(status);
     const indicatorData: IStatusIndicatorData = {
       label: "Success",
       statusProps: { ...Statuses.Success, ariaLabel: "Success" }

diff --git a/src/models/Deployment.ts b/src/models/Deployment.ts
@@ -76,18 +76,22 @@ class Deployment {
         if (!error) {
           const srcBuildIds: Set<string> = new Set<string>();
           const manifestBuildIds: Set<string> = new Set<string>();
+          const releaseIds: Set<string> = new Set<string>();
           for (const entry of result.entries) {
             if (entry.p1) {
               srcBuildIds.add(entry.p1._);
             }
             if (entry.p3) {
               manifestBuildIds.add(entry.p3._);
             }
+            if (entry.p2) {
+              releaseIds.add(entry.p2._);
+            }
           }
 
           const p1 = srcPipeline.getListOfBuilds(undefined, srcBuildIds);
           // TODO: send releaseIds to below after bug in release API is fixed
-          const p2 = hldPipeline.getListOfReleases();
+          const p2 = hldPipeline.getListOfReleases(undefined, releaseIds);
           const p3 = manifestPipeline.getListOfBuilds(
             undefined,
             manifestBuildIds
@@ -176,6 +180,7 @@ class Deployment {
     let hldCommitId = "";
     let manifestCommitId = "";
     let env = "";
+    let service = "";
     if (entry.p2 != null) {
       p2 = hldPipeline.releases[entry.p2._];
     }
@@ -193,6 +198,9 @@ class Deployment {
     if (entry.env != null) {
       env = entry.env._;
     }
+    if (entry.service != null) {
+      service = entry.service._;
+    }
 
     const deployment = new Deployment(
       entry.RowKey._,
@@ -201,6 +209,7 @@ class Deployment {
       imageTag,
       entry.Timestamp._,
       env,
+      service,
       manifestCommitId,
       p1,
       p2,
@@ -220,6 +229,7 @@ class Deployment {
   public manifestCommitId?: string;
   public author?: Author;
   public environment: string;
+  public service: string;
 
   constructor(
     deploymentId: string,
@@ -228,6 +238,7 @@ class Deployment {
     imageTag: string,
     timeStamp: string,
     environment: string,
+    service: string,
     manifestCommitId?: string,
     srcToDockerBuild?: Build,
     dockerToHldRelease?: Release,
@@ -243,6 +254,7 @@ class Deployment {
     this.timeStamp = timeStamp;
     this.manifestCommitId = manifestCommitId;
     this.environment = environment;
+    this.service = service;
   }
 
   public duration(): string {