Merge pull request #63 from microsoft/markphip/add-oidc-support

Add support for OIDC during prebuild

src/external-repository/NOTES.md

@@ -59,17 +59,49 @@ that the token only have this scope.
 This would clone the repository to `/workspaces/ado-repos` during the Prebuild process
 using the PAT stored in a Codespaces secret. At runtime, when a user opens the Codespace
 the `workspaceFolder` feature would open VS Code to this folder automatically and it
-would be configured to prompt the user to login to Azure DevOps when they open the Codespace.
+will prompt the user to login to Azure DevOps when they open the Codespace. The user of
+the Codespaces does not need a PAT as the runtime will use their own login provided by
+the VS Code extension.
 
-If you want to allow your users to use their own token, then you can add this to the configuration:
+### Secret-less Azure DevOps Prebuilds
+
+As of version 4, it is possible to avoid using PATs entirely and dynamically obtain a token during prebuild using
+OIDC. This requires creating a Managed Identity or App Registration in Entra, and creating a
+Federated Identity Credential on the Service Principal for the branch you are prebuilding. The 
+Service Principal created must also be added to Azure DevOps and given permission to the repositories
+and feeds you will be accessing during the prebuild process. The configuration replaces the `cloneSecret`
+with parameters for the Azure `clientID` and `tenantID` and also requires adding the feature for
+the azure-cli:
 
 ```json
-        "userSecret": "ADO_SECRET"
+{
+"image": "mcr.microsoft.com/devcontainers/universal:ubuntu",
+"features": {
+    "ghcr.io/devcontainers/features/azure-cli:1": {},
+    "ghcr.io/microsoft/codespace-features/external-repository:latest": {
+        "cloneUrl": "https://dev.azure.com/contoso/_git/reposname",
+        "clientID": "xxxx-yyyy-zzzz",
+        "tenantID": "1111-2222-3333",
+        "folder": "/workspaces/ado-repos"
+    }
+},
+"workspaceFolder": "/workspaces/ado-repos",
+"initializeCommand": "mkdir -p ${localWorkspaceFolder}/../ado-repos",
+"onCreateCommand": "external-git clone",
+"postStartCommand": "external-git config"     
+}
 ```
 
-If a user configures a Codespaces User Secret named `ADO_SECRET` and assigns this secret to the
-Codespace, then the value of that secret will be used as a PAT for authentication. If the secret
-is not defined by the user it will fallback to the browser login.
+In this scenario, during the prebuild process an ADO token will be obtained via OIDC and the Federated Identity Credential.
+This token will be used during the git clone process only. If you have other scripts you are running during
+`onCreateCommand` you can run the command `external-git prebuild` and the ADO token will be sent to stdout for you
+to use in your scripts to install dependencies from feeds or anything else you may need. The token will only be
+available during the prebuild process and this has to be done after the clone command so that the OIDC login has
+already happened. Install and use the [artifacts-helper](../artifacts-helper/README.md) feature to provide support
+at runtime for artifacts and feeds.
+
+> [!NOTE]
+> You MUST install the Azure CLI feature in your devcontainer.json if using this option
 
 ### Interactive authentication only (avoids PAT token)
 
@@ -92,6 +124,11 @@ Codespace loads. This means the repository will be cloned only after the Codespa
 }
 ```
 
+> [!NOTE]
+> Obtaining the credentials from the user will not work from `onCreateCommand` because the required
+> VS Code extension is not available during this phase of the process. In this example, we are running
+> the clone during `postStartCommand`. This is important.
+
 ## Multiple Repository Support
 
 As of version 3, you can clone multiple repositories by separating the URL's with a comma. In this

src/external-repository/devcontainer-feature.json

@@ -1,7 +1,7 @@
 {
     "name": "External Git Repository in Codespace",
     "id": "external-repository",
-    "version": "3.0.3",
+    "version": "4.0.0",
     "description": "Configures Codespace to work with an external Git repository",
     "options": {
         "gitProvider": {
@@ -73,10 +73,21 @@
             ],
             "default": "none",
             "description": "Configure source of Git commit telemetry"
+        },
+        "clientID": {
+            "type": "string",
+            "default": "",
+            "description": "Azure Client ID for OIDC token acquisition during prebuild"
+        },
+        "tenantID": {
+            "type": "string",
+            "default": "",
+            "description": "Azure Tenant ID for OIDC token acquisition during prebuild"
         }
     },
     "installsAfter": [
-        "ghcr.io/devcontainers/features/common-utils"
+        "ghcr.io/devcontainers/features/common-utils",
+        "ghcr.io/devcontainers/features/azure-cli"
     ],
     "customizations": {
         "vscode": {

src/external-repository/install.sh

@@ -14,7 +14,13 @@ EXT_GIT_OPTIONS="${OPTIONS:-""}"
 EXT_GIT_SCALAR="${SCALAR:-"false"}"
 EXT_GIT_SPARSECHECKOUT="${SPARSECHECKOUT:-""}"
 EXT_GIT_TELEMETRY="${TELEMETRYSOURCE:-"none"}"
+EXT_GIT_AZURE_CLIENT_ID="${CLIENTID:-""}"
+EXT_GIT_AZURE_TENANT_ID="${TENANTID:-""}"
 
+EXT_GIT_OIDC_PREBUILD="false"
+if [ -n "${EXT_GIT_AZURE_CLIENT_ID}" ] && [ -n "${EXT_GIT_AZURE_TENANT_ID}" ]; then
+    EXT_GIT_OIDC_PREBUILD="true"
+fi
 
 if [ "$(id -u)" -ne 0 ]; then
     echo 'Script must be run as root. Use sudo, su, or add "USER root" to your Dockerfile before running this script.'
@@ -47,6 +53,9 @@ echo "EXT_GIT_OPTIONS=\"${EXT_GIT_OPTIONS}\"" >> /usr/local/external-repository-
 echo "EXT_GIT_SCALAR=\"${EXT_GIT_SCALAR}\"" >> /usr/local/external-repository-feature/variables.sh
 echo "EXT_GIT_SPARSECHECKOUT=\"${EXT_GIT_SPARSECHECKOUT}\"" >> /usr/local/external-repository-feature/variables.sh
 echo "EXT_GIT_TELEMETRY=\"${EXT_GIT_TELEMETRY}\"" >> /usr/local/external-repository-feature/variables.sh
+echo "EXT_GIT_AZURE_CLIENT_ID=\"${EXT_GIT_AZURE_CLIENT_ID}\"" >> /usr/local/external-repository-feature/variables.sh
+echo "EXT_GIT_AZURE_TENANT_ID=\"${EXT_GIT_AZURE_TENANT_ID}\"" >> /usr/local/external-repository-feature/variables.sh
+echo "EXT_GIT_OIDC_PREBUILD=\"${EXT_GIT_OIDC_PREBUILD}\"" >> /usr/local/external-repository-feature/variables.sh
 
 # Make the scripts executable
 chmod +rx /usr/local/external-repository-feature/*.sh
@@ -57,4 +66,4 @@ cp ./scripts/*.config /usr/local/external-repository-feature
 
 chmod +r /usr/local/external-repository-feature/*.config
 
-exit 0
+exit 0

src/external-repository/scripts/ado-git.config

@@ -1,3 +1,3 @@
 [credential]
 	helper =
-	helper = ADO_HELPER_PATH
+	helper = /usr/local/bin/external-git ado-helper

src/external-repository/scripts/clone.sh

@@ -6,6 +6,18 @@ cd "$(dirname "$0")"
 # Load the variables
 source ./variables.sh
 
+check_prebuild() {
+    # If EXT_GIT_OIDC_PREBUILD is true and ACTIONS_ID_TOKEN_REQUEST_URL is set then return 0
+    if [[ "${EXT_GIT_OIDC_PREBUILD}" == "true" && "${ACTIONS_ID_TOKEN_REQUEST_URL}" != "" ]]; then
+        return 0
+    fi
+    # If EXT_GIT_PREBUILD_PAT is set and the value of the variable is not empty then return 0
+    if [[ "${EXT_GIT_PREBUILD_PAT}" != "" && "${!EXT_GIT_PREBUILD_PAT}" != "" ]]; then
+        return 0
+    fi
+    return 1
+}
+
 clone_repository() {
     set -e
     GIT_REPO_URL=${1}
@@ -18,13 +30,12 @@ clone_repository() {
             mv ${HOME}/.gitconfig ${HOME}/.gitconfig.external_git_feature
         fi
 
-        if [[ "${EXT_GIT_PREBUILD_PAT}" == "" ]]; then
-            # Put the ado-auth-helper git config in place
-            ADO_HELPER=$(echo ~)/ado-auth-helper
-            sed "s|ADO_HELPER_PATH|${ADO_HELPER}|g" "./ado-git.config" > ${HOME}/.gitconfig
-        else
+        if check_prebuild; then
             # Put the prebuild git config in place
             cp /usr/local/external-repository-feature/prebuild-git.config ${HOME}/.gitconfig
+        else
+            # Put the ado-auth-helper git config in place
+            cp /usr/local/external-repository-feature/ado-git.config ${HOME}/.gitconfig
         fi
 
         # Perform a git clone
@@ -98,21 +109,13 @@ if [[ "${EXT_GIT_LOCAL_PATH}" == "" ]]; then
     exit 0;
 fi
 
-if [[ "${EXT_GIT_PREBUILD_PAT}" == "" ]]; then
-    echo "Prebuild secret is not set, attempting to clone with ado-auth-helper"
-    if [ ! -f ${HOME}/ado-auth-helper ]; then
-        echo "Waiting up to 180 seconds for ado-auth-helper extension to be installed"
-    fi    
-    # Wait up to 3 minutes for the ado-auth-helper to be installed
-    for i in {1..180}; do
-        if [ -f ${HOME}/ado-auth-helper ]; then
-            break
-        fi
-        sleep 1
-    done
-else
-    # Get the value from environment variable whose name is set in EXT_GIT_PREBUILD_PAT
-    EXT_GIT_PAT_VALUE=${!EXT_GIT_PREBUILD_PAT}
+
+if check_prebuild; then
+    if [[ "${EXT_GIT_OIDC_PREBUILD}" == "true" ]]; then
+        EXT_GIT_PAT_VALUE="OIDC"
+    else
+        EXT_GIT_PAT_VALUE=${!EXT_GIT_PREBUILD_PAT}
+    fi
 
     if [[ "${EXT_GIT_PAT_VALUE}" == "" ]]; then
         echo "There is no secret stored in ${EXT_GIT_PREBUILD_PAT}"
@@ -140,4 +143,4 @@ if [ ${#EXT_GIT_REPO_URL_ARRAY[@]} -gt 1 ]; then
     exit 0
 fi
 # There was only one repository specified
-clone_repository ${EXT_GIT_REPO_URL} ${EXT_GIT_LOCAL_PATH}
+clone_repository ${EXT_GIT_REPO_URL} ${EXT_GIT_LOCAL_PATH}

src/external-repository/scripts/external-git

@@ -14,11 +14,29 @@ case "${SUBCOMMAND}" in
         exec /bin/bash "/usr/local/external-repository-feature/setup-user.sh"
         ;;
     "prebuild")
-        if [ "$2" = "get" ]; then
-            source /usr/local/external-repository-feature/variables.sh
-            PREBUILD_PAT=${!EXT_GIT_PREBUILD_PAT}
-            echo "username=${EXT_GIT_USERNAME}"
-            echo "password=${PREBUILD_PAT}"
+        source /usr/local/external-repository-feature/variables.sh
+        if [ "${EXT_GIT_OIDC_PREBUILD}" == "true" ]; then
+          # Get token using OIDC
+            if [ "$2" = "get" ]; then
+                FIC=$(curl -s -H "Authorization: bearer ${ACTIONS_ID_TOKEN_REQUEST_TOKEN}" "${ACTIONS_ID_TOKEN_REQUEST_URL}&audience=api://AzureADTokenExchange" | jq -r '.value')
+                if az login --service-principal -u ${EXT_GIT_AZURE_CLIENT_ID} --tenant ${EXT_GIT_AZURE_TENANT_ID} --federated-token "${FIC}" --allow-no-subscriptions --only-show-errors &>/dev/null; then
+                    ADO_TOKEN=$(az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 | jq -r .accessToken)
+                else
+                    ADO_TOKEN="ERROR"
+                fi
+                echo "username=${EXT_GIT_USERNAME}"
+                echo "password=${ADO_TOKEN}"
+            else
+               # This allows the prebuild command to be called and just get the token on stdout
+                echo $(az account get-access-token --resource 499b84ac-1321-427f-aa17-267ca6975798 | jq -r .accessToken)
+            fi
+        else
+            # Get token using PAT
+            if [ "$2" = "get" ]; then
+                PREBUILD_PAT=${!EXT_GIT_PREBUILD_PAT}
+                echo "username=${EXT_GIT_USERNAME}"
+                echo "password=${PREBUILD_PAT}"
+            fi
         fi
         exit 0
         ;;
@@ -31,6 +49,22 @@ case "${SUBCOMMAND}" in
         fi
         exit 0
         ;;
+    "ado-helper")
+        if [ "$2" = "get" ]; then
+            # Wait up to 120 seconds for the ado-auth-helper to be installed
+            for i in {1..120}; do
+                if [ -f ${HOME}/ado-auth-helper ]; then
+                    ~/ado-auth-helper get
+                    exit 0
+                fi
+                sleep 1
+            done
+            # Helper is not available
+            echo "username=helper"
+            echo "password=notinstalled"
+        fi
+        exit 0
+        ;;        
     *)
         echo "devcontainer.json example:"
         echo "  "\"onCreateCommand\": \"external-git clone\",""

src/external-repository/scripts/setup-user.sh

@@ -59,8 +59,7 @@ configure_git_for_user() {
 
     if [ "$ADO" = "true" ]; then
         echo "Configuring ADO Authorization Helper"
-        ADO_HELPER=$(echo ~)/ado-auth-helper
-        sed "s|ADO_HELPER_PATH|${ADO_HELPER}|g" "./ado-git.config" >> "${GIT_PATH}/config"
+        cat "./ado-git.config" >> "${GIT_PATH}/config"
         # See if there was a request to checkout an AzDO branch by checking the branch name of
         # the Codespaces bridge repository. If the branch name begins with azdo/ then the
         # rest of the branch name is the branch to checkout.

src/external-repository/scripts/variables.sh

@@ -15,4 +15,7 @@ EXT_GIT_BRANCH="main"
 EXT_GIT_OPTIONS=
 EXT_GIT_SCALAR="false"
 EXT_GIT_SPARSECHECKOUT=
-EXT_GIT_TELEMETRY="none"
+EXT_GIT_TELEMETRY="none"
+EXT_GIT_AZURE_CLIENT_ID=""
+EXT_GIT_AZURE_TENANT_ID=""
+EXT_GIT_OIDC_PREBUILD="false"

test/_global/debian_tests.sh

@@ -7,7 +7,7 @@ source dev-container-features-test-lib
 
 check "git-version" bash -c "git --version | grep 'vfs'"
 check "no-dirs"  grep -v "drwxr" <(ls -l /tmp/debian_tests)
-check "git-config" grep "ado-auth-helper" <(cat /tmp/debian_tests/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/debian_tests/.git/config)
 check "dotnet" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-dotnet.sh)
 check "nuget" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-nuget.sh)
 

test/_global/mariner_tests.sh

@@ -7,7 +7,7 @@ source dev-container-features-test-lib
 
 check "git-version" bash -c "git --version | grep 'vfs'"
 check "no-dirs"  grep -v "drwxr" <(ls -l /tmp/mariner_tests)
-check "git-config" grep "ado-auth-helper" <(cat /tmp/mariner_tests/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/mariner_tests/.git/config)
 check "dotnet" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-dotnet.sh)
 check "nuget" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-nuget.sh)
 check "docfx-version" bash -c "docfx --version"

test/_global/ubuntu_tests.sh

@@ -7,7 +7,7 @@ source dev-container-features-test-lib
 
 check "git-version" bash -c "git --version | grep 'vfs'"
 check "no-dirs"  grep -v "drwxr" <(ls -l /tmp/debian_tests)
-check "git-config" grep "ado-auth-helper" <(cat /tmp/debian_tests/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/debian_tests/.git/config)
 check "dotnet" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-dotnet.sh)
 check "nuget" grep "pkgs.dev.azure.com" <(cat /usr/local/bin/run-nuget.sh)
 

test/external-repository/basic-install.sh

@@ -6,7 +6,7 @@ set -e
 source dev-container-features-test-lib
 
 # Definition specific tests
-check "git-config" grep "ado-auth-helper" <(cat /tmp/basic-repos/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/basic-repos/.git/config)
 
 # Report result
 reportResults

test/external-repository/branch-install.sh

@@ -7,7 +7,7 @@ source dev-container-features-test-lib
 
 # Definition specific tests
 touch ${HOME}/ado-auth-helper
-check "git-config" grep "ado-auth-helper" <(cat /tmp/branch-repos/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/branch-repos/.git/config)
 check "branch" grep "joshaber/parallel-execution-schema" <(git -C /tmp/branch-repos branch --show-current)
 
 # Report result

test/external-repository/multi-install.sh

@@ -6,9 +6,9 @@ set -e
 source dev-container-features-test-lib
 
 # Definition specific tests
-check "git-config-1" grep "ado-auth-helper" <(cat /tmp/multi-repos/community/.git/config)
-check "git-config-2" grep "ado-auth-helper" <(cat /tmp/multi-repos/spec/.git/config)
-check "git-config-3" grep "ado-auth-helper" <(cat /tmp/multi-repos/devcontainers.github.io/.git/config)
+check "git-config-1" grep "ado-helper" <(cat /tmp/multi-repos/community/.git/config)
+check "git-config-2" grep "ado-helper" <(cat /tmp/multi-repos/spec/.git/config)
+check "git-config-3" grep "ado-helper" <(cat /tmp/multi-repos/devcontainers.github.io/.git/config)
 
 # Report result
 reportResults

test/external-repository/scalar-basic.sh

@@ -7,7 +7,7 @@ source dev-container-features-test-lib
 
 # Definition specific tests
 
-check "git-config" grep "ado-auth-helper" <(cat /tmp/scalar-basic/src/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/scalar-basic/src/.git/config)
 check "no-dirs"  grep -v "drwxr" <(ls -l /tmp/scalar-basic/src)
 
 # Report result

test/external-repository/scalar-folder.sh

@@ -7,7 +7,7 @@ source dev-container-features-test-lib
 
 # Definition specific tests
 
-check "git-config" grep "ado-auth-helper" <(cat /tmp/scalar-folder/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/scalar-folder/.git/config)
 check "no-dirs"  grep -v "drwxr" <(ls -l /tmp/scalar-folder)
 
 # Report result

test/external-repository/scalar-no-src.sh

@@ -7,7 +7,7 @@ source dev-container-features-test-lib
 
 # Definition specific tests
 
-check "git-config" grep "ado-auth-helper" <(cat /tmp/scalar-no-src/.git/config)
+check "git-config" grep "ado-helper" <(cat /tmp/scalar-no-src/.git/config)
 check "no-dirs"  grep -v "drwxr" <(ls -l /tmp/scalar-no-src)