Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Adding workspace/account logic for python #81

Merged
merged 3 commits into from
Mar 20, 2025
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions .gitattributes
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@ sdk/go/api_esc_test.go linguist-generated=false

sdk/python/**/* linguist-generated=true
sdk/python/pulumi_esc_sdk/esc_client.py linguist-generated=false
sdk/python/pulumi_esc_sdk/workspace.py linguist-generated=false
sdk/python/pulumi_esc_sdk/workspace_models.py linguist-generated=false
sdk/python/test/**/* linguist-generated=false
sdk/python/pyproject.toml linguist-generated=false

Expand Down
2 changes: 2 additions & 0 deletions CHANGELOG_PENDING.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,8 @@
- All SDK now automatically read in configuration environment variables
- Go SDK also automatically picks up configuration from CLI Pulumi accounts
[#76](https://github.com/pulumi/esc-sdk/pull/76)
- Adds support for reading credentials from disk to Python SDK
[#81](https://github.com/pulumi/esc-sdk/pull/81)

### Bug Fixes

Expand Down
9 changes: 9 additions & 0 deletions sdk/python/pulumi_esc_sdk/esc_client.py
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@
import yaml
import os
from urllib.parse import urlparse, urlunparse
import pulumi_esc_sdk.workspace as workspace


class EscClient:
Expand Down Expand Up @@ -441,6 +442,14 @@ def default_config(host=None,
access_token = os.getenv("PULUMI_ACCESS_TOKEN")
if not host:
host = os.getenv("PULUMI_BACKEND_URL")

if not access_token or not host:
account, backend_url = workspace.get_current_account(False)
if not access_token:
access_token = account.accessToken if account else None
if not host:
host = backend_url

return configuration.Configuration(
host=host,
access_token=access_token,
Expand Down
93 changes: 93 additions & 0 deletions sdk/python/pulumi_esc_sdk/workspace.py
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lets mark this and the workspace_models.py as linguist-generated=false in .gitattributes

Original file line number Diff line number Diff line change
@@ -0,0 +1,93 @@
# Copyright 2025, Pulumi Corporation. All rights reserved.

"""
Pulumi workspace and account logic for python SDK.
This is a partial port of ESC and Pulumi CLI code found in
https://github.com/pulumi/esc/tree/main/cmd/esc/cli/workspace
"""

from dataclasses import dataclass, field
import json
import os
import pathlib
from typing import Optional

from pulumi_esc_sdk.workspace_models import Account, Credentials

def get_pulumi_home_dir() -> str:
"""
Returns the path of the '.pulumi' folder where Pulumi puts its artifacts.
"""
# Allow the folder we use to be overridden by an environment variable
dir_env = os.getenv("PULUMI_HOME")
if dir_env:
return dir_env

# Otherwise, use the current user's home dir + .pulumi
home_dir = pathlib.Path.home()

return str(home_dir.joinpath(".pulumi"))

def get_pulumi_path(*elem: str) -> str:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see this method is only called with ".esc" and with no args. can the no args call be replaced with get_pulumi_home_dir()? If so, then I don't think this method has any purpose

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fair point, I was porting over method by method, and in CLI there's more uses for it. But in this case, I don't expect more usage, so can eliminate this for sure.

"""
Returns the path to a file or directory under the '.pulumi' folder.
It joins the path of the '.pulumi' folder with elements passed as arguments.
"""
home_dir = get_pulumi_home_dir()
return str(pathlib.Path(home_dir).joinpath(*elem))

def get_esc_bookkeeping_dir() -> str:
"""
Returns the path of the '.esc' folder inside Pulumi home dir.
"""
return get_pulumi_path(".esc")
Comment on lines +39 to +43
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What is that directory for? I don't see it on my machine.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This one appears if you login via esc cli, not the pulumi one. It is a single-value file like this

{
    "name": "https://api.pulumi.com/"
}

that allows you to override current account. So if your current account in normal credentials file is moolumi.com, and you use pulumi CLI, that's what it will use. But if using esc CLI you then login to boolumi.com, it will use that as default in ESC CLI. See this method as go reference.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay sweet can we just add a comment to that effect?


def get_path_to_creds_file(dir: str) -> str:
"""
Returns the path to the esc credentials file on disk.
"""
return str(pathlib.Path(dir).joinpath("credentials.json"))

def get_esc_current_account_name() -> Optional[str]:
"""
Returns the current account name from the ESC credentials file.
"""
creds_file = get_path_to_creds_file(get_esc_bookkeeping_dir())
try:
with open(creds_file, 'r') as f:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nit: let's be consistent with using double quotes, here and elsewhere

data = json.loads(f.read())
return data['name']
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we should add a case for if name doesn't exist. whether that's a conditional here or caught as an except ..., either is fine to me

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, no reason to print "unexpected error" when it is rather expected. I'll make it return None on KeyError

except FileNotFoundError:
return None
except Exception as e:
print(f"An unexpected error occured: {e}")
return None

def get_stored_credentials() -> Credentials:
"""
Reads and parses credentials from the Pulumi credentials file.
"""
creds_file = get_path_to_creds_file(get_pulumi_path())
try:
with open(creds_file, 'r') as f:
data = f.read()
creds = Credentials.from_json(data)
return creds
except FileNotFoundError:
return None
except Exception as e:
print(f"An unexpected error occured: {e}")
return None

def get_current_account(shared) -> tuple[Account, str]:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

what is shared for?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If Shared is true, we would ignore the ESC override and use the account selected in main credentials file - same thing as with the path method, not strictly necessary in this implementation, I was just keeping it as in the CLI.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if we're not using it, i would opt to remove it. but if you want to keep it to match the CLI, i would at least document it

"""
Gets current account values from credentials file.
"""
backend_url = get_esc_current_account_name()
pulumi_credentials = get_stored_credentials()
if not pulumi_credentials:
return None, None
if backend_url is None or shared:
backend_url = pulumi_credentials.current
pulumi_account = pulumi_credentials.accounts[backend_url]
return pulumi_account, backend_url
45 changes: 45 additions & 0 deletions sdk/python/pulumi_esc_sdk/workspace_models.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
# Copyright 2025, Pulumi Corporation. All rights reserved.

"""
Models for Pulumi workspace and account logic for python SDK.
This is a partial port of ESC and Pulumi CLI code found in
https://github.com/pulumi/esc/tree/main/cmd/esc/cli/workspace
"""

from dataclasses import dataclass, field
import json
from typing import Dict, List, Optional

@dataclass
class Account:
accessToken: str
username: str
organizations: List[str]

@classmethod
def from_json(self, data: dict):
return self(
accessToken=data.get("accessToken"),
username=data.get("username"),
organizations=data.get("organizations", []),
)

@dataclass
class Credentials:
current: str
accessTokens: Dict[str, str]
accounts: Dict[str, Account]

@classmethod
def from_json(self, dataJson: str):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Account.from_json reads from a dict, but this reads from a json string? maybe we can name this from_file and pass the file? (json.loads can accept a file so we can skip the extra f.read() step)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, not great that they are different. I think it would be better to keep the methods the same, both from_jsons receiving dict with values, and we can follow the same pattern of data = json.loads(f.read()) on file loading

data = json.loads(dataJson)
accounts: Dict[str, Account] = {}
accounts_dict = data.get("accounts")
Copy link
Contributor

@seanyeh seanyeh Mar 20, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this seems more natural to me, but what you have is fine

if "accounts" in data:
  for ...

Also, should we take into account if the creds file is in the wrong format? I assume this will throw an ambiguous attribute error, so if we can have a helpful error that might be better. (do we give a helpful error in pulumi/pulumi for the equivalent?)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ty, I've no idea what's natural or not in python 😅

Hmm, creds files are not meant to be edited by hand, so I think it's fine to just throw an ambiguous "unexpected error". But I think it might be worth adding a test that makes sure that's what happens and SDK doesn't break.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would prefer a clearer error but not a huge deal

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In most scenarios it won't throw an error at all, just return None, it's only if you screw us the schema in very specific ways (like for example adding "accounts" key, but making it a string and not an object). I just think it would be overkill to try to catch all potential errors on bad deserializations

if accounts_dict:
for account_name, account_data in accounts_dict.items():
accounts[account_name] = Account.from_json(account_data)
return self(
current=data.get("current"),
accessTokens=data.get("accessTokens"),
accounts=accounts
)
2 changes: 1 addition & 1 deletion sdk/python/test/test_esc_api.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ class TestEscApi(unittest.TestCase):
"""EscApi unit test stubs"""

def setUp(self) -> None:
self.orgName = "pulumi"
self.orgName = os.getenv("PULUMI_ORG")
self.assertIsNotNone(self.orgName, "PULUMI_ORG must be set")

self.client = esc.esc_client.default_client()
Expand Down
25 changes: 25 additions & 0 deletions sdk/python/test/test_pulumi_home/credentials.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"current": "https://api.moolumi.com",
"accessTokens": {
"https://api.moolumi.com": "pul-fake-token-moo",
"https://api.boolumi.com": "pul-fake-token-boo"
},
"accounts": {
"https://api.moolumi.com": {
"accessToken": "pul-fake-token-moo",
"username": "moolumipus",
"organizations": [
"moolumipus-org"
],
"lastValidatedAt": "2025-03-17T11:19:28.392525488-06:00"
},
"https://api.boolumi.com": {
"accessToken": "pul-fake-token-boo",
"username": "boolumipus",
"organizations": [
"boolumipus-org2"
],
"lastValidatedAt": "2026-03-17T11:19:28.392525488-06:00"
}
}
}
3 changes: 3 additions & 0 deletions sdk/python/test/test_pulumi_home_esc/.esc/credentials.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"name": "https://api.boolumi.com"
}
25 changes: 25 additions & 0 deletions sdk/python/test/test_pulumi_home_esc/credentials.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,25 @@
{
"current": "https://api.moolumi.com",
"accessTokens": {
"https://api.moolumi.com": "pul-fake-token-moo",
"https://api.boolumi.com": "pul-fake-token-boo"
},
"accounts": {
"https://api.moolumi.com": {
"accessToken": "pul-fake-token-moo",
"username": "moolumipus",
"organizations": [
"moolumipus-org"
],
"lastValidatedAt": "2025-03-17T11:19:28.392525488-06:00"
},
"https://api.boolumi.com": {
"accessToken": "pul-fake-token-boo",
"username": "boolumipus",
"organizations": [
"boolumipus-org2"
],
"lastValidatedAt": "2026-03-17T11:19:28.392525488-06:00"
}
}
}
51 changes: 51 additions & 0 deletions sdk/python/test/test_workspace_accounts.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,51 @@
# coding: utf-8

# Copyright 2024, Pulumi Corporation. All rights reserved.

from typing import Optional
import unittest
import os

import pulumi_esc_sdk as esc

class TestWorkspaceAccounts(unittest.TestCase):
"""WorkspaceAccounts unit test stubs"""
tokenBefore: Optional[str]
backendBefore: Optional[str]
homeBefore: Optional[str]

def setUp(self) -> None:
self.tokenBefore = os.getenv("PULUMI_ACCESS_TOKEN")
self.backendBefore = os.getenv("PULUMI_BACKEND_URL")
self.homeBefore = os.getenv("PULUMI_HOME")
os.environ["PULUMI_ACCESS_TOKEN"] = ""
os.environ["PULUMI_BACKEND_URL"] = ""

def tearDown(self) -> None:
os.environ["PULUMI_ACCESS_TOKEN"] = self.tokenBefore or ''
os.environ["PULUMI_BACKEND_URL"] = self.backendBefore or ''
os.environ["PULUMI_HOME"] = self.homeBefore or ''

def test_no_creds_at_all(self):
os.environ["PULUMI_HOME"] = "/not_real_dir"
self.client = esc.esc_client.default_client()
self.config = self.client.esc_api.api_client.configuration
self.assertEqual(self.config.host, "https://api.pulumi.com/api/esc")
self.assertTrue('Authorization' not in self.config.api_key)

def test_just_pulumi_creds(self):
os.environ["PULUMI_HOME"] = os.getcwd() + "/test/test_pulumi_home"
self.client = esc.esc_client.default_client()
self.config = self.client.esc_api.api_client.configuration
self.assertEqual(self.config.host, "https://api.moolumi.com/api/esc")
self.assertTrue(self.config.api_key['Authorization'], "pul-fake-token-moo")

def test_pulumi_creds_with_esc(self):
os.environ["PULUMI_HOME"] = os.getcwd() + "/test/test_pulumi_home_esc"
self.client = esc.esc_client.default_client()
self.config = self.client.esc_api.api_client.configuration
self.assertEqual(self.config.host, "https://api.boolumi.com/api/esc")
self.assertTrue(self.config.api_key['Authorization'], "pul-fake-token-boo")

if __name__ == '__main__':
unittest.main()
Loading