Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add external id config for role assumption #36725

Open
wants to merge 3 commits into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
27 changes: 27 additions & 0 deletions .chloggen/awscloudwatchexporter_external_id.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
# Use this changelog template to create an entry for release notes.

# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
change_type: enhancement

# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
component: awscloudwatchlogsexporter, awsemfexporter, awsxrayexporter

# A brief description of the change. Surround your text with quotes ("") if it needs to start with a backtick (`).
note: Adding external id support when assuming a role for AWS credentials.

# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
issues: [36725]

# (Optional) One or more lines of additional information to render under the primary note.
# These lines will be padded with 2 spaces and then inserted directly into the document.
# Use pipe (|) for multiline entries.
subtext:

# If your change doesn't affect end users or the exported elements of any package,
# you should instead start your pull request title with [chore] or use the "Skip Changelog" label.
# Optional: The change log or logs in which this entry should be included.
# e.g. '[user]' or '[user, api]'
# Include 'user' if the change is relevant to end users.
# Include 'api' if there is a change to a library API.
# Default: '[user]'
change_logs: [user]
2 changes: 2 additions & 0 deletions exporter/awscloudwatchlogsexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,8 @@ The following settings can be optionally configured:
- `log_retention`: LogRetention is the option to set the log retention policy for only newly created CloudWatch Log Groups. Defaults to Never Expire if not specified or set to 0. Possible values for retention in days are 1, 3, 5, 7, 14, 30, 60, 90, 120, 150, 180, 365, 400, 545, 731, 1827, 2192, 2557, 2922, 3288, or 3653.
- `tags`: Tags is the option to set tags for the CloudWatch Log Group. If specified, please add at most 50 tags. Input is a string to string map like so: { 'key': 'value' }. Keys must be between 1-128 characters and follow the regex pattern: `^([\p{L}\p{Z}\p{N}_.:/=+\-@]+)$`(alphanumerics, whitespace, and _.:/=+-!). Values must be between 1-256 characters and follow the regex pattern: `^([\p{L}\p{Z}\p{N}_.:/=+\-@]*)$`(alphanumerics, whitespace, and _.:/=+-!). [Link to tagging restrictions](https://docs.aws.amazon.com/AmazonCloudWatchLogs/latest/APIReference/API_CreateLogGroup.html#:~:text=Required%3A%20Yes-,tags,-The%20key%2Dvalue)
- `raw_log`: Boolean default false. If set to true, only the log message will be exported to CloudWatch Logs. This needs to be set to true for [EMF logs](https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Specification.html).
- `role_arn`: IAM role to upload logs to a different account.
- `external_id`: Shared identitier used when assuming an IAM role in an external AWS account.
- `sending_queue`: [Parameters for the sending queue](https://github.com/open-telemetry/opentelemetry-collector/blob/main/exporter/exporterhelper/README.md), where you can control parallelism and the size of the sending buffer. Obs.: this component will always have a sending queue enabled.
- `num_consumers`: Number of consumers that will consume from the sending queue. This parameter controls how many consumers will consume from the sending queue in parallel.
- `queue_size`: Maximum number of batches kept in memory before dropping; ignored if enabled is false
Expand Down
1 change: 1 addition & 0 deletions exporter/awsemfexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -39,6 +39,7 @@ The following exporter configuration parameters are supported.
| `proxy_address` | Upload Structured Logs to AWS CloudWatch through a proxy. | |
| `region` | Send Structured Logs to AWS CloudWatch in a specific region. If this field is not present in config, environment variable "AWS_REGION" can then be used to set region. | determined by metadata |
| `role_arn` | IAM role to upload segments to a different account. | |
| `external_id` | Shared identitier used when assuming an IAM role in an external AWS account. | |
| `max_retries` | Maximum number of retries before abandoning an attempt to post data. | 1 |
| `dimension_rollup_option` | DimensionRollupOption is the option for metrics dimension rollup. Three options are available: `NoDimensionRollup`, `SingleDimensionRollupOnly` and `ZeroAndSingleDimensionRollup`. The default value is `ZeroAndSingleDimensionRollup`. Enabling feature gate `awsemf.nodimrollupdefault` will set default to `NoDimensionRollup`. |"ZeroAndSingleDimensionRollup" (Enable both zero dimension rollup and single dimension rollup)|
| `resource_to_telemetry_conversion` | "resource_to_telemetry_conversion" is the option for converting resource attributes to telemetry attributes. It has only one config onption- `enabled`. For metrics, if `enabled=true`, all the resource attributes will be converted to metric labels by default. See `Resource Attributes to Metric Labels` section below for examples. | `enabled=false` |
Expand Down
1 change: 1 addition & 0 deletions exporter/awsxrayexporter/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -65,6 +65,7 @@ comparable AWS X-Ray Daemon configuration values.
| `local_mode` | Local mode to skip EC2 instance metadata check. | false |
| `resource_arn` | Amazon Resource Name (ARN) of the AWS resource running the collector. | |
| `role_arn` | IAM role to upload segments to a different account. | |
| `external_id` | Shared identitier used when assuming an IAM role in an external AWS account. | |
| `indexed_attributes` | List of attribute names to be converted to X-Ray annotations. | |
| `index_all_attributes` | Enable or disable conversion of all OpenTelemetry attributes to X-Ray annotations. | false |
| `aws_log_groups` | List of log group names for CloudWatch. | [] |
Expand Down
2 changes: 2 additions & 0 deletions internal/aws/awsutil/awsconfig.go
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ type AWSSessionSettings struct {
ResourceARN string `mapstructure:"resource_arn"`
// IAM role to upload segments to a different account.
RoleARN string `mapstructure:"role_arn"`
// External ID to verify third party role assumption
ExternalID string `mapstructure:"external_id"`
}

func CreateDefaultSessionConfig() AWSSessionSettings {
Expand Down
32 changes: 19 additions & 13 deletions internal/aws/awsutil/conn.go
Original file line number Diff line number Diff line change
Expand Up @@ -25,7 +25,7 @@ import (
)

type ConnAttr interface {
newAWSSession(logger *zap.Logger, roleArn string, region string) (*session.Session, error)
newAWSSession(logger *zap.Logger, roleArn string, externalID string, region string) (*session.Session, error)
getEC2Region(s *session.Session) (string, error)
}

Expand Down Expand Up @@ -145,7 +145,7 @@ func GetAWSConfigSession(logger *zap.Logger, cn ConnAttr, cfg *AWSSessionSetting
logger.Error(msg)
return nil, nil, awserr.New("NoAwsRegion", msg, nil)
}
s, err = cn.newAWSSession(logger, cfg.RoleARN, awsRegion)
s, err = cn.newAWSSession(logger, cfg.RoleARN, cfg.ExternalID, awsRegion)
if err != nil {
return nil, nil, err
}
Expand Down Expand Up @@ -193,7 +193,7 @@ func ProxyServerTransport(logger *zap.Logger, config *AWSSessionSettings) (*http
return transport, nil
}

func (c *Conn) newAWSSession(logger *zap.Logger, roleArn string, region string) (*session.Session, error) {
func (c *Conn) newAWSSession(logger *zap.Logger, roleArn, externalID string, region string) (*session.Session, error) {
var s *session.Session
var err error
if roleArn == "" {
Expand All @@ -202,7 +202,7 @@ func (c *Conn) newAWSSession(logger *zap.Logger, roleArn string, region string)
return s, err
}
} else {
stsCreds, _ := getSTSCreds(logger, region, roleArn)
stsCreds, _ := getSTSCreds(logger, region, roleArn, externalID)

s, err = session.NewSession(&aws.Config{
Credentials: stsCreds,
Expand All @@ -218,13 +218,13 @@ func (c *Conn) newAWSSession(logger *zap.Logger, roleArn string, region string)
// getSTSCreds gets STS credentials from regional endpoint. ErrCodeRegionDisabledException is received if the
// STS regional endpoint is disabled. In this case STS credentials are fetched from STS primary regional endpoint
// in the respective AWS partition.
func getSTSCreds(logger *zap.Logger, region string, roleArn string) (*credentials.Credentials, error) {
func getSTSCreds(logger *zap.Logger, region string, roleArn, externalID string) (*credentials.Credentials, error) {
t, err := GetDefaultSession(logger)
if err != nil {
return nil, err
}

stsCred := getSTSCredsFromRegionEndpoint(logger, t, region, roleArn)
stsCred := getSTSCredsFromRegionEndpoint(logger, t, region, roleArn, externalID)
// Make explicit call to fetch credentials.
_, err = stsCred.Get()
if err != nil {
Expand All @@ -234,7 +234,7 @@ func getSTSCreds(logger *zap.Logger, region string, roleArn string) (*credential

if awsErr.Code() == sts.ErrCodeRegionDisabledException {
logger.Error("Region ", zap.String("region", region), zap.Error(awsErr))
stsCred = getSTSCredsFromPrimaryRegionEndpoint(logger, t, roleArn, region)
stsCred = getSTSCredsFromPrimaryRegionEndpoint(logger, t, roleArn, externalID, region)
}
}
}
Expand All @@ -245,7 +245,7 @@ func getSTSCreds(logger *zap.Logger, region string, roleArn string) (*credential
// AWS STS recommends that you provide both the Region and endpoint when you make calls to a Regional endpoint.
// Reference: https://docs.aws.amazon.com/IAM/latest/UserGuide/id_credentials_temp_enable-regions.html#id_credentials_temp_enable-regions_writing_code
func getSTSCredsFromRegionEndpoint(logger *zap.Logger, sess *session.Session, region string,
roleArn string,
roleArn, externalID string,
) *credentials.Credentials {
regionalEndpoint := getSTSRegionalEndpoint(region)
// if regionalEndpoint is "", the STS endpoint is Global endpoint for classic regions except ap-east-1 - (HKG)
Expand All @@ -254,23 +254,29 @@ func getSTSCredsFromRegionEndpoint(logger *zap.Logger, sess *session.Session, re
c := &aws.Config{Region: aws.String(region), Endpoint: &regionalEndpoint}
st := sts.New(sess, c)
logger.Info("STS Endpoint ", zap.String("endpoint", st.Endpoint))
return stscreds.NewCredentialsWithClient(st, roleArn)
options := []func(*stscreds.AssumeRoleProvider){}
if externalID != "" {
options = append(options, func(arp *stscreds.AssumeRoleProvider) {
arp.ExternalID = aws.String(externalID)
})
}
return stscreds.NewCredentialsWithClient(st, roleArn, options...)
}

// getSTSCredsFromPrimaryRegionEndpoint fetches STS credentials for provided roleARN from primary region endpoint in
// the respective partition.
func getSTSCredsFromPrimaryRegionEndpoint(logger *zap.Logger, t *session.Session, roleArn string,
func getSTSCredsFromPrimaryRegionEndpoint(logger *zap.Logger, t *session.Session, roleArn, externalID string,
region string,
) *credentials.Credentials {
logger.Info("Credentials for provided RoleARN being fetched from STS primary region endpoint.")
partitionID := getPartition(region)
switch partitionID {
case endpoints.AwsPartitionID:
return getSTSCredsFromRegionEndpoint(logger, t, endpoints.UsEast1RegionID, roleArn)
return getSTSCredsFromRegionEndpoint(logger, t, endpoints.UsEast1RegionID, roleArn, externalID)
case endpoints.AwsCnPartitionID:
return getSTSCredsFromRegionEndpoint(logger, t, endpoints.CnNorth1RegionID, roleArn)
return getSTSCredsFromRegionEndpoint(logger, t, endpoints.CnNorth1RegionID, roleArn, externalID)
case endpoints.AwsUsGovPartitionID:
return getSTSCredsFromRegionEndpoint(logger, t, endpoints.UsGovWest1RegionID, roleArn)
return getSTSCredsFromRegionEndpoint(logger, t, endpoints.UsGovWest1RegionID, roleArn, externalID)
}

return nil
Expand Down
16 changes: 9 additions & 7 deletions internal/aws/awsutil/conn_test.go
Original file line number Diff line number Diff line change
Expand Up @@ -32,7 +32,7 @@ func (c *mockConn) getEC2Region(_ *session.Session) (string, error) {
return ec2Region, nil
}

func (c *mockConn) newAWSSession(_ *zap.Logger, _ string, _ string) (*session.Session, error) {
func (c *mockConn) newAWSSession(_ *zap.Logger, _ string, _ string, _ string) (*session.Session, error) {
return c.sn, nil
}

Expand Down Expand Up @@ -104,15 +104,16 @@ func TestGetAWSConfigSessionWithEC2RegionErr(t *testing.T) {
func TestNewAWSSessionWithErr(t *testing.T) {
logger := zap.NewNop()
roleArn := "fake_arn"
externalID := ""
region := "fake_region"
t.Setenv("AWS_EC2_METADATA_DISABLED", "true")
t.Setenv("AWS_STS_REGIONAL_ENDPOINTS", "fake")
conn := &Conn{}
se, err := conn.newAWSSession(logger, roleArn, region)
se, err := conn.newAWSSession(logger, roleArn, externalID, region)
assert.Error(t, err)
assert.Nil(t, se)
roleArn = ""
se, err = conn.newAWSSession(logger, roleArn, region)
se, err = conn.newAWSSession(logger, roleArn, externalID, region)
assert.Error(t, err)
assert.Nil(t, se)
t.Setenv("AWS_SDK_LOAD_CONFIG", "true")
Expand All @@ -132,10 +133,10 @@ func TestGetSTSCredsFromPrimaryRegionEndpoint(t *testing.T) {
regions := []string{"us-east-1", "us-gov-west-1", "cn-north-1"}

for _, region := range regions {
creds := getSTSCredsFromPrimaryRegionEndpoint(logger, session, "", region)
creds := getSTSCredsFromPrimaryRegionEndpoint(logger, session, "", "", region)
assert.NotNil(t, creds)
}
creds := getSTSCredsFromPrimaryRegionEndpoint(logger, session, "", "fake_region")
creds := getSTSCredsFromPrimaryRegionEndpoint(logger, session, "", "", "fake_region")
assert.Nil(t, creds)
}

Expand All @@ -150,9 +151,10 @@ func TestGetSTSCreds(t *testing.T) {
logger := zap.NewNop()
region := "fake_region"
roleArn := ""
_, err := getSTSCreds(logger, region, roleArn)
externalID := ""
_, err := getSTSCreds(logger, region, roleArn, externalID)
assert.NoError(t, err)
t.Setenv("AWS_STS_REGIONAL_ENDPOINTS", "fake")
_, err = getSTSCreds(logger, region, roleArn)
_, err = getSTSCreds(logger, region, roleArn, externalID)
assert.Error(t, err)
}
Loading