Upgrade Contrib to v0.103.0 #1247

okankoAMZ · 2024-07-11T20:52:06Z

Description of the issue

Contrib was last updated 2 months ago. Contrib repository PR can be found here. Please do not merge before that PR.

Description of changes

Upgrade contrib to v0.103.0 stable version. To do that I had do:

Update CloudWatch agent command, where I am using now passing in the Provider Settings and not just Provider.
Update Config Provider because the inputs have changed to Factories and Settings from directly feeding in the Converters and Providers.
Update our logger, so when a NewLogger is created now there is a option to retrieve the Logger Options with the Logger itself.
component.UnmarshalConfig was deprecated so I had to migrate it to confmap.Unmarshal
Updated translator sample configs.

License

By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.

Tests

Ran integration test here: https://github.com/aws/amazon-cloudwatch-agent/actions/runs/10063006520
More recent integration test here:https://github.com/aws/amazon-cloudwatch-agent/actions/runs/10101543388

Latest agent logs:

[ec2-user@ip-172-31-46-210 amazon-cloudwatch-agent]$ tail -n 100 /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log 
        logs:
            development: false
            disable_caller: false
            disable_stacktrace: false
            encoding: console
            level: debug
            output_paths:
                - /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log
            sampling:
                enabled: true
                initial: 2
                thereafter: 500
                tick: 10s
        metrics:
            address: ""
            level: None
        traces: {}
2024/08/05 18:27:58 Reading json config file path: /opt/aws/amazon-cloudwatch-agent/etc/amazon-cloudwatch-agent.json ...
2024/08/05 18:27:58 I! Valid Json input schema.
2024/08/05 18:27:58 I! Detected runAsUser: root
2024/08/05 18:27:58 I! Changing ownership of [/opt/aws/amazon-cloudwatch-agent/logs /opt/aws/amazon-cloudwatch-agent/etc /opt/aws/amazon-cloudwatch-agent/var] to 0:0
2024-08-05T18:27:58Z I! Starting AmazonCloudWatchAgent CWAgent/1.300041.1 (go1.22.4; linux; amd64) with log file /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log with log target lumberjack
2024-08-05T18:27:58Z I! AWS SDK log level not set
2024-08-05T18:27:58Z I! creating new logs agent
2024-08-05T18:27:58Z I! [logagent] starting
2024-08-05T18:27:58Z I! {"caller":"[email protected]/telemetry.go:47","msg":"Skipping telemetry setup.","address":"","level":"None"}
2024-08-05T18:27:58Z D! {"caller":"[email protected]/extension.go:165","msg":"Alpha component. May change in the future.","kind":"extension","name":"agenthealth/metrics"}
2024-08-05T18:27:58Z D! {"caller":"[email protected]/exporter.go:273","msg":"Alpha component. May change in the future.","kind":"exporter","data_type":"metrics","name":"awscloudwatch"}
2024-08-05T18:27:58Z D! {"caller":"[email protected]/receiver.go:308","msg":"Stable component.","kind":"receiver","name":"telegraf_mem","data_type":"metrics"}
2024-08-05T18:27:58Z I! {"caller":"[email protected]/service.go:143","msg":"Starting CWAgent...","Version":"1.300041.1","NumCPU":2}
2024-08-05T18:27:58Z I! {"caller":"extensions/extensions.go:34","msg":"Starting extensions..."}
2024-08-05T18:27:58Z I! {"caller":"extensions/extensions.go:37","msg":"Extension is starting...","kind":"extension","name":"agenthealth/metrics"}
2024-08-05T18:27:58Z I! {"caller":"extensions/extensions.go:52","msg":"Extension started.","kind":"extension","name":"agenthealth/metrics"}
2024-08-05T18:27:58Z D! Fallback shared config file(s): [.aws/credentials]
2024-08-05T18:27:58Z D! Successfully created credential sessions
2024-08-05T18:27:58Z E! Failed to get credential from session: NoCredentialProviders: no valid providers in chain
caused by: EnvAccessKeyNotFound: failed to find credentials in the environment.
SharedCredsLoad: failed to load profile, .
EC2RoleRequestError: no EC2 instance role found
caused by: EC2MetadataError: failed to make EC2Metadata request
<?xml version="1.0" encoding="iso-8859-1"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Transitional//EN"
                 "http://www.w3.org/TR/xhtml1/DTD/xhtml1-transitional.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en" lang="en">
 <head>
  <title>404 - Not Found</title>
 </head>
 <body>
  <h1>404 - Not Found</h1>
 </body>
</html>

        status code: 404, request id: 
2024-08-05T18:27:58Z D! {"caller":"[email protected]/helper.go:18","msg":"Configured middleware on AWS client","kind":"exporter","data_type":"metrics","name":"awscloudwatch","middleware":"agenthealth/metrics"}
2024-08-05T18:27:58Z I! cloudwatch: get unique roll up list []
2024-08-05T18:27:58Z D! {"caller":"adapter/receiver.go:41","msg":"Starting adapter","kind":"receiver","name":"telegraf_mem","data_type":"metrics","receiver":"mem"}
2024-08-05T18:27:58Z I! {"caller":"[email protected]/service.go:169","msg":"Everything is ready. Begin running and processing data."}
2024-08-05T18:27:58Z W! {"caller":"localhostgate/featuregate.go:63","msg":"The default endpoints for all servers in components will change to use localhost instead of 0.0.0.0 in a future version. Use the feature gate to preview the new default.","feature gate ID":"component.UseLocalHostAsDefaultHost"}
2024-08-05T18:27:58Z I! cloudwatch: publish with ForceFlushInterval: 1m0s, Publish Jitter: 29.991730558s
2024-08-05T18:27:59Z D! [logagent] open file count, 0
2024-08-05T18:27:59Z D! {"caller":"adapter/receiver.go:61","msg":"Begin scraping metrics with adapter","kind":"receiver","name":"telegraf_mem","data_type":"metrics","receiver":"mem"}
2024-08-05T18:28:00Z D! [logagent] open file count, 0
2024-08-05T18:28:01Z D! [logagent] open file count, 0
2024-08-05T18:28:02Z D! [logagent] open file count, 0
2024-08-05T18:28:03Z D! [logagent] open file count, 0
2024-08-05T18:28:04Z D! [logagent] open file count, 0
2024-08-05T18:28:05Z D! [logagent] open file count, 0
2024-08-05T18:28:06Z D! [logagent] open file count, 0
2024-08-05T18:28:07Z D! [logagent] open file count, 0
2024-08-05T18:28:08Z D! [logagent] open file count, 0
2024-08-05T18:28:09Z D! [logagent] open file count, 0
2024-08-05T18:28:10Z D! [logagent] open file count, 0
2024-08-05T18:28:11Z D! [logagent] open file count, 0
2024-08-05T18:28:12Z D! [logagent] open file count, 0
2024-08-05T18:28:13Z D! [logagent] open file count, 0
2024-08-05T18:28:14Z D! [logagent] open file count, 0
2024-08-05T18:28:15Z D! [logagent] open file count, 0
2024-08-05T18:28:16Z D! [logagent] open file count, 0
2024-08-05T18:28:17Z D! [logagent] open file count, 0
2024-08-05T18:28:18Z D! [logagent] open file count, 0
2024-08-05T18:28:19Z D! [logagent] open file count, 0
2024-08-05T18:28:20Z D! [logagent] open file count, 0
2024-08-05T18:28:21Z D! [logagent] open file count, 0
2024-08-05T18:28:22Z D! [logagent] open file count, 0
2024-08-05T18:28:23Z D! [logagent] open file count, 0
2024-08-05T18:28:24Z D! [logagent] open file count, 0
2024-08-05T18:28:25Z D! [logagent] open file count, 0
2024-08-05T18:28:26Z D! [logagent] open file count, 0
2024-08-05T18:28:27Z D! [logagent] open file count, 0
2024-08-05T18:28:28Z D! [logagent] open file count, 0
2024-08-05T18:28:28Z I! CWAGENT_LOG_LEVEL is set to "DEBUG"

after my changes:

 2024-08-05T20:02:44Z D! Using credential [REDACTED] from EC2RoleProvider
2024-08-05T20:02:44Z I! {"caller":"[email protected]/service.go:182","msg":"Starting CWAgent...","Version":"1.300042.1-32-ge7225958-untracked","NumCPU":2}
2024-08-05T20:02:44Z I! {"caller":"extensions/extensions.go:34","msg":"Starting extensions..."}
2024-08-05T20:02:44Z I! {"caller":"extensions/extensions.go:37","msg":"Extension is starting...","kind":"extension","name":"agenthealth/metrics"}
2024-08-05T20:02:44Z I! {"caller":"extensions/extensions.go:52","msg":"Extension started.","kind":"extension","name":"agenthealth/metrics"}
2024-08-05T20:02:44Z D! Fallback shared config file(s): [/home/cwagent/.aws/credentials]
2024-08-05T20:02:44Z D! Successfully created credential sessions
2024-08-05T20:02:44Z D! Using credential [REDACTED]from EC2RoleProvider
2024-08-05T20:02:44Z I! cloudwatch: get unique roll up list []
2024-08-05T20:02:44Z I! {"caller":"ec2tagger/ec2tagger.go:415","msg":"ec2tagger: Check EC2 Metadata.","kind":"processor","name":"ec2tagger","pipeline":"metrics/host"}
2024-08-05T20:02:44Z I! cloudwatch: publish with ForceFlushInterval: 1m0s, Publish Jitter: 5.078351757s
2024-08-05T20:02:44Z D! Fallback shared config file(s): [/home/cwagent/.aws/credentials]
2024-08-05T20:02:44Z D! Successfully created credential sessions
2024-08-05T20:02:44Z D! Using credential [REDACTED] from EC2RoleProvider
2024-08-05T20:02:44Z I! {"caller":"ec2tagger/ec2tagger.go:333","msg":"ec2tagger: EC2 tagger has started initialization.","kind":"processor","name":"ec2tagger","pipeline":"metrics/host"}
2024-08-05T20:02:44Z I! {"caller":"[email protected]/service.go:208","msg":"Everything is ready. Begin running and processing data."}
2024-08-05T20:02:44Z W! {"caller":"localhostgate/featuregate.go:63","msg":"The default endpoints for all servers in components will change to use localhost instead of 0.0.0.0 in a future version. Use the feature gate to preview the new default.","feature gate ID":"component.UseLocalHostAsDefaultHost"}
2024-08-05T20:02:44Z W! {"caller":"ec2tagger/ec2tagger.go:465","msg":"ec2tagger: Unable to describe ec2 tags for initial retrieval","kind":"processor","name":"ec2tagger","pipeline":"metrics/host","error":"UnauthorizedOperation: You are not authorized to perform this operation. User: arn:aws:sts::956457624121:assumed-role/SSMDefaultRoleForPVREReporting/i-06aad6834bc794769 is not authorized to perform: ec2:DescribeTags because no identity-based policy allows the ec2:DescribeTags action\n\tstatus code: 403, request id: 0b615dc1-b942-4c17-b77d-84ffff632f4d"}
2024-08-05T20:02:45Z D! [logagent] open file count, 0
2024-08-05T20:02:46Z D! [logagent] open file count, 0
2024-08-05T20:02:47Z D! [logagent] open file count, 0
2024-08-05T20:02:48Z D! [logagent] open file count, 0
2024-08-05T20:02:49Z D! [logagent] open file count, 0
2024-08-05T20:02:50Z D! [logagent] open file count, 0
2024-08-05T20:02:51Z D! [logagent] open file count, 0
2024-08-05T20:02:52Z D! [logagent] open file count, 0
2024-08-05T20:02:53Z D! [logagent] open file count, 0
2024-08-05T20:02:54Z D! [logagent] open file count, 0
2024-08-05T20:02:55Z D! [logagent] open file count, 0
2024-08-05T20:02:56Z D! [logagent] open file count, 0
2024-08-05T20:02:57Z D! [logagent] open file count, 0
2024-08-05T20:02:58Z D! [logagent] open file count, 0
2024-08-05T20:02:59Z D! [logagent] open file count, 0
2024-08-05T20:03:00Z D! [logagent] open file count, 0
2024-08-05T20:03:01Z D! [logagent] open file count, 0
2024-08-05T20:03:02Z D! [logagent] open file count, 0
2024-08-05T20:03:03Z D! [logagent] open file count, 0
2024-08-05T20:03:04Z D! [logagent] open file count, 0
2024-08-05T20:03:05Z D! [logagent] open file count, 0
2024-08-05T20:03:06Z D! [logagent] open file count, 0
2024-08-05T20:03:07Z D! [logagent] open file count, 0
2024-08-05T20:03:08Z D! [logagent] open file count, 0
2024-08-05T20:03:09Z D! [logagent] open file count, 0
2024-08-05T20:03:10Z D! [logagent] open file count, 0
2024-08-05T20:03:11Z D! [logagent] open file count, 0
2024-08-05T20:03:12Z D! [logagent] open file count, 0
2024-08-05T20:03:13Z D! [logagent] open file count, 0
2024-08-05T20:03:14Z I! CWAGENT_LOG_LEVEL is set to "DEBUG"

Requirements

Before commit the code, please do the following steps.

Run make fmt and make fmt-sh
Run make lint

github-actions · 2024-07-20T00:09:34Z

This PR was marked stale due to lack of activity.

This reverts commit 1d3d2ef.

ZhenyuTan-amz · 2024-07-26T15:28:51Z

branch named with v102 but eventually bumped to v103. Is it just us trying to pulling in whatever is the latest? (just curious if it's fully intended, not against it at all)

okankoAMZ · 2024-07-29T15:59:40Z

branch named with v102 but eventually bumped to v103. Is it just us trying to pulling in whatever is the latest? (just curious if it's fully intended, not against it at all)

Initially it we were migrating to v102 but due to parity issue between v102.1 and v102.0 we have decided to bump to v103. I renamed the PR but did not recreate the branch

translator/translate/otel/exporter/debug/translator.go

cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go

…oudwatch-agent into otel-bump-v0.102.1

cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go

service/configprovider/provider.go

translator/tocwconfig/sampleConfig/advanced_config_darwin.yaml

cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go

go.mod

logger/logger.go

service/configprovider/provider.go

okankoAMZ changed the title ~~Otel bump v0.102.1~~ Upgrade Contrib to v0.102.1 Jul 11, 2024

okankoAMZ added 2 commits July 12, 2024 11:54

v0.98.0 to v0.102.1

1e7f5a6

Clean-up

29ab202

okankoAMZ force-pushed the otel-bump-v0.102.1 branch from b402a86 to 29ab202 Compare July 12, 2024 15:54

okankoAMZ added 2 commits July 12, 2024 11:57

Merge branch 'main' into otel-bump-v0.102.1

0952c6c

Clean up

b81435c

github-actions bot added the Stale label Jul 20, 2024

okankoAMZ force-pushed the otel-bump-v0.102.1 branch from 69ff0c6 to adf4ac1 Compare July 23, 2024 16:19

103 bump

8d7cf3a

okankoAMZ force-pushed the otel-bump-v0.102.1 branch from adf4ac1 to 8d7cf3a Compare July 23, 2024 16:49

okankoAMZ changed the title ~~Upgrade Contrib to v0.102.1~~ Upgrade Contrib to v0.103.0 Jul 23, 2024

okankoAMZ added 3 commits July 23, 2024 14:23

Fix lint errors

d7f0653

Fix lint errors

d89cd29

Fixed windows tests

4ac3e3b

github-actions bot removed the Stale label Jul 24, 2024

okankoAMZ and others added 5 commits July 24, 2024 12:02

Windows Translator fix

affa9c8

Updated unmarshler

1d3d2ef

Revert "Updated unmarshler"

592a86a

This reverts commit 1d3d2ef.

edited metric statement name

276a564

Merge branch 'main' into otel-bump-v0.102.1

0967c52

okankoAMZ marked this pull request as ready for review July 25, 2024 21:10

okankoAMZ requested a review from a team as a code owner July 25, 2024 21:10

jefchien reviewed Jul 29, 2024

View reviewed changes

translator/translate/otel/exporter/debug/translator.go Outdated Show resolved Hide resolved

cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go Outdated Show resolved Hide resolved

cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go Outdated Show resolved Hide resolved

musa-asad previously approved these changes Jul 30, 2024

View reviewed changes

okankoAMZ added 2 commits July 30, 2024 15:25

removed second setting creation

00e73c6

Merge branch 'otel-bump-v0.102.1' of https://github.com/aws/amazon-cl…

15ac90d

…oudwatch-agent into otel-bump-v0.102.1

okankoAMZ dismissed musa-asad’s stale review via 15ac90d July 30, 2024 19:25

Merge branch 'main' into otel-bump-v0.102.1

b7e928a

jefchien reviewed Jul 31, 2024

View reviewed changes

cmd/amazon-cloudwatch-agent/amazon-cloudwatch-agent.go Outdated Show resolved Hide resolved

service/configprovider/provider.go Outdated Show resolved Hide resolved

okankoAMZ added 2 commits July 31, 2024 17:26

addressed comments

444db20

changed default sampling to static

4e42bbe

okankoAMZ force-pushed the otel-bump-v0.102.1 branch from ba5852d to 4e42bbe Compare July 31, 2024 23:00

jefchien reviewed Aug 1, 2024

View reviewed changes

service/configprovider/provider.go Outdated Show resolved Hide resolved

jefchien reviewed Aug 1, 2024

View reviewed changes

translator/tocwconfig/sampleConfig/advanced_config_darwin.yaml Show resolved Hide resolved

okankoAMZ and others added 5 commits August 1, 2024 12:57

removed redundant error check

03fa986

switched to aws-cwa-dev for contrib usage

ee2387c

Merge branch 'main' into otel-bump-v0.102.1

3970292

fixed lint

2e63315

Merge branch 'main' into otel-bump-v0.102.1

a33f109

sky333999 previously approved these changes Aug 5, 2024

View reviewed changes

jefchien reviewed Aug 5, 2024

View reviewed changes

Addressed comments...

40d2b8f

okankoAMZ dismissed sky333999’s stale review via 40d2b8f August 5, 2024 19:42

Merge branch 'main' into otel-bump-v0.102.1

e24a216

okankoAMZ force-pushed the otel-bump-v0.102.1 branch from e722595 to e24a216 Compare August 5, 2024 20:07

jefchien approved these changes Aug 5, 2024

View reviewed changes

sky333999 approved these changes Aug 5, 2024

View reviewed changes

okankoAMZ merged commit f9beaec into main Aug 5, 2024
6 checks passed

okankoAMZ deleted the otel-bump-v0.102.1 branch August 5, 2024 20:33

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Upgrade Contrib to v0.103.0 #1247

Upgrade Contrib to v0.103.0 #1247

okankoAMZ commented Jul 11, 2024 •

edited

Loading

github-actions bot commented Jul 20, 2024

ZhenyuTan-amz commented Jul 26, 2024

okankoAMZ commented Jul 29, 2024

Upgrade Contrib to v0.103.0 #1247

Upgrade Contrib to v0.103.0 #1247

Conversation

okankoAMZ commented Jul 11, 2024 • edited Loading

Description of the issue

Description of changes

License

Tests

Requirements

github-actions bot commented Jul 20, 2024

ZhenyuTan-amz commented Jul 26, 2024

okankoAMZ commented Jul 29, 2024

okankoAMZ commented Jul 11, 2024 •

edited

Loading