Skip to content

Latest commit

 

History

History
117 lines (87 loc) · 7.35 KB

README.md

File metadata and controls

117 lines (87 loc) · 7.35 KB
Raw

Cloudwatch Receiver

Status
Stability alpha: logs
Distributions contrib
Issues Open issues Closed issues
Code Owners @schmikei | Seeking more code owners!
Emeritus @djaglowski

Receives Cloudwatch events from AWS Cloudwatch via the AWS SDK for Cloudwatch Logs

Getting Started

This receiver uses the AWS SDK as mode of authentication, which includes Credentials File and IMDS authentication for EC2 instances.

Configuration

Top Level Parameters

Parameter Notes type Description
region required string The AWS recognized region string
profile optional string The AWS profile used to authenticate, if none is specified the default is chosen from the list of profiles
imds_endpoint optional string A way of specifying a custom URL to be used by the EC2 IMDS client to validate the session. If unset, and the environment variable AWS_EC2_METADATA_SERVICE_ENDPOINT has a value the client will use the value of the environment variable as the endpoint for operation calls.
logs optional Logs Configuration for Logs ingestion of this receiver

Logs Parameters

Parameter Notes type Description
poll_interval default=1m duration The duration waiting in between requests.
max_events_per_request default=50 int The maximum number of events to process per request to Cloudwatch
groups optional See Group Parameters Configuration for Log Groups, by default all Log Groups and Log Streams will be collected.

Group Parameters

autodiscover and named are ways to control and filter which log groups and log streams which are collected from. They are mutually exclusive and are incompatible to be configured at the same time.

  • autodiscover
    • limit: (optional; default = 50) Limits the number of discovered log groups. This does not limit how large each API call to discover the log groups will be.
    • prefix: (optional) A prefix for log groups to limit the number of log groups discovered.
      • if omitted, all log streams up to the limit are collected from
    • streams: (optional) If streams is omitted, then all streams will be attempted to retrieve events from.
      • names: A list of full log stream names to filter the discovered log groups to collect from.
      • prefixes: A list of prefixes to filter the discovered log groups to collect from.
  • named
    • This is a map of log group name to stream filtering options
      • streams: (optional)
        • names: A list of full log stream names to filter the discovered log groups to collect from.
        • prefixes: A list of prefixes to filter the discovered log groups to collect from.

Autodiscovery Example Configuration

awscloudwatch:
  region: us-west-1
  logs:
    poll_interval: 1m
    groups:
      autodiscover:
        limit: 100
        prefix: /aws/eks/
        streams:
          prefixes: [kube-api-controller]

Named Example

awscloudwatch:
  region: us-west-1
  logs:
    poll_interval: 5m
    groups:
      named:
        /aws/eks/dev-0/cluster: 
          names: [kube-apiserver-ea9c831555adca1815ae04b87661klasdj]

Sample Configs

This receiver has a number of sample configs for reference.

  1. Default

    • Minimal configuration of the receiver
    • Performs autodiscovery
    • Collects all log groups and log streams

  2. Autodiscover Filtering Log Groups

    • Performs autodiscovery
    • Only collects log groups matching a prefix
    • Limits the number of discovered Log Groups

  3. Autodiscover Filtering Log Streams

    • Performs autodiscovery for all Log Groups
    • Filters log streams

  4. Named Groups

    • Specifies and only collects from the desired Log Groups
    • Does not attempt autodiscovery

  5. Named Groups Filter Log Streams

    • Specifies the names of the log groups to collect
    • Does not attempt autodiscovery
    • Only collects from log streams matching a prefix