Skip to content

Commit

Permalink
Add support for lambda package
Browse files Browse the repository at this point in the history
  • Loading branch information
nitin-bhadauria committed Nov 9, 2024
1 parent 8b50f29 commit 0f6cc03
Show file tree
Hide file tree
Showing 8 changed files with 254 additions and 7 deletions.
45 changes: 45 additions & 0 deletions .github/workflows/build_lambda_package.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
name: Build and Release Lambda Package

on:
release:
types: [published]

jobs:
build:
runs-on: ubuntu-latest

steps:
- name: Checkout code
uses: actions/checkout@v3

- name: Set up Python
uses: actions/setup-python@v4
with:
python-version: '3.12'

- name: Install dependencies
run: |
mkdir package
pip install -r lambda/requirements.txt -t package
- name: Copy source code
run: |
cp -r src/aws_resource_scheduler package/aws_resource_scheduler
cp lambda/lambda_function.py package/
# Uncomment if including config.yml
# cp config.yml package/
- name: Create zip package
run: |
cd package
zip -r ../aws_resource_scheduler_lambda_package.zip .
- name: Upload Release Asset
uses: actions/upload-release-asset@v1
env:
GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}
with:
upload_url: ${{ github.event.release.upload_url }}
asset_path: aws_resource_scheduler_lambda_package.zip
asset_name: aws_resource_scheduler_lambda_package.zip
asset_content_type: application/zip
105 changes: 105 additions & 0 deletions README-lambda.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,105 @@
# AWS Resource Scheduler Lambda Deployment

[![Build Status](https://github.com/cloudstaff-apps/aws-resource-scheduler/actions/workflows/build_lambda_package.yml/badge.svg)](https://github.com/cloudstaff-apps/aws-resource-scheduler/actions/workflows/build_lambda_package.yml)
[![Latest Release](https://img.shields.io/github/v/release/cloudstaff-apps/aws-resource-scheduler?style=flat-square)](https://github.com/cloudstaff-apps/aws-resource-scheduler/releases/latest)
[![PyPI Latest Release](https://img.shields.io/pypi/v/aws-resource-scheduler?style=flat-square)](https://pypi.org/project/aws-resource-scheduler/)

This Lambda function package allows you to deploy and execute the `aws-resource-scheduler` on AWS Lambda.

## Setup Instructions

1. **Deployment Package**: Download the latest [`aws_resource_scheduler_lambda_package.zip`](https://github.com/cloudstaff-apps/aws-resource-scheduler/releases/latest/download/aws_resource_scheduler_lambda_package.zip) from the GitHub release.

2. **Upload to AWS Lambda**:
- Go to the AWS Lambda Console.
- Create a new Lambda function (e.g., `aws-resource-scheduler`).
- Choose Python 3.12 runtime.
- Upload the `aws_resource_scheduler_lambda_package.zip` file as the Lambda code.
- Set the handler to `lambda_function.lambda_handler`.

3. **Environment Variables**:
- `CONFIG_FILE`: Path to the config file included in the Lambda package (e.g., `example/config-default-tags.yml`).
- `CONFIG_S3_BUCKET`: (Optional) S3 bucket name for configuration.
- `CONFIG_S3_KEY`: (Optional) Key within S3 bucket to fetch configuration from.
- `WORKSPACE`: Workspace name (e.g., `stage`).
- `RESOURCES`: Comma-separated list of resources to manage (e.g., `ec2,rds,asg,ecs,aurora`).
- `ACTION`: Action to perform (`start`, `stop`, `status`).
- `NO_WAIT`: Set to `true` or `false` (default: `false`).
- `THREADS`: Number of threads for execution (default: `10`).
- `LOG_LEVEL`: Logging level (e.g., `INFO`, `DEBUG`).

4. **Configuration Options**:
- You can use the default `config-default-tags.yml` included in the package or fetch the configuration file dynamically from S3 by setting `CONFIG_S3_BUCKET` and `CONFIG_S3_KEY`.
- Alternatively, create a local `config.yml` file and add it to the Lambda package using:
```bash
zip -u aws_resource_scheduler_lambda_package.zip config.yml
```

5. **CloudWatch Event Trigger**:
- Set up a CloudWatch Event rule to trigger the Lambda function on a schedule, such as every hour or daily, depending on your needs.

6. **Testing**:
- Create a test event in the AWS Lambda Console:
```json
{
"config_file": "example/config-default-tags.yml",
"workspace": "stage",
"resources": "ec2,rds,asg,aurora",
"action": "status",
"no_wait": true,
"threads": 10
}
```
- Invoke the function and check CloudWatch Logs for the output.

## Permissions

Ensure the Lambda function role has the necessary permissions assume role that are configured in the config:
```json
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Action": [
"sts:AssumeRole"
],
"Resource": "arn:aws:iam::123456789012:role/SchedulerRole"
},
{
"Effect": "Allow",
"Action": [
"logs:CreateLogGroup",
"logs:CreateLogStream",
"logs:PutLogEvents"
],
"Resource": "*"
}
]
}
```

And the role should allow lambda to assume

```json
{
"Version": "2012-10-17",
"Statement": [
{
"Sid": "Statement1",
"Effect": "Allow",
"Principal": {
"AWS": [
"arn:aws:sts::123456789012:assumed-role/aws_resource_scheduler-role-nxnytew9/aws_resource_scheduler"
]
},
"Action": "sts:AssumeRole"
}
]
}
```

## Notes

- If your configuration file changes frequently, using S3 for config storage is recommended.
- Make sure to adjust the IAM role permissions to the principle of least privilege.
7 changes: 5 additions & 2 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,9 @@
# AWS Resource Scheduler

[![Build Status](https://github.com/cloudstaff-apps/aws-resource-scheduler/actions/workflows/pip_publish.yml/badge.svg)](https://github.com/cloudstaff-apps/aws-resource-scheduler/actions/workflows/pip_publish.yml)
[![Latest Release](https://img.shields.io/github/v/release/cloudstaff-apps/aws-resource-scheduler?style=flat-square)](https://github.com/cloudstaff-apps/aws-resource-scheduler/releases/latest)
[![PyPI Latest Release](https://img.shields.io/pypi/v/aws-resource-scheduler?style=flat-square)](https://pypi.org/project/aws-resource-scheduler/)

AWS Resource Scheduler is an open-source Python module that automates the start and stop operations for various AWS resources, including EC2 instances, Auto Scaling Groups (ASG), ECS services, RDS databases, and Aurora clusters.

## Features
Expand Down Expand Up @@ -86,8 +90,7 @@ aws-resource-scheduler -f config-stage.yml -w stage -r ec2,asg,ecs -a start

### IAM Role and Permission

To securely interact with AWS resources, create an IAM role with the necessary permissions. Follow these steps:

To securely interact with AWS resources, create an IAM role with the necessary permissions listed below:

```json
{
Expand Down
21 changes: 21 additions & 0 deletions example/config-default-tags.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
workspaces:
stage:
aws_region: us-west-2
role_arn: arn:aws:iam::123456789012:role/SchedulerRole
notification:
enable: false
platform: google
webhook_url: https://chat.googleapis.com/v1/spaces/XXX/messages?key=YYY&token=ZZZ
ec2:
name:
tags:
scheduler: "true"
asg:
tags:
scheduler: "true"
rds:
tags:
scheduler: "true"
aurora:
tags:
scheduler: "true"
2 changes: 0 additions & 2 deletions example/config.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,8 +7,6 @@ workspaces:
platform: google
webhook_url: https://chat.googleapis.com/v1/spaces/XXX/messages?key=YYY&token=ZZZ
ec2:
name:
- i-005937bc14e7d576e
tags:
"Env": "Dev"
"Scheduler": "True"
Expand Down
69 changes: 69 additions & 0 deletions lambda/lambda_function.py
Original file line number Diff line number Diff line change
@@ -0,0 +1,69 @@
# lambda_function.py
import logging
import json
import os
import boto3
from aws_resource_scheduler.scheduler import main as scheduler_main

# Explicitly set the root logger level
log_level = os.environ.get('LOG_LEVEL', 'INFO')
logging.basicConfig(level=getattr(logging, log_level, logging.INFO), format='%(asctime)s - %(levelname)s - %(message)s')
logging.getLogger().setLevel(getattr(logging, log_level, logging.INFO))

def lambda_handler(event, context):
logging.info("Lambda handler started with log level: %s", log_level)
# Set up parameters
config_file = os.environ.get('CONFIG_FILE', 'example/config-default-tags.yml')
s3_bucket = os.environ.get('CONFIG_S3_BUCKET')
s3_key = os.environ.get('CONFIG_S3_KEY')

if s3_bucket and s3_key:
# Fetch config from S3
s3 = boto3.client('s3')
response = s3.get_object(Bucket=s3_bucket, Key=s3_key)
config_content = response['Body'].read().decode('utf-8')
with open('/tmp/config.yml', 'w') as f:
f.write(config_content)
config_file = '/tmp/config.yml'

# Get parameters from event or environment variables
config_file = os.environ.get('CONFIG_FILE', 'config.yml')
workspace = os.environ.get('WORKSPACE', 'default')
resources = os.environ.get('RESOURCES', 'ec2,rds,asg,ecs,aurora')
action = os.environ.get('ACTION', 'status')
no_wait = os.environ.get('NO_WAIT', 'true').lower() == 'true'
threads = int(os.environ.get('THREADS', '10'))

# If parameters are provided in the event, they override environment variables
config_file = event.get('config_file', config_file)
workspace = event.get('workspace', workspace)
resources = event.get('resources', resources)
action = event.get('action', action)
no_wait = event.get('no_wait', no_wait)
threads = event.get('threads', threads)

# Prepare arguments for the scheduler
class Args:
pass

args = Args()
args.file = config_file
args.workspace = workspace
args.resource = resources
args.action = action
args.no_wait = no_wait
args.threads = threads

# Run the scheduler
try:
scheduler_main(args)
return {
'statusCode': 200,
'body': json.dumps('Scheduler executed successfully.')
}
except Exception as e:
logging.exception(f"An error occurred: {e}")
return {
'statusCode': 500,
'body': json.dumps(f"An error occurred: {str(e)}")
}
3 changes: 3 additions & 0 deletions lambda/requirements.txt
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
boto3==1.28.76
PyYAML==6.0.1
requests==2.31.0
9 changes: 6 additions & 3 deletions src/aws_resource_scheduler/scheduler.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,15 +9,18 @@
from aws_resource_scheduler.utils.ecs import EcsModule
from aws_resource_scheduler.utils.common import parse_arguments, evaluate, aws_login, send_chat_notification, Storage, ParameterStoreStorage, DynamoDBStorage

# Set up the logger
logger = logging.getLogger(__name__)
if not logger.hasHandlers():
# Configure logging only if not already set (useful for CLI mode)
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

def main(args=None):
"""
Main function to parse arguments, evaluate configuration, and perform actions
on AWS resources such as EC2, ASG, RDS, Aurora, and ECS. Also sends notifications
to the specified chat platform (Google Chat, Slack, Teams) if enabled.
"""
# Setup logging
logging.basicConfig(level=logging.INFO, format='%(asctime)s - %(levelname)s - %(message)s')

if args is None:
# Parse command-line arguments and fetch configuration
args = parse_arguments()
Expand Down

0 comments on commit 0f6cc03

Please sign in to comment.