Skip to content

Commit

Permalink
Update documentation and Setup (#285)
Browse files Browse the repository at this point in the history 
* Fix some parts of the readme

* Update documentation

* chore: Update documentation and fix some parts of the readme

* updated setup documentation

* updated readme files

* Update poetry

* Update quick start and readme

* updated naming restrictions.

* Added additional restrictions to sourrce_code limitation regarding reserved log levels and invoke_serverless_function levels.

* Updated metrics manager with workflow collector and region collector changes.

* Remote Client Caching and Benchmark Updates (#288)

* Updated benchmarks, and caching created remove clients.

* Removed override permission for benchmarks.

* Added a unit test for _get_remote_client

* linting

* Slightly improved benchmarks labeling.

* Caribou-Go: Moving Monte-Carlo Simulation to Go (#282)

* paper information added

* Update QUICK_START.md

* move docker installation to INSTALL.md

* minor refine documentations

* Added caching to client CLI (#295)

* Added caching to client CLI

* linting

* Also modified remove_function instance to also use the cached remote client.

* Disabled all deployment package upload to S3 and S3 table creation (#296)

* Disabled all deployment package upload to S3 and S3 table creation, fixed issue of deployment_manager crashing IF some region does not possess carbon data.

* Fixed small comment mistake.

* Patch integration test to still function with this change.

* Add contributing

* chore: Retry table creation on error

* Fix some last docu issues

---------

Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Danidite <[email protected]>
Co-authored-by: Daniel Long <[email protected]>
Co-authored-by: pjavanrood <[email protected]>
Co-authored-by: Mohammad Shahrad <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
Co-authored-by: Jerry Sun <[email protected]>
  • Loading branch information
@vGsteiger @pjavanrood
@Danidite @engshahrad @jerryyiransun
8 people committed Aug 21, 2024
1 parent f34ea08 commit ecd7ada
Show file tree
Hide file tree
Showing 19 changed files with 869 additions and 229 deletions.
106 changes: 106 additions & 0 deletions CONTRIBUTING.md
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,106 @@
# Contributing to Caribou

Thank you for your interest in contributing to Caribou! We’re excited to have you join our community of developers and researchers working together to improve serverless workflows on AWS. This document will guide you through the process of contributing to the project.

## Code of Conduct

We are committed to creating a welcoming and inclusive environment for everyone. Please be respectful, professional, and collaborative when engaging in discussions, reviewing code, and contributing to the project. Any form of harassment or discrimination will not be tolerated.

## How to Contribute

### 1. Reporting Issues

If you encounter a bug, have a feature request, or want to share ideas for improvement, feel free to open an issue in the repository.
Feel free to use the issue templates provided to ensure that your issue is well-structured and easy to understand.
Please ensure that your issue is clear and provides as much detail as possible.

- **Bug Reports**: Provide a detailed description of the issue, steps to reproduce it, and any relevant logs or error messages.
- **Feature Requests**: Describe the feature or enhancement you'd like to see and explain why it would be beneficial.

### 2. Working on Issues

Before you start working on an issue:

- **Assign it to yourself**: This helps avoid duplication of effort and allows others to see that the issue is being actively worked on.
If the issue is not assigned, feel free to claim it.
- **Communication**: If you have any questions or need clarification, don’t hesitate to start a discussion in the issue thread.

### 3. Submitting Pull Requests (PRs)

When you are ready to submit a Pull Request (PR):

- **Assign the PR to yourself**: This indicates that you are the primary author of the changes.
- **Request Reviews**: Ask for at least one approval from a project maintainer or a contributor.
Ensure your code follows the project's guidelines and passes all tests.
- **Do Not Close Comments**: When receiving feedback, do not close comments on the PR yourself.
Leave it to the commenter to decide whether their concerns have been addressed and resolved.
- **Add a good description**: Provide a clear and concise description of the changes made in the PR.
Mention any relevant issues or PRs that are related to your changes.
If an issue is being addressed and resolved, include `Closes #issue-number` in the PR description.
- **Squash Commits**: If your PR has multiple commits, always squash them into a single commit before merging.
GitHub provides an option to squash commits when merging a PR.
This also reduces the need for overly verbose commit messages and keeps the commit history clean.

### 4. Code Style

Please follow the existing code style used in the repository.
We have automated checks in place to ensure that the code adheres to the style guidelines.
Please always run the linting and formatting checks before submitting your PR using the `./scripts/compliance.sh` script.
Ensure that your code is:

- **Readable**: Write code that is easy to understand and maintain.
- **Documented**: Add comments where necessary to explain complex logic or workflows.
- **Tested**: Write tests for new features and bug fixes. Ensure that existing tests pass before submitting your PR.

### 5. Be Friendly and Welcoming

We value a positive and collaborative atmosphere. Always be friendly, patient, and considerate towards others.
If you are reviewing code or responding to an issue, provide constructive feedback and help contributors learn and grow.

## Getting Started

To get started with contributing to Caribou, follow these steps:

1. **Fork the repository**: Create a personal copy of the repository by forking it.
2. **Clone the repository**: Clone your fork to your local machine:

```bash
git clone https://github.com/your-username/caribou.git
cd caribou
```

3. **Install dependencies**: Ensure you have the necessary dependencies installed by following the instructions in the [`INSTALL.md`](./INSTALL.md) file.
4. **Create a branch**: Create a new branch for your work:

```bash
git checkout -b username/type-of-work/short-description
```

The type of work can be `feature`, `bugfix`, `refactor`, etc.

5. **Make your changes**: Work on your changes in your branch. Make sure to test them thoroughly.
6. **Push your changes**: Once you are happy with your changes, push them to your forked repository:

```bash
git push origin username/type-of-work/short-description
```

7. **Open a PR**: Open a Pull Request against the main branch, following the PR submission guidelines mentioned above.

## Review Process

- PRs will be reviewed by maintainers or contributors who have expertise in the relevant area.
- Feedback will be provided, and changes may be requested before approval.
- Once approved by at least one reviewer, the PR will be merged by a maintainer.

## Questions and Discussions

If you have any questions or ideas, feel free to reach out through the issue tracker or by starting a discussion. We encourage open communication and collaboration!

Thank you for contributing to Caribou. Your contributions help make the project stronger and more impactful.

Happy coding!

---

**Caribou Maintainers**
71 changes: 59 additions & 12 deletions INSTALL.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -37,36 +37,83 @@ Run any file with the following command:
poetry run <executable> <args>
```

## Why do we use poetry?
### Why do we use poetry?

Poetry is a tool for dependency management and packaging in Python.
It allows you to declare the libraries your project depends on, and it will manage (install/update) them for you.

You can either always use `poetry run` before running a file or activate the virtual environment created by poetry.
We always have to use `poetry run` because poetry creates a virtual environment for the project.
You can alternatively find the virtual environment path using `poetry env info` and activate it with `source <path>/bin/activate`, removing the need for `poetry run`.
This means all the dependencies are installed in a virtual environment, not your global one.
If you execute a file without `poetry run`, it will not find the dependencies as they are most likely not installed in your global environment.

For more information, see the [poetry documentation](https://python-poetry.org/docs/).

## Install dependencies

To install the dependencies, run the following command:

```bash
poetry install
```

This will install all the dependencies required to run the framework. To check the dependencies, you can run:

```bash
poetry show
```

To open a shell with the dependencies installed, you can run:

```bash
poetry shell
```

If you opened a shell, you can run all the caribou commands without the `poetry run` prefix.

```bash
caribou --help
```

## AWS Account Access

To run the framework, you need an AWS account and the necessary permissions to create and manage the required resources.
The `docs/iam_policy.json` file lists the required permissions for any user wanting to interact with a deployed framework.
In [IAM Policies](docs/iam_policies.md) we list the required permissions for any user wanting to interact with a deployed framework.

## Setup AWS Environment
The fastest way to set up the necessary permissions is to [create a new AWS user](https://docs.aws.amazon.com/IAM/latest/UserGuide/id_users_create.html) under your account with the necessary permissions and use the access key and secret key to [login the AWS CLI](https://docs.aws.amazon.com/signin/latest/userguide/command-line-sign-in.html) of this user to interact with the framework.

### Setup AWS Environment

First of all, make sure to have [AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/getting-started-install.html) installed.
To set up the required tables in AWS required for the framework to run, you can use the following command:

```bash
caribou setup_tables
poetry run caribou setup_tables
```

**Note:** The bucket that Caribou uses to store the resources (a feature for future provider compatibility) needs to be manually created.
Since AWS bucket names need to be unique, the currently configured bucket might already exist and be used by another version of the framework deployed somewhere else.
In this case, adapt the bucket name for the variable `DEPLOYMENT_RESOURCES_BUCKET` in the `caribou/common/constants.py` file.

##  Installing dependencies for the framework components
## Docker

The Deployment Utility has an additional dependency on `docker`.
To install it, follow the instructions on the [docker website](https://docs.docker.com/engine/install/).
Ensure you have the docker daemon running before running the deployment utility.

To verify that Docker is installed correctly, you can try running:

```bash
docker --version
```

## Google Maps API Key

For the server side component, more specifically the data collectors, we use the Google Maps Geocoding API to resolve the location of the data centers.
To use this API, you need to have a Google Maps API key.
You can get one by following the instructions on the [Google Maps Platform](https://developers.google.com/maps/documentation/geocoding/get-api-key) website.

## Electricity Map API Key

For the server side component, more specifically the data collectors, we use the Electricity Map API to get the carbon intensity of the electricity in the regions.
To use this API, you need to have an Electricity Map API key.
You can get one by following the instructions on the [Electricity Map](https://api-portal.electricitymaps.com) website.

## Other dependencies

Since the AWS lambda environment restricts us from using Docker, we have to deploy the workflows using [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane). For the following step to work, please install the crane as described in the [crane documentation](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md).
Since the AWS lambda environment restricts us from using Docker, we have to migrate the workflows using [crane](https://github.com/google/go-containerregistry/tree/main/cmd/crane). If you plan on running the framework locally instead of deploying it to the cloud, please install the crane as described in the [crane documentation](https://github.com/google/go-containerregistry/blob/main/cmd/crane/README.md).
Loading

0 comments on commit ecd7ada

Please sign in to comment.