Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prototype with Step Function #11

Open
jterry64 opened this issue Sep 24, 2019 · 1 comment
Open

Prototype with Step Function #11

jterry64 opened this issue Sep 24, 2019 · 1 comment
Assignees

Comments

@jterry64
Copy link
Member

Create a prototype using this lambda function with Step Function's Map State to see if it can work as intended (calling 100s-1000s in parallel with high performance) and to see if Step Function could work to connect the components of this service.

It seems like the Map State feature was only released a week ago, so this will also let us see if it's ready to use at our scale yet.

@jterry64 jterry64 self-assigned this Sep 24, 2019
@jterry64
Copy link
Member Author

jterry64 commented Oct 8, 2019

It works, although we need to do some workarounds with S3 (or possibly DynamoDB) to deal with the number of geometries we want to process. It also is not incredibly stable at this moment, and usually will just fail with an internal error if you try more than 50 lambdas at once.

Currently an open ticket on AWS support center discussing these issues.

@logoul logoul added this to the Improve on-the-fly Analyses milestone Nov 25, 2019
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants