-
Notifications
You must be signed in to change notification settings - Fork 177
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add the InfluxDB Timestream connector #201
base: mainline
Are you sure you want to change the base?
Add the InfluxDB Timestream connector #201
Commits on Sep 25, 2024
-
Initial commit for the InfluxDB Timestream Connector.
Signed-off-by: forestmvey <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for cfce8f0 - Browse repository at this point
Copy the full SHA cfce8f0View commit details
Commits on Oct 1, 2024
-
Add pre-commit hook for secrets scanning
*Issue #, if available:* N/A. *Description of changes:* - A pre-commit hook has been added that uses `aws-secrets` in order to prevent secrets from being committed. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Configuration menu - View commit details
-
Copy full SHA for eea8a7c - Browse repository at this point
Copy the full SHA eea8a7cView commit details
Commits on Oct 7, 2024
-
*Issue #, if available:* N/A *Description of changes:* - SAM template `template.yml` added. - Asynchronous and synchronous invocation supported. - Documentation for how the connector can be deployed using the SAM template added. - `lambda_runtime` crate added. - `LambdaEvent<serde_json::Value>` used instead of `lambda_http::Request`, in order to support more types of requests, instead of just AWS service integrations. - Tests added for handling different kinds of `queryParameters` keys in requests. - DLQ implemented for asynchronous invocation. - Integration tests changed to check the newly returned `serde_json::Value` struct. - [x] Unit tests passed. - [x] Integration tests passed. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Configuration menu - View commit details
-
Copy full SHA for da989ef - Browse repository at this point
Copy the full SHA da989efView commit details
Commits on Oct 11, 2024
-
Add support for customer-defined partition keys
- Users can now define partition keys for their newly-created tables. Partition key configuration is controlled using three environment variables, `custom_partition_key_type`, `custom_partition_key_dimension`, and `enforce_custom_partition_key`. - Environment variables added for CDPK support. - SAM template parameters added for CDPK support. - Documentation added for CDPK support. - Integration tests added for CDPK support. - Integration test asserts moved to end of tests, in order to ensure resources are cleaned up. - [x] Unit tests passed. - [x] Integration tests passed.
Configuration menu - View commit details
-
Copy full SHA for 8c8bc2f - Browse repository at this point
Copy the full SHA 8c8bc2fView commit details -
*Issue #, if available:* N/A. *Description of changes:* - Deployment permissions have been updated according to the required permissions as discovered by testing deploying the connector using its SAM template. - `samconfig.toml` added with default stack deployment options. - "Troubleshooting" section added to the README with two known errors users may encounter. - Issue with cross-platform Rust compilation. - ConflictExceptions occurring with concurrent instances of the connector. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Configuration menu - View commit details
-
Copy full SHA for 3070ee6 - Browse repository at this point
Copy the full SHA 3070ee6View commit details -
Return 2.0 formatted Lambda responses for local invocation
- The environment variable `local_invocation` has been added and is set to `true` by default when the Lambda function is run with `cargo lambda watch`. This environment enables responses to be returned in a format `cargo lambda` expects, in the 2.0 formatting. Otherwise, the 1.0 format will be returned, which the synchronous API Gateway expects. - `cargo fmt` run. - [x] Tested locally. - [x] Tested with synchronous invocation. - [x] Tested with asynchronous invocation.
Configuration menu - View commit details
-
Copy full SHA for 996d3f7 - Browse repository at this point
Copy the full SHA 996d3f7View commit details
Commits on Oct 22, 2024
-
*Issue #, if available:* N/A. *Description of changes:* - Ingestion to multiple tables done in parallel. - Ingestion of 100 records to a single table done in parallel. - Chunking of records into batches of 100 done in parallel. - Limit on maximum possible number of threads added. - Print statement for each 100 records removed. - Logging option added to SAM template. - All `println!` calls changed to `info!`. - Default logging level for the connector changed to `INFO`. - Trace statements that measure the execution time of each function added. - Instructions added to README for how to configure logging levels. - Database and tables are no longer checked if their corresponding environment variables for creation are not set. - The checking of whether tables exist has been moved within the asynchronous code block used to ingest records to a table. This checking is now done in parallel and the hashmap is looped through once, instead of twice. - Instructions added to README for how to reduce stack costs. - [x] Integration tests passed. - [x] Unit tests passed. By submitting this pull request, I confirm that you can use, modify, copy, and redistribute this contribution, under the terms of your choice.
Configuration menu - View commit details
-
Copy full SHA for 0e6fdf7 - Browse repository at this point
Copy the full SHA 0e6fdf7View commit details
Commits on Oct 31, 2024
-
Add Single Table Mapping for InfluxDB Timestream Connector (#23)
* Initial implementation for adding single-table mapping. Signed-off-by: forestmvey <[email protected]> * Adding environment variables and updating documentation. Signed-off-by: forestmvey <[email protected]> * Adding tests and revising single-table mapping. Signed-off-by: forestmvey <[email protected]> * Adding single table example to README. Signed-off-by: forestmvey <[email protected]> * Fixing measure-name using line protocol metric name for single table mapping. Signed-off-by: forestmvey <[email protected]> * Fix typo in README. Signed-off-by: forestmvey <[email protected]> * Fixing documentation typos and test setting wrong environment variable. Signed-off-by: forestmvey <[email protected]> * Fixing table for single table multi measure records example in README. Signed-off-by: forestmvey <[email protected]> * Fix invalid line protocol examples with additional comma separating the timestamp. Signed-off-by: forestmvey <[email protected]> --------- Signed-off-by: forestmvey <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 4c51820 - Browse repository at this point
Copy the full SHA 4c51820View commit details
Commits on Nov 15, 2024
-
Multi-table Mapping Default for InfluxDB Timestream Connector (#25)
* Set default table mapping to multi-table for the InfluxDB Timestream Connector. Signed-off-by: forestmvey <[email protected]> * Fixing table formatting in README for InfluxDB Timestream Connector. Signed-off-by: forestmvey <[email protected]> * Updating README to reference multi-table for the default table mapping. Signed-off-by: forestmvey <[email protected]> --------- Signed-off-by: forestmvey <[email protected]>
Configuration menu - View commit details
-
Copy full SHA for 4487277 - Browse repository at this point
Copy the full SHA 4487277View commit details