Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Q] How to increase kinesis_firehose PutRecordBatch size ? #2

Open
azman0101 opened this issue Jan 14, 2020 · 0 comments
Open

[Q] How to increase kinesis_firehose PutRecordBatch size ? #2

azman0101 opened this issue Jan 14, 2020 · 0 comments

Comments

@azman0101
Copy link

Our average PutBatchRecord size is 1Ko bytes.

D, [2020-01-14T10:16:30.981466 #1] DEBUG -- : [Aws::Firehose::Client 200 0.033437 0 retries] put_record_batch(delivery_stream_name:"kxxxGe" ... (1429 bytes)>}])

Regarding to AWS FAQ:
https://aws.amazon.com/kinesis/data-firehose/faqs/#Pricing_and_billing

Q: When I use PutRecordBatch operation to send data to Amazon Kinesis Data Firehose, how is the 5KB roundup calculated?

The 5KB roundup is calculated at the record level rather than the API operation level. For example, if your PutRecordBatch call contains two 1KB records, the data volume from that call is metered as 10KB. (5KB per record)

We are not optimizing our costs.

I don't see which kinesis_firehose plugin option can help to increate PutRecordBatch size.

Regards,

JB

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant