DynamoDB WCU is not utilized

[fryce]

Dears:

First I have to say, this is a really helpful tool.

However when I used it to copy my source table to my target table, on CloudWatch I found that only 25 WCU was consumed per second. (Both my source and target tables have more than 150 RCU and more than 1000 WCU)

I am assuming the limitation of 25 request per BatchWriteItem API was met. Is it as expected?

Kind Regard :')

[abbathaw]

I noticed the same thing from the metrics that its using on average 55 WCU. I have a table with over 60k items around 11kb per item size, and its taking at least 10 minutes to copy the first 3k items even though I've defined 1000 WCU.

For now, I ended up testing using this AWS Labs library (its in Java though). It performed 68k items in 18 mins with 100 RCU and 1000 WCU.

[enGMzizo]

I believe the library needs to be optimized for some cases to support parallel writing to take full capacity of the selected WCU.
I will mark this issue as feature and will be implemented in the next version of this lib.

[asuresh26]

This tool is exactly what I need - I have close to 500K records to be migrated and would be very helpful if this tool took into consideration the source and target Read/Write capacities.
Do you have an idea when you will be able to add that feature?

[enGMzizo]

@asuresh26 still trying to find time for updating this repo.

[asuresh26]

@enGMzizo - Thanks much for this amazing tool. I migrated 500k and was hoping to scale up to a few millions if performance was good.
The 500K took 45 mins with 200 RCU and 1000 WCU.
Looking forward to this feature.

[shamilasallayAuxenta]

I set the minimum RCU and WCU to 1000 and it worked for me