Optimize resource utilization (EC2, LBs, Network and RDS) for RWB

[himeshr]

Optimize resource utilization (EC2, LBs, Network and RDS) for RWB.

Step 1:

Perform analysis on cost saving as well as rsource utilization optimizations that could be done

• Instance type changes for resource utilization optimizations
• Reserve instances for billing discounts
• Discarding redundant instances
• etc..

Step 2:

Submit recommendations with information on

• change
• impact (postivie / negative)
• cost-savings
• Action-plan and any follow-up activity needed

We'll review the recommendations and provide approval for specific changes

Step 3:

Implement approved changes. Perform any other follow-up activity needed, which was identified during analysis.

[Balamuruganjeevi]

1. EC2 Instance Type Changes:

Change: After my analyses based on our current EC2 instance types and their utilization metrics. Consider switching to instance types family from T3 to T3a because both offers same performance the only difference is their processors. While T3 works on Intel and T3a works on AMD and i am not sure our application works flawlessly in AMD need to test and it offers lower cost than T3 type instances and performance is also offered same comopared to Intel, However it depends on our workload it is my consideration.

• If we are not decided to change instance types to t3a family, we can change jasper-server instance type to t3.small as my recommendation.

Impact: Positive impact on resource efficiency by aligning instances with workload demands. However, If its not fulfill our performance and worload and other needs may negatively affect performance. Need to test it if we decided to change type.

Cost-Savings: By using T3a family we can save upto 10% to 20% of cost.

Action Plan:
Need to regularly monitor instance metrics.
We can Perform certain load testing first to validate performance after instance type changes.
And by analysing regularly in Future our needs increases have to change their types again to fulfill the requirements.

Follow-Up: Need to regularly monitor and adjust instance types based on changing workload demands.

2. RDS Instance Optimization:

Change: After my analyses based on our current RDS instance types and their utilization metrics. Consider switching to instance types family from T3 to T4g like EC2 also the performance and all are same the only diffreence is their processors while T3 is works under Intel and T4g works under AWS Graviton both offers same performance. And i also nit sure about the performance and can work flawlessly under this processor and It offers lower cost than T3 type instances and performance is also offered same comopared to Intel, However it depends on our workload it is my consideration.

Impact: Positive impact is cost savings where negative impact depends on our workloads we have to implement and check first to know about the right performance.

Cost-Savings: Definitely a cost savings from 10% to 20% on the changes.

Action Plan:
Need to regularly Monitor RDS performance metrics after changes happens.
Regularly Analyze database workload patterns to determine optimal instance sizes and storage types.
And analysing regularly in Future workloads changes have to change their types again also.

Follow-Up: After changing Instance types have to regularly review and adjust RDS instance configurations when needed based on our workload and database performance.

3. Reserve Instances for Billing Discounts:

Change: Purchase Reserved Instances (RIs) for EC2 and RDS is a better way to reduce cost. where if we decided to change the instance types as mentioned above and need to again review about the RI instance types to works cost effectively, And we are not decided to change instance types then we can use RI instances as mentioned below

• And we can purchase Reserved Instances for better cost optimisation for EC2 we can purchase 16 t3.nano RI's which will give better cost savings.

• We can purchase Reserved instances for RDS as it will reduce the cost more than on demand, we can purchase 7 db.t3.micro RI's to meet requirements of the instances.

Impact: Positive impact on cost savings through discounted pricing.

Cost-Savings: For EC2 it will give 30% to 40% cost savings and for RDS it gives 20% to 30% cost savings compared to On-Demand pricing.

Action Plan:
Need to Analyze historical EC2 usage to identify instances suitable for reservations.
Utilize AWS Cost Explorer to forecast RI purchases.

Follow-Up: Regularly review RI coverage and adjust reservations based on changing usage patterns.

And however above all are just the recommendations to change the type of the instances, If we need better confirmation we need to implement it and testing it.

[himeshr]

@Balamuruganjeevi Have ready through your recommendations in the earlier comment.

• Regarding the action-items that recommend switching from Intel Processor instance to AWS Graviton ARM Processor instance types, there are technical considerations with using them for Dockerization and / or some other package installations, which we had encountered sometime ago. Therefore, let us disregard these action-items related to change in Instance types, for now.
• Regarding creation of RI for for EC2 and RDS, please go ahead and book the instances for 1 year, with no upfront payment option, for as per your recommendations.

Review Summary:

1. EC2 Instance Type Changes to AWS Graviton or AMD: //No go
2. RDS Instance Optimization to AWS Graviton or AMD: //No go
3. Reserve Instances for Billing Discounts: //Proceed

Further optimization recommendations:

1. Explore right-sizing of EC2 / RDS instances to other Intel Processor instance types based on Performance and usage metrics

[Balamuruganjeevi]

@himeshr Please see below comments:

For EC2 we are already in a cost efficient instance type based on our utilisation for Prod and staging servers and have some recommendations are here below

• we can change jasper-server instance type from t2.medium to t3.small, it will reduce little amount of cost and it will not impact our performance for now but we need to be regularly monitor to avoid any latency.

• For staging-etl-server recommended to reduce the allocated storage from 50GB to 20GB as there is more free space available and will reduce some small amount of cost.

• same for prod-etl-server and prod-avni-server recommended to reduce the storage to 20GB as also there is more free space available

• If you people think storage is not necessary to reduce and we need that allocated storage as it is then we can leave it.

• RDS we already have our more cost efficient instance type, so no recommendation on that.

[1t5j0y]

• Go ahead for jasper-server instance type change to t3.small. We will need to keep an eye if this causes issues. For the jasper instance on Avni cloud, we were often dealing with memory issues but this doesn't seem to be the case on RWB since it is used by a single org.
• Don't change allocated storage size for any of the servers since the application has storage needs that will increase over time and the cost benefit is very low.

[Balamuruganjeevi]

• Changed jasper-server instance type from t2.medium to t3.small and it works as expected, will regularly monitorize metrics and will aware on any performance break down.

• Purchased RI's for EC2 instances and RDS instances as we discussed earlier.

Activity pending:

• Requested to increase EC2 RI's Service Quota and once that completed will purchase again some RI for our current usage. NOw we have only limit of 20 RI per month.

[1t5j0y]

@Balamuruganjeevi why do we need so many RIs when we don't have these many instances?

[Balamuruganjeevi]

@1t5j0y Because we have overall of memory usage is 12 GB and we used to create RI with t3.nano and its memory storage is only 0.5 GB so to cover all instances as Reserved we have to purchase 24 RI with t3.nano instance type, so our current service quota limit is only 20 and i have purchased RI with 20 instances and still we need additional 4 instances to cover all on demand instances.

[1t5j0y]

ok. We could have purchased higher RIs for larger instances than t3.nano to avoid this.

[Balamuruganjeevi]

Ok, then decided to modify RWB EC2 RI to 12 t3.micro instances. Since there is only minimal amount of cost difference, So i think it will works.