Skip to content

Commit

Permalink
Merge branch 'features/aoe' of https://github.com/helderpinto/finops-…
Browse files Browse the repository at this point in the history
…toolkit-hp-fork into features/aoe
  • Loading branch information
Helder Pinto committed Apr 12, 2024
2 parents 145d6bb + 6782465 commit 24c2fbd
Show file tree
Hide file tree
Showing 4 changed files with 20 additions and 2 deletions.
6 changes: 6 additions & 0 deletions docs/optimization-engine/configuring-workspaces.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,8 @@ Include the VM performance logs available in your Log Analytics workspaces to ge

</details>

---

## 🔢 Configuring performance counters

If you want to fully leverage the VM right-size augmented recommendation, you need to have your VMs sending logs to a Log Analytics workspace (it should normally be the one you chose at AOE installation time, but it can be a different one) and you need them to send specific performance counters. The list of required counters is defined in the `perfcounters.json` file (available in the [AOE root folder](https://aka.ms/AzureOptimizationEngine/code)). AOE provides a couple of tools that help you validate and fix the configured Log Analytics performance counters, depending on the type of agent you are using to collect logs from your machines.
Expand Down Expand Up @@ -75,6 +77,8 @@ Install-Module -Name Az.OperationalInsights
./Setup-LogAnalyticsWorkspaces.ps1 -AutoFix -WorkspaceIds "d69e840a-2890-4451-b63c-bcfc5580b90f","961550b2-2c4a-481a-9559-ddf53de4b455" -IntervalSeconds 30
```

<br>

## 💰 Additional performance logs cost estimation

Each performance counter entry in the `Perf` table has different sizings, depending on the 7 required counters per OS type. The following table enumerates the size (in bytes) per performance counter entry.
Expand All @@ -98,6 +102,8 @@ Linux | Network | Total Bytes | 200 | network adapters count

In summary, a Windows VM generates, in average, 245 bytes per performance counter entry, while a Linux consumes a bit less, 230 bytes per entry. However, depending on the number of CPU cores, data disks or network adapters, a VM will generate more or less Log Analytics entries. For example, a Windows VM with 4 vCPUs, 1 data disk and 5 network adapters will generate 5 * 200 + 220 + 4 * 250 + 4 * 250 + 4 * 250 + 4 * 250 + 5 * 290 = 6670 bytes (6.5 KB) per collection interval. If you set your Performance Counters interval to 60 seconds, then you'll have 60 * 24 * 30 * 6.5 = 280800 KB (274 MB) of ingestion data per month, which means less than 0.70 EUR/month at the Log Analytics ingestion retail price (Pay As You Go).

<br>

## 🎯 Using multiple workspaces for performance logs

If you have VMs onboarded to multiple Log Analytics workspaces and you want them to be fully included in the VM right-size recommendations report, you can add those workspaces to the solution just by adding a new variable to the AOE Azure Automation account. In the Automation Account _Shared Resources - Variables_ menu option, click on the _Add a variable button_ and enter `AzureOptimization_RightSizeAdditionalPerfWorkspaces` as the variable name and fill in the comma-separated list of workspace IDs (see example below). Finally, click on _Create_.
Expand Down
12 changes: 10 additions & 2 deletions docs/optimization-engine/customizing-aoe.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ parent: Optimization Engine
title: Customizations
nav_order: 20
description: 'Customize the Azure Optimization Engine settings according to your organization requirements.'
permalink: /optimization-engine/customizing-aoe
permalink: /optimization-engine/customize
---

<span class="fs-9 d-block mb-4">Customizations</span>
Expand All @@ -21,12 +21,16 @@ Customize the Azure Optimization Engine settings according to your organization

</details>

---

## 🧿 Widen the engine scope

By default, the Azure Automation Managed Identity is assigned the Reader role only over the respective subscription. However, you can widen the scope of its recommendations just by granting the same Reader role to other subscriptions or, even simpler, to a top-level Management Group.

In the context of augmented VM right-size recommendations, you may have your VMs reporting to multiple workspaces. If you need to include other workspaces - besides the main one AOE is using - in the recommendations scope, you just have to add their workspace IDs to the `AzureOptimization_RightSizeAdditionalPerfWorkspaces` variable (see more details in [Configuring workspaces](./configuring-workspaces.md)).

<br>

## ⏰ Adjust schedules

By default, the base time for the AOE Automation schedules is set as the deployment time. Soon after the initial deployment completes, the exports, ingests and recommendations runbooks will run according to the engine's default schedules. For example, if you deploy AOE on a Monday at 11:00 a.m., you will get new recommendations every Monday at 2:30 p.m.. If this schedule, for some reason, does not fit your needs, you can reset it to the time that better suits you, by using the `Reset-AutomationSchedules.ps1` script (available in the [AOE root folder](https://aka.ms/AzureOptimizationEngine/code)). You just have to call the script following the syntax below and answer the input requests:
Expand All @@ -39,6 +43,8 @@ The base time you choose must be in UTC and must be defined according to the wee

The script will also ask you to enter, **if needed**, the Hybrid Worker Group you want the runbooks to run in (see the next sub-section).

<br>

## 🦹 Scale AOE runbooks with Hybrid Worker

By default, AOE Automation runbooks are executed in the context of the Azure Automation sandbox. If you face performance issues due to the memory limits of the Automation sandbox or decide to implement private endpoints for the Storage Account or SQL Database, to harden AOE's security, you will need to execute runbooks from a Hybrid Worker (an Azure or on-premises Virtual Machine with the Automation Hybrid Worker extension). To change the execution context for the AOE runbooks, you must use the `Reset-AutomationSchedules.ps1` script. See how to use the script in the previous sub-section and, after setting the runbooks execution base time, enter the Hybrid Worker Group name you want the runbooks to run in.
Expand All @@ -48,9 +54,11 @@ By default, AOE Automation runbooks are executed in the context of the Azure Aut
* Once you change the runbook execution context to Hybrid Worker, you will have to always use the `DoPartialUpgrade` flag whenever you upgrade AOE, or else you will lose the runbook schedule settings and revert to the default sandbox configuration.
* The Managed Identity used to authenticate against Azure, Microsoft Entra ID and Billing Account scopes is still the Azure Automation's one, even if the Hybrid Worker machine has a Managed Identity assigned ([see details](https://learn.microsoft.com/en-us/azure/automation/automation-hrw-run-runbooks?#runbook-auth-managed-identities)). User-assigned Managed Identities are supported in the context of Hybrid Workers only if 1) the Automation Account does not have any associated Managed Identity, i.e., only the Hybrid Worker machine can have a User-Assigned Managed Identity; 2) all runbooks run in the context of the Hybrid Worker. In this case, you must create an `AzureOptimization_UAMIClientID` Automation Variable with the User-Assigned Managed Identity Client ID as value; and 3) the `AzureOptimization_AuthenticationOption` Automation variable value is updated to `UserAssignedManagedIdentity`.

<br>

## 🚥 Adjust thresholds

For Advisor Cost recommendations, the AOE's default configuration produces percentile 99th VM metrics aggregations, but you can adjust those to be less conservative. There are also adjustable metrics thresholds that are used to compute the fit score. The default thresholds values are 30% for CPU (5% for shutdown recommendations), 50% for memory (100% for shutdown) and 750 Mbps for network bandwidth (10 Mbps for shutdown). All the adjustable configurations are available as Azure Automation variables. The list below is a highlight of the most relevant configuration variables. To access them, go to the Automation Account _Shared Resources - Variables_ menu option.
For Advisor cost recommendations, the AOE's default configuration produces percentile 99th VM metrics aggregations, but you can adjust those to be less conservative. There are also adjustable metrics thresholds that are used to compute the fit score. The default thresholds values are 30% for CPU (5% for shutdown recommendations), 50% for memory (100% for shutdown) and 750 Mbps for network bandwidth (10 Mbps for shutdown). All the adjustable configurations are available as Azure Automation variables. The list below is a highlight of the most relevant configuration variables. To access them, go to the Automation Account _Shared Resources - Variables_ menu option.

Variable | Description
--- | --- |
Expand Down
2 changes: 2 additions & 0 deletions docs/optimization-engine/faq.md
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,8 @@ permalink: /optimization-engine/faq
All the frequently asked questions about AOE in one place.
{: .fs-6 .fw-300 }

---

* **Is AOE supported by Microsoft?** No, the Azure Optimization Engine is not supported under any Microsoft standard support program or service. The scripts are provided AS IS without warranty of any kind. The entire risk arising out of the use or performance of the scripts and documentation remains with you.

* **What type of Azure subscriptions/clouds are supported?** AOE has been deployed and tested against EA, MCA and MSDN subscriptions in the Azure commercial cloud (AzureCloud). Although not tested yet, it should also work in MOSA subscriptions. It was designed to also operate in the US Government cloud, though it was never tested there. Sponsorship (MS-AZR-0036P and MS-AZR-0143P), CSP (MS-AZR-0145P, MS-AZR-0146P, and MS-AZR-159P) DreamSpark (MS-AZR-0144P) and Internal subscriptions should also work, but due to lack of availability or disparities in their consumption (billing) exports models, some of the Workbooks may not fully work.
Expand Down
2 changes: 2 additions & 0 deletions docs/optimization-engine/reports.md
Original file line number Diff line number Diff line change
Expand Up @@ -26,6 +26,8 @@ Visualize the Azure Optimization Engine rich recommendations and insights.

</details>

---

## 📒 Power BI recommendations report

AOE includes a [Power BI report](http://aka.ms/AzureOptimizationEngine/powerbi) for visualizing recommendations. To use it, you have first to change the data source connection to the SQL Database you deployed with the AOE. In the Power BI top menu, choose Transform Data > Data source settings.
Expand Down

0 comments on commit 24c2fbd

Please sign in to comment.