Merge pull request #1016 from wavefrontHQ/docs-219

Docs 219

pages/doc/authorization_faq.md

@@ -30,6 +30,7 @@ When a Super Admin user [enables Super Admin mode](users_account_managing.html#e
 * Can [restore orphan dashboards and alerts](access.html#make-orphan-dashboards-or-alerts-visible).
 * Can invite other Super Admin users.
 * Can [sign out a user](user-accounts.html#sign-out-a-user).
+* Can [convert metrics](metrics_managing.html#change-the-retention-period-of-metrics) from persistent to ephemeral and the reverse.
 * Can purchase more PPS.
 
 To invite other Super Admin users:

pages/doc/cardinality.md

@@ -101,7 +101,7 @@ Although Operations for Applications supports high cardinality for time series d
 
    1. Ensure that the metric names are stable and do not change.
    2. Keep source names stable. Source names change over time, but make sure that they don't change frequently.
-   3. Use point tags for data that are ephemeral.
+   3. Use point tags and [ephemeral metrics](metric_types.html#metric-types-per-retention-period) for data that is ephemeral.
    4. In Kubernetes, where point tags are usually called labels, add only the point tags that you really need.
 
 For information about metric, source, and point tag names, see [Operations for Applications Data Format Best Practices](wavefront_data_format.html#operations-for-applications-data-format-best-practices). You can also understand more about the metrics structure, sources and the sources browser, and tags, by exploring [Metrics and the Metrics Browser](metrics_managing.html), [Sources](sources_managing.html), and [Organizing with Tags](tags_overview.html).

pages/doc/metric_types.md

@@ -18,12 +18,14 @@ summary: Learn about gauges, counters, delta counters, histograms, and spans.
 
 ## Summary of Metric Types
 
+### Metric Types per Data Type
+
 The following table gives an overview of metric types. We introduce each type in more detail below.
 
 <table style="width: 100%;">
 <tbody>
 <thead>
-<tr><th width="20%">Metric</th><th width="40%">Description</th><th width="40%">Example</th></tr>
+<tr><th width="20%">Metric Type</th><th width="40%">Description</th><th width="40%">Example</th></tr>
 </thead>
 <tr>
 <td>Gauge</td>
@@ -32,7 +34,7 @@ The following table gives an overview of metric types. We introduce each type in
 </tr>
 <tr>
 <td>Counter</td>
-<td>Shows values as they increase (and decrease).</td>
+<td>Shows values as they increase. Counters only accumulate or reset to zero (do not decrease).</td>
 <td>Number of failed connections, registered users.</td>
 </tr>
 <tr>
@@ -55,7 +57,28 @@ The following table gives an overview of metric types. We introduce each type in
 <td>Spans are the fundamental units of trace data. Each span corresponds to a distinct invocation of an operation that executes as part of the request.</td>
 <td>For example, in our BeachShirts sample application, we have the <code>beachshirts.shopping</code> operation, which includes many invocations of the <code>Printing.getAvailableColors</code> span. </td>
 </tr>
+</tbody>
+</table>
+
+### Metric Types per Retention Period
 
+With the 2024-07 release, we introduce **ephemeral** metrics, which have a short [retention period](terms_of_service.html#data-retention).
+
+<table style="width: 100%;">
+<tbody>
+<thead>
+<tr><th width="20%">Metric Type</th><th width="40%">Description</th></tr>
+</thead>
+<tr>
+<td>Persistent</td>
+<td>18 months of data retention. By default, all metrics and counters are persistent. Metrics are convertible to ephemeral.</td>
+</tr>
+<tr>
+<td>Ephemeral</td>
+<td>28 days of data retention. Suitable for metrics that are relevant for a short time and that have high cardinality, such as the Kubernetes metrics (<code>kubernetes.</code>).
+<p><a href="metrics_managing.html#change-the-retention-period-of-metric">Converting</a> metrics from persistent to ephemeral improves the <a href="query_language_performance.html">query performance</a> and reduces the <a href="cardinality.html">cardinality</a>.</p>
+</td>
+</tr>
 </tbody>
 </table>
 

pages/doc/metrics_managing.md

@@ -97,41 +97,26 @@ In the Metrics browser and Query Editor, obsolete metrics are no longer shown in
 
 Select **Browse > Metrics** to display the Metrics Browser. Use the Metrics Browser to find metrics that are actively sending data points.
 
-{% include note.html content="The Metrics browser filters out the obsolete metrics." %}
+{% include note.html content="The Metrics Browser filters out the obsolete metrics." %}
 
-To make search easier, you can
-* Drill down and go up the hierarchy.
-* Filter by source.
-* Hide and redisplay metrics or groups of metrics to unclutter your page.
-
-{% include tip.html content="If you select **Browse > Delta Counters** you can use the same browser to examine [delta counters](delta_counters.html)." %}
-
-![metrics browser with pointers to folder & chart icon for selection, source filter, and info button which displays sources and point tags for a metric](images/metrics_browser.png)
-
-### Examine Metrics
+![An annotated screenshot of the Metrics Browser. The information is listed below.](images/metrics_browser.png)
 
-<table style="width: 100%;">
-<tbody>
-<tr>
-<td width="60%">
-<strong>To examine metrics</strong>
-<ol>
-<li>Select <strong>Browse > Metrics</strong></li>
-<li>Select folder icons to drill down to individual metrics.</li>
-<li>With a metric selected, click <strong>Expand Info</strong> to show sources and point tags for that metric.</li>
-<li>Click the metric name to show a chart with that metric.</li>
-</ol></td>
-<td width="40%"><img src="/images/browse_metrics.png" alt="browse metrics"></td>
-</tr>
-</tbody>
-</table>
+On the Metrics Browser, you can:
+* Drill down and go up the hierarchy.
+* Filter by name or source.
+* Hide and redisplay individual metrics or metrics namespaces to unclutter your page.
+* View the metric type in terms of retention period - persistent or ephemeral.
+* Convert persistent metrics to ephemeral and the reverse.
+* Create a chart or dashboard for an individual metric or for the current set of metrics.
+* View the sources and point tags for an individual metric.
 
+{% include tip.html content="If you select **Browse > Delta Counters** you can use the same browser to examine [delta counters](delta_counters.html). The only difference is that counters are persistent and not convertible to ephemeral." %}
 
-## Hide and Redisplay Metrics
+### Hide and Redisplay Metrics
 
-While obsolete metrics are automatically hidden, you can manually hide metrics from the Metrics browser. Manually hiding metrics does not permanently delete a metric or metric namespace.
+While [obsolete metrics](metrics_managing.html#obsolete-metrics) are automatically hidden, you can manually hide metrics from the Metrics browser. Manually hiding metrics does not permanently delete a metric or metric namespace.
 
-{% include shared/permissions.html entity="metrics" entitymgmt="Metric" %}
+{% include shared/permissions.html entity="metrics" entitymgmt="Metrics" %}
 
 {% include note.html content="Hidden metrics are removed from the autocomplete drop-down lists, but you can still use these metrics in queries as long as data points exist." %}
 
@@ -141,14 +126,13 @@ While obsolete metrics are automatically hidden, you can manually hide metrics f
 <td width="60%">
 <strong>To hide one or more metrics:</strong>
 <ol>
-<li>Select <strong>Browse > Metrics</strong></li>
-<li>Click the <strong>Manage Hidden Metrics</strong> button</li>
-<li>In the dialog type a complete metrics name (e.g. <code>requests.latency</code>) or a metric prefix (e.g. <code>requests.</code>, <code>cpu.loadavg.</code>).
+<li>Select <strong>Browse > Metrics</strong>.</li>
+<li>Click the <strong>Manage Hidden Metrics</strong> button.</li>
+<li>In the dialog box, type a complete metric name (e.g. <code>requests.latency</code>) or a metric prefix (e.g. <code>requests.</code>, <code>cpu.loadavg.</code>).
 <ul>
 <li>This field does not support auto-complete, so you have to type the entire metric name or metric prefix.</li>
 <li>The text is case sensitive.</li>
 <li>Wildcards are not supported. The star <code>*</code> character is considered part of the text string.</li>
-
 </ul></li>
 <li>Press Enter to add the metric(s) to the list and click <strong>Save</strong>.</li>
 </ol> </td>
@@ -163,7 +147,7 @@ While obsolete metrics are automatically hidden, you can manually hide metrics f
 <td width="60%">
 <strong>To view hidden metrics:</strong>
 <ol>
-<li>Select <strong>Browse > Metrics</strong></li>
+<li>Select <strong>Browse > Metrics</strong>.</li>
 <li>Click the <strong>Manage Hidden Metrics</strong> button.</li>
 <li>Click the <strong>Unhide</strong> button to the right of the metric or metric prefix to unhide and click <strong>Save</strong>.</li>
 </ol>
@@ -174,6 +158,37 @@ The selected metrics and metric prefixes appear again as long as they are not ob
 </tbody>
 </table>
 
+### Change the Retention Period of Metrics
+
+With the 2024-07 release, we introduce **ephemeral** metrics, which have short [retention period](terms_of_service.html#data-retention). By default, all ingested metrics are persistent but are convertible to ephemeral.
+
+Converting persistent metrics to ephemeral can significantly improve the [query performance](query_language_performance.html) and reduce the [cardinality](cardinality.html).
+
+{% include note.html content="To change the retention period of a metric or metrics namespace, you must be a Super Admin user with [enabled Super Admin mode](users_account_managing.html#enable-or-disable-super-admin-mode)." %}
+
+{% include important.html content="Converting a persistent metric to ephemeral **permanently deletes** the data points of this metric that are older than 28 days." %}
+
+<table style="width: 100%;">
+<tbody>
+<tr>
+<td width="60%">
+<ol>
+<li>Select <strong>Browse > Metrics</strong>.</li>
+<li>Click the <strong>Change Ephemerality</strong> button.</li>
+<li>To convert a persistent metric or metrics namespace to ephemeral, in the <strong>Select Metric Prefix</strong> text box, enter the target metric name or namespace prefix, and press Enter.
+<p>The metric name or namespace prefix appears in the <strong>Ephemeral Metrics</strong> table below. You can repeat this step for multiple metrics and metrics namespaces.</p></li>
+<li>To convert an ephemeral metric or metrics namespace to persistent, in the <strong>Ephemeral Metrics</strong> table, locate the target metric or namespace prefix and click the corresponding <strong>Convert to Persistent Metric</strong> action.<p>The metric name or namespace prefix disappears from the <strong>Ephemeral Metrics</strong> table. You can repeat this step for multiple metrics and metrics namespaces.</p></li>
+<li>Click <strong>Save</strong>.</li>
+</ol> </td>
+<td width="40%"><img src="/images/change_ephemerality.png" alt="A screenshot of the Change Ephemerality dialog box."></td>
+</tr>
+</tbody>
+</table>
+
+Changing the retention period of a metric or metrics namespace creates a [System event](events.html):
+* Converting a persistent metric to ephemeral creates a System event with the name `Ephemeral Prefix: Added <metric_name>`.
+* Converting an ephemeral metric to persistent creates a System event with the name `Ephemeral Prefix: Removed <metric_name>`.
+
 ## Learn More!
 
 * [Optimizing the Data Shape to Improve Performance](optimize_data_shape.html)

pages/doc/missing_data_troubleshooting.md

@@ -7,7 +7,7 @@ permalink: missing_data_troubleshooting.html
 summary: Learn how to troubleshoot when you expect to see data but it doesn't appear in charts.
 ---
 
-Sometimes you expect to see certain data in VMware Aria Operations for Applications (formerly known as Tanzu Observability by Wavefront) but, for some reason, it doesn't show up!  This can be a frustrating and confusing experience, especially when you urgently need the data. Operations for Applications does not delete data, and retains [metric data for 18 months](terms_of_service.html#data-retention). What could be the problem?
+Sometimes you expect to see certain data in VMware Aria Operations for Applications (formerly known as Tanzu Observability by Wavefront) but, for some reason, it doesn't show up! This can be a frustrating and confusing experience, especially when you urgently need the data. Operations for Applications does not delete data - the [retention period](terms_of_service.html#data-retention) is 18 months for persistent metrics and 28 days for ephemeral metrics. What could be the problem?
 
 This doc page, based on the extensive experience of our customer success team, helps you investigate, understand, and remedy possible causes. In addition to manually investigating and troubleshooting your issues, you can use the [Query Analyzer](query_language_performance.html#use-the-query-analyzer) which helps you identify where exactly the problem is.
 

pages/doc/proxies_configuring.md

@@ -145,7 +145,7 @@ This section gives details on the proxy configuration properties. All properties
 <tbody>
 <tr>
 <td>agentMetricsPointTags</td>
-<td>Point tags and their values to be passed along with <code>~agent.*</code> metrics. <br/>Default: none</td>
+<td>Point tags and their values to be passed along with <code>~proxy.*</code> metrics. <br/>Default: none</td>
 <td>Comma-separated list of key-value pairs.<br/>
 Ex: <code>dc=west,env=prod</code></td>
 <td>3.24</td>

pages/doc/query_language_performance.md

@@ -40,7 +40,10 @@ Watch this video to learn how to optimize dashboard and query performance. Note
 ## Use Performance Statistics
 
 You can see performance statistics for the whole chart and for each query of the chart. For the performance statistics, we measure the following characteristics:
-- **Cardinality**: Number of unique time series. A unique time series has unique metric name, source name and point tags (key and value). For example, you might receive `networks_bytes_received` from multiple sources and with multiple point tags (e.g. `availability_zone`). You can lower cardinality for each query (and the chart) by filtering, for example, limiting the query to certain sources, certain availability zones, etc.
+- **Cardinality**: Number of unique time series. A unique time series has unique metric name, source name and point tags (key and value). For example, you might receive `networks_bytes_received` from multiple sources and with multiple point tags (e.g. `availability_zone`). You can lower cardinality for each query (and the chart) by:
+
+  - Filtering, for example, limiting the query to certain sources, certain availability zones, etc.
+  - Using [ephemeral metrics](metric_types.html#metric-types-per-retention-period), which have a shorter retention period.
 - **Points Scanned**: Number of data points that were queried to show the chart on the screen. You can affect this number by including the time window in the query or by changing the time window interactively.
 - **Duration**: Time between query start and return of result.
 

pages/doc/query_language_series_matching.md

@@ -289,15 +289,15 @@ You can use `groupRight` and `groupLeft` operators to achieve many-to-one and on
 For example, when we use the `groupRight` operator in the query below, we see results with the `processId` tag which is on the right side of the operator.
 
 ```
-sum(ts(~agent.listeners.connections.*), port, processId) + by(processId) groupRight sum(ts(~agent.points.2878.blocked), processId)
+sum(ts(~proxy.listeners.connections.*), port, processId) + by(processId) groupRight sum(ts(~proxy.points.2878.blocked), processId)
 ```
 
 ![A chart created with the above query with one dimension shown in the pinned legend - processId.](images/groupRight.png)
 
 In the example below, we use the `groupLeft` operator. Therefore, we see results with the `port` and `processId` tags, which are on the left side of the operator.
 
 ```
-sum(ts(~agent.listeners.connections.*), port, processId) + by (processId) groupLeft sum(ts(~agent.points.2878.blocked), processId)
+sum(ts(~proxy.listeners.connections.*), port, processId) + by (processId) groupLeft sum(ts(~proxy.points.2878.blocked), processId)
 ```
 
 ![A chart created with the above query with two dimensions shown in the pinned legend - port and processId.](images/groupLeft.png)

pages/doc/terms_of_service.md

@@ -12,27 +12,32 @@ The terms of service and data retention for VMware Aria Operations for Applicati
 
 ## Data Retention
 
-A production instance retains different types data for different amounts of time. While this is subject to change, here are the default settings:
+A production instance retains different types of data for different amounts of time. While this is subject to change, here are the default settings:
 
 <table>
 <tbody>
 <thead>
 <tr><th width="20%">Type of Data</th><th width="80%">Retention</th></tr>
 </thead>
-<tr><td><strong>metrics and counters</strong></td>
-<td>18 months of full-resolution (no downsampling)</td></tr>
+<tr><td><strong>metrics</strong></td>
+<td><ul><li>For persistent (default) metrics, 18 months of full-resolution (no downsampling).</li>
+<li>For ephemeral metrics, 28 days of data detention.</li></ul>
+See <a href="metric_types.html#metric-types-per-retention-period">Metric Types per Retention Period</a>.</td></tr>
+<tr><td><strong>counters</strong></td>
+<td>18 months of data retention.
+</td></tr>
 <tr><td><strong>histograms</strong></td>
-<td>6 months of data retention</td></tr>
+<td>6 months of data retention.</td></tr>
 <tr><td><strong>spans</strong></td>
-<td>7 days retention. With spans, we use <a href="trace_data_sampling.html#intelligent-sampling">Intelligent Sampling</a>. Use trace sampling policies explicitly exclude certain spans.  </td></tr>
+<td>7 days of retention. With spans, we use <a href="trace_data_sampling.html#intelligent-sampling">Intelligent Sampling</a>. Use trace sampling policies explicitly exclude certain spans.  </td></tr>
 </tbody>
 </table>
 
 ## Terms of Service
 
 Your Terms of Service are different depending on when you became a customer.
 
-Production clusters currently offer 18 months of full-resolution (no downsampling) data retention for metrics, 6 months for histograms, and 7 days for spans. We also have an uptime guarantee, as well as High Availability (HA) and Disaster Recovery (DR) options.
+Production clusters currently offer 18 months of full-resolution (no downsampling) data retention for persistent metrics, 28 days for ephemeral metrics, 6 months for histograms, and 7 days for spans. We also have an uptime guarantee, as well as High Availability (HA) and Disaster Recovery (DR) options.
 
 If you became a customer on or after August 17, 2017:
 

pages/doc/wavefront_internal_metrics.md

@@ -12,7 +12,6 @@ You can:
 * Clone and modify one of the Operations for Applications Usage integration dashboards.
 * Create your own dashboard, query these metrics in charts, and create alerts for some of these metrics.
 
-
 ## Internal Metrics Overview
 
 We collect the following sets of metrics.

pages/doc/wavefront_kubernetes.md

@@ -89,11 +89,11 @@ To use our Kubernetes Metrics Collector, you must set up our Kubernetes integrat
 
 ## Monitor Kubernetes
 
-Our Kubernetes Metrics Collector supports monitoring for your Kubernetes infrastructure at all levels of the stack.
+Our Kubernetes Metrics Collector supports monitoring for your Kubernetes infrastructure at all levels of the stack. See the [list of metrics collected by the Kubernetes Metrics Collector](kubernetes.html#metrics).
 * Set up the Kubernetes Collector to have much of the monitoring happen automatically.
 * Fine-tune and customize the solution with configuration options available in the Operations for Applications Kubernetes Metrics Collector.
 
-{% include note.html content="See the [list of metrics collected by the Kubernetes Metrics Collector](kubernetes.html#metrics)." %}
+{% include tip.html content="To avoid [high-cardinality](cardinality.html) and ensure optimal [query performance](query_language_performance.html), consider reducing the retention period of the Kubernetes metrics from 18 months (default) to 28 days. For that purpose, convert the `kubernetes.` metrics namespace from persistent to ephemeral. For details, see [Change the Retention Period of Metrics](metrics_managing.html#change-the-retention-period-of-metrics)." %}
 
 ### Infrastructure Monitoring