From 563a551f3c47079bebdc0e1d043ced0cf4bfc349 Mon Sep 17 00:00:00 2001 From: Leif Madsen Date: Tue, 12 Sep 2023 11:07:05 -0400 Subject: [PATCH] Don't enable event collection by default on OSP (#488) * Don't enable event collection by default on OSP Closes STF-1498 * Remove events configurations and use defaults The defaults for events pipeline and Ceilometer QDR publish events is disabled and does not need to be called out specifically. --- ...or-director-operator-for-the-overcloud.adoc | 8 -------- ...g-the-stf-connection-for-the-overcloud.adoc | 12 +----------- ...k-environment-file-for-multiple-clouds.adoc | 18 ++++-------------- ...guration-for-director-operator-for-stf.adoc | 4 +--- ...reating-the-base-configuration-for-stf.adoc | 14 +++++--------- .../modules/proc_deploying-the-overcloud.adoc | 2 +- .../proc_planning-amqp-address-prefixes.adoc | 4 ++-- 7 files changed, 14 insertions(+), 48 deletions(-) diff --git a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc index 6bd7550d..892d1317 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-director-operator-for-the-overcloud.adoc @@ -44,19 +44,11 @@ data: MetricsQdrSSLProfiles: - name: sslProfile - CeilometerQdrEventsConfig: - driver: amqp - topic: cloud1-event - CeilometerQdrMetricsConfig: driver: amqp topic: cloud1-metering CollectdAmqpInstances: - cloud1-notify: - notify: true - format: JSON - presettle: false cloud1-telemetry: format: JSON presettle: false diff --git a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc index 44cc9545..60455cde 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc @@ -2,7 +2,7 @@ = Configuring the {ProjectShort} connection for the overcloud [role="_abstract"] -To configure the {Project} ({ProjectShort}) connection, you must create a file that contains the connection configuration of the {MessageBus} for the overcloud to the {ProjectShort} deployment. Enable the collection of events and storage of the events in {ProjectShort} and deploy the overcloud. The default configuration is for a single cloud instance with the default message bus topics. For configuration of multiple cloud deployments, see xref:configuring-multiple-clouds_assembly-completing-the-stf-configuration[]. +To configure the {Project} ({ProjectShort}) connection, you must create a file that contains the connection configuration of the {MessageBus} for the overcloud to the {ProjectShort} deployment. Enable the collection of metrics and storage of the metrics in {ProjectShort} and deploy the overcloud. The default configuration is for a single cloud instance with the default message bus topics. For configuration of multiple cloud deployments, see xref:configuring-multiple-clouds_assembly-completing-the-stf-configuration[]. .Prerequisites @@ -51,19 +51,11 @@ ifdef::include_when_13,include_when_17[] -----END CERTIFICATE----- endif::include_when_13,include_when_17[] - CeilometerQdrEventsConfig: - driver: amqp - topic: cloud1-event - CeilometerQdrMetricsConfig: driver: amqp topic: cloud1-metering CollectdAmqpInstances: - cloud1-notify: - notify: true - format: JSON - presettle: false cloud1-telemetry: format: JSON presettle: false @@ -80,9 +72,7 @@ ifdef::include_when_13,include_when_17[] * Replace the `caCertFileContent` parameter with the contents retrieved in xref:getting-ca-certificate-from-stf-for-overcloud-configuration_assembly-completing-the-stf-configuration[]. endif::include_when_13,include_when_17[] * Replace the `host` sub-parameter of `MetricsQdrConnectors` with the value that you retrieved in xref:retrieving-the-qdr-route-address_assembly-completing-the-stf-configuration[]. -* Set `topic` value of `CeilometerQdrEventsConfig` to define the topic for Ceilometer events. The value is a unique topic idenifier for the cloud such as `cloud1-event`. * Set `topic` value of `CeilometerQdrMetricsConfig.topic` to define the topic for Ceilometer metrics. The value is a unique topic identifier for the cloud such as `cloud1-metering`. -* Set `CollectdAmqpInstances` sub-paramter to define the topic for collectd events. The section name is a unique topic identifier for the cloud such as `cloud1-notify`. * Set `CollectdAmqpInstances` sub-parameter to define the topic for collectd metrics. The section name is a unique topic identifier for the cloud such as `cloud1-telemetry`. ifndef::include_when_13[] * Set `CollectdSensubilityResultsChannel` to define the topic for collectd-sensubility events. The value is a unique topic identifier for the cloud such as `sensubility/cloud1-telemetry`. diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc index 2ae2d0d7..c0c60399 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-openstack-environment-file-for-multiple-clouds.adoc @@ -24,7 +24,7 @@ = Creating the {OpenStack} environment file for multiple clouds [role="_abstract"] -To label traffic according to the cloud of origin, you must create a configuration with cloud-specific instance names. Create an `stf-connectors.yaml` file and adjust the values of `CeilometerQdrEventsConfig`, `CeilometerQdrMetricsConfig` and `CollectdAmqpInstances` to match the AMQP address prefix scheme. +To label traffic according to the cloud of origin, you must create a configuration with cloud-specific instance names. Create an `stf-connectors.yaml` file and adjust the values of `CeilometerQdrMetricsConfig` and `CollectdAmqpInstances` to match the AMQP address prefix scheme. ifdef::include_when_16[] [NOTE] @@ -46,10 +46,10 @@ ifdef::include_when_13,include_when_17[* You have retrieved the CA certificate f . Create a configuration file called `stf-connectors.yaml` in the `/home/stack` directory. ifdef::include_when_13[] -. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrEventsConfig`, `CeilometerQdrMetricsConfig`, and `CollectdAmqpInstances` topic values to match the AMQP address that you want for this cloud deployment. +. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrMetricsConfig`, and `CollectdAmqpInstances` topic values to match the AMQP address that you want for this cloud deployment. endif::include_when_13[] ifndef::include_when_13[] -. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrEventsConfig`, `CeilometerQdrMetricsConfig`, `CollectdAmqpInstances`, and `CollectdSensubilityResultsChannel` topic values to match the AMQP address that you want for this cloud deployment. +. In the `stf-connectors.yaml` file, configure the `MetricsQdrConnectors` address to connect to the {MessageBus} on the overcloud deployment. Configure the `CeilometerQdrMetricsConfig`, `CollectdAmqpInstances`, and `CollectdSensubilityResultsChannel` topic values to match the AMQP address that you want for this cloud deployment. endif::include_when_13[] + // The following configuration should match the contents in modules/proc_configuring-the-stf-connection-for-the-overcloud.adoc. If you have changes to make, please make the same changes to both files. @@ -80,19 +80,11 @@ ifdef::include_when_13,include_when_17[] -----END CERTIFICATE----- endif::include_when_13,include_when_17[] - CeilometerQdrEventsConfig: - driver: amqp - topic: cloud1-event - CeilometerQdrMetricsConfig: driver: amqp topic: cloud1-metering CollectdAmqpInstances: - cloud1-notify: - notify: true - format: JSON - presettle: false cloud1-telemetry: format: JSON presettle: false @@ -109,9 +101,7 @@ ifdef::include_when_13,include_when_17[] * Replace the `caCertFileContent` parameter with the contents retrieved in xref:getting-ca-certificate-from-stf-for-overcloud-configuration_assembly-completing-the-stf-configuration[]. endif::include_when_13,include_when_17[] * Replace the `host` sub-parameter of `MetricsQdrConnectors` with the value that you retrieved in xref:retrieving-the-qdr-route-address_assembly-completing-the-stf-configuration[]. -* Set `topic` value of `CeilometerQdrEventsConfig` to define the topic for Ceilometer events. The value is a unique topic idenifier for the cloud such as `cloud1-event`. * Set `topic` value of `CeilometerQdrMetricsConfig.topic` to define the topic for Ceilometer metrics. The value is a unique topic identifier for the cloud such as `cloud1-metering`. -* Set `CollectdAmqpInstances` sub-paramter to define the topic for collectd events. The section name is a unique topic identifier for the cloud such as `cloud1-notify`. * Set `CollectdAmqpInstances` sub-parameter to define the topic for collectd metrics. The section name is a unique topic identifier for the cloud such as `cloud1-telemetry`. ifndef::include_when_13[] * Set `CollectdSensubilityResultsChannel` to define the topic for collectd-sensubility events. The value is a unique topic identifier for the cloud such as `sensubility/cloud1-telemetry`. @@ -127,7 +117,7 @@ ifndef::include_when_13[The value for sensubility is the full topic path and has For an example of a cloud configuration in the `ServiceTelemetry` object referring to the full topic address, see xref:clouds_assembly-installing-the-core-components-of-stf[]. ==== -. Ensure that the naming convention in the `stf-connectors.yaml` file aligns with the `spec.bridge.amqpUrl` field in the Smart Gateway configuration. For example, configure the `CeilometerQdrEventsConfig.topic` field to a value of `cloud1-event`. +. Ensure that the naming convention in the `stf-connectors.yaml` file aligns with the `spec.bridge.amqpUrl` field in the Smart Gateway configuration. For example, configure the `CeilometerQdrMetricsConfig.topic` field to a value of `cloud1-metering`. . Log in to the undercloud host as the `stack` user. diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc index f99f2fca..ee04f485 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-director-operator-for-stf.adoc @@ -32,7 +32,6 @@ data: enable-stf.yaml: | parameter_defaults: # only send to STF, not other publishers - EventPipelinePublishers: [] PipelinePublishers: [] # manage the polling and pipeline configuration files for Ceilometer agents @@ -41,7 +40,6 @@ data: # enable Ceilometer metrics and events CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: true # enable collection of API status CollectdEnableSensubility: true @@ -58,7 +56,7 @@ data: CollectdExtraPlugins: - vmem - # set standard prefixes for where metrics and events are published to QDR + # set standard prefixes for where metrics are published to QDR MetricsQdrAddresses: - prefix: 'collectd' distribution: multicast diff --git a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc index 47090273..537bee27 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_creating-the-base-configuration-for-stf.adoc @@ -12,7 +12,7 @@ To configure the base parameters to provide a compatible data collection and tra + [IMPORTANT] ==== -Setting `EventPipelinePublishers` and `PipelinePublishers` to empty lists results in no event or metric data passing to {OpenStackShort} telemetry components, such as Gnocchi or Panko. If you need to send data to additional pipelines, the Ceilometer polling interval of `30` seconds, that you specify in `ExtraConfig`, might overwhelm the {OpenStackShort} telemetry components. You must increase the interval to a larger value, such as `300`, which results in less telemetry resolution in {ProjectShort}. +Setting `PipelinePublishers` to an empty list results in no metric data passing to {OpenStackShort} telemetry components, such as Gnocchi or Panko. If you need to send data to additional pipelines, the Ceilometer polling interval of `30` seconds, that you specify in `ExtraConfig`, might overwhelm the {OpenStackShort} telemetry components. You must increase the interval to a larger value, such as `300`, which results in less telemetry resolution in {ProjectShort}. ==== + .enable-stf.yaml @@ -21,16 +21,14 @@ ifdef::include_when_13[] ---- parameter_defaults: # only send to STF, not other publishers - EventPipelinePublishers: [] PipelinePublishers: [] # manage the polling and pipeline configuration files for Ceilometer agents ManagePolling: true ManagePipeline: true - # enable Ceilometer metrics and events + # enable Ceilometer metrics CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: true # set collectd overrides for higher telemetry resolution and extra plugins to load CollectdConnectionType: amqp1 @@ -39,7 +37,7 @@ parameter_defaults: CollectdExtraPlugins: - vmem - # set standard prefixes for where metrics and events are published to QDR + # set standard prefixes for where metrics are published to QDR MetricsQdrAddresses: - prefix: 'collectd' distribution: multicast @@ -97,16 +95,14 @@ ifdef::include_when_16[] ---- parameter_defaults: # only send to STF, not other publishers - EventPipelinePublishers: [] PipelinePublishers: [] # manage the polling and pipeline configuration files for Ceilometer agents ManagePolling: true ManagePipeline: true - # enable Ceilometer metrics and events + # enable Ceilometer metrics CeilometerQdrPublishMetrics: true - CeilometerQdrPublishEvents: true # enable collection of API status CollectdEnableSensubility: true @@ -123,7 +119,7 @@ parameter_defaults: CollectdExtraPlugins: - vmem - # set standard prefixes for where metrics and events are published to QDR + # set standard prefixes for where metrics are published to QDR MetricsQdrAddresses: - prefix: 'collectd' distribution: multicast diff --git a/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc b/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc index a8a3ba7b..bbea3297 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_deploying-the-overcloud.adoc @@ -27,7 +27,7 @@ $ source ~/stackrc -e /home/stack/stf-connectors.yaml ---- -* Include the `ceilometer-write-qdr.yaml` file to ensure that Ceilometer telemetry and events are sent to {ProjectShort}. +* Include the `ceilometer-write-qdr.yaml` file to ensure that Ceilometer telemetry is sent to {ProjectShort}. * Include the `qdr-edge-only.yaml` file to ensure that the message bus is enabled and connected to {ProjectShort} message bus routers. * Include the `enable-stf.yaml` environment file to ensure that the defaults are configured correctly. * Include the `stf-connectors.yaml` environment file to define the connection to {ProjectShort}. diff --git a/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc b/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc index 3783254a..82d9c0ce 100644 --- a/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc +++ b/doc-Service-Telemetry-Framework/modules/proc_planning-amqp-address-prefixes.adoc @@ -4,7 +4,7 @@ [role="_abstract"] ifdef::include_when_16[] -By default, {OpenStack} ({OpenStackShort}) nodes receive data through two data collectors; collectd and Ceilometer. The collectd-sensubility plugin requires a unique address. These components send telemetry data or notifications to the respective AMQP addresses, for example, `collectd/telemetry`. {ProjectShort} Smart Gateways listen on those AMQP addresses for data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: +By default, {OpenStack} ({OpenStackShort}) nodes retrieve data through two data collectors; collectd and Ceilometer. The collectd-sensubility plugin requires a unique address. These components send telemetry data or notifications to the respective AMQP addresses, for example, `collectd/telemetry`. {ProjectShort} Smart Gateways listen on those AMQP addresses for data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: * `collectd/cloud1-telemetry` * `collectd/cloud1-notify` @@ -21,7 +21,7 @@ By default, {OpenStack} ({OpenStackShort}) nodes receive data through two data c endif::include_when_16[] ifdef::include_when_13[] -By default, {OpenStack} nodes receive data through two data collectors; collectd and Ceilometer. These components send telemetry data or notifications to the respective AMQP addresses, for example, collectd/telemetry. {ProjectShort} Smart Gateways listen on the AMQP addresses for monitoring data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: +By default, {OpenStack} nodes retrieve data through two data collectors; collectd and Ceilometer. These components send telemetry data or notifications to the respective AMQP addresses, for example, collectd/telemetry. {ProjectShort} Smart Gateways listen on the AMQP addresses for monitoring data. To support multiple clouds and to identify which cloud generated the monitoring data, configure each cloud to send data to a unique address. Add a cloud identifier prefix to the second part of the address. The following list shows some example addresses and identifiers: * `collectd/cloud1-telemetry` * `collectd/cloud1-notify`