Add updated architecture diagrams (#499)

* Add updated architecture diagrams * Use updated architecture diagrams * Update architecture overview to focus on metrics Update the architecture overview to make it clear STF is focused on delivery of metrics from RHOSP. Provide information about use_redhat observability strategy, and note that prior versions of STF would manage Elasticsearch instances. Note that new installations use the observability strategy of use_redhat, and that the guide will focus on that deployment model. Provide a placeholder for a new xref that would guide the user towards the deprecated architecture using events, where our updated metrics-and-events architecture diagram would live. * Link to observability strategy * Remove community components from core overview * Use ObservabilityOperator parameter to refer to OBO/COO * Update doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc Co-authored-by: mickogeary <[email protected]> * Update doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc Co-authored-by: mickogeary <[email protected]> * Update doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc Co-authored-by: mickogeary <[email protected]> * Update doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc Co-authored-by: mickogeary <[email protected]> * Update doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc * Update doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc Co-authored-by: mickogeary <[email protected]> * Update doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc Co-authored-by: mickogeary <[email protected]> --------- Co-authored-by: mickogeary <[email protected]>
infrawatch · Nov 3, 2023 · e2f1961 · e2f1961
1 parent c8386ab
commit e2f1961
Show file tree

Hide file tree

Showing 14 changed files with 37 additions and 17 deletions.
diff --git a/common/global/stf-attributes.adoc b/common/global/stf-attributes.adoc
@@ -37,6 +37,7 @@ ifeval::[{ProductVersion} < 1.5]
 endif::[]
 
 ifeval::["{build}" == "upstream"]
+:ObservabilityOperator: Observability{nbsp}Operator
 :OpenShift: OpenShift
 :OpenShiftShort: OKD
 :OpenStack: OpenStack
@@ -55,6 +56,7 @@ ifeval::["{build}" == "upstream"]
 endif::[]
 
 ifeval::["{build}" == "downstream"]
+:ObservabilityOperator: Cluster{nbsp}Observability{nbsp}Operator
 :OpenShift: Red{nbsp}Hat{nbsp}OpenShift{nbsp}Container{nbsp}Platform
 :OpenShiftShort: OCP
 :OpenStack: Red{nbsp}Hat{nbsp}OpenStack{nbsp}Platform

diff --git a/...emetry-Framework/assemblies/assembly_installing-the-core-components-of-stf.adoc b/...emetry-Framework/assemblies/assembly_installing-the-core-components-of-stf.adoc
@@ -6,13 +6,12 @@ ifdef::context[:parent-context: {context}]
 :context: assembly-installing-the-core-components-of-stf
 
 [role="_abstract"]
-You can use Operators to load the {Project} ({ProjectShort}) components and objects. Operators manage each of the following {ProjectShort} core and community components:
+You can use Operators to load the {Project} ({ProjectShort}) components and objects. Operators manage each of the following {ProjectShort} core components:
 
 * cert-manager
 * {MessageBus}
 * Smart Gateway
 * Prometheus and AlertManager
-* Grafana
 
 .Prerequisites
 

diff --git a/doc-Service-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_arch.png b/doc-Service-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_arch.png
diff --git a/doc-Service-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_arch_2.png b/doc-Service-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_arch_2.png
diff --git a/...lemetry-Framework/images/363_OpenStack_STF_updates_0923_deployment_manually.png b/...lemetry-Framework/images/363_OpenStack_STF_updates_0923_deployment_manually.png
diff --git a/...Telemetry-Framework/images/363_OpenStack_STF_updates_0923_deployment_prereq.png b/...Telemetry-Framework/images/363_OpenStack_STF_updates_0923_deployment_prereq.png
diff --git a/doc-Service-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_topology.png b/doc-Service-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_topology.png
diff --git a/...ervice-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_topology_2.png b/...ervice-Telemetry-Framework/images/363_OpenStack_STF_updates_0923_topology_2.png
diff --git a/doc-Service-Telemetry-Framework/modules/con_configuring-multiple-clouds.adoc b/doc-Service-Telemetry-Framework/modules/con_configuring-multiple-clouds.adoc
@@ -6,7 +6,7 @@ You can configure multiple {OpenStack} ({OpenStackShort}) clouds to target a sin
 
 [[osp-stf-multiple-clouds]]
 .Two {OpenStackShort} clouds connect to {ProjectShort}
-image::OpenStack_STF_Overview_37_0919_topology.png[An example of two {OpenStackShort} clouds connecting to {ProjectShort}]
+image::363_OpenStack_STF_updates_0923_topology_2.png[An example of two {OpenStackShort} clouds connecting to {ProjectShort}]
 
 To configure the {OpenStackShort} overcloud for a multiple cloud scenario, complete the following tasks:
 

diff --git a/doc-Service-Telemetry-Framework/modules/con_observability-strategy.adoc b/doc-Service-Telemetry-Framework/modules/con_observability-strategy.adoc
@@ -15,13 +15,13 @@ The following values are available:
 |value |meaning
 
 | use_redhat
-| Red Hat supported components are requested by {ProjectShort}. This includes Prometheus and Alertmanager from the Observability Operator, but no resource requests to Elastic Cloud on Kubernetes (ECK) Operator. If enabled, resources are also requested from the Grafana Operator (community component).
+| Red Hat supported components are requested by {ProjectShort}. This includes Prometheus and Alertmanager from the {ObservabilityOperator}, but no resource requests to Elastic Cloud on Kubernetes (ECK) Operator. If enabled, resources are also requested from the Grafana Operator (community component).
 
 | use_hybrid
 | In addition to the Red Hat supported components, Elasticsearch and Grafana resources are also requested (if specified in the ServiceTelemetry object)
 
 | use_community
-| The community version of Prometheus Operator is used instead of Observability Operator. Elasticsearch and Grafana resources are also requested (if specified in the ServiceTelemetry object)
+| The community version of Prometheus Operator is used instead of {ObservabilityOperator}. Elasticsearch and Grafana resources are also requested (if specified in the ServiceTelemetry object)
 
 | none
 | No storage or alerting components are deployed

diff --git a/doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc b/doc-Service-Telemetry-Framework/modules/con_stf-architecture.adoc
@@ -11,19 +11,25 @@
 [role="_abstract"]
 {Project} ({ProjectShort}) uses a client-server architecture, in which {OpenStack} ({OpenStackShort}) is the client and {OpenShift} is the server.
 
-{ProjectShort} consists of the following components:
+By default, {ProjectShort} collects, transports, and stores metrics information.
+
+You can collect {OpenStackShort} events data, transport it with the message bus, and forward it to a user-provided Elasticsearch from the Smart Gateways, but this option is deprecated.
+// For more information about {ProjectShort} when configured with events, see <xref-TODO>. Tracked via https://issues.redhat.com/browse/STF-1552
 
+{ProjectShort} consists of the following components:
 
 * Data collection
-** collectd: Collects infrastructure metrics and events.
-** Ceilometer: Collects {OpenStackShort} metrics and events.
+** collectd: Collects infrastructure metrics and events on {OpenStackShort}.
+** Ceilometer: Collects {OpenStackShort} metrics and events on {OpenStackShort}.
 * Transport
-** {MessageBus}: An AMQP 1.x compatible messaging bus that provides fast and reliable data transport to transfer the metrics to {ProjectShort} for storage.
+** {MessageBus}: An AMQP 1.x compatible messaging bus that provides fast and reliable data transport to transfer the metrics from {OpenStackShort} to {ProjectShort} for storage or forwarding.
 ** Smart Gateway: A Golang application that takes metrics and events from the AMQP 1.x bus to deliver to Prometheus or an external Elasticsearch.
 * Data storage
 ** Prometheus: Time-series data storage that stores {ProjectShort} metrics received from the Smart Gateway.
 ** Alertmanager: An alerting tool that uses Prometheus alert rules to manage alerts.
+* User provided components
 ** Grafana: A visualization and analytics application that you can use to query, visualize, and explore data.
+** Elasticsearch: Events data storage that stores {OpenStackShort} events received and forwarded by the Smart Gateway.
 
 The following table describes the application of the client and server components:
 
@@ -49,6 +55,10 @@ The following table describes the application of the client and server component
 |no
 |yes
 
+|Grafana
+|no
+|yes
+
 |collectd
 |yes
 |no
@@ -64,11 +74,20 @@ To ensure that the monitoring platform can report operational problems with your
 
 [[osp-stf-overview]]
 .Service Telemetry Framework architecture overview
-image::OpenStack_STF_Overview_37_1019_arch.png[Service Telemetry Framework architecture overview]
+image::363_OpenStack_STF_updates_0923_arch_2.png[Service Telemetry Framework architecture overview]
 
 For client side metrics, collectd provides infrastructure metrics without project data, and Ceilometer provides {OpenStackShort} platform data based on projects or user workload. Both Ceilometer and collectd deliver data to Prometheus by using the {MessageBus} transport, delivering the data through the message bus. On the server side, a Golang application called the Smart Gateway takes the data stream from the bus and exposes it as a local scrape endpoint for Prometheus.
 
-If you plan to collect and store events, collectd and Ceilometer deliver event data to the server side by using the {MessageBus} transport. Another Smart Gateway writes the data to the Elasticsearch datastore.
+When you collect and store events, collectd and Ceilometer deliver event data to the server side by using the {MessageBus} transport. Another Smart Gateway forwards the data to a user-provided Elasticsearch datastore.
+
+[NOTE]
+====
+In previous releases of {ProjectShort}, the Service Telemetry Operator requested instances of Elasticsearch from the Elastic Cloud on Kubernetes (ECK) Operator. {ProjectShort} now uses a forwarding model, where events are forwarded from a Smart Gateway instance to a user-provided instance of Elasticsearch. The management of an Elasticsearch instance by Service Telemetry Operator is deprecated.
+
+In new `ServiceTelemetry`deployments, the `observabilityStrategy` parameter has a value of `use_redhat`, that does not request Elasticsearch instances from ECK. Deployments of `ServiceTelemetry` that are version {ProjectShort} 1.5.3 or older have the `observabilityStrategy` parameter set to `use_community`, which matches the previous architecture. If a user deployed an Elasticsearch instance with {ProjectShort}, the Service Telemetry Operator updates the `ServiceTelemetry` custom resource object to have the `observabilityStrategy` parameter set to `use_community`, and functions similar to previous releases. For more information about observability strategies, see xref:observability-strategy-in-service-telemetry-framework_assembly-preparing-your-ocp-environment-for-stf[].
+
+For more information about migration to the `use_redhat` observability strategy, see link:https://access.redhat.com/articles/7011708[Migrating Service Telemetry Framework to fully supported operators].
+====
 
 Server-side {ProjectShort} monitoring infrastructure consists of the following layers:
 
@@ -83,4 +102,4 @@ endif::[]
 
 [[osp-stf-server-side-monitoring]]
 .Server-side STF monitoring infrastructure
-image::STF_Overview_37_0819_deployment_prereq.png[Server-side STF monitoring infrastructure]
+image::363_OpenStack_STF_updates_0923_deployment_prereq.png[Server-side STF monitoring infrastructure]
diff --git a/doc-Service-Telemetry-Framework/modules/con_support-for-stf.adoc b/doc-Service-Telemetry-Framework/modules/con_support-for-stf.adoc
@@ -2,7 +2,7 @@
 = Support for {Project}
 
 [role="_abstract"]
-Red Hat supports the core Operators and workloads, including {MessageBus}, Observability Operator (Prometheus, Alertmanager), Service Telemetry Operator, and Smart Gateway Operator. Red Hat does not support the community Operators or workload components, inclusive of Elasticsearch, Grafana, and their Operators.
+Red Hat supports the core Operators and workloads, including {MessageBus}, {ObservabilityOperator} (Prometheus, Alertmanager), Service Telemetry Operator, and Smart Gateway Operator. Red Hat does not support the community Operators or workload components, inclusive of Elasticsearch, Grafana, and their Operators.
 
 You can only deploy {ProjectShort} in a fully connected network environment. You cannot deploy {ProjectShort} in {OpenShift}-disconnected environments or network proxy environments.
 

diff --git a/doc-Service-Telemetry-Framework/modules/proc_deploying-observability-operator.adoc b/doc-Service-Telemetry-Framework/modules/proc_deploying-observability-operator.adoc
@@ -1,15 +1,15 @@
 :_content-type: PROCEDURE
 
 [id="deploying-observability-operator_{context}"]
-= Deploying Observability Operator
+= Deploying {ObservabilityOperator}
 
 [role="_abstract"]
 // TODO: https://access.redhat.com/articles/7011708 covers migration to OBO from community-operators Prometheus Operator. This documentation references community-operators as the installation CatalogSource. It is hoping OBO is available from redhat-operators CatalogSource prior to STF 1.5.3. If so, then we will need to update this.
-{Project} ({ProjectShort}) uses other supporting Operators as part of the deployment. {ProjectShort} can resolve most dependencies automatically, but you need to pre-install some Operators, such as Observability Operator, which provides an instance of Prometheus.
+{Project} ({ProjectShort}) uses other supporting Operators as part of the deployment. {ProjectShort} can resolve most dependencies automatically, but you need to pre-install some Operators, such as {ObservabilityOperator}, which provides an instance of Prometheus.
 
 .Procedure
 
-. To store metrics in Prometheus, enable the Observability Operator by using the community-operators CatalogSource:
+. To store metrics in Prometheus, enable the {ObservabilityOperator} by using the community-operators CatalogSource:
 +
 [WARNING]
 ====

diff --git a/...vice-Telemetry-Framework/modules/proc_deploying-service-telemetry-operator.adoc b/...vice-Telemetry-Framework/modules/proc_deploying-service-telemetry-operator.adoc
@@ -8,7 +8,7 @@ Deploy Service Telemetry Operator on {OpenShift} to provide the supporting Opera
 
 .Prerequisites
 
-* Installation of Observability Operator which uses Prometheus to store metrics. For more information, see xref:deploying-observability-operator_assembly-installing-the-core-components-of-stf[].
+* Installation of {ObservabilityOperator} which uses Prometheus to store metrics. For more information, see xref:deploying-observability-operator_assembly-installing-the-core-components-of-stf[].
 ifeval::[{SupportedOpenShiftVersion} < 4.12]
 * Installation of Certificate Manager for OpenShift on {OpenShift} versions prior to 4.12. For more information, see xref:deploying-certificate-manager-for-openshift-operator_assembly-installing-the-core-components-of-stf[]
 endif::[]