[connector/failover] Support unlimited retries in failover connector (#…

…32017)

**Description:** <Describe what has changed.>

This adds to the conditional for max_retries that
short circuits for the magic value of 0. This allows effectively turning
off retry limiting so that
the failover connector will continue to poll higher priority pipelines
indefinitely.

**Link to tracking Issue:** #9868

**Documentation:** Added line to document the magic value

Signed-off-by: sinkingpoint <[email protected]>

.chloggen/sinkingpoint_failover-max-retries.yaml

@@ -0,0 +1,13 @@
+# Use this changelog template to create an entry for release notes.
+
+# One of 'breaking', 'deprecation', 'new_component', 'enhancement', 'bug_fix'
+change_type: enhancement
+
+# The name of the component, or a single word describing the area of concern, (e.g. filelogreceiver)
+component: failoverconnector
+
+# A brief description of the change.  Surround your text with quotes ("") if it needs to start with a backtick (`).
+note: Support ignoring `max_retries` setting in failover connector
+
+# Mandatory: One or more tracking issues related to the change. You can use the PR number here if no issue exists.
+issues: [9868]

connector/failoverconnector/README.md

@@ -33,7 +33,7 @@ The following settings are available:
 - `priority_levels (required)`: list of pipeline level priorities in a 1 - n configuration, multiple pipelines can sit at a single priority level.
 - `retry_interval (optional)`: the frequency at which the pipeline levels will attempt to reestablish connection with all higher priority levels. Default value is 10 minutes. (See Example below for further explanation)
 - `retry_gap (optional)`: the amount of time between trying two separate priority levels in a single retry_interval timeframe. Default value is 30 seconds. (See Example below for further explanation)
-- `max_retries (optional)`: the maximum retries per level. Default value is 10.
+- `max_retries (optional)`: the maximum retries per level. Default value is 10. Set to 0 to allow unlimited retries.
 
 The connector intakes a list of `priority_levels` each of which can contain multiple pipelines.
 If any pipeline at a stable level fails, the level is considered unhealthy and the connector will move down one priority level and route all data to the new level (assuming it is stable).

connector/failoverconnector/internal/state/pipeline_selector.go

@@ -119,7 +119,7 @@ func (p *PipelineSelector) checkContinueRetry(index int) bool {
 }
 
 func (p *PipelineSelector) exceededMaxRetries(idx int) bool {
-	return idx < len(p.pipelineRetries) && (p.loadRetryCount(idx) >= p.constants.MaxRetries)
+	return p.constants.MaxRetries > 0 && idx < len(p.pipelineRetries) && (p.loadRetryCount(idx) >= p.constants.MaxRetries)
 }
 
 // SetToStableIndex returns the CurrentIndex to the known Stable Index