Engine: Implement Awk mapping

[NassimBtk]

Required connector model

  memory:
    keys: [id] # key to identify the monitor instance
    simple:
      sources:
        source1: 
        ... 
        source2:
        ... 
      mapping:
        type: awk # csv, single, jsonArray, jsonObject, awk, keyValueMap
        sources:
        - ${source::source1}
        - ${source::source2}
        # Manifests the mapping of the metrics for the documentation (Optional)
        metrics:
        - system.memory.usage{system.memory.state="free"}
        - system.memory.limit
        script: |
            /^Mem:/ { print "system.memory.usage{system.memory.state=\"free\", id=\"$3\"}:" $2 }
            /^Total online memory:/ { print "system.memory.limit{id=\"$3\"}:" $4 }

The monitor identified should be managed using the keys field and the attributes defined in the metric outputs (Output of the awk script).

[bertysentry]

Instead of print() statements in the AWK script, we could provide a dedicated function through the Jawk extension mechanism (as we did for HTTP requests, SNMP, etc.):

void pushMetric(String name, AssocArray attributes, Double value) { /* push metric to telemetry manager */ }

This way, there is no need to parse the output of the AWK script and handle syntax errors.

[bertysentry]

Also, please note that we can't provide multiple sources to AWK (because they are not input files, but just stdin). So the YAML should be with source: <source ref> and not sources: [ array of source refs ].

[bertysentry]

I'm thinking: if we implement this pushMetric() function for AWK scripts, there is no need to have a dedicated mapping: awk.

Instead we could call the pushMetric() function directly in an AWK source, or even in an AWK compute step.

The YAML then becomes:

  memory:
    keys: [id] # key to identify the monitor instance
    simple:
      sources:
        source1: 
          # run command
          compute:
          - type: awk
            script: |
              /^Mem:/ {
                attrs["id"] = $3
                attrs["system.memory.state"] = "free"
                pushMetric("system.memory.usage", attrs, $4);
              }
      mapping:
        type: none # yes, none! This mapping section becomes totally optional!
        # Manifests the mapping of the metrics for the documentation (Optional)
        metrics:
        - system.memory.usage{system.memory.state="free"}
        - system.memory.limit

[NassimBtk]

I'm currently considering this proposal: MetricsHub manages attributes both at the monitor level and at the metric level itself. We know that before metrics are pushed, monitor attributes and metric attributes are merged, with everything applied to the metric. If we consider the current MetricsHub model, perhaps we should update the extension to define createMonitor(attr). Given that Otel is experimenting with prototypes for Resource entities, it seems we might be on the right track by creating monitors within our internal model.