Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Bug] [chunjun-metrics-prometheus] Invalid metric name: org/apache/flink_taskmanager_job_task_operator_chunjun_startLocation #1786

Open
3 tasks done
tangguol opened this issue Aug 9, 2023 · 1 comment · May be fixed by #1894
Open
3 tasks done

[Bug] [chunjun-metrics-prometheus] Invalid metric name: org/apache/flink_taskmanager_job_task_operator_chunjun_startLocation #1786

tangguol opened this issue Aug 9, 2023 · 1 comment · May be fixed by #1894
Labels
bug Something isn't working

Comments

@tangguol
Copy link
Contributor

tangguol commented Aug 9, 2023 •
edited
Loading

Search before asking

  • I had searched in the issues and found no similar issues.

What happened

在我使用master分之使用LocalTest进行测试时报错。

What you expected to happen

解决方法：
修改PrometheusReport类内容
修改前：private static final String SCOPE_PREFIX = "org/apache/flink" + SCOPE_SEPARATOR;
修改后：private static final String SCOPE_PREFIX = "flink" + SCOPE_SEPARATOR;

修改后就可以正常运行啦。

不知道这个显而易见的错误是不是因为我使用的监控版本导致的。
我不确定修改SCOPE_PREFIX参数内容会不会影响其它功能正常使用。

How to reproduce

1.在chunjun-local-test模块pom中加

<dependency>
	<groupId>com.dtstack.chunjun</groupId>
	<artifactId>chunjun-metrics-prometheus</artifactId>
	<version>${project.version}</version>
</dependency>

<dependency>
	<groupId>com.dtstack.chunjun</groupId>
	<artifactId>chunjun-connector-oracle</artifactId>
	<version>${project.version}</version>
</dependency>
  1. oracle同步到oracle json 脚本
{
  "job": {
    "content": [
      {
        "reader": {
          "parameter": {
            "username": "yh_dmp",
            "password": "yh_dmp_yh",
            "connection": [{
              "jdbcUrl": ["jdbc:oracle:thin:@//xxxxxxxxxx"],
              "table": ["TB_STUDENT"],
              "schema": "xxxxx"
            }],
            "increColumn": "ID",
            "startLocation": "0",
            "polling": false,
            "pollingInterval": 3000,
            "column": [{
              "name": "ID",
              "type": "decimal"
            },{
              "name": "NAME",
              "type": "string"
            }]
          },
          "name": "oraclereader"
        },
        "writer": {
          "name": "oraclewriter",
          "parameter": {
            "mode": "update",
            "allReplace": true,
            "username": "yh_dmp",
            "password": "yh_dmp_yh",
            "connection": [
              {
                "jdbcUrl": "jdbc:oracle:thin:@//xxxxxxx",
                "table": ["TB_STUDENT_1"],
                "schema": "xxxxxx"
              }
            ],
            "uniqueKey": ["ID"],
            "column": [{
              "name": "ID",
              "type": "decimal"
            },{
              "name": "NAME",
              "type": "string"
            }]
          }
        }
      }
    ],
    "setting": {
      "speed": {
        "channel": 1,
        "bytes": 0
      },
      "errorLimit": {
        "record": 1
      },
      "restore": {
        "maxRowNumForCheckpoint": 0,
        "isRestore": true,
        "restoreColumnName": "ID",
        "restoreColumnIndex": 0
      },
      "log" : {
        "isLogger": false,
        "level" : "debug",
        "path" : "",
        "pattern":""
      },
      "metricPluginConf" : {
        "pluginName": ""
      }
    }
  }
}
  1. flink-conf.yaml 配置加参数
metrics.reporter.promgateway.factory.class: org.apache.flink.metrics.prometheus.PrometheusPushGatewayReporterFactory
metrics.reporter.promgateway.hostUrl: http://xxxxxx:9091
metrics.reporter.promgateway.jobName: myJob
metrics.reporter.promgateway.randomJobNameSuffix: true
metrics.reporter.promgateway.deleteOnShutdown: false
#metrics.reporter.promgateway.groupingKey: k1=v1;k2=v2
metrics.reporter.promgateway.interval: 60 SECONDS

  1. LocalTest测试类修改内容
    image

  2. 报错信息

image

Anything else

监控安装版本：
prometheus-2.38.0.linux-arm64.tar.gz
pushgateway-1.6.0.linux-arm64.tar.gz

Version

master

Are you willing to submit PR?

  • Yes I am willing to submit a PR!

Code of Conduct
@tangguol tangguol added the bug Something isn't working label Aug 9, 2023
@Icarus-Alpha
Copy link

修改以后，执行增量同步，您的Prometheus可以查询到任务的endlocation吗

@Icarus-Alpha Icarus-Alpha linked a pull request Apr 12, 2024 that will close this issue
Open
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

[hotfix-1786][chunjun-metrics-prometheus] Fix uploading of startLocat… Icarus-Alpha/chunjun
2 participants
@tangguol @Icarus-Alpha