Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

运行Linear.java python报dl_on_flink_framework依赖不存在 #764

Open
I-am-DJ opened this issue Aug 29, 2023 · 0 comments
Open

运行Linear.java python报dl_on_flink_framework依赖不存在 #764

I-am-DJ opened this issue Aug 29, 2023 · 0 comments

Comments

@I-am-DJ
Copy link

I-am-DJ commented Aug 29, 2023

我执行的步骤

按照初始化步骤对dl-on-flink进行打包
新建了一个服务用来跑flink任务
服务中引入dl-on-flink-dist

          <dependency>
            <groupId>org.flinkextended</groupId>
            <artifactId>dl-on-flink-dist</artifactId>
            <version>0.6.0-SNAPSHOT</version>
        </dependency>
        <dependency>
            <groupId>org.apache.flink</groupId>
            <artifactId>flink-python_2.12</artifactId>
            <version>${flink.version}</version>
        </dependency>

复制dl-on-flink-examples-tensorflow项目下的Linear.java到新服务中运行
对应报错信息

[08-29 17:52:28] [INFO] [org.mortbay.log:67] Start logging process worker:0 with org.flinkextended.flink.ml.cluster.node.runner.python.log.StdOutErrorProcessOutputConsumer@610546a2
[08-29 17:52:28] [INFO] [org.mortbay.log:67] Start logging process worker:1 with org.flinkextended.flink.ml.cluster.node.runner.python.log.StdOutErrorProcessOutputConsumer@4ea7c222
Traceback (most recent call last):
  File "/Users/lsz/code/AppAIRealTimeDataProvider/startup.py", line 22, in <module>
Traceback (most recent call last):
  File "/Users/lsz/code/AppAIRealTimeDataProvider/startup.py", line 22, in <module>
    from dl_on_flink_framework import context
ModuleNotFoundError: No module named 'dl_on_flink_framework'
    from dl_on_flink_framework import context
ModuleNotFoundError: No module named 'dl_on_flink_framework'
[08-29 17:52:29] [ERROR] [org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner:262] Got exception during python running
org.flinkextended.flink.ml.util.MLException: worker:0 python process exited with code 1
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runProcess(ProcessPythonRunner.java:145)
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runScript(ProcessPythonRunner.java:111)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.runScript(CommonMLRunner.java:276)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.run(CommonMLRunner.java:252)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[08-29 17:52:29] [ERROR] [org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner:262] Got exception during python running
org.flinkextended.flink.ml.util.MLException: worker:1 python process exited with code 1
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runProcess(ProcessPythonRunner.java:145)
	at org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner.runScript(ProcessPythonRunner.java:111)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.runScript(CommonMLRunner.java:276)
	at org.flinkextended.flink.ml.cluster.node.runner.CommonMLRunner.run(CommonMLRunner.java:252)
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
	at java.util.concurrent.FutureTask.run$$$capture(FutureTask.java:266)
	at java.util.concurrent.FutureTask.run(FutureTask.java)
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
	at java.lang.Thread.run(Thread.java:748)
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.AbstractScriptRunner:41] worker:0 mark input queue finished.
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.AbstractScriptRunner:41] worker:1 mark input queue finished.
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner:248] Python scriptRunner for worker:1 closed
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.python.ProcessPythonRunner:248] Python scriptRunner for worker:0 closed
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.NodeHeartBeatRunner:86] worker:1 heartbeat thread interrupted
[08-29 17:52:29] [INFO] [org.flinkextended.flink.ml.cluster.node.runner.NodeHeartBeatRunner:86] worker:0 heartbeat thread interrupted
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant