Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Dev #482

Open
wants to merge 2 commits into
base: dev
Choose a base branch
from
Open

Dev #482

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
c42f00f
增加权限管控（项目、任务和日志权限管理）
gaigaikuaipao Aug 24, 2021
0695921
增加权限管控（项目、任务、日志 只有创建者可以看）
gaigaikuaipao Aug 24, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 10 additions & 6 deletions .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,16 +1,20 @@
*/target/

/datax-executor/target/
/datax-executor/datax-executor.iml
/datax-core/datax-core.iml
/datax-core/target/
/datax-admin/datax-admin.iml
/datax-admin/target/
/.idea/
/datax-admin/.idea/
/datax-registry/datax-registry.iml
/datax-registry/target/
/datax-rpc/datax-rpc.iml
/datax-rpc/target/
/datax-all.iml
/logs/
*-dev*
/data/
/build/
/packages/
/datax-assembly/target/
/datax-assembly/datax-assembly.iml
/datax-web.iml
*.iml
/datax-transformer/target/
/applogs/
61 changes: 5 additions & 56 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
[![Stargazers over time](https://starchart.cc/WeiYe-Jing/datax-web.svg)](https://starchart.cc/WeiYe-Jing/datax-web)

DataX Web是在DataX之上开发的分布式数据同步工具，提供简单易用的
操作界面，降低用户使用DataX的学习成本，缩短任务配置时间，避免配置过程中出错。用户可通过页面选择数据源即可创建数据同步任务，RDBMS数据源可批量创建数据同步任务，支持实时查看数据同步进度及日志并提供终止同步功能，集成并二次开发xxl-job可根据时间、自增主键增量同步数据。
操作界面，降低用户使用DataX的学习成本，缩短任务配置时间，避免配置过程中出错。用户可通过页面选择数据源即可创建数据同步任务，支持RDBMS、Hive、HBase、ClickHouse、MongoDB等数据源，RDBMS数据源可批量创建数据同步任务，支持实时查看数据同步进度及日志并提供终止同步功能，集成并二次开发xxl-job可根据时间、自增主键增量同步数据。

任务"执行器"支持集群部署，支持执行器多节点路由策略选择，支持超时控制、失败重试、失败告警、任务依赖，执行器CPU.内存.负载的监控等等。后续还将提供更多的数据源支持、数据转换UDF、表结构同步、数据同步血缘等更为复杂的业务场景。

Expand Down Expand Up @@ -270,63 +270,12 @@ Copyright (c) 2020 WeiYe

# Contact us

### QQ交流群 776939467
### 个人微信
![](https://datax-web.oss-cn-hangzhou.aliyuncs.com/doc/WechatIMG2.jpeg?x-oss-process=image/resize,w_230,h_230)

![](https://datax-web.oss-cn-hangzhou.aliyuncs.com/doc/qrcode.png)
### QQ交流群
![](https://datax-web.oss-cn-hangzhou.aliyuncs.com/doc/qrcode3.jpeg?x-oss-process=image/resize,w_250,h_300)


# 《阿里巴巴 DataX 源码解读专栏》系类文章

[DataX(1):编译打包使用](https://waterwang.blog.csdn.net/article/details/110952589)

[DataX(2): 通过idea搭建源码阅读+调试环境](https://waterwang.blog.csdn.net/article/details/111034241)

[DataX(3): win环境cmd乱码](https://waterwang.blog.csdn.net/article/details/111036461)

[DataX(4): DataX.py解读](https://waterwang.blog.csdn.net/article/details/112234435)

[DataX(5):改造升级-自动识别py环境，执行DataX任务](https://waterwang.blog.csdn.net/article/details/112241390)

[DataX(6):启动步骤解析](https://waterwang.blog.csdn.net/article/details/112617108)

[DataX(7):JobContainer源码解读](https://waterwang.blog.csdn.net/article/details/112718493)

[DataX(8):TaskGroupContainer源码解读](https://waterwang.blog.csdn.net/article/details/112794237)

[DataX(9):Job和TaskGroup的通讯机制](https://waterwang.blog.csdn.net/article/details/112795294)

[DataX(10):源码解读Communication（DataX通讯类） ](https://waterwang.blog.csdn.net/article/details/112850175)

[DataX(11):源码解读 ContainerCommunicator ](https://waterwang.blog.csdn.net/article/details/112855025)

[DataX(12):调度源码解读AbstractScheduler ](https://waterwang.blog.csdn.net/article/details/112911089)

[DataX(13)：源码解读Column-dDataX中的数据类型 ](https://waterwang.blog.csdn.net/article/details/112911316)

[DataX(14):源码解读Channel ](https://waterwang.blog.csdn.net/article/details/112956397)

[DataX(15):源码解读AbstractCollector](https://waterwang.blog.csdn.net/article/details/112983377)

[DataX(16):源码解读AbstractReporter](https://waterwang.blog.csdn.net/article/details/112985829)

[DataX(17):源码解读AbstractRunner ](https://waterwang.blog.csdn.net/article/details/113033822)

[DataX(18):源码解读Transformer ](https://waterwang.blog.csdn.net/article/details/113084048)

[DataX(19):源码解读内置Transformer ](https://waterwang.blog.csdn.net/article/details/113090919)

[DataX(20):内置transformer使用 ](https://waterwang.blog.csdn.net/article/details/113093413)

[DataX(21):编写自己的Transformer ](https://waterwang.blog.csdn.net/article/details/113095114)

[DataX(22):任务分配规则 ](https://waterwang.blog.csdn.net/article/details/112968136)

[DataX(23):dDataX调优 ](https://waterwang.blog.csdn.net/article/details/112947868)

[DataX(24):远程调试dDataX ](https://waterwang.blog.csdn.net/article/details/113353200)

[DataX(25):插件加载原理 ](https://waterwang.blog.csdn.net/article/details/113357535)

[DataX(26):各个数据库与DataX字段映射 ](https://waterwang.blog.csdn.net/article/details/113499474)

[DataX(27):不太常见配置项querySql、preSql、postSql、splitPk](https://waterwang.blog.csdn.net/article/details/113505791)
12 changes: 12 additions & 0 deletions datax-web.iml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
<?xml version="1.0" encoding="UTF-8"?>
<module org.jetbrains.idea.maven.project.MavenProjectsManager.isMavenModule="true" type="JAVA_MODULE" version="4">
<component name="NewModuleRootManager" LANGUAGE_LEVEL="JDK_1_8">
<output url="file://$MODULE_DIR$/target/classes" />
<output-test url="file://$MODULE_DIR$/target/test-classes" />
<content url="file://$MODULE_DIR$">
<excludeFolder url="file://$MODULE_DIR$/target" />
</content>
<orderEntry type="inheritedJdk" />
<orderEntry type="sourceFolder" forTests="false" />
</component>
</module>
4 changes: 0 additions & 4 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,6 @@
<module>datax-executor</module>
<module>datax-rpc</module>
<module>datax-assembly</module>
<module>datax-transformer</module>
</modules>

<properties>
Expand Down Expand Up @@ -55,9 +54,6 @@
<oshi.core.version>3.5.0</oshi.core.version>

<phoenix.version>5.0.0-HBase-2.0</phoenix.version>

<druid.version>1.2.3</druid.version>
<db2jcc.version>db2jcc4</db2jcc.version>
</properties>

</project>
2 changes: 1 addition & 1 deletion userGuid.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -200,7 +200,7 @@
logging:
level:
com.wugui.datax.admin.mapper: error
path: ./data/applogs/admin
path: ./data/applogs/admin
```

修改日志路径path
Expand Down