Skip to content

Commit c14c49c

Browse files
committed
[FLINK-37522] Add description for the config about file system connection limit
1 parent 452b648 commit c14c49c

File tree

3 files changed

+29
-11
lines changed

3 files changed

+29
-11
lines changed

docs/content.zh/docs/deployment/filesystems/common.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -50,9 +50,9 @@ fs.default-scheme: <default-fs>
5050
要限制文件系统的连接数,可将下列配置添加至 Flink 配置中。设置限制的文件系统由其 scheme 指定:
5151

5252
```yaml
53-
fs.<scheme>.limit.total: (数量,0/-1 表示无限制)
54-
fs.<scheme>.limit.input: (数量,0/-1 表示无限制)
55-
fs.<scheme>.limit.output: (数量,0/-1 表示无限制)
53+
fs.<scheme>.limit.total: (数量,0/-1 表示无连接限制)
54+
fs.<scheme>.limit.input: (数量,0/-1 表示无输入连接限制)
55+
fs.<scheme>.limit.output: (数量,0/-1 表示无输出连接限制)
5656
fs.<scheme>.limit.timeout: (毫秒,0 表示无穷)
5757
fs.<scheme>.limit.stream-timeout: (毫秒,0 表示无穷)
5858
```
@@ -63,4 +63,4 @@ fs.<scheme>.limit.stream-timeout: (毫秒,0 表示无穷)
6363

6464
连接数是按每个 TaskManager/文件系统来进行限制的。因为文件系统的创建是按照 scheme 和 authority 进行的,所以不同的 authority 具有独立的连接池,例如 `hdfs://myhdfs:50010/` 和 `hdfs://anotherhdfs:4399/` 会有单独的连接池。
6565

66-
{{< top >}}
66+
{{< top >}}

docs/content/docs/deployment/filesystems/common.md

+4-4
Original file line numberDiff line numberDiff line change
@@ -52,9 +52,9 @@ To limit a specific file system's connections, add the following entries to the
5252
its scheme.
5353

5454
```yaml
55-
fs.<scheme>.limit.total: (number, 0/-1 mean no limit)
56-
fs.<scheme>.limit.input: (number, 0/-1 mean no limit)
57-
fs.<scheme>.limit.output: (number, 0/-1 mean no limit)
55+
fs.<scheme>.limit.total: (number, 0/-1 mean there is no connection limit)
56+
fs.<scheme>.limit.input: (number, 0/-1 mean there is no input connection limit)
57+
fs.<scheme>.limit.output: (number, 0/-1 mean there is no output connection limit)
5858
fs.<scheme>.limit.timeout: (milliseconds, 0 means infinite)
5959
fs.<scheme>.limit.stream-timeout: (milliseconds, 0 means infinite)
6060
```
@@ -69,4 +69,4 @@ Limit enforcement on a per TaskManager/file system basis.
6969
Because file systems creation occurs per scheme and authority, different
7070
authorities have independent connection pools. For example `hdfs://myhdfs:50010/` and `hdfs://anotherhdfs:4399/` will have separate pools.
7171

72-
{{< top >}}
72+
{{< top >}}

flink-core/src/main/java/org/apache/flink/configuration/CoreOptions.java

+21-3
Original file line numberDiff line numberDiff line change
@@ -527,23 +527,41 @@ public static String[] mergeListsToArray(List<String> base, List<String> append)
527527
* open. Unlimited be default.
528528
*/
529529
public static ConfigOption<Integer> fileSystemConnectionLimit(String scheme) {
530-
return ConfigOptions.key("fs." + scheme + ".limit.total").intType().defaultValue(-1);
530+
return ConfigOptions.key("fs." + scheme + ".limit.total")
531+
.intType()
532+
.defaultValue(-1)
533+
.withDescription(
534+
"The connection limit for the file system "
535+
+ scheme
536+
+ ". The valid value must >= -1. Values of -1 or 0 mean there is no connection limit.");
531537
}
532538

533539
/**
534540
* The total number of input connections that a file system for the given scheme may open.
535541
* Unlimited be default.
536542
*/
537543
public static ConfigOption<Integer> fileSystemConnectionLimitIn(String scheme) {
538-
return ConfigOptions.key("fs." + scheme + ".limit.input").intType().defaultValue(-1);
544+
return ConfigOptions.key("fs." + scheme + ".limit.input")
545+
.intType()
546+
.defaultValue(-1)
547+
.withDescription(
548+
"Input connection limit for the file system "
549+
+ scheme
550+
+ ". The valid value must >= -1. Values of -1 or 0 mean there is no input connection limit.");
539551
}
540552

541553
/**
542554
* The total number of output connections that a file system for the given scheme may open.
543555
* Unlimited be default.
544556
*/
545557
public static ConfigOption<Integer> fileSystemConnectionLimitOut(String scheme) {
546-
return ConfigOptions.key("fs." + scheme + ".limit.output").intType().defaultValue(-1);
558+
return ConfigOptions.key("fs." + scheme + ".limit.output")
559+
.intType()
560+
.defaultValue(-1)
561+
.withDescription(
562+
"Output connection limit for the file system "
563+
+ scheme
564+
+ ". The valid value must >= -1. Values of -1 or 0 mean there is no output connection limit.");
547565
}
548566

549567
/**

0 commit comments

Comments
 (0)