Update

cpp/include/cudf/io/data_sink.hpp

@@ -1,5 +1,5 @@
 /*
- * Copyright (c) 2020-2024, NVIDIA CORPORATION.
+ * Copyright (c) 2020-2025, NVIDIA CORPORATION.
  *
  * Licensed under the Apache License, Version 2.0 (the "License");
  * you may not use this file except in compliance with the License.
@@ -122,7 +122,7 @@ class data_sink {
    *
    * In the case where the sink type is itself a memory buffered write, this ends up
    * being effectively a second memcpy.  So a useful optimization for a "smart"
-   * custom data_sink is to do it's own internal management of the movement
+   * custom data_sink is to do its own internal management of the movement
    * of data between cpu and gpu; turning the internals of the writer into simply
    *
    * sink->device_write(device_buffer, size)

cpp/src/io/utilities/data_sink.cpp

@@ -53,8 +53,9 @@ class file_sink : public data_sink {
 
   void flush() override
   {
-    // KvikIO's pwrite() is a system call that reaches the kernel buffer. Flushing the application
-    // buffer is therefore not needed.
+    // kvikio::FileHandle::pwrite() makes system calls that reach the kernel buffer cache. This
+    // process does not involve application buffer. Therefore calls to ::fflush() or
+    // ofstream::flush() do not apply.
   }
 
   size_t bytes_written() override { return _bytes_written; }
@@ -77,7 +78,7 @@ class file_sink : public data_sink {
 
     // KvikIO's `pwrite()` returns a `std::future<size_t>` so we convert it
     // to `std::future<void>`
-    return std::async(std::launch::deferred, [this, gpu_data, size, offset] {
+    return std::async(std::launch::deferred, [this, gpu_data, size, offset]() -> void {
       _kvikio_file.pwrite(gpu_data, size, offset).get();
     });
   }
@@ -141,7 +142,7 @@ class void_sink : public data_sink {
                                        rmm::cuda_stream_view stream) override
   {
     _bytes_written += size;
-    return std::async(std::launch::deferred, [] {});
+    return std::async(std::launch::deferred, []() -> void {});
   }
 
   void flush() override {}

docs/cudf/source/user_guide/io/io.md

@@ -80,7 +80,10 @@ IO format.
 - \[¹\] - Not all orientations are GPU-accelerated.
 - \[²\] - Not GPU-accelerated.
 
-## Magnum IO GPUDirect Storage Integration
+## KvikIO Integration
+
+cuDF leverages the [KvikIO](https://github.com/rapidsai/kvikio) library for high-performance
+I/O features, such as parallel I/O operations and Nvidia Magnum IO GPUDirect Storage (GDS).
 
 Many IO APIs can use Magnum IO GPUDirect Storage (GDS) library to optimize
 IO operations.  GDS enables a direct data path for direct memory access
@@ -91,33 +94,27 @@ SDK is available for download
 [here](https://developer.nvidia.com/gpudirect-storage). GDS is also
 included in CUDA Toolkit 11.4 and higher.
 
-Use of GPUDirect Storage in cuDF is disabled by default, but can be
-enabled through the environment variable `LIBCUDF_CUFILE_POLICY` and `KVIKIO_COMPAT_MODE`.
-This variable also controls the GDS compatibility mode.
-
-There are two valid values for the environment variable:
-
-- "KVIKIO": Enable GDS compatibility mode through [KvikIO](https://github.com/rapidsai/kvikio).
-Note that KvikIO also provides the environment variable `KVIKIO_COMPAT_MODE` for GDS
-control that may alter the effect of "KVIKIO" option in cuDF:
-  - By default, `KVIKIO_COMPAT_MODE` is unset. In this case, cuDF enforces
-    the GDS compatibility mode, and the system configuration check for GDS I/O
-    is never performed.
-  - If `KVIKIO_COMPAT_MODE=ON`, this is the same with the above case.
-  - If `KVIKIO_COMPAT_MODE=OFF`, KvikIO enforces GDS I/O without system
-    configuration check, and will error out if GDS requirements are not met. The
-    only exceptional case is that if the system does not support files being
-    opened with the `O_DIRECT` flag, the GDS compatibility mode will be used.
-- "OFF": Completely disable GDS and KvikIO use.
-
-If no value is set, behavior will be the same as the "KVIKIO" option.
-
-This environment variable also affects how cuDF treats GDS errors.
-
-- When `LIBCUDF_CUFILE_POLICY` is set to "KVIKIO" and a KvikIO API
-  call fails for any reason (unlikely, given that KvikIO implements
-  its own compatibility mode) cuDF throws an exception to propagate
-  the error to the user.
+Use of GDS in cuDF is controlled by KvikIO's environment variable `KVIKIO_COMPAT_MODE`. It has
+3 options (case-insensitive):
+
+- `ON` (aliases: `TRUE`, `YES`, `1`): Enable the compatibility mode, which enforces KvikIO POSIX I/O path.
+  This is the default option in cuDF.
+- `OFF` (aliases: `FALSE`, `NO`, `0`): Force-enable KvikIO cuFile (the underlying API for GDS) I/O path.
+  GDS will be activated if the system requirements for cuFile are met and cuFile is properly
+  configured. However, if the system is not suited for cuFile, I/O operations under the `OFF`
+  option may error out, crash or hang.
+- `AUTO`: Try KvikIO cuFile I/O path first, and fall back to KvikIO POSIX I/O if the system requirements
+  for cuFile are not met.
+
+Note that:
+- Even if KvikIO cuFile I/O path is taken, it is possible that GDS is still not activated, where cuFile falls back
+  to its internal compatibility mode. This will happen, for example, on an ext4 file system whose journaling
+  mode has not been explicitly set to `data=ordered`. This may also happen if cuFile's environment variable
+  `CUFILE_FORCE_COMPAT_MODE` is set to true. For more details, refer to
+  [cuFile compatibility mode](https://docs.nvidia.com/gpudirect-storage/api-reference-guide/index.html#cufile-compatibility-mode)
+  and [cuFile environment variables](https://docs.nvidia.com/gpudirect-storage/troubleshooting-guide/index.html#environment-variables).
+- Details of the GDS system requirements can be found in the [GDS documentation](https://docs.nvidia.com/gpudirect-storage/index.html).
+- If a KvikIO API call fails for any reason, cuDF throws an exception to propagate the error to the user.
 
 For more information about error handling, compatibility mode, and
 tuning parameters in KvikIO see: <https://github.com/rapidsai/kvikio>
@@ -133,15 +130,6 @@ Operations that support the use of GPUDirect Storage:
 - {py:meth}`cudf.DataFrame.to_parquet`
 - {py:meth}`cudf.DataFrame.to_orc`
 
-Several parameters that can be used to tune the performance of
-GDS-enabled I/O are exposed through environment variables:
-
-- `LIBCUDF_CUFILE_THREAD_COUNT`: Integral value, maximum number of
-  parallel reads/writes per file (default 16);
-- `LIBCUDF_CUFILE_SLICE_SIZE`: Integral value, maximum size of each
-  GDS read/write, in bytes (default 4MB).  Larger I/O operations are
-  split into multiple calls.
-
 ## nvCOMP Integration
 
 Some types of compression/decompression can be performed using either