Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Unmount actions getting denied on vector file source #22297

Open
LinTechSo opened this issue Jan 26, 2025 · 1 comment
Open

Unmount actions getting denied on vector file source #22297

LinTechSo opened this issue Jan 26, 2025 · 1 comment
Labels
source: file Anything `file` source related type: bug A code related bug.

Comments

@LinTechSo
Copy link

LinTechSo commented Jan 26, 2025

A note for the community

No response

Problem

I'm trying to collect log files using Vector's file source. However, this keeps the file open indefinitely, resulting in umount actions getting denied. This issue occurs even when the file writer process has been stopped, as Vector continues to read the file. Is there any way to follow the logs without preventing umount actions?

Configuration

Reproduction Steps

mkdir -p /opt/test/
dd if=/dev/zero of=/opt/test/ bs=1M count=1024
losetup /dev/loop0 /opt/test/
mkfs.ext4 /dev/loop0
mkdir -p ./opt/test/data
mount ./dev/loop0 /opt/test/data

vector writer:

sources:
  demo_logs:
    type: demo_logs
    format: json
    interval: 1

sinks:
  log_writer:
    type: file
    encoding:
      codec: json
    inputs:
      - demo_logs
    path: ./data/test.log

vector reader:

sources:
  tail-test:
    type: file
    ignore_older_secs: 600
    include:
      - "./data/test.log"
    read_from: beginning

sinks:
  console_all:
    type: console
    inputs:
      - tail-test
    encoding:
      codec: json

When I stop the Vector writer and try to umount ./data, I get the error: umount: /opt/test/data: target is busy.

Version

vector 0.44.0 (x86_64-unknown-linux-gnu 3cdc7c3 2025-01-13 21:26:04.735691656)

@LinTechSo LinTechSo added the type: bug A code related bug. label Jan 26, 2025
@LinTechSo
Copy link
Author

LinTechSo commented Jan 26, 2025

Assuming I am using tail, the above problem is partially fixed when I use inotifywait. However, how can I fix this issue with Vector?

#!/bin/bash

logfile="/path/to/logfile"

# Monitor the file for changes and output new content when modified
inotifywait -m -e modify "$logfile" |
while read path action file; do
    echo "File $file was modified. Showing new content:"
    tail -n 10 "$logfile"
done

@pront pront added the source: file Anything `file` source related label Jan 27, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
source: file Anything `file` source related type: bug A code related bug.
Projects
None yet
Development

No branches or pull requests

2 participants