Repository navigation
in_tail: fix log duplication when a rotated file is appended during rotate_wait with follow_inodes - #5499
Open
Watson1978 wants to merge 5 commits into
Open
in_tail: fix log duplication when a rotated file is appended during rotate_wait with follow_inodes#5499Watson1978 wants to merge 5 commits into
Watson1978 wants to merge 5 commits into
Conversation
Watson1978
force-pushed
the
in_tail-skip-rotating-inode-on-refresh
branch
from
September 29, 2026 08:20
82f5f8a to
ceb58ad
Compare
Watson1978
marked this pull request as ready for review
October 1, 2026 08:45
Watson1978
marked this pull request as draft
October 1, 2026 09:00
kenhys
pushed a commit
that referenced
this pull request
Oct 6, 2026
… is removed from the watch list (#5515) **Which issue(s) this PR fixes**: Fixes # **What this PR does / why we need it**: A file under a `<group>` limit that left the watch list before EOF, for example through `limit_recently_modified`, never had its remaining lines collected, because `stop_watchers` dropped the path from the group watcher at once and `limit_lines_reached?` then rejected the watcher waiting for `rotate_wait` forever. This PR keeps the group registration and the position entry of such a watcher until it is detached, so it drains the file under the group limit and is closed normally. It also gives every path at least one line of allotment, because a path that had not started reading raised `TypeError` and stopped the refresh timer when the shared limit left it nothing (found while reviewing #5499). **Docs Changes**: N/A **Release Note**: in_tail: fix unread lines never being collected when a throttled file is removed from the watch list 🤖 Generated with [Claude Code](https://claude.com/claude-code) --------- Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Watson1978
force-pushed
the
in_tail-skip-rotating-inode-on-refresh
branch
3 times, most recently
from
October 6, 2026 07:16
b85ed03 to
d5b257a
Compare
…otate_wait with follow_inodes With follow_inodes, a TailWatcher detached on rotation keeps reading its inode until rotate_wait elapses, but refresh_watchers only looked at @Tails and started a second TailWatcher for the same inode. Skip inodes still read by a watcher waiting for rotate_wait. Fixes fluent#4243 Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…opped watcher drains With fluent#5515 the stopped watcher keeps reading the file, and the skip in refresh_watchers must not start a second watcher for the same inode while it does. Check that the appended line is collected once and that the file is followed again after the watcher is closed. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
…h timer is disabled With enable_watch_timer false, a TailWatcher is only notified through the stat watcher on its path. Once the path is rotated away, appends to the old inode no longer notify the watcher, so a watcher stopped by a group limit before EOF never reads again. The close timer only checked eof? and left it in @tails_rotate_wait forever, and the skip in refresh_watchers then blocked a new watcher for that inode. Let the close timer read more of the file for such a watcher until it reaches EOF. Only the read is triggered, not the rotation check, because the path no longer belongs to this watcher. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Watson1978
force-pushed
the
in_tail-skip-rotating-inode-on-refresh
branch
from
October 6, 2026 07:42
d5b257a to
d7ab20a
Compare
…ails A timer started with timer_execute detaches itself when its block raises. A transient BufferOverflowError from the multiline flush in read_more therefore stopped the close timer, and the watcher stayed in @tails_rotate_wait with its remaining lines unread. Log the error and retry on the next tick, as TimerTrigger does for on_notify. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Match TimerTrigger, which calls on_notify every second regardless of EOF, so lines appended to the rotated file during rotate_wait are picked up as they arrive instead of only by the final read at detach. Signed-off-by: Shizuo Fujita <fujita@clear-code.com> Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Watson1978
marked this pull request as ready for review
October 6, 2026 08:46
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Which issue(s) this PR fixes:
Fixes #4243
What this PR does / why we need it:
With
follow_inodes true, a TailWatcher detached on rotation keeps reading its inode untilrotate_waitelapses, butrefresh_watchersonly looks at@tails, so it started a second TailWatcher for the same inode and lines appended to the rotated file in that window were collected twice. This PR makesrefresh_watchersskip inodes still read by a watcher waiting forrotate_wait, and the rotated file is followed again from the recorded position on the next refresh after that watcher is closed. It builds on #5515, which lets such a watcher keep reading under a<group>limit and keeps its position entry, so a file that leaves and re-enters the watch list is neither stalled nor read twice. Withenable_watch_timer falsethe close timer now notifies the waiting watcher itself, because the stat watcher on the rotated path no longer does.Docs Changes:
N/A
Release Note:
in_tail: fix log duplication when a rotated file is appended during
rotate_waitwithfollow_inodes🤖 Generated with Claude Code