libuv update in 18.18.0 breaks webpack's thread-loader

Version

v18.18.0

Platform

Linux abulia 6.5.0-1-amd64 #1 SMP PREEMPT_DYNAMIC Debian 6.5.3-1 (2023-09-13) x86_64 GNU/Linux

Subsystem

No response

What steps will reproduce the bug?

Starting in an empty directory,

Create the following files:

package.json:

{
	"dependencies": {
		"thread-loader": "^4.0.2",
		"webpack": "^5.88.2",
		"webpack-cli": "^4.10.0"
	}
}

webpack.config.js:

const path = require( 'path' );

module.exports = {
    mode: 'production',
    entry: './index.js',
    output: {
        path: path.resolve( __dirname, 'dist' ),
        filename: 'foo.bundle.js',
    },
    module: {
        rules: [
            {
                test: /\.(js)$/,
                exclude: /node_modules/,
                use: [
                    {
                        loader: require.resolve( 'thread-loader' ),
                    },
                ],
            }
        ]
    },
};

Run node -e 'console.log( "console.log( typeof {} );" ); for(let i=0; i<5000; i++){ console.log( "// we need a large file in order to trigger the bug" ); }' > index.js. Increase the "5000" if necessary, I don't know whether it depends on the system in some manner.
npm install
npm exec webpack

How often does it reproduce? Is there a required condition?

Always. No requirements I know of beyond what's described above and being on Linux.

What is the expected behavior? Why is that the expected behavior?

asset foo.bundle.js 22 bytes [emitted] [minimized] (name: main)
./index.js 254 KiB [built] [code generated]
webpack 5.88.2 compiled successfully in 376 ms

Because that's what happens when webpack runs successfully.

What do you see instead?

Process hangs before producing any output. CPU is not being used, processes seem to be waiting in epoll_pwait.

Additional information

It worked in 18.17.1. fb2b80f appears to be the first failing commit.

Setting UV_USE_IO_URING=0 works around the problem. Of course, that's undocumented and temporary so a real fix would be good.

I found this problem exists in 20.3.0 through 20.6.1 as well, but is not present in 20.7.0. A git bisect turned up 88ba79b as the commit that fixed it. Cherry-picking that commit onto 18.18.0 fixes it for me there too.

So my guess is that the new io_uring stuff started returning short writes when passed a lot of data, which WriteStream didn't handle correctly until 88ba79b fixed it.

See also webpack/thread-loader#191

The bug fix will be backported to v18.x automatically, it's just a matter of waiting until it shows up in a release.

In Astro we had to pin to 18.17.1 after GitHub upgraded its CI to use 18.18. Not getting good stacktraces or failures, just seems to hang. Sounds similar to this issue.

I found this problem exists in 20.3.0 through 20.6.1 as well, but is not present in 20.7.0. A git bisect turned up 88ba79b as the commit that fixed it. Cherry-picking that commit onto 18.18.0 fixes it for me there too.

I've cherry-picked 88ba79b (and a4928b0 which it depends on to land cleanly) onto v18.x-staging so it's ready for the next non-security Node.js 18 release. Tentatively we have a release planned for October, see nodejs/Release#737 for the plan, but bear in mind the plan is subject to change based on releaser availability.

FYI @nodejs/lts @nodejs/releasers

I'm wondering, since this appears to be a known bug (for a while?) in 20.x, and v18 is currently the only LTS line, shouldn't have the offending commit be either omitted from the v18.18.0 release, or the v18.18.0 release be post-poned while waiting for the fix, or a bugfix release being made now with v18.18.1, as opposed to be waiting for a 18.19.0 release sometime in October?

It's just a bit surprising, but I also don't have the entire context.

Node 18.18 is broken, see: nodejs/node#49911

2023-10-10, Version 18.18.1 'Hydrogen' (LTS)

I think here the label would also have to be supplemented with Node20.
Node > 20.2.0 also has the bug

2023-10-10, Version 18.18.1 'Hydrogen' (LTS)

v18.18.1 fix the Problem...
In Node 20 the problem is still present

Use newer version of Node

I think here the label would also have to be supplemented with Node20. Node > 20.2.0 also has the bug

It was fixed in 20.7.0 in my testing.

In my testing is the same Bug in 20.7.0 and 20.8.0

I just it tested again, 20.6.1 reproduce following the instructions here while 20.7.0 and 20.8.0 do not. Are you following the instructions on this bug, or are you doing something different?

For us, the error occurs in connection with writestream and CIFS mount

#50061

The fix for the issue reported by OP was released in v18.18.1 and v20.7.0. I think some of you may be talking about different-but-similar-looking bugs that are tracked elsewhere so I'll go ahead and close this report.

@simatec

Oh, and @simatec, if you or anyone still experiences issues, then please open (or reopen) an issue but - and this is critical - include full steps to reproduce, without third-party code. That means no npm modules.

I think the problem still exists in Node 20.8.0. It has only been fixed in 18.18.1 so far. The last working version of Node 20 was 20.2.0

@simatec

Oh, and @simatec, if you or anyone still experiences issues, then please open (or reopen) an issue but - and this is critical - include full steps to reproduce, without third-party code.

I had already done that, see issue in the link

#50061

Missing the critical bit though: steps to reproduce, see #50061 (comment).

Update frontend to Node 18 (#861)

2023-10-10, Version 18.18.1 'Hydrogen' (LTS)

Navigation Menu

Repository navigation

libuv update in 18.18.0 breaks webpack's `thread-loader` #49911

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

libuv update in 18.18.0 breaks webpack's thread-loader #49911

Description

Version

Platform

Subsystem

What steps will reproduce the bug?

How often does it reproduce? Is there a required condition?

What is the expected behavior? Why is that the expected behavior?

What do you see instead?

Additional information

Activity

bnoordhuis commented on Sep 28, 2023

matthewp commented on Sep 28, 2023

richardlau commented on Sep 28, 2023

dnalborczyk commented on Sep 28, 2023

simatec commented on Oct 9, 2023

simatec commented on Oct 10, 2023

anomiex commented on Oct 11, 2023

simatec commented on Oct 11, 2023

anomiex commented on Oct 11, 2023

simatec commented on Oct 11, 2023

bnoordhuis commented on Oct 11, 2023

bnoordhuis commented on Oct 11, 2023

simatec commented on Oct 11, 2023

simatec commented on Oct 11, 2023

bnoordhuis commented on Oct 11, 2023

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions

libuv update in 18.18.0 breaks webpack's `thread-loader` #49911