Sitelet https://web.archive.org/web/20240427165628/https://github.com/node-fetch/node-fetch/issues/1153
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Connection pooling/reuse for multiple back-to-back requests to the same host #1153

Open
stbrody opened this issue May 10, 2021 · 1 comment

Comments

@stbrody
Copy link

stbrody commented May 10, 2021

Is your feature request related to a problem? Please describe.

We are using the cross-fetch package as part of the http-client for the Ceramic protocol. We've noticed that if we try to fetch from the same http endpoint a few thousand times in a row (as part of the client loading a few thousand streams from the same Ceramic server node) that the number of open sockets/file descriptors on the client machine performing the request goes up according to how many requests we make. It seems like each new request is making a brand new TCP connection even though all the requests are going to the exact same server.

Describe the solution you'd like

Ideally if we have 1000 back to back requests to the same server, they should all be done over the same connection, rather than requiring 1000 brand new TCP connections.

Describe alternatives you've considered

We can work around the issues we are seeing in our tests by just upping the max open file descriptors limit in the OS, but that's not ideal as there's still the overhead of establishing all those unnecessary connections.

Additional context

@epiphone
Copy link

Is this solved by passing in a custom HTTP(S) agent? You can specify keepAlive: true and maxSockets if necessary.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants