Repository navigation
Node fails to download entire (large) file via http/https when agent: false is used. Fails on 0.10, 0.12, 4.1.1. #3055
Description
Activity
I'm not seeing what you're describing. I've ran the non-pipe and pipe versions of your script against the master branch with the same url for ~20 mins (with many, many requests during that time period) and did not see any errors.
@mscdex Thanks. You are right. It appears that we (at my office) can only repro this when running over our office network. Yuck. FWIW, I cannot repro when doing (at least as close as I can get) similar with
curl:$ while true; do echo ""; curl -sSf https://nodejs.org/dist/v4.1.1/node-v4.1.1-darwin-x64.tar.gz -o out.file --no-keepalive; \ls -l out.file; done -rw-r--r-- 1 trentm staff 9544550 Sep 24 15:05 out.file -rw-r--r-- 1 trentm staff 9544550 Sep 24 15:05 out.file -rw-r--r-- 1 trentm staff 9544550 Sep 24 15:05 out.file ...That makes me think it is a node issue, but something to do with a funky interaction with my office network. I'll follow up if I can find something to go on.
Possible duplicate of nodejs/node-v0.x-archive#8488.
is this still an issue?
Closing as there does not appear to be anything further to do. Can reopen if necessary
When attempting to download a (largish) file via
https.requestand usingagent: false, the end of the file will frequently be missing.https://gist.github.com/trentm/80409ded30b4e0418e9c has a demo script and sample runs showing it failing on node 0.10.40 and node 4.1.1. In my experience I get a failure between 10% - 80% of the time.
There is a sample run with
agent: falsecommented out which shows successful downloads a dozen or more times in a row.My sample script just does manual writing of 'data' chunks from the
res(response) to a file (fs.createWriteStream): https://gist.github.com/trentm/80409ded30b4e0418e9c#file-download-sans-agent-js-L83-L86This was me trying to eliminate
.pipe()usage as a possible cause.https://gist.github.com/trentm/5d3c4d9dbcb7c3b8dbaa is an equivalent script (and sample runs) using
res.pipe(outStream). It fails frequently as well.