Prompting a file download from get request python

John Zwinck John Zwinck k 31 31 gold badges silver badges bronze badges. Note that you may need to adjust when streaming gzipped responses per issue THIS should be the correct answer! A small caveat for using. Mentioned in the docs here: docs. EricCousineau You can patch up this behaviour replacing the read method: response. Adding length param got me better download speeds shutil. Show 18 more comments. Am I missing something? For Python 2. Community Bot 1 1 1 silver badge.

I use System Monitor in Kubuntu. It shows me that python process memory increases up to 1. That memory bloat sucks, maybe f. That's what's causing the memory bloat. Show 1 more comment. Ben Moskovitch Ben Moskovitch 1 1 silver badge 3 3 bronze badges. Here is a snippet import wget wget. Sign up or log in Sign up using Google. Sign up using Facebook. Sign up using Email and Password. Post as a guest Name. Email Required, but never shown.

The Overflow Blog. Email me if you're having that issue, because it likely means you probably don't have Anaconda installed properly. The get method of the requests module is the one we will use most frequently — which corresponds to how the majority of the HTTP requests your browser makes involve the GET method. The get method requires one argument: a web URL, e. The URL's scheme — i. But it turns out there's a lot more to getting a webpage than just getting what you see rendered in your browser.

What each of those various attributes mean isn't important to figure out now, it's just enough to know that they exist as part of every request for a web resource, whether it's a webpage, image file, data file, etc. Returning to our previous code snippet, let's assign the result of the requests. When the URL linked to a webpage rather than a binary, I had to not download that file and just keep the link as is.

To solve this, what I did was inspecting the headers of the URL. Headers usually contain a Content-Type parameter which tells us about the type of data the url is linking to. A naive way to do it will be -. It works but is not the optimum way to do so as it involves downloading the file for checking the header.

So if the file is large, this will do nothing but waste bandwidth. I looked into the requests documentation and found a better way to do it. That way involved just fetching the headers of a url before actually downloading it.

This allows us to skip downloading files which weren't meant to be downloaded. To restrict download by file size, we can get the filesize from the Content-Length header and then do suitable comparisons. We can parse the url to get the filename. This will be give the filename in some cases correctly. However, there are times when the filename information is not present in the url. In that case, the Content-Disposition header will contain the filename information.

Here is how to fetch it.