How to Download a List of URLs to a Directory in Python

This script downloads a list of URLs into a specified directory, creating the folder if needed and keeping original filenames.

Easy Python 3.9+ Aug 9, 2026 Automation & scripting 16 views 0 copies

urllib download file-io scripting automation

Python code

25 lines

Python 3.9+

import urllib.request
from pathlib import Path

def download_urls(url_list, directory):
    """Download each URL in url_list into directory, keeping original filenames."""
    save_dir = Path(directory)
    save_dir.mkdir(parents=True, exist_ok=True)
    
    for url in url_list:
        filename = url.rstrip('/').split('/')[-1] or 'index.html'
        target = save_dir / filename
        
        try:
            urllib.request.urlretrieve(url, target)
            print(f"Downloaded: {filename}")
        except Exception as e:
            print(f"Failed: {url} -> {e}")

if __name__ == "__main__":
    urls = [
        "https://www.example.com/index.html",
        "https://example.com/robots.txt",
        "https://example.com/favicon.ico"
    ]
    download_urls(urls, "downloads")

Output

stdout

Downloaded: index.html
Downloaded: robots.txt
Downloaded: favicon.ico

How it works

The urllib.request.urlretrieve function downloads a file from a URL to a local path in one call. Path.mkdir(parents=True, exist_ok=True) creates the destination directory and any missing parent folders. The filename is extracted from the URL tail using .split('/')[-1], with a fallback to index.html for URLs ending in a slash. Network or HTTP errors are caught per-file, so one failure does not stop the remaining downloads.

Common mistakes

Forgetting to strip trailing slashes, which produces empty filenames
Assuming the directory exists — always call `mkdir` before writing
Letting one failed URL abort the whole batch instead of handling errors per URL

Variations

Use `requests` for more control over headers, timeouts, and redirects
Add URL-parsing with `urlparse` to derive filenames from the path component

Real-world use cases

Downloading daily report files or nightly data exports from internal servers into an archive folder.
Grabbing static assets like favicons or logos from a list of client sites for a branding audit.
Fetching reference documents from a CMS export list during an automated data migration script.

How to Download a List of URLs to a Directory in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Automation & scripting

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Automation & scripting

Keep learning

Tutorials

Quizzes