Why Python's Packaging Still Frustrates Developers

You know that feeling when you just want to install a simple library and suddenly you're in dependency hell? If you've been using Python for more than a week, you've probably experienced this. And honestly, it shouldn't be this hard in 2024.

I work at PythonSkillset, and I've seen countless developers—from beginners to seasoned pros—hit their heads against Python's packaging wall. The good news is that things are getting better. The bad news? We're not quite there yet.

The Core Problem: Too Many Tools, Too Little Clarity

Let's be real for a second. Python has pip, pipenv, poetry, conda, venv, virtualenv, pyenv, setup.py, setup.cfg, pyproject.toml, requirements.txt... and that's just scratching the surface.

When I started at PythonSkillset, a colleague joked that Python packaging tools multiply faster than rabbits. And honestly? He wasn't wrong.

The real issue isn't that any single tool is bad. It's that there are too many options, and no clear "this is the one you should use" standard. Compare this to JavaScript's npm, Rust's cargo, or Go's modules. Those ecosystems have clear, singular package managers that "just work."

pip: The Good, The Bad, The Ugly

pip does its job. It installs packages from PyPI. But here's where it falls short:

• No dependency resolution for conflicts — pip will happily install conflicting packages and let you deal with the mess
• Global installs by default — until you learn about virtual environments
• No lockfiles natively — pip freeze gives you a list, but not reproducible builds

I can't count how many times I've seen a new PythonSkillset contributor clone a project, run pip install -r requirements.txt, only to face version conflicts that take hours to sort out.

The rise and fall of setup.py

For years, setup.py was the standard way to define Python packages. But there's a dirty secret: setup.py is actually executable code, not just configuration. This means:

from setuptools import setup
import os

# This runs arbitrary code during installation
if os.name == 'nt':
    # Windows-specific setup
    pass

setup(name='mypackage', version='1.0')

This flexibility sounds nice until someone's setup.py downloads malware or deletes files. It's happened. More than once.

Why pyproject.toml matters (and why we're still not there)

PEP 517 and PEP 518 introduced pyproject.toml as a standardized configuration file. This was supposed to be the savior. And in many ways, it is:

[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"

[project]
name = "mypackage"
version = "1.0.0"
dependencies = [
    "requests>=2.28.0",
    "numpy>=1.21.0"
]

Clean, declarative, no code execution. But here's the problem: adoption has been slow. Many popular packages still use setup.py, documentation is inconsistent, and different tools interpret the same pyproject.toml differently.

The poetry experiment

Poetry came in as a breath of fresh air. It handles dependencies, virtual environments, packaging, and publishing all in one tool. It has lockfiles. It just makes sense.

At PythonSkillset, we actually switched to Poetry for internal projects. But it has its own issues:

• Slow dependency resolution — sometimes taking minutes for complex projects
• Incompatibility with pip — you can't just pip install a Poetry project without extra steps
• Plugin ecosystem still developing

One PythonSkillset contributor showed me a project with 50 dependencies that took Poetry 4 minutes to resolve. With pip, it was 30 seconds. But pip's solution was broken.

What actually works (for now)

After years of trial and error at PythonSkillset, here's what I'd recommend:

For beginners: - Use venv (it comes built into Python 3) - Stick with pip and requirements.txt - Don't overthink it

For serious projects: - Use poetry or pipenv for dependency management - Commit your lockfile - Test your builds in CI

For library authors: - Use pyproject.toml with setuptools - Keep dependencies minimal - Test your package installs cleanly

The elephant in the room: conda

Conda solves many of these problems beautifully. It handles non-Python dependencies (like C libraries), has excellent environment management, and resolves dependencies properly. But it's heavy. And it's not always the right tool for the job.

At PythonSkillset, we use conda for data science projects and poetry for web applications. It works, but it's not ideal.

Where we're heading

The Python packaging authority (PyPA) is working on a unified tool called pipenv that should combine the best of all worlds. But it's been in development for years with no stable release in sight.

The truth is: Python's packaging will probably always be more complex than other ecosystems because Python itself is more flexible. You can package Jupyter notebooks, C extensions, Rust bindings, and pure Python code. That flexibility comes at a cost.

What you can do right now

Instead of waiting for the perfect solution:

1. Pick one tool and stick with it — I don't care which one, just commit to it
2. Always use virtual environments — this solves 80% of packaging problems
3. Pin your dependencies — use a lockfile if possible, or at least pin versions in requirements.txt
4. Test in a clean environment — CI is your friend here

At PythonSkillset, we've spent years building guides around these practices because we know how frustrating packaging can be. And honestly? It's getting better. Just slowly.

The packaging experience will improve. But until then, knowing the tools, their limitations, and how to work around them is your best bet. And remember: you're not alone in this struggle. Even the Python core developers have admitted packaging is a mess they're still cleaning up.

So, yes, Python's packaging still frustrates developers. But with a little knowledge and the right workflow, you can avoid most of the pain. And hopefully, in a few years, we'll look back at this era the way we look at Python 2: with a mix of nostalgia and relief that we don't have to deal with it anymore.