How to implement round-robin load balancing in Python

Implement a client-side round-robin load balancer that distributes requests sequentially across a list of mock servers using itertools.cycle.

Easy Python 3.6+ Aug 9, 2026 Microservices patterns 14 views 0 copies

load balancing round robin microservices itertools mock

Python code

30 lines

Python 3.6+

import itertools
import random


class MockServer:
    def __init__(self, name):
        self.name = name

    def handle_request(self, request_id):
        return f"Server {self.name} handled request #{request_id}"


class RoundRobinLoadBalancer:
    def __init__(self, servers):
        self.servers = servers
        self.pool = itertools.cycle(servers)
        self.request_counter = itertools.count(1)

    def dispatch(self):
        server = next(self.pool)
        request_id = next(self.request_counter)
        return server.handle_request(request_id)


if __name__ == "__main__":
    servers = [MockServer("A"), MockServer("B"), MockServer("C")]
    lb = RoundRobinLoadBalancer(servers)

    for _ in range(7):
        print(lb.dispatch())

Output

stdout

Server A handled request #1
Server B handled request #2
Server C handled request #3
Server A handled request #4
Server B handled request #5
Server C handled request #6
Server A handled request #7

How it works

The core of this pattern is itertools.cycle, which infinitely iterates over the list of servers, yielding each one in order and wrapping back to the beginning. A separate counter from itertools.count assigns a unique request ID to each dispatch, simulating real request tracking. This combination gives you a deterministic, stateless round-robin algorithm without manually tracking indices. The MockServer class encapsulates the server behavior, making it easy to swap in real HTTP client calls in production.

Common mistakes

Modifying the server list while the cycle is active, which can cause inconsistent behavior
Forgetting to handle the case of an empty server list, which raises StopIteration
Using a mutable iterator state that is not thread-safe when sharing across threads

Variations

Replace itertools.cycle with an index counter that resets to 0 after reaching the end
Add weight support by repeating heavier servers in the list before building the cycle

Real-world use cases

Distributing outgoing HTTP requests across multiple API gateway instances from a client library.
Rotating through a list of database read replicas for query balancing in a service layer.
Sending tasks to a pool of worker nodes in a message producer without centralized coordination.

How to implement round-robin load balancing in Python

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Microservices patterns

Tutorials

Quizzes

Python code

Output

How it works

Common mistakes

Variations

Real-world use cases

More from Microservices patterns

Keep learning

Tutorials

Quizzes