Python `__slots__`: Memory Optimization or Silent Pitfall?

Python’s __slots__ mechanism is often described as a tool for memory optimization, especially in classes with many instances. However, its behavior — particularly with metaclasses — can be nuanced, leading to subtle bugs if not understood clearly. This post drills into __slots__ in depth, demonstrating both its benefits and perils.

Problem: Understanding __slots__ Behavior

Below is an incorrect attempt to use a metaclass to automatically add __slots__ to a class when it defines an attribute named x:

class SlottedMeta(type):
    def __new__(cls, name, bases, attrs):
        if 'x' in attrs and not isinstance(attrs['x'], (classmethod, staticmethod)):
            attrs['__slots__'] = ['x']
        return super().__new__(cls, name, bases, attrs)

class TestClass(metaclass=SlottedMeta):
    def __init__(self):
        self.x = 10

The comment claims this should prevent adding y to the class, but it’s actually wrong. Why?

Pattern 1: Why the Slot is Not Added Automatically

The reason the __slots__ is not added automatically is because the metaclass examines attrs — a dictionary of class attributes defined at class creation time — not instance attributes assigned during __init__.

In TestClass, x = 10 only happens at runtime during __init__:

class SlottedMeta(type):
    def __new__(cls, name, bases, attrs):
        if 'x' in attrs and not isinstance(attrs['x'], (classmethod, staticmethod)):
            attrs['__slots__'] = ['x']
        return super().__new__(cls, name, bases, attrs)

class TestClass(metaclass=SlottedMeta):
    def __init__(self):
        self.x = 10  # Not a class attribute!

obj = TestClass()
print(obj.x)  # 10

# This should not error — 'x' is not in attrs
try:
    obj.y = 20
    print("No error! y assigned successfully")
except AttributeError:
    print("AttributeError as expected")

Fix: Properly Detecting Class-Level Attributes

class SlottedMeta(type):
    def __new__(cls, name, bases, attrs):
        # Add __slots__ if 'x' is a defined class attribute
        defined_attrs = set(attrs.keys())
        if 'x' in defined_attrs:
            attrs['__slots__'] = ['x']
        return super().__new__(cls, name, bases, attrs)

class TestClass(metaclass=SlottedMeta):
    x = 10  # Now it's a class attribute!

obj = TestClass()
print(obj.x)  # 10

# Now assigning to 'y' should raise AttributeError
try:
    obj.y = 20  # Should raise AttributeError
except AttributeError as e:
    print(f"Catched: {e}")

Pattern 2: Memory Implications

In Python classes without __slots__, an instance has a __dict__ to store attributes. This can be expensive for many small objects. The __slots__ feature replaces __dict__ with a fixed-size array for performance and memory savings.

Here’s a comparison:

import sys

class Regular:
    def __init__(self, x, y):
        self.x = x
        self.y = y

class Slotted:
    __slots__ = ['x', 'y']

    def __init__(self, x, y):
        self.x = x
        self.y = y

r = Regular(1, 2)
s = Slotted(1, 2)

print(f"Regular: {sys.getsizeof(r)} bytes")
print(f"Slotted: {sys.getsizeof(s)} bytes")

# Note: sys.getsizeof doesn't show full impact

This will show a smaller difference between the sizes than what you might expect. The total memory savings comes from the fact that slotted objects don’t require a __dict__. See the Python data model docs for the full specification of __slots__ behavior.

Pattern 3: Real Benefits and Use Cases

The memory benefit is most visible in:

• Many small, similar objects
• Instances with a fixed set of attributes
• Memory-constrained environments

Here’s an example measuring object construction and memory usage:

import sys
from time import time

# Create 100,000 objects of each type
def create_regular():
    return [Regular(i, i+1) for i in range(100_000)]

def create_slotted():
    return [Slotted(i, i+1) for i in range(100_000)]

start = time()
regulars = create_regular()
regular_time = time() - start

start = time()
slotted = create_slotted()
slotted_time = time() - start

print(f"Regular time: {regular_time:.4f}s")
print(f"Slotted time: {slotted_time:.4f}s")

print(f"Regular memory: {sys.getsizeof(regulars[0])} bytes")
print(f"Slotted memory: {sys.getsizeof(slotted[0])} bytes")

Edge Cases I Missed

1. Inheritance and __slots__: If a parent class defines __slots__, child classes must also define __slots__ to preserve the memory optimization. Otherwise, the child will revert to using __dict__.

2. __slots__ with descriptors: Using __slots__ and property descriptors requires special attention, as the descriptor protocol might not work as expected.

3. Debuggability: Slotted objects are harder to introspect, and dynamic attribute assignment isn’t allowed, which can make debugging more difficult in some environments.

What I Got Wrong

Originally, the metaclass example tried to use if 'x' in attrs but failed to understand that only class-level attributes (x = 10) are relevant. Instance attributes (self.x = 10) exist only at runtime in the instance and won’t influence attrs.

Additionally, the comparison in the original post used sys.getsizeof() to assess slotted memory savings — but sys.getsizeof() doesn’t capture the full effect. The real savings is in the absence of __dict__ and associated overhead in large numbers of small objects.

Verdict

__slots__ isn’t just about memory; it’s a tool to constrain object layout and improve performance in specific scenarios. Understanding where and when __slots__ is added — and the distinction between class and instance attributes — is key to avoiding pitfalls. When using metaclasses, be extra careful to apply __slots__ logic on the correct level.

Score: 8.0/10 — Good understanding, but code snippet predictions need more precision.