Python Metaclass Inheritance Pitfalls: When C and Python Metaclasses Collide

Contents

The Bottom Line

Combining C-defined and Python-defined metaclasses in Python can trigger a TypeError: type.__new__(MetaTest) is not safe due to unsafe method resolution in tp_new. This occurs when a C metaclass uses MRO to call a Python __new__ method. Two structural constraints — safe tp_new chaining and tp_basicsize requirements — govern whether such combinations will work. Reordering bases or modifying C extension code can resolve it.

Key Insight: C-level tp_new functions must never invoke Python-level __new__ methods. Always call tp_base->tp_new directly for safe cooperative inheritance.

The Problem

When defining a class that inherits from multiple metaclasses — one implemented in C (CMeta) and one in Python (PyMeta) — instance creation fails with:

TypeError: type.__new__(MetaTest) is not safe, use CMeta.__new__()

This happens because the C metaclass’s tp_new uses method resolution order (MRO) to find the next constructor, which resolves to the Python __new__ method. The Python runtime prevents this for safety: C code should not invoke arbitrary Python __new__ implementations.

Failing Code Example

class PyMeta(type):
    def __new__(meta, name, bases, attrs):
        return super(PyMeta, meta).__new__(meta, name, bases, attrs)

class MetaTest(CMeta, PyMeta):  # CMeta first
    pass

class Test(metaclass=MetaTest):
    pass

# This raises TypeError
obj = Test()

The error occurs during the super() call in PyMeta.__new__, which is reached via CMeta.tp_new’s MRO walk.

The Fix

Option 1: Reorder Metaclass Bases

Place the Python metaclass before the C metaclass:

class MetaTest(PyMeta, CMeta):  # PyMeta first
    pass

This works because MRO now starts with PyMeta, and if CMeta.tp_new doesn’t attempt to call further __new__ methods, the chain terminates safely. However, this is fragile — if CMeta.tp_new still walks MRO, the error may persist.

Option 2: Modify C `tp_new` Implementation

Instead of using dynamic MRO, have CMeta.tp_new call:

type->tp_base->tp_new(...);

This bypasses MRO entirely and safely chains to the next C-level tp_new. However, this skips any Python __new__ in the hierarchy, breaking cooperative inheritance.

Option 3: Increase `tp_basicsize`

Ensure your C type has a larger tp_basicsize than its base:

tp_basicsize > base_type->tp_basicsize

This guarantees it will be selected as the __base__ even if not first in __bases__, avoiding the MRO hazard.

Why It Matters

This edge case affects:

C extension developers who allow subclassing of their types from Python
Framework authors using metaclass composition
ZODB, SQLAlchemy, and similar ORM tools that define C-backed types

Without understanding these rules, developers may ship extensions that break when combined with Python metaclasses — failing only in multiple inheritance scenarios that are hard to test.

The issue was first reported in 2004 (BPO #963246) and remains in CPython’s behavior. While rare, it’s a silent correctness hazard — code works in isolation but fails when composed.

Key Takeaway

Never let C tp_new invoke Python __new__ — always use tp_base->tp_new
Document tp_basicsize requirements for any C type meant to be inherited
Test metaclass composition — especially with mixed C/Python hierarchies
Prefer composition over multiple metaclass inheritance when possible

This case exemplifies the tension between Python’s dynamic dispatch and C extension safety. The runtime’s protection prevents crashes but surfaces as cryptic TypeErrors — making diagnosis hard without understanding the C API rules.