C Extension Switching¶
Overview¶
NumPy + SciPy Stress Test¶
The Holy Grail of Python Conflicts
This demo performs an action considered “impossible” in standard Python: Swapping C-extension modules (NumPy/SciPy) mid-execution without crashing the interpreter.
This demo is the ultimate stress test for OmniPkg’s LibResolver. It forces the interpreter to load, unload, and reload incompatible C-extension binaries (shared objects/DLLs) within the same process.
Usage¶
Prerequisite: This demo requires Python 3.11 (due to specific binary wheels used in the test). Try it now: Cloud users can click the play button to run this demo instantly:
To run this demo locally:What You’ll Learn¶
- C-Extension Hot-Swapping: How OmniPkg manages the memory lifecycle of shared libraries (
.so/.pyd). - Binary Compatibility: How it ensures
numpy==1.24.3(linked to older BLAS) doesn’t crash when swapped for1.26.4. - The “Nuclear” Swap: Watch it combine incompatible pairs (
numpy==1.24.3 + scipy==1.12.0) vs (numpy==1.26.4 + scipy==1.16.1) in real-time.
The Code¶
This script performs a series of high-speed swaps between incompatible versions of NumPy and SciPy, verifying that the math operations (which rely on C code) return correct results every time.
from omnipkg.loader import omnipkgLoader
import numpy as np
# 1. Baseline: NumPy 1.26.4 (Main Env)
print(f"Baseline Version: {np.__version__}")
# Output: 1.26.4
# 2. The "Impossible" Swap: NumPy 1.24.3
# This involves unloading C-extensions and reloading older binaries
with omnipkgLoader("numpy==1.24.3"):
import numpy as np
# Verify we are running C-code from 1.24.3
print(f"Swapped Version: {np.__version__}")
# Output: 1.24.3
# Verify math accuracy (ensure memory isn't corrupted)
print(f"Array Sum: {np.array([1, 2, 3]).sum()}")
# Output: 6
# 3. Restore: Back to 1.26.4
# OmniPkg handles the cleanup and re-linking automatically
import numpy as np
print(f"Restored Version: {np.__version__}")
# Output: 1.26.4
Live Execution Log¶
Pay attention to the Activation Time. OmniPkg swaps these massive C-libraries in ~118ms.
(evocoder_env) minds3t@aiminingrig:~/omnipkg$ omnipkg demo 3
============================================================
๐ STEP 3: Executing the Nuclear Test
============================================================
๐ฅ NUMPY VERSION JUGGLING:
โก Switching to numpy==1.24.3
๐งน Purging 1 module(s) from memory...
- ๐ STRICT mode
- ๐ฉ Activating binary path: .../numpy-1.24.3/bin
โก HEALED in 21,684.8 ฮผs
โ
Version: 1.24.3
๐ข Array sum: 6
โก Activation time: 118.52ms
๐ฏ Version verification: PASSED
๐ omnipkg loader: Deactivating numpy==1.24.3...
โ
Environment restored.
โฑ๏ธ Swap Time: 47,614.668 ฮผs
โก Switching to numpy==1.26.4
โ
Version: 1.26.4
๐ข Array sum: 6
โก Activation time: 119.48ms
๐ฏ Version verification: PASSED
๐คฏ NUMPY + SCIPY VERSION MIXING:
๐ COMBO: numpy==1.24.3 + scipy==1.12.0
๐งช numpy: 1.24.3, scipy: 1.12.0
๐ Compatibility check: [1. 2. 3.]
๐ฏ Version verification: BOTH PASSED!
โก Total combo time: 162.69ms
๐ COMBO: numpy==1.26.4 + scipy==1.16.1
๐งช numpy: 1.26.4, scipy: 1.16.1
๐ Compatibility check: [1. 2. 3.]
๐ฏ Version verification: BOTH PASSED!
โก Total combo time: 130.39ms
๐จ OMNIPKG SURVIVED NUCLEAR TESTING! ๐
How It Works¶
- Memory Purge: Before swapping, OmniPkg forces a garbage collection and manually unregisters the C-extension modules from
sys.modules. - Binary Path Injection: It injects the
binpath of the target version intoLD_LIBRARY_PATH(Linux) orPATH(Windows) to ensure the dynamic linker finds the correct.so/.dllfiles. - Strict Mode: For packages identified as “C-Heavy” (like NumPy), OmniPkg enables Strict Mode, which performs deeper verification to prevent segmentation faults.