Multi-context support for C extensions

[timfel]

TL;DR

Native extension libraries such as for popular DS and ML packages cannot be loaded multiple times into the same process, precluding their use in embeddings that use multiple GraalPy contexts. We will try to alleviate this issue for all native extensions.

Goals

The primary goal for allowing native extensions in multiple contexts is to enable the scaling of GraalPy embeddings. While each GraalPy context is subject to the Python global interpreter lock (GIL), multiple GraalPy contexts can be used to execute Python code in parallel on different Java threads while sharing compiled code caches. If we managed to allow native extensions to be loaded multiple times and used in different contexts, this scaling strategy could be extended to the vast majority of Python packages.

Non-Goals

• Complete isolation between native extensions. The goal is to be able to load and run native extensions across contexts, but native code will have full access to the process memory and resources. Thus, ill-behaved native extensions can still affect other GraalPy contexts.