Layout API

Data needed by GMP-like import-export functions.

struct PyLongLayout

Layout of an array of digits, used by Python int object.

Use PyLong_GetNativeLayout() to get the native layout of Python int objects.

See also sys.int_info which exposes similar information to Python.

uint8_t bits_per_digit

Bits per digit.

uint8_t digit_size

Digit size in bytes.

int8_t digits_order

Digits order:

• 1 for most significant digit first
• -1 for least significant digit first

int8_t endianness

Digit endianness:

• 1 for most significant byte first (big endian)
• -1 for least significant first (little endian)

const PyLongLayout *PyLong_GetNativeLayout(void)

Get the native layout of Python int objects.

See the PyLongLayout structure.

The function must not be called before Python initialization nor after Python finalization. The returned layout is valid until Python is finalized. The layout is the same for all Python sub-interpreters and so it can be cached.

Export API

struct PyLongExport

Export of a Python int object.

There are two cases:

• If digits is NULL, only use the value member. Calling PyLong_FreeExport() is optional in this case.
• If digits is not NULL, use negative, ndigits and digits members. Calling PyLong_FreeExport() is mandatory in this case.

int64_t value

The native integer value of the exported int object. Only valid if digits is NULL.

uint8_t negative

1 if the number is negative, 0 otherwise. Only valid if digits is not NULL.

Py_ssize_t ndigits

Number of digits in digits array. Only valid if digits is not NULL.

const void *digits

Read-only array of unsigned digits. Can be NULL.

int PyLong_Export(PyObject *obj, PyLongExport *export_long)

Export a Python int object.

On success, set *export_long and return 0. On error, set an exception and return -1.

This function always succeeds if obj is a Python int object or a subclass.

If export_long->digits is not NULL, PyLong_FreeExport() must be called when the export is no longer needed.

On CPython 3.14, no memory copy is needed, it’s just a thin wrapper to expose Python int internal digits array.

If digits not NULL, a private field of the PyLongExport structure stores a strong reference to the Python int object to make sure that that structure remains valid until PyLong_FreeExport() is called.

void PyLong_FreeExport(PyLongExport *export_long)

Release the export export_long created by PyLong_Export().

Import API

The PyLongWriter API can be used to import an integer: create a Python int object from a digits array.

struct PyLongWriter

A Python int writer instance.

The instance must be destroyed by PyLongWriter_Finish().

PyLongWriter *PyLongWriter_Create(int negative, Py_ssize_t ndigits, void **digits)

Create a PyLongWriter.

On success, set *digits and return a writer. On error, set an exception and return NULL.

negative is 1 if the number is negative, or 0 otherwise.

ndigits is the number of digits in the digits array. It must be greater than or equal to 0.

The caller must initialize the array of digits digits and then call PyLongWriter_Finish() to get a Python int. Digits must be in the range [0; PyLong_BASE - 1]. Unused digits must be set to 0.

On CPython 3.14, the implementation is a thin wrapper to the private _PyLong_New() function.

PyObject *PyLongWriter_Finish(PyLongWriter *writer)

Finish a PyLongWriter created by PyLongWriter_Create().

On success, return a Python int object. On error, set an exception and return NULL.

The function takes care of normalizing the digits and converts the object to a compact integer if needed.

void PyLongWriter_Discard(PyLongWriter *writer)

Discard the internal object and destroy the writer instance.

Export: PyLong_Export() with gmpy2

Code:

static void
mpz_set_PyLong(mpz_t z, PyObject *obj)
{
    const PyLongLayout* layout = PyLong_GetNativeLayout();
    static PyLongExport long_export;

    PyLong_Export(obj, &long_export);
    if (long_export.digits) {
        mpz_import(z, long_export.ndigits, layout->digits_order,
                   layout->digit_size, layout->endianness,
                   layout->digit_size*8 - layout->bits_per_digit,
                   long_export.digits);
        if (long_export.negative) {
            mpz_neg(z, z);
        }
        PyLong_FreeExport(&long_export);
    }
    else {
        if (LONG_MIN <= long_export.value && long_export.value <= LONG_MAX) {
            mpz_set_si(z, long_export.value);
        }
        else {
            mpz_import(z, 1, -1, sizeof(int64_t), 0, 0,
                       &long_export.value);
            if (long_export.value < 0) {
                mpz_t tmp;
                mpz_init(tmp);
                mpz_ui_pow_ui(tmp, 2, 64);
                mpz_sub(z, z, tmp);
                mpz_clear(tmp);
            }
        }
    }
}

Reference code: mpz_set_PyLong() in the gmpy2 master for commit 9177648.

Benchmark:

import pyperf
from gmpy2 import mpz

runner = pyperf.Runner()
runner.bench_func('1<<7', mpz, 1 << 7)
runner.bench_func('1<<38', mpz, 1 << 38)
runner.bench_func('1<<300', mpz, 1 << 300)
runner.bench_func('1<<3000', mpz, 1 << 3000)

Results on Linux Fedora 40 with CPU isolation, Python built in release mode:

Import: PyLongWriter_Create() with gmpy2

Code:

static PyObject *
GMPy_PyLong_From_MPZ(MPZ_Object *obj, CTXT_Object *context)
{
    if (mpz_fits_slong_p(obj->z)) {
        return PyLong_FromLong(mpz_get_si(obj->z));
    }

    const PyLongLayout *layout = PyLong_GetNativeLayout();
    size_t size = (mpz_sizeinbase(obj->z, 2) +
                   layout->bits_per_digit - 1) / layout->bits_per_digit;
    void *digits;
    PyLongWriter *writer = PyLongWriter_Create(mpz_sgn(obj->z) < 0, size,
                                               &digits);
    if (writer == NULL) {
        return NULL;
    }

    mpz_export(digits, NULL, layout->endianness,
               layout->digit_size, layout->digits_order,
               layout->digit_size*8 - layout->bits_per_digit,
               obj->z);

    return PyLongWriter_Finish(writer);
}

Reference code: GMPy_PyLong_From_MPZ() in the gmpy2 master for commit 9177648.

Benchmark:

import pyperf
from gmpy2 import mpz

runner = pyperf.Runner()
runner.bench_func('1<<7', int, mpz(1 << 7))
runner.bench_func('1<<38', int, mpz(1 << 38))
runner.bench_func('1<<300', int, mpz(1 << 300))
runner.bench_func('1<<3000', int, mpz(1 << 3000))

Results on Linux Fedora 40 with CPU isolation, Python built in release mode:

Benchmark hidden because not significant (2): 1<<38, 1<<3000.

Support arbitrary layout

It would be convenient to support arbitrary layout to import-export Python integers.

For example, it was proposed to add a layout parameter to PyLongWriter_Create() and a layout member to the PyLongExport structure.

The problem is that it’s more complex to implement and not really needed. What’s strictly needed is only an API to import-export using the Python “native” layout.

If later there are use cases for arbitrary layouts, new APIs can be added.