cpp_and_numpy.rst

.. toctree::
    :maxdepth: 2

C++ and the Numpy C API

Numpy is a powerful arrary based data structure with fast vector and array operations. It has a fully featured C API. This section describes some aspects of using Numpy with C++.

Initialising Numpy

The Numpy C API must be setup so that a number of static data structures are initialised correctly. The way to do this is to call import_array() which makes a number of Python import statements so the Python interpreter must be initialised first. This is described in detail in the Numpy documentation so this document just presents a cookbook approach.

Verifying Numpy is Initialised

import_array() always returns NUMPY_IMPORT_ARRAY_RETVAL regardless of success instead we have to check the Python error status:

#include <Python.h>
#include "numpy/arrayobject.h" // Include any other Numpy headers, UFuncs for example.

// Initialise Numpy
import_array();
if (PyErr_Occurred()) {
    std::cerr << "Failed to import numpy Python module(s)." << std::endl;
    return NULL; // Or some suitable return value to indicate failure.
}

In other running code where Numpy is expected to be initialised then PyArray_API should be non-NULL and this can be asserted:

assert(PyArray_API);

Numpy Initialisation Techniques

Initialising Numpy in a CPython Module

Taking the simple example of a module from the Python documentation we can add Numpy access just by including the correct Numpy header file and calling import_numpy() in the module initialisation code:

#include <Python.h>

#include "numpy/arrayobject.h" // Include any other Numpy headers, UFuncs for example.

static PyMethodDef SpamMethods[] = {
    ...
    {NULL, NULL, 0, NULL}        /* Sentinel */
};

static struct PyModuleDef spammodule = {
   PyModuleDef_HEAD_INIT,
   "spam",   /* name of module */
   spam_doc, /* module documentation, may be NULL */
   -1,       /* size of per-interpreter state of the module,
                or -1 if the module keeps state in global variables. */
   SpamMethods
};

PyMODINIT_FUNC
PyInit_spam(void) {
    ...
    assert(! PyErr_Occurred());
    import_numpy(); // Initialise Numpy
    if (PyErr_Occurred()) {
        return NULL;
    }
    ...
    return PyModule_Create(&spammodule);
}

That is fine for a singular translation unit but you have multiple translation units then each has to initialise the Numpy API which is a bit extravagant. The following sections describe how to manage this with multiple translation units.

Initialising Numpy in Pure C++ Code

This is mainly for development and testing of C++ code that uses Numpy. Your code layout might look something like this where main.cpp has a main() entry point and class.h has your class declarations and class.cpp has their implementations, like this:

.
└── src
    └── cpp
        ├── class.cpp
        ├── class.h
        └── main.cpp

The way of managing Numpy initialisation and access is as follows. In class.h choose a unique name such as awesome_project then include:

#define PY_ARRAY_UNIQUE_SYMBOL awesome_project_ARRAY_API
#include "numpy/arrayobject.h"

In the implementation file class.cpp we do not want to import Numpy as that is going to be handled by main() in main.cpp so we put this at the top:

#define NO_IMPORT_ARRAY
#include "class.h"

Finally in main.cpp we initialise Numpy:

#include "Python.h"
#include "class.h"

int main(int argc, const char * argv[]) {
    // ...
    // Initialise the Python interpreter
    wchar_t *program = Py_DecodeLocale(argv[0], NULL);
    if (program == NULL) {
        fprintf(stderr, "Fatal error: cannot decode argv[0]\n");
        exit(1);
    }
    Py_SetProgramName(program);  /* optional but recommended */
    Py_Initialize();
    // Initialise Numpy
    import_array();
    if (PyErr_Occurred()) {
        std::cerr << "Failed to import numpy Python module(s)." << std::endl;
        return -1;
    }
    assert(PyArray_API);
    // ...
}

If you have multiple .h, .cpp files then it might be worth having a single .h file, say numpy_init.h with just this in:

#define PY_ARRAY_UNIQUE_SYMBOL awesome_project_ARRAY_API
#include "numpy/arrayobject.h"

Then each implementation .cpp file has:

#define NO_IMPORT_ARRAY
#include "numpy_init.h"
#include "class.h" // Class declarations

And main.cpp has:

#include "numpy_init.h"
#include "class_1.h"
#include "class_2.h"
#include "class_3.h"

int main(int argc, const char * argv[]) {
    // ...
    import_array();
    if (PyErr_Occurred()) {
        std::cerr << "Failed to import numpy Python module(s)." << std::endl;
        return -1;
    }
    assert(PyArray_API);
    // ...
}

Initialising Numpy in a CPython Module using C++ Code

Supposing you have laid out your source code in the following fashion:

.
└── src
    ├── cpp
    │   ├── class.cpp
    │   └── class.h
    └── cpython
        └── module.c

This is a hybrid of the above and typical for CPython C++ extensions where module.c contains the CPython code that allows Python to access the pure C++ code.

The code in class.h and class.cpp is unchanged and the code in module.c is essentially the same as that of a CPython module as described above where import_array() is called from within the PyInit_<module> function.