RLAI Reinforcement Learning and Artificial Intelligence (RLAI)

Mixing Python and C
The ambition of this page is to provide information to the RL group about crossing the language boundary between Python and C. Simply put, this page is meant to be a tutorial on calling C from Python and vice versa. Any questions can be posted as an extension, which I will attempt to answer quickly.
Back to /RLAI/RLsoftware/PythonLinks.html

Creating a Python module in C

In order to call C functions from Python, you must write an interface between the two languages. This interface will take the form of a C source file, which will be compiled into a shared library (a .so file). The shared library can then be used as a normal Python module. In the interface, you create a set of Python methods. Basically, you create one Python method for every C function that you wish call from Python.

For example, if you have a C function:

int sum(int a, int b)

that you wish to call from Python:

import mod
s = mod.sum(3,5)

Then, you would create a file mod.c which exports a Python method named sum:

#include <Python.h>
#include "sum.h"

static PyObject* mod_sum(PyObject *self, PyObject *args)
{
    int a;
    int b;
    int s;
    if (!PyArg_ParseTuple(args,"ii",&a,&b))
       return NULL;
    s = sum(a,b);
    return Py_BuildValue("i",s);
}

static PyMethodDef ModMethods[] = {
    {"sum", mod_sum, METH_VARARGS, "Description.."},
    {NULL,NULL,0,NULL}
};

PyMODINIT_FUNC initmod(void)
{
    PyObject *m;
    m = Py_InitModule("mod",ModMethods);
    if (m == NULL)
       return;
}

Assuming the C version of sum() is declared in sum.h, this code will make it accessible from Python in a module named mod.

There are 3 steps to creating a Python module in C:
  1. Implement each method
  2. Declare a list of the methods
  3. Initialize the module
As well the C implementation must include Python.h to make available the Python API. All names made available by Python.h begin with Py.

Implementing the Methods

Implement each method that the module will export. Each method will have the following signature:

static PyObject* modname_funcname(PyObject *self, PyObject *args)

where modname is the name of the module, and funcname is the name of the method.

The implementation must extract the arguments, convert them to C data types, perform some computation (this is where you call the C function which you are trying to access from Python), convert the result to a Python data type, and return it.

To extract the arguments and convert them to C data types use the function PyArg_ParseTuple:

int arg1;
float arg2;
char* arg3;
if (!PyArg_ParseTuple(args, "ifs", &arg1, &arg2, &arg3))
    return NULL;

This code extracts an integer, a float, and a character string from the argument list and fails if the method has been called with different arguments. It then places these arguments into C variables. Non-simple data like objects are a bit more complicated. See "Defining New Types" for an in depth explanation.

So, at this point you have the arguments in C variables. This is where you perform whatever computation you'd like to do on those arguments and produce some sort of return value (or none). Converting the return value to a Python data type is relatively similar using the Py_BuildValue function:

int res;
...
return Py_BuildValue("i",res);

Or, if there is no return value then the method must return the Python value None:

return Py_BuildValue("");

Below is an example of a method implementation that calls a C function named sum to add 2 numbers together:

static PyObject* mod_sum(PyObject *self, PyObject *args)
{
    int a;
    int b;
    int s;
    if (!PyArg_ParseTuple(args,"ii",&a,&b))
       return NULL;
    s = sum(a,b);
    return Py_BuildValue("i",s);
}

See "A Simple Example" from the Python document titled, "Extending and Embedding the Python Interpreter" for a different example.

Declaring the List of Methods

Once you have all of the methods implemented you must declare a special list of methods which matches up the name of the implementation function to the name the method will have in Python. For example, if you are creating a Python method name f1 and it is in module m1, then the implementation function will likely be called m1_f1 (though it can be called whatever you like). So, you must tell Python that when f1 is called, it should look for the C function m1_f1 instead.

The way to do this is to declare a list like so:

static PyMethodDef modNameMethods[] = {
    {"methName", modName_methName, METH_VARARGS, "Description.."},
    ...
    {NULL,NULL,0,NULL}
};

Where modName is the name of the module, and methName is the name of the method.

Each entry of the list contains the name of the method in Python (methName), the name of the C implementation function (modName_methName), a flag telling Python how to call the implementation function (METH_VARARGS works most of the time. There are other options, but they are beyond the scope of this page), and a description of the method (for Python's help and documentation facilities).

The list must be terminated with a null entry as shown above.

Below is an example for a module named mod that has one method named sum:

static PyMethodDef ModMethods[] = {
    {"sum", mod_sum, METH_VARARGS, "Description.."},
    {NULL,NULL,0,NULL}
};

Initializing the Module

The last step in implementing a Python module in C is to write a function that will initialize the module. This function will be called automatically by Python when the module is imported. Therefore, the name of the function is important as that is the only way for Python to find it.

Assuming you aren't using any non-simple data types or anything, the initialization function is pretty simple:

PyMODINIT_FUNC initmodName(void)
{
    PyObject *m;
    m = Py_InitModule("modName",modNameMethods);
    if (m == NULL)
       return;
}

where modName is the name of the module, and modNameMethods is the name of the list of methods (declared in the previous step).

The function Py_InitModule initializes the module by declaring the name and providing the list of methods. Below is an example initialization function for a module named mod:

PyMODINIT_FUNC initmod(void)
{
    PyObject *m;
    m = Py_InitModule("mod",ModMethods);
    if (m == NULL)
       return;
}

Extend this Page   How to edit   Style   Subscribe   Notify   Suggest   Help   This open web page hosted at the University of Alberta.   Terms of use  4135/0