diff options
Diffstat (limited to 'sphinx/using.rst')
| -rw-r--r-- | sphinx/using.rst | 662 |
1 files changed, 662 insertions, 0 deletions
diff --git a/sphinx/using.rst b/sphinx/using.rst new file mode 100644 index 0000000..ff121ce --- /dev/null +++ b/sphinx/using.rst @@ -0,0 +1,662 @@ +.. _ref-using: + +Using SIP +========= + +Bindings are generated by the SIP code generator from a number of specification +files, typically with a ``.sip`` extension. Specification files look very +similar to C and C++ header files, but often with additional information (in +the form of a *directive* or an *annotation*) and code so that the bindings +generated can be finely tuned. + + +.. _ref-simple-c++-example: + +A Simple C++ Example +-------------------- + +We start with a simple example. Let's say you have a (fictional) C++ library +that implements a single class called ``Word``. The class has one constructor +that takes a ``\0`` terminated character string as its single argument. The +class has one method called ``reverse()`` which takes no arguments and returns +a ``\0`` terminated character string. The interface to the class is defined in +a header file called ``word.h`` which might look something like this:: + + // Define the interface to the word library. + + class Word { + const char *the_word; + + public: + Word(const char *w); + + char *reverse() const; + }; + +The corresponding SIP specification file would then look something like this:: + + // Define the SIP wrapper to the word library. + + %Module word 0 + + class Word { + + %TypeHeaderCode + #include <word.h> + %End + + public: + Word(const char *w); + + char *reverse() const; + }; + +Obviously a SIP specification file looks very much like a C++ (or C) header +file, but SIP does not include a full C++ parser. Let's look at the +differences between the two files. + + - The :directive:`%Module` directive has been added [#]_. This is used to + name the Python module that is being created and to give it a + *generation* number. In this example these are ``word`` and ``0`` + respectively. The generation number is effectively the version number of + the module. + + - The :directive:`%TypeHeaderCode` directive has been added. The text + between this and the following :directive:`%End` directive is included + literally in the code that SIP generates. Normally it is used, as in + this case, to ``#include`` the corresponding C++ (or C) header file [#]_. + + - The declaration of the private variable ``this_word`` has been removed. + SIP does not support access to either private or protected instance + variables. + +If we want to we can now generate the C++ code in the current directory by +running the following command:: + + sip -c . word.sip + +However, that still leaves us with the task of compiling the generated code and +linking it against all the necessary libraries. It's much easier to use the +:ref:`SIP build system <ref-build-system>` to do the whole thing. + +Using the SIP build system is simply a matter of writing a small Python script. +In this simple example we will assume that the ``word`` library we are wrapping +and it's header file are installed in standard system locations and will be +found by the compiler and linker without having to specify any additional +flags. In a more realistic example your Python script may take command line +options, or search a set of directories to deal with different configurations +and installations. + +This is the simplest script (conventionally called ``configure.py``):: + + import os + import sipconfig + + # The name of the SIP build file generated by SIP and used by the build + # system. + build_file = "word.sbf" + + # Get the SIP configuration information. + config = sipconfig.Configuration() + + # Run SIP to generate the code. + os.system(" ".join([config.sip_bin, "-c", ".", "-b", build_file, "word.sip"])) + + # Create the Makefile. + makefile = sipconfig.SIPModuleMakefile(config, build_file) + + # Add the library we are wrapping. The name doesn't include any platform + # specific prefixes or extensions (e.g. the "lib" prefix on UNIX, or the + # ".dll" extension on Windows). + makefile.extra_libs = ["word"] + + # Generate the Makefile itself. + makefile.generate() + +Hopefully this script is self-documenting. The key parts are the +``Configuration`` and ``SIPModuleMakefile`` classes. The build system contains +other Makefile classes, for example to build programs or to call other +Makefiles in sub-directories. + +After running the script (using the Python interpreter the extension module is +being created for) the generated C++ code and ``Makefile`` will be in the +current directory. + +To compile and install the extension module, just run the following +commands [#]_:: + + make + make install + +That's all there is to it. + +See :ref:`ref-distutils` for an example of how to build this example using +distutils. + +.. [#] All SIP directives start with a ``%`` as the first non-whitespace + character of a line. +.. [#] SIP includes many code directives like this. They differ in where the + supplied code is placed by SIP in the generated code. +.. [#] On Windows you might run ``nmake`` or ``mingw32-make`` instead. + + +A Simple C Example +------------------ + +Let's now look at a very similar example of wrapping a fictional C library:: + + /* Define the interface to the word library. */ + + struct Word { + const char *the_word; + }; + + struct Word *create_word(const char *w); + char *reverse(struct Word *word); + +The corresponding SIP specification file would then look something like this:: + + /* Define the SIP wrapper to the word library. */ + + %CModule word 0 + + struct Word { + + %TypeHeaderCode + #include <word.h> + %End + + const char *the_word; + }; + + struct Word *create_word(const char *w) /Factory/; + char *reverse(struct Word *word); + +Again, let's look at the differences between the two files. + + - The :directive:`%CModule` directive has been added. This has the same + syntax as the :directive:`%Module` directive used in the previous example + but tells SIP that the library being wrapped is implemented in C rather + than C++. + + - The :directive:`%TypeHeaderCode` directive has been added. + + - The :fanno:`Factory` annotation has been added to the ``create_word()`` + function. This tells SIP that a newly created structure is being + returned and it is owned by Python. + +The ``configure.py`` build system script described in the previous example can +be used for this example without change. + + +A More Complex C++ Example +-------------------------- + +In this last example we will wrap a fictional C++ library that contains a class +that is derived from a Qt class. This will demonstrate how SIP allows a class +hierarchy to be split across multiple Python extension modules, and will +introduce SIP's versioning system. + +The library contains a single C++ class called ``Hello`` which is derived from +Qt's ``QLabel`` class. It behaves just like ``QLabel`` except that the text +in the label is hard coded to be ``Hello World``. To make the example more +interesting we'll also say that the library only supports Qt v4.2 and later, +and also includes a function called ``setDefault()`` that is not implemented +in the Windows version of the library. + +The ``hello.h`` header file looks something like this:: + + // Define the interface to the hello library. + + #include <qlabel.h> + #include <qwidget.h> + #include <qstring.h> + + class Hello : public QLabel { + // This is needed by the Qt Meta-Object Compiler. + Q_OBJECT + + public: + Hello(QWidget *parent = 0); + + private: + // Prevent instances from being copied. + Hello(const Hello &); + Hello &operator=(const Hello &); + }; + + #if !defined(Q_OS_WIN) + void setDefault(const QString &def); + #endif + +The corresponding SIP specification file would then look something like this:: + + // Define the SIP wrapper to the hello library. + + %Module hello 0 + + %Import QtGui/QtGuimod.sip + + %If (Qt_4_2_0 -) + + class Hello : QLabel { + + %TypeHeaderCode + #include <hello.h> + %End + + public: + Hello(QWidget *parent /TransferThis/ = 0); + + private: + Hello(const Hello &); + }; + + %If (!WS_WIN) + void setDefault(const QString &def); + %End + + %End + +Again we look at the differences, but we'll skip those that we've looked at in +previous examples. + + - The :directive:`%Import` directive has been added to specify that we are + extending the class hierarchy defined in the file ``QtGui/QtGuimod.sip``. + This file is part of PyQt. The build system will take care of finding + the file's exact location. + + - The :directive:`%If` directive has been added to specify that everything + [#]_ up to the matching :directive:`%End` directive only applies to Qt + v4.2 and later. ``Qt_4_2_0`` is a *tag* defined in ``QtCoremod.sip`` + [#]_ using the :directive:`%Timeline` directive. :directive:`%Timeline` + is used to define a tag for each version of a library's API you are + wrapping allowing you to maintain all the different versions in a single + SIP specification. The build system provides support to ``configure.py`` + scripts for working out the correct tags to use according to which + version of the library is actually installed. + + - The ``public`` keyword used in defining the super-classes has been + removed. This is not supported by SIP. + + - The :aanno:`TransferThis` annotation has been added to the constructor's + argument. It specifies that if the argument is not 0 (i.e. the ``Hello`` + instance being constructed has a parent) then ownership of the instance + is transferred from Python to C++. It is needed because Qt maintains + objects (i.e. instances derived from the ``QObject`` class) in a + hierachy. When an object is destroyed all of its children are also + automatically destroyed. It is important, therefore, that the Python + garbage collector doesn't also try and destroy them. This is covered in + more detail in :ref:`ref-object-ownership`. SIP provides many other + annotations that can be applied to arguments, functions and classes. + Multiple annotations are separated by commas. Annotations may have + values. + + - The ``=`` operator has been removed. This operator is not supported by + SIP. + + - The :directive:`%If` directive has been added to specify that everything + up to the matching :directive:`%End` directive does not apply to Windows. + ``WS_WIN`` is another tag defined by PyQt, this time using the + :directive:`%Platforms` directive. Tags defined by the + :directive:`%Platforms` directive are mutually exclusive, i.e. only one + may be valid at a time [#]_. + +One question you might have at this point is why bother to define the private +copy constructor when it can never be called from Python? The answer is to +prevent the automatic generation of a public copy constructor. + +We now look at the ``configure.py`` script. This is a little different to the +script in the previous examples for two related reasons. + +Firstly, PyQt includes a pure Python module called ``pyqtconfig`` that extends +the SIP build system for modules, like our example, that build on top of PyQt. +It deals with the details of which version of Qt is being used (i.e. it +determines what the correct tags are) and where it is installed. This is +called a module's configuration module. + +Secondly, we generate a configuration module (called ``helloconfig``) for our +own ``hello`` module. There is no need to do this, but if there is a chance +that somebody else might want to extend your C++ library then it would make +life easier for them. + +Now we have two scripts. First the ``configure.py`` script:: + + import os + import sipconfig + from PyQt4 import pyqtconfig + + # The name of the SIP build file generated by SIP and used by the build + # system. + build_file = "hello.sbf" + + # Get the PyQt configuration information. + config = pyqtconfig.Configuration() + + # Get the extra SIP flags needed by the imported PyQt modules. Note that + # this normally only includes those flags (-x and -t) that relate to SIP's + # versioning system. + pyqt_sip_flags = config.pyqt_sip_flags + + # Run SIP to generate the code. Note that we tell SIP where to find the qt + # module's specification files using the -I flag. + os.system(" ".join([config.sip_bin, "-c", ".", "-b", build_file, "-I", config.pyqt_sip_dir, pyqt_sip_flags, "hello.sip"])) + + # We are going to install the SIP specification file for this module and + # its configuration module. + installs = [] + + installs.append(["hello.sip", os.path.join(config.default_sip_dir, "hello")]) + + installs.append(["helloconfig.py", config.default_mod_dir]) + + # Create the Makefile. The QtGuiModuleMakefile class provided by the + # pyqtconfig module takes care of all the extra preprocessor, compiler and + # linker flags needed by the Qt library. + makefile = pyqtconfig.QtGuiModuleMakefile( + configuration=config, + build_file=build_file, + installs=installs + ) + + # Add the library we are wrapping. The name doesn't include any platform + # specific prefixes or extensions (e.g. the "lib" prefix on UNIX, or the + # ".dll" extension on Windows). + makefile.extra_libs = ["hello"] + + # Generate the Makefile itself. + makefile.generate() + + # Now we create the configuration module. This is done by merging a Python + # dictionary (whose values are normally determined dynamically) with a + # (static) template. + content = { + # Publish where the SIP specifications for this module will be + # installed. + "hello_sip_dir": config.default_sip_dir, + + # Publish the set of SIP flags needed by this module. As these are the + # same flags needed by the qt module we could leave it out, but this + # allows us to change the flags at a later date without breaking + # scripts that import the configuration module. + "hello_sip_flags": pyqt_sip_flags + } + + # This creates the helloconfig.py module from the helloconfig.py.in + # template and the dictionary. + sipconfig.create_config_module("helloconfig.py", "helloconfig.py.in", content) + +Next we have the ``helloconfig.py.in`` template script:: + + from PyQt4 import pyqtconfig + + # These are installation specific values created when Hello was configured. + # The following line will be replaced when this template is used to create + # the final configuration module. + # @SIP_CONFIGURATION@ + + class Configuration(pyqtconfig.Configuration): + """The class that represents Hello configuration values. + """ + def __init__(self, sub_cfg=None): + """Initialise an instance of the class. + + sub_cfg is the list of sub-class configurations. It should be None + when called normally. + """ + # This is all standard code to be copied verbatim except for the + # name of the module containing the super-class. + if sub_cfg: + cfg = sub_cfg + else: + cfg = [] + + cfg.append(_pkg_config) + + pyqtconfig.Configuration.__init__(self, cfg) + + class HelloModuleMakefile(pyqtconfig.QtGuiModuleMakefile): + """The Makefile class for modules that %Import hello. + """ + def finalise(self): + """Finalise the macros. + """ + # Make sure our C++ library is linked. + self.extra_libs.append("hello") + + # Let the super-class do what it needs to. + pyqtconfig.QtGuiModuleMakefile.finalise(self) + +Again, we hope that the scripts are self documenting. + +.. [#] Some parts of a SIP specification aren't subject to version control. +.. [#] Actually in ``versions.sip``. PyQt uses the :directive:`%Include` + directive to split the SIP specification for Qt across a large number of + separate ``.sip`` files. +.. [#] Tags can also be defined by the :directive:`%Feature` directive. These + tags are not mutually exclusive, i.e. any number may be valid at a time. + + +.. _ref-object-ownership: + +Ownership of Objects +-------------------- + +When a C++ instance is wrapped a corresponding Python object is created. The +Python object behaves as you would expect in regard to garbage collection - it +is garbage collected when its reference count reaches zero. What then happens +to the corresponding C++ instance? The obvious answer might be that the +instance's destructor is called. However the library API may say that when the +instance is passed to a particular function, the library takes ownership of the +instance, i.e. responsibility for calling the instance's destructor is +transferred from the SIP generated module to the library. + +Ownership of an instance may also be associated with another instance. The +implication being that the owned instance will automatically be destroyed if +the owning instance is destroyed. SIP keeps track of these relationships to +ensure that Python's cyclic garbage collector can detect and break any +reference cycles between the owning and owned instances. The association is +implemented as the owning instance taking a reference to the owned instance. + +The TransferThis, Transfer and TransferBack annotations are used to specify +where, and it what direction, transfers of ownership happen. It is very +important that these are specified correctly to avoid crashes (where both +Python and C++ call the destructor) and memory leaks (where neither Python and +C++ call the destructor). + +This applies equally to C structures where the structure is returned to the +heap using the ``free()`` function. + +See also :cfunc:`sipTransferTo()`, :cfunc:`sipTransferBack()` and +:cfunc:`sipTransferBreak()`. + + +.. _ref-types-metatypes: + +Types and Meta-types +-------------------- + +Every Python object (with the exception of the :class:`object` object itself) +has a meta-type and at least one super-type. By default an object's meta-type +is the meta-type of its first super-type. + +SIP implements two super-types, :class:`sip.simplewrapper` and +:class:`sip.wrapper`, and a meta-type, :class:`sip.wrappertype`. + +:class:`sip.simplewrapper` is the super-type of :class:`sip.wrapper`. The +super-type of :class:`sip.simplewrapper` is :class:`object`. + +:class:`sip.wrappertype` is the meta-type of both :class:`sip.simplewrapper` +and :class:`sip.wrapper`. The super-type of :class:`sip.wrappertype` is +:class:`type`. + +:class:`sip.wrapper` supports the concept of object ownership described in +:ref:`ref-object-ownership` and, by default, is the super-type of all the types +that SIP generates. + +:class:`sip.simplewrapper` does not support the concept of object ownership but +SIP generated types that are sub-classed from it have Python objects that take +less memory. + +SIP allows a class's meta-type and super-type to be explicitly specified using +the :canno:`Metatype` and :canno:`Supertype` class annotations. + +SIP also allows the default meta-type and super-type to be changed for a module +using the :directive:`%DefaultMetatype` and :directive:`%DefaultSupertype` +directives. Unlike the default super-type, the default meta-type is inherited +by importing modules. + +If you want to use your own meta-type or super-type then they must be +sub-classed from one of the SIP provided types. Your types must be registered +using :cfunc:`sipRegisterPyType()`. This is normally done in code specified +using the :directive:`%InitialisationCode` directive. + +As an example, PyQt4 uses :directive:`%DefaultMetatype` to specify a new +meta-type that handles the interaction with Qt's own meta-type system. It also +uses :directive:`%DefaultSupertype` to specify that the smaller +:class:`sip.simplewrapper` super-type is normally used. Finally it uses +:canno:`Supertype` as an annotation of the ``QObject`` class to override the +default and use :class:`sip.wrapper` as the super-type so that the parent/child +relationships of ``QObject`` instances are properly maintained. + + +.. _ref-lazy-type-attributes: + +Lazy Type Attributes +-------------------- + +Instead of populating a wrapped type's dictionary with its attributes (or +descriptors for those attributes) SIP only creates objects for those attributes +when they are actually needed. This is done to reduce the memory footprint and +start up time when used to wrap large libraries with hundreds of classes and +tens of thousands of attributes. + +SIP allows you to extend the handling of lazy attributes to your own attribute +types by allowing you to register an attribute getter handler (using +:cfunc:`sipRegisterAttributeGetter()`). This will be called just before a +type's dictionary is accessed for the first time. + + +Support for Python's Buffer Interface +------------------------------------- + +SIP supports Python's buffer interface in that whenever C/C++ requires a +``char`` or ``char *`` type then any Python type that supports the buffer +interface (including ordinary Python strings) can be used. + +If a buffer is made up of a number of segments then all but the first will be +ignored. + + +Support for Wide Characters +--------------------------- + +SIP v4.6 introduced support for wide characters (i.e. the ``wchar_t`` type). +Python's C API includes support for converting between unicode objects and wide +character strings and arrays. When converting from a unicode object to wide +characters SIP creates the string or array on the heap (using memory allocated +using :cfunc:`sipMalloc()`). This then raises the problem of how this memory +is subsequently freed. + +The following describes how SIP handles this memory in the different situations +where this is an issue. + + - When a wide string or array is passed to a function or method then the + memory is freed (using :cfunc:`sipFree()`) after than function or method + returns. + + - When a wide string or array is returned from a virtual method then SIP + does not free the memory until the next time the method is called. + + - When an assignment is made to a wide string or array instance variable + then SIP does not first free the instance's current string or array. + + +.. _ref-gil: + +The Python Global Interpreter Lock +---------------------------------- + +Python's Global Interpretor Lock (GIL) must be acquired before calls can be +made to the Python API. It should also be released when a potentially +blocking call to C/C++ library is made in order to allow other Python threads +to be executed. In addition, some C/C++ libraries may implement their own +locking strategies that conflict with the GIL causing application deadlocks. +SIP provides ways of specifying when the GIL is released and acquired to +ensure that locking problems can be avoided. + +SIP always ensures that the GIL is acquired before making calls to the Python +API. By default SIP does not release the GIL when making calls to the C/C++ +library being wrapped. The :fanno:`ReleaseGIL` annotation can be used to +override this behaviour when required. + +If SIP is given the :option:`-g <sip -g>` command line option then the default +behaviour is changed and SIP releases the GIL every time is makes calls to the +C/C++ library being wrapped. The :fanno:`HoldGIL` annotation can be used to +override this behaviour when required. + + +.. _ref-incompat-apis: + +Managing Incompatible APIs +-------------------------- + +.. versionadded:: 4.9 + +Sometimes it is necessary to change the way something is wrapped in a way that +introduces an incompatibility. For example a new feature of Python may +suggest that something may be wrapped in a different way to exploit that +feature. + +SIP's :directive:`%Feature` directive could be used to provide two different +implementations. However this would mean that the choice between the two +implementations would have to be made when building the generated module +potentially causing all sorts of deployment problems. It may also require +applications to work out which implementation was available and to change +their behaviour accordingly. + +Instead SIP provides limited support for providing multiple implementations +(of classes, mapped types and functions) that can be selected by an +application at run-time. It is then up to the application developer how they +want to manage the migration from the old API to the new, incompatible API. + +This support is implemented in three parts. + +Firstly the :directive:`%API` directive is used to define the name of an API +and its default version number. The default version number is the one used if +an application doesn't explicitly set the version number to use. + +Secondly the :canno:`API class <API>`, :manno:`mapped type <API>` or +:fanno:`function <API>` annotation is applied accordingly to specify the API +and range of version numbers that a particular class, mapped type or function +implementation should be enabled for. + +Finally the application calls :func:`sip.setapi` to specify the version number +of the API that should be enabled. This call must be made before any module +that has multiple implementations is imported for the first time. + +Note this mechanism is not intended as a way or providing equally valid +alternative APIs. For example:: + + %API MyAPI 1 + + class Foo + { + public: + void bar(); + }; + + class Baz : Foo + { + public: + void bar() /API=MyAPI:2-/; + }; + +If the following Python code is executed then an exception will be raised:: + + b = Baz() + b.bar() + +This is because when version 1 of the *MyAPI* API (the default) is enabled +there is no *Baz.bar()* implementation and *Foo.bar()* will not be called +instead as might be expected. |