PEP 229 – Using Distutils to Build Python
- A.M. Kuchling <amk at amk.ca>
- Standards Track
Modules/Setup mechanism has some flaws:
- People have to remember to uncomment bits of
Modules/Setupin order to get all the possible modules.
Setupto a new version of Python is tedious; new modules have been added, so you can’t just copy the older version, but have to reconcile the two versions.
- Users have to figure out where the needed libraries, such as
zlib, are installed.
Use the Distutils to build the modules that come with Python.
The changes can be broken up into several pieces:
- The Distutils needs some Python modules to be able to build
modules. Currently I believe the minimal list is
These modules will have to be built before the Distutils can be used, so they’ll simply be hardwired into
Modules/Makefileand be automatically built.
- A top-level setup.py script will be written that checks the libraries installed on the system and compiles as many modules as possible.
Modules/Setupwill be kept and settings in it will override
setup.py’s usual behavior, so you can disable a module known to be buggy, or specify particular compilation or linker flags. However, in the common case where
setup.pyworks correctly, everything in
Setupwill remain commented out. The other
Setup.*become unnecessary, since nothing will be generating
The patch was checked in for Python 2.1, and has been subsequently modified.
Patch #102588 on SourceForge contains the proposed patch. Currently the patch tries to be conservative and to change as few files as possible, in order to simplify backing out the patch. For example, no attempt is made to rip out the existing build mechanisms. Such simplifications can wait for later in the beta cycle, when we’re certain the patch will be left in, or they can wait for Python 2.2.
The patch makes the following changes:
- Makes some required changes to distutils/sysconfig (these will be checked in separately)
- In the top-level
Makefile.in, the “sharedmods” target simply runs
"./python setup.py build", and “sharedinstall” runs
"./python setup.py install". The “clobber” target also deletes the
build/subdirectory where Distutils puts its output.
Modules/Setup.config.inonly contains entries for the
dbmodules are removed because it’s now
setup.py’s job to handle them.
Modules/Setup.distnow contains entries for only 3 modules –
setup.cfg.in. This is needed for two reasons: to make building in subdirectories work, and to get the configured installation prefix.
setup.pyto the top directory of the source tree.
setup.pyis the largest piece of the puzzle, though not the most complicated.
setup.pycontains a subclass of the
BuildExtclass, and extends it with a
detect_modules()method that does the work of figuring out when modules can be compiled, and adding them to the ‘exts’ list.
Do we need to make it possible to disable the 3 hard-wired modules without manually hacking the Makefiles? [Answer: No.]
The Distutils always compile modules as shared libraries. How do we support compiling them statically into the resulting Python binary?
[Answer: building a Python binary with the Distutils should be
feasible, though no one has implemented it yet. This should be
done someday, but isn’t a pressing priority as messing around with
Makefile.pre.in is good enough.]
This document has been placed in the public domain.
Last modified: 2017-11-11 19:28:55 GMT