PEP 433 – Easier suppression of file descriptor inheritance
- Victor Stinner <vstinner at python.org>
- Standards Track
Table of Contents
- Applications using inheritance of file descriptors
- Backward compatibility
- Appendix: Operating system support
Add a new optional cloexec parameter on functions creating file descriptors, add different ways to change default values of this parameter, and add four new functions:
A file descriptor has a close-on-exec flag which indicates if the file descriptor will be inherited or not.
On UNIX, if the close-on-exec flag is set, the file descriptor is not inherited: it will be closed at the execution of child processes; otherwise the file descriptor is inherited by child processes.
On Windows, if the close-on-exec flag is set, the file descriptor is not
inherited; the file descriptor is inherited by child processes if the
close-on-exec flag is cleared and if
CreateProcess() is called with
the bInheritHandles parameter set to
subprocess.Popen is created with
close_fds=False for example).
Windows does not have “close-on-exec” flag but an inheritance flag which
is just the opposite value. For example, setting close-on-exec flag
means clearing the
HANDLE_FLAG_INHERIT flag of a handle.
Status in Python 3.3
On UNIX, the subprocess module closes file descriptors greater than 2 by default since Python 3.2 . All file descriptors created by the parent process are automatically closed in the child process.
xmlrpc.server.SimpleXMLRPCServer sets the close-on-exec flag of
the listening socket, the parent class
does not set this flag.
There are other cases creating a subprocess or executing a new program
where file descriptors are not closed: functions of the
os.exec*() families and third party modules calling
exec(). In this case, file descriptors
are shared between the parent and the child processes which is usually
unexpected and causes various issues.
This PEP proposes to continue the work started with the change in the subprocess in Python 3.2, to fix the issue in any code, and not just code using subprocess.
Inherited file descriptors issues
Closing the file descriptor in the parent process does not close the related resource (file, socket, …) because it is still open in the child process.
The listening socket of TCPServer is not closed on
exec(): the child
process is able to get connection from new clients; if the parent closes
the listening socket and create a new listening socket on the same
address, it would get an “address already is used” error.
Not closing file descriptors can lead to resource exhaustion: even if the parent closes all files, creating a new file descriptor may fail with “too many files” because files are still open in the child process.
See also the following issues:
- Issue #2320: Race condition in subprocess using stdin (2008)
- Issue #3006: subprocess.Popen causes socket to remain open after close (2008)
- Issue #7213: subprocess leaks open file descriptors between Popen instances causing hangs (2009)
- Issue #12786: subprocess wait() hangs when stdin is closed (2011)
Leaking file descriptors is a major security vulnerability. An untrusted child process can read sensitive data like passwords and take control of the parent process though leaked file descriptors. It is for example a known vulnerability to escape from a chroot.
See also the CERT recommendation: FIO42-C. Ensure files are properly closed when they are no longer needed.
Example of vulnerabilities:
- OpenSSH Security Advisory: portable-keysign-rand-helper.adv (April 2011)
- CWE-403: Exposure of File Descriptor to Unintended Control Sphere (2008)
- Hijacking Apache https by mod_php (Dec 2003)
- Apache: Apr should set FD_CLOEXEC if APR_FOPEN_NOCLEANUP is not set (fixed in 2009)
- PHP: system() (and similar) don’t cleanup opened handles of Apache (not fixed in January 2013)
fcntl() to set the close-on-exec flag is not safe in a
multithreaded application. If a thread calls
between the creation of the file descriptor and the call to
fcntl(fd, F_SETFD, new_flags): the file descriptor will be
inherited by the child process. Modern operating systems offer
functions to set the flag during the creation of the file descriptor,
which avoids the race condition.
Python 3.2 added
socket.SOCK_CLOEXEC flag, Python 3.3 added
os.O_CLOEXEC flag and
os.pipe2() function. It is already
possible to set atomically close-on-exec flag in Python 3.3 when
opening a file and creating a pipe or socket.
The problem is that these flags and functions are not portable: only
recent versions of operating systems support them.
SOCK_CLOEXEC flags are ignored by old Linux versions and so
FD_CLOEXEC flag must be checked using
fcntl(fd, F_GETFD). If
the kernel ignores
SOCK_CLOEXEC flag, a call to
fcntl(fd, F_SETFD, flags) is required to set close-on-exec flag.
OpenBSD older 5.2 does not close the file descriptor with
close-on-exec flag set if
fork() is used before
it works correctly if
exec() is called without
Applications still have to close explicitly file descriptors after a
fork(). The close-on-exec flag only closes file descriptors after
exec(), and so after
This PEP only change the close-on-exec flag of file descriptors
created by the Python standard library, or by modules using the
standard library. Third party modules not using the standard library
should be modified to conform to this PEP. The new
os.set_cloexec() function can be used for example.
See Close file descriptors after fork for a possible solution
Add a new optional cloexec parameter on functions creating file descriptors and different ways to change default value of this parameter.
Add new functions:
os.get_cloexec(fd:int) -> bool: get the close-on-exec flag of a file descriptor. Not available on all platforms.
os.set_cloexec(fd:int, cloexec:bool=True): set or clear the close-on-exec flag on a file descriptor. Not available on all platforms.
sys.getdefaultcloexec() -> bool: get the current default value of the cloexec parameter
sys.setdefaultcloexec(cloexec: bool): set the default value of the cloexec parameter
Add a new optional cloexec parameter to:
The default value of the cloexec parameter is
Add a new command line option
-e and an environment variable
PYTHONCLOEXEC to the set close-on-exec flag by default.
subprocess clears the close-on-exec flag of file descriptors of the
All functions creating file descriptors in the standard library must
respect the default value of the cloexec parameter:
File descriptors 0 (stdin), 1 (stdout) and 2 (stderr) are expected to be
inherited, but Python does not handle them differently. When
os.dup2() is used to replace standard streams,
must be specified explicitly.
Drawbacks of the proposal:
- It is not more possible to know if the close-on-exec flag will be set or not on a newly created file descriptor just by reading the source code.
- If the inheritance of a file descriptor matters, the cloexec parameter must now be specified explicitly, or the library or the application will not work depending on the default value of the cloexec parameter.
Inheritance enabled by default, default no configurable
Add a new optional parameter cloexec on functions creating file
descriptors. The default value of the cloexec parameter is
and this default cannot be changed. File descriptor inheritance enabled by
default is also the default on POSIX and on Windows. This alternative is
the most conservative option.
This option does not solve issues listed in the Rationale section, it only provides a helper to fix them. All functions creating file descriptors have to be modified to set cloexec=True in each module used by an application to fix all these issues.
Inheritance enabled by default, default can only be set to True
This alternative is based on the proposal: the only difference is that
sys.setdefaultcloexec() does not take any argument, it can only be
used to set the default value of the cloexec parameter to
Disable inheritance by default
This alternative is based on the proposal: the only difference is that
the default value of the cloexec parameter is
True (instead of
If a file must be inherited by child processes,
parameter can be used.
Advantages of setting close-on-exec flag by default:
- There are far more programs that are bitten by FD inheritance upon exec (see Inherited file descriptors issues and Security) than programs relying on it (see Applications using inheritance of file descriptors).
Drawbacks of setting close-on-exec flag by default:
- It violates the principle of least surprise. Developers using the os module may expect that Python respects the POSIX standard and so that close-on-exec flag is not set by default.
- The os module is written as a thin wrapper to system calls (to functions of the C standard library). If atomic flags to set close-on-exec flag are not supported (see Appendix: Operating system support), a single Python function call may call 2 or 3 system calls (see Performances section).
- Extra system calls, if any, may slow down Python: see Performances.
Backward compatibility: only a few programs rely on inheritance of file
descriptors, and they only pass a few file descriptors, usually just
one. These programs will fail immediately with
EBADF error, and it
will be simple to fix them: add
cloexec=False parameter or use
subprocess module will be changed anyway to clear
close-on-exec flag on file descriptors listed in the
parameter of Popen constructor. So it possible that these programs will
not need any fix if they use the
Close file descriptors after fork
This PEP does not fix issues with applications using
exec(). Python needs a generic process to register
callbacks which would be called after a fork, see #16500:
Add an atfork module. Such registry could be used to close file
descriptors just after a
- It does not solve the problem on Windows:
fork()does not exist on Windows
- This alternative does not solve the problem for programs using
- A third party module may call directly the C function
fork()which will not call “atfork” callbacks.
- All functions creating file descriptors must be changed to register a callback and then unregister their callback when the file is closed. Or a list of all open file descriptors must be maintained.
- The operating system is a better place than Python to close automatically file descriptors. For example, it is not easy to avoid a race condition between closing the file and unregistering the callback closing the file.
open(): add “e” flag to mode
A new “e” mode would set close-on-exec flag (best-effort).
This alternative only solves the problem for
socket.socket() and os.pipe() do not have a
mode parameter for
Since its version 2.7, the GNU libc supports
"e" flag for
fopen(). It uses
O_CLOEXEC if available, or use
F_SETFD, FD_CLOEXEC). With Visual Studio, fopen() accepts a “N”
flag which uses
Bikeshedding on the name of the new parameter
inherited: closer to Windows definition
sterile: “Does not produce offspring.”
Applications using inheritance of file descriptors
Most developers don’t know that file descriptors are inherited by
default. Most programs do not rely on inheritance of file descriptors.
subprocess.Popen was changed in Python 3.2 to close
all file descriptors greater than 2 in the child process by default.
No user complained about this behavior change yet.
Network servers using fork may want to pass the client socket to the
child process. For example, on UNIX a CGI server pass the socket
client through file descriptors 0 (stdin) and 1 (stdout) using
To access a restricted resource like creating a socket listening on a TCP port lower than 1024 or reading a file containing sensitive data like passwords, a common practice is: start as the root user, create a file descriptor, create a child process, drop privileges (ex: change the current user), pass the file descriptor to the child process and exit the parent process.
Security is very important in such use case: leaking another file descriptor would be a critical security vulnerability (see Security). The root process may not exit but monitors the child process instead, and restarts a new child process and pass the same file descriptor if the previous child process crashed.
Example of programs taking file descriptors from the parent process using a command line option:
--logger-fd <fd>, etc.
On Linux, it is possible to use
"/dev/fd/<fd>" filename to pass a
file descriptor to a program expecting a filename.
Setting close-on-exec flag may require additional system calls for each creation of new file descriptors. The number of additional system calls depends on the method used to set the flag:
O_NOINHERIT: no additional system call
O_CLOEXEC: one additional system call, but only at the creation of the first file descriptor, to check if the flag is supported. If the flag is not supported, Python has to fallback to the next method.
ioctl(fd, FIOCLEX): one additional system call per file descriptor
fcntl(fd, F_SETFD, flags): two additional system calls per file descriptor, one to get old flags and one to set new flags
On Linux, setting the close-on-flag has a low overhead on performances. Results of bench_cloexec.py on Linux 3.6:
- close-on-flag not set: 7.8 us
O_CLOEXEC: 1% slower (7.9 us)
ioctl(): 3% slower (8.0 us)
fcntl(): 3% slower (8.0 us)
Get the close-on-exec flag of a file descriptor.
if os.name == 'nt': def get_cloexec(fd): handle = _winapi._get_osfhandle(fd); flags = _winapi.GetHandleInformation(handle) return not(flags & _winapi.HANDLE_FLAG_INHERIT) else: try: import fcntl except ImportError: pass else: def get_cloexec(fd): flags = fcntl.fcntl(fd, fcntl.F_GETFD) return bool(flags & fcntl.FD_CLOEXEC)
Set or clear the close-on-exec flag on a file descriptor. The flag is set after the creation of the file descriptor and so it is not atomic.
if os.name == 'nt': def set_cloexec(fd, cloexec=True): handle = _winapi._get_osfhandle(fd); mask = _winapi.HANDLE_FLAG_INHERIT if cloexec: flags = 0 else: flags = mask _winapi.SetHandleInformation(handle, mask, flags) else: fnctl = None ioctl = None try: import ioctl except ImportError: try: import fcntl except ImportError: pass if ioctl is not None and hasattr('FIOCLEX', ioctl): def set_cloexec(fd, cloexec=True): if cloexec: ioctl.ioctl(fd, ioctl.FIOCLEX) else: ioctl.ioctl(fd, ioctl.FIONCLEX) elif fnctl is not None: def set_cloexec(fd, cloexec=True): flags = fcntl.fcntl(fd, fcntl.F_GETFD) if cloexec: flags |= FD_CLOEXEC else: flags &= ~FD_CLOEXEC fcntl.fcntl(fd, fcntl.F_SETFD, flags)
ioctl is preferred over fcntl because it requires only one syscall, instead of two syscalls for fcntl.
fcntl(fd, F_SETFD, flags) only supports one flag
FD_CLOEXEC), so it would be possible to avoid
F_GETFD). But it may drop other flags in the future, and so it is
safer to keep the two functions calls.
fopen() function of the GNU libc ignores the error if
fcntl(fd, F_SETFD, flags) failed.
fcntl(fd, F_DUP2FD_CLOEXEC, fd2)[atomic]
There is no backward incompatible change. The default behaviour is unchanged: the close-on-exec flag is not set by default.
Appendix: Operating system support
Windows has an
O_NOINHERIT flag: “Do not inherit in child
For example, it is supported by
The flag can be cleared using
SetHandleInformation(fd, HANDLE_FLAG_INHERIT, 0).
CreateProcess() has an
bInheritHandles parameter: if it is
FALSE, the handles are not inherited. If it is
HANDLE_FLAG_INHERIT flag set are inherited.
close_fds option to define
ioctl(fd, FIOCLEX, 0): set the close-on-exec flag
ioctl(fd, FIONCLEX, 0): clear the close-on-exec flag
Availability: Linux, Mac OS X, QNX, NetBSD, OpenBSD, FreeBSD.
flags = fcntl(fd, F_GETFD); fcntl(fd, F_SETFD, flags | FD_CLOEXEC): set the close-on-exec flag
flags = fcntl(fd, F_GETFD); fcntl(fd, F_SETFD, flags & ~FD_CLOEXEC): clear the close-on-exec flag
Availability: AIX, Digital UNIX, FreeBSD, HP-UX, IRIX, Linux, Mac OS X, OpenBSD, Solaris, SunOS, Unicos.
O_CLOEXEC: available on Linux (2.6.23), FreeBSD (8.3), OpenBSD 5.0, Solaris 11, QNX, BeOS, next NetBSD release (6.1?). This flag is part of POSIX.1-2008.
socketpair(), available on Linux 2.6.27, OpenBSD 5.2, NetBSD 6.0.
WSASocket(): supported on Windows 7 with SP1, Windows Server 2008 R2 with SP1, and later
F_DUPFD_CLOEXECflag, available on Linux 2.6.24, OpenBSD 5.0, FreeBSD 9.1, NetBSD 6.0, Solaris 11. This flag is part of POSIX.1-2008.
F_DUP2FD_CLOEXECflag, available on FreeBSD 9.1 and Solaris 11.
MSG_CMSG_CLOEXEC, available on Linux 2.6.23, NetBSD 6.0.
On Linux older than 2.6.23,
O_CLOEXEC flag is simply ignored. So
we have to check that the flag is supported by calling
it does not work, we have to set the flag using
On Linux older than 2.6.27, if the
SOCK_CLOEXEC flag is set in the
socketpair() fail and
errno is set
On Windows XPS3,
WSA_FLAG_NO_HANDLE_INHERIT flag is used.
dup3(): available on Linux 2.6.27 (and glibc 2.9)
pipe2(): available on Linux 2.6.27 (and glibc 2.9)
accept4(): available on Linux 2.6.28 (and glibc 2.10)
accept4() is called on Linux older than 2.6.28,
-1 (fail) and
errno is set to
- On UNIX since Python 3.2, subprocess.Popen()
closes all file descriptors by default:
close_fds=True. It closes file descriptors in range 3 inclusive to
fcntl(0, F_MAXFD)on NetBSD, or
sysconf(_SC_OPEN_MAX)otherwise. If the error pipe has a descriptor smaller than 3,
This document has been placed in the public domain.
Last modified: 2021-02-09 16:54:26 GMT