4. Asynchronous Files and Pipes (asyncfile)

asyncfile module provides API for asynchronous files and pipes to be used in tasks.

4.1. Examples

Following is a brief description of the examples included relevant to this section (unlike examples in other sections which tend to be a bit trivial, these examples are more complicated than necessary, to introduce different features / approaches):

• examples/pipe_csum.py uses asynchronous pipes to write data to and read data from a system program (that computes checksum of data).

• examples/pipe_grep.py uses chained pipes with asynchronous read and write interface to count number of lines matching a pattern.

4.2. Asynchronous File

AsyncFile class wraps given file so it can be used for asynchronous I/O. Note that regular on-disk files are non-blocking and can’t be used for polling and asynchronous I/O under Linux, OS X and other Unix variants. Under Windows asynchronous I/O of on-disk files is supported, but it may not be very useful. However, under Linux, OS X and other Unix variants, network sockets can be converted to asynchronous file interface so read(), write(), readline() in AsyncFile can be used. With Windows, though, network sockets don’t use file in implementation so sockets can’t be converted to AsyncFile under Windows.

AsyncFile is used in implementing AsyncPipe.

class asyncfile.AsyncFile(fd)

Note

This interface is for Linux, OS X and other Unix variants.

Sets up given file descriptor (that was created with open() or socket.socket) for asynchronous I/O.

4.2.1. Examples

See socket_afile.py in the examples directory.

4.3. Asynchronous Pipe

AsyncPipe provides asynchronous API for pipes.

Under Windows, Popen in asyncfile module must be used instead of subprocess.Popen.

class asyncfile.AsyncPipe(first, last=None)

Sets up (chained) pipe for asynchronous I/O. first must be subprocess.Popen object under Linux, OS X and other Unix variants and asyncfile.Popen object under Windows. If last is None, the pipe is not chained and in the description below, last is same as first; otherwise, last must also be a Popen object (representing end of chained pipe). write() operations send data to first's stdin and read() operations get data from last's stdout/stderr.

Asynchronous pipes support following methods:

write(buf, full=False, timeout=None)

Note

This method must be used with yield as n = yield apipe.write(buf)

Writes data in buf to stdin of first. full, timeout and operation of this method are same as that for write() of AsyncFile.

read(size=0, timeout=None)

Note

This method must be used with yield as buf = yield apipe.read()

Reads data from stdout of last. timeout and operation of this method are same as that for read() of AsyncFile.

readline(size=0, sizehint=100, timeout=None)

Note

This method must be used with yield as line = yield apipe.readline()

Reads a line from stdout of last. sizehint, timeout and operation of this method are same as that for readline() of AsyncFile.

read_stderr(size=0, timeout=None)

Note

This method must be used with yield as buf = yield apipe.read_stderr()

Reads data from stderr of last. timeout and operation of this method are same as that for read() of AsyncFile.

readline_stderr(size=0, sizehint=100, timeout=None)

Note

This method must be used with yield as line = yield apipe.readline_stderr()

Reads a line from stderr of last. sizehint, timeout and operation of this method are same as that for readline() of AsyncFile.

communicate(input=None)

Note

This method must be used with yield as stdoutdata, stderrdata = yield apipe.communicate()

Similar to Popen.communicate, except that input can be either data (as per Popen.communicate) or a file descriptor. The file descriptor can be synchronous (obtained with open) or asynchronous (created with AsyncFile).

close()

Closes pipe.