16.2.1.1.Text I/O¶

Text I/O expects and producesstr objects. This means that wheneverthe backing store is natively made of bytes (such as in the case of a file),encoding and decoding of data is made transparently as well as optionaltranslation of platform-specific newline characters.

The easiest way to create a text stream is withopen(), optionallyspecifying an encoding:

f=open("myfile.txt","r",encoding="utf-8")

In-memory text streams are also available asStringIO objects:

f=io.StringIO("some initial text data")

The text stream API is described in detail in the documentation ofTextIOBase.

16.2.1.2.Binary I/O¶

Binary I/O (also calledbuffered I/O) expectsbytes-like objects and producesbytesobjects. No encoding, decoding, or newline translation is performed. Thiscategory of streams can be used for all kinds of non-text data, and also whenmanual control over the handling of text data is desired.

The easiest way to create a binary stream is withopen() with'b' inthe mode string:

f=open("myfile.jpg","rb")

In-memory binary streams are also available asBytesIO objects:

f=io.BytesIO(b"some initial binary data:\x00\x01")

The binary stream API is described in detail in the docs ofBufferedIOBase.

Other library modules may provide additional ways to create text or binarystreams. Seesocket.socket.makefile() for example.

16.2.1.3.Raw I/O¶

Raw I/O (also calledunbuffered I/O) is generally used as a low-levelbuilding-block for binary and text streams; it is rarely useful to directlymanipulate a raw stream from user code. Nevertheless, you can create a rawstream by opening a file in binary mode with buffering disabled:

f=open("myfile.jpg","rb",buffering=0)

The raw stream API is described in detail in the docs ofRawIOBase.

16.2.2.1.In-memory streams¶

It is also possible to use astr orbytes-like object as afile for both reading and writing. For stringsStringIO can be usedlike a file opened in text mode.BytesIO can be used like a fileopened in binary mode. Both provide full read-write capabilities with randomaccess.

16.2.3.1.I/O Base Classes¶

classio.IOBase¶

The abstract base class for all I/O classes, acting on streams of bytes.There is no public constructor.

This class provides empty abstract implementations for many methodsthat derived classes can override selectively; the defaultimplementations represent a file that cannot be read, written orseeked.

Even thoughIOBase does not declareread(),readinto(),orwrite() because their signatures will vary, implementations andclients should consider those methods part of the interface. Also,implementations may raise aValueError (orUnsupportedOperation)when operations they do not support are called.

The basic type used for binary data read from or written to a file isbytes. Otherbytes-like objects areaccepted as method arguments too. In some cases, such asreadinto(), a writable object such asbytearrayis required. Text I/O classes work withstr data.

Note that calling any method (even inquiries) on a closed stream isundefined. Implementations may raiseValueError in this case.

IOBase (and its subclasses) supports the iterator protocol, meaningthat anIOBase object can be iterated over yielding the lines in astream. Lines are defined slightly differently depending on whether thestream is a binary stream (yielding bytes), or a text stream (yieldingcharacter strings). Seereadline() below.

IOBase is also a context manager and therefore supports thewith statement. In this example,file is closed after thewith statement’s suite is finished—even if an exception occurs:

withopen('spam.txt','w')asfile:file.write('Spam and eggs!')

IOBase provides these data attributes and methods:

close()¶

Flush and close this stream. This method has no effect if the file isalready closed. Once the file is closed, any operation on the file(e.g. reading or writing) will raise aValueError.

As a convenience, it is allowed to call this method more than once;only the first call, however, will have an effect.

closed¶

True if the stream is closed.

fileno()¶

Return the underlying file descriptor (an integer) of the stream if itexists. AnOSError is raised if the IO object does not use a filedescriptor.

flush()¶

Flush the write buffers of the stream if applicable. This does nothingfor read-only and non-blocking streams.

isatty()¶

ReturnTrue if the stream is interactive (i.e., connected toa terminal/tty device).

readable()¶

ReturnTrue if the stream can be read from. IfFalse,read()will raiseOSError.

readline(size=-1)¶

Read and return one line from the stream. Ifsize is specified, atmostsize bytes will be read.

The line terminator is alwaysb'\n' for binary files; for text files,thenewline argument toopen() can be used to select the lineterminator(s) recognized.

readlines(hint=-1)¶

Read and return a list of lines from the stream.hint can be specifiedto control the number of lines read: no more lines will be read if thetotal size (in bytes/characters) of all lines so far exceedshint.

Note that it’s already possible to iterate on file objects usingforlineinfile:... without callingfile.readlines().

seek(offset[,whence])¶

Change the stream position to the given byteoffset.offset isinterpreted relative to the position indicated bywhence. The defaultvalue forwhence isSEEK_SET. Values forwhence are:

• SEEK_SET or0 – start of the stream (the default);offset should be zero or positive

• SEEK_CUR or1 – current stream position;offset maybe negative

• SEEK_END or2 – end of the stream;offset is usuallynegative

Return the new absolute position.

New in version 3.1:TheSEEK_* constants.

New in version 3.3:Some operating systems could support additional values, likeos.SEEK_HOLE oros.SEEK_DATA. The valid valuesfor a file could depend on it being open in text or binary mode.

seekable()¶

ReturnTrue if the stream supports random access. IfFalse,seek(),tell() andtruncate() will raiseOSError.

tell()¶

Return the current stream position.

truncate(size=None)¶

Resize the stream to the givensize in bytes (or the current positionifsize is not specified). The current stream position isn’t changed.This resizing can extend or reduce the current file size. In case ofextension, the contents of the new file area depend on the platform(on most systems, additional bytes are zero-filled). The new file sizeis returned.

Changed in version 3.5:Windows will now zero-fill files when extending.

writable()¶

ReturnTrue if the stream supports writing. IfFalse,write() andtruncate() will raiseOSError.

writelines(lines)¶

Write a list of lines to the stream. Line separators are not added, so itis usual for each of the lines provided to have a line separator at theend.

__del__()¶

Prepare for object destruction.IOBase provides a defaultimplementation of this method that calls the instance’sclose() method.

classio.RawIOBase¶

Base class for raw binary I/O. It inheritsIOBase. There is nopublic constructor.

Raw binary I/O typically provides low-level access to an underlying OSdevice or API, and does not try to encapsulate it in high-level primitives(this is left to Buffered I/O and Text I/O, described later in this page).

In addition to the attributes and methods fromIOBase,RawIOBase provides the following methods:

read(size=-1)¶

Read up tosize bytes from the object and return them. As a convenience,ifsize is unspecified or -1, all bytes until EOF are returned.Otherwise, only one system call is ever made. Fewer thansize bytes maybe returned if the operating system call returns fewer thansize bytes.

If 0 bytes are returned, andsize was not 0, this indicates end of file.If the object is in non-blocking mode and no bytes are available,None is returned.

The default implementation defers toreadall() andreadinto().

readall()¶

Read and return all the bytes from the stream until EOF, using multiplecalls to the stream if necessary.

readinto(b)¶

Read bytes into a pre-allocated, writablebytes-like objectb, and return thenumber of bytes read. If the object is in non-blocking mode and no bytesare available,None is returned.

write(b)¶

Write the givenbytes-like object,b, to theunderlying raw stream, and return the number ofbytes written. This can be less than the length ofb inbytes, depending on specifics of the underlying rawstream, and especially if it is in non-blocking mode.None isreturned if the raw stream is set not to block and no single byte couldbe readily written to it. The caller may release or mutateb afterthis method returns, so the implementation should only accessbduring the method call.

classio.BufferedIOBase¶

Base class for binary streams that support some kind of buffering.It inheritsIOBase. There is no public constructor.

The main difference withRawIOBase is that methodsread(),readinto() andwrite() will try (respectively) to read as muchinput as requested or to consume all given output, at the expense ofmaking perhaps more than one system call.

In addition, those methods can raiseBlockingIOError if theunderlying raw stream is in non-blocking mode and cannot take or giveenough data; unlike theirRawIOBase counterparts, they willnever returnNone.

Besides, theread() method does not have a defaultimplementation that defers toreadinto().

A typicalBufferedIOBase implementation should not inherit from aRawIOBase implementation, but wrap one, likeBufferedWriter andBufferedReader do.

BufferedIOBase provides or overrides these methods and attribute inaddition to those fromIOBase:

raw¶

The underlying raw stream (aRawIOBase instance) thatBufferedIOBase deals with. This is not part of theBufferedIOBase API and may not exist on some implementations.

detach()¶

Separate the underlying raw stream from the buffer and return it.

After the raw stream has been detached, the buffer is in an unusablestate.

Some buffers, likeBytesIO, do not have the concept of a singleraw stream to return from this method. They raiseUnsupportedOperation.

New in version 3.1.

read(size=-1)¶

Read and return up tosize bytes. If the argument is omitted,None,or negative, data is read and returned until EOF is reached. An emptybytes object is returned if the stream is already at EOF.

If the argument is positive, and the underlying raw stream is notinteractive, multiple raw reads may be issued to satisfy the byte count(unless EOF is reached first). But for interactive raw streams, at mostone raw read will be issued, and a short result does not imply that EOF isimminent.

ABlockingIOError is raised if the underlying raw stream is innon blocking-mode, and has no data available at the moment.

read1(size=-1)¶

Read and return up tosize bytes, with at most one call to theunderlying raw stream’sread() (orreadinto()) method. This can be useful if you areimplementing your own buffering on top of aBufferedIOBaseobject.

readinto(b)¶

Read bytes into a pre-allocated, writablebytes-like objectb and return the number of bytes read.

Likeread(), multiple reads may be issued to the underlying rawstream, unless the latter is interactive.

ABlockingIOError is raised if the underlying raw stream is in nonblocking-mode, and has no data available at the moment.

readinto1(b)¶

Read bytes into a pre-allocated, writablebytes-like objectb, using at most one call tothe underlying raw stream’sread() (orreadinto()) method. Return the number of bytes read.

ABlockingIOError is raised if the underlying raw stream is in nonblocking-mode, and has no data available at the moment.

New in version 3.5.

write(b)¶

Write the givenbytes-like object,b, and return the numberof bytes written (always equal to the length ofb in bytes, since ifthe write fails anOSError will be raised). Depending on theactual implementation, these bytes may be readily written to theunderlying stream, or held in a buffer for performance and latencyreasons.

When in non-blocking mode, aBlockingIOError is raised if thedata needed to be written to the raw stream but it couldn’t acceptall the data without blocking.

The caller may release or mutateb after this method returns,so the implementation should only accessb during the method call.

16.2.3.2.Raw File I/O¶

classio.FileIO(name,mode='r',closefd=True,opener=None)¶

FileIO represents an OS-level file containing bytes data.It implements theRawIOBase interface (and therefore theIOBase interface, too).

Thename can be one of two things:

• a character string orbytes object representing the path to thefile which will be opened. In this case closefd must beTrue (the default)otherwise an error will be raised.

• an integer representing the number of an existing OS-level file descriptorto which the resultingFileIO object will give access. When theFileIO object is closed this fd will be closed as well, unlessclosefdis set toFalse.

Themode can be'r','w','x' or'a' for reading(default), writing, exclusive creation or appending. The file will becreated if it doesn’t exist when opened for writing or appending; it will betruncated when opened for writing.FileExistsError will be raised ifit already exists when opened for creating. Opening a file for creatingimplies writing, so this mode behaves in a similar way to'w'. Add a'+' to the mode to allow simultaneous reading and writing.

Theread() (when called with a positive argument),readinto()andwrite() methods on this class will only make one system call.

A custom opener can be used by passing a callable asopener. The underlyingfile descriptor for the file object is then obtained by callingopener with(name,flags).opener must return an open file descriptor (passingos.open asopener results in functionality similar to passingNone).

The newly created file isnon-inheritable.

See theopen() built-in function for examples on using theopenerparameter.

Changed in version 3.3:Theopener parameter was added.The'x' mode was added.

Changed in version 3.4:The file is now non-inheritable.

In addition to the attributes and methods fromIOBase andRawIOBase,FileIO provides the following dataattributes:

mode¶

The mode as given in the constructor.

name¶

The file name. This is the file descriptor of the file when no name isgiven in the constructor.

16.2.3.3.Buffered Streams¶

Buffered I/O streams provide a higher-level interface to an I/O devicethan raw I/O does.

classio.BytesIO([initial_bytes])¶

A stream implementation using an in-memory bytes buffer. It inheritsBufferedIOBase. The buffer is discarded when theclose() method is called.

The optional argumentinitial_bytes is abytes-like object thatcontains initial data.

BytesIO provides or overrides these methods in addition to thosefromBufferedIOBase andIOBase:

getbuffer()¶

Return a readable and writable view over the contents of the bufferwithout copying them. Also, mutating the view will transparentlyupdate the contents of the buffer:

>>>b=io.BytesIO(b"abcdef")>>>view=b.getbuffer()>>>view[2:4]=b"56">>>b.getvalue()b'ab56ef'

Note

As long as the view exists, theBytesIO object cannot beresized or closed.

New in version 3.2.

getvalue()¶

Returnbytes containing the entire contents of the buffer.

read1()¶

InBytesIO, this is the same asread().

readinto1()¶

InBytesIO, this is the same asreadinto().

New in version 3.5.

classio.BufferedReader(raw,buffer_size=DEFAULT_BUFFER_SIZE)¶

A buffer providing higher-level access to a readable, sequentialRawIOBase object. It inheritsBufferedIOBase.When reading data from this object, a larger amount of data may berequested from the underlying raw stream, and kept in an internal buffer.The buffered data can then be returned directly on subsequent reads.

The constructor creates aBufferedReader for the given readableraw stream andbuffer_size. Ifbuffer_size is omitted,DEFAULT_BUFFER_SIZE is used.

BufferedReader provides or overrides these methods in addition tothose fromBufferedIOBase andIOBase:

peek([size])¶

Return bytes from the stream without advancing the position. At most onesingle read on the raw stream is done to satisfy the call. The number ofbytes returned may be less or more than requested.

read([size])¶

Read and returnsize bytes, or ifsize is not given or negative, untilEOF or if the read call would block in non-blocking mode.

read1(size)¶

Read and return up tosize bytes with only one call on the raw stream.If at least one byte is buffered, only buffered bytes are returned.Otherwise, one raw stream read call is made.

classio.BufferedWriter(raw,buffer_size=DEFAULT_BUFFER_SIZE)¶

A buffer providing higher-level access to a writeable, sequentialRawIOBase object. It inheritsBufferedIOBase.When writing to this object, data is normally placed into an internalbuffer. The buffer will be written out to the underlyingRawIOBaseobject under various conditions, including:

• when the buffer gets too small for all pending data;

• whenflush() is called;

• when aseek() is requested (forBufferedRandom objects);

• when theBufferedWriter object is closed or destroyed.

The constructor creates aBufferedWriter for the given writeableraw stream. If thebuffer_size is not given, it defaults toDEFAULT_BUFFER_SIZE.

BufferedWriter provides or overrides these methods in addition tothose fromBufferedIOBase andIOBase:

flush()¶

Force bytes held in the buffer into the raw stream. ABlockingIOError should be raised if the raw stream blocks.

write(b)¶

Write thebytes-like object,b, and return thenumber of bytes written. When in non-blocking mode, aBlockingIOError is raised if the buffer needs to be written out butthe raw stream blocks.

classio.BufferedRandom(raw,buffer_size=DEFAULT_BUFFER_SIZE)¶

A buffered interface to random access streams. It inheritsBufferedReader andBufferedWriter, and further supportsseek() andtell() functionality.

The constructor creates a reader and writer for a seekable raw stream, givenin the first argument. If thebuffer_size is omitted it defaults toDEFAULT_BUFFER_SIZE.

BufferedRandom is capable of anythingBufferedReader orBufferedWriter can do.

classio.BufferedRWPair(reader,writer,buffer_size=DEFAULT_BUFFER_SIZE)¶

A buffered I/O object combining two unidirectionalRawIOBaseobjects – one readable, the other writeable – into a single bidirectionalendpoint. It inheritsBufferedIOBase.

reader andwriter areRawIOBase objects that are readable andwriteable respectively. If thebuffer_size is omitted it defaults toDEFAULT_BUFFER_SIZE.

BufferedRWPair implements all ofBufferedIOBase’s methodsexcept fordetach(), which raisesUnsupportedOperation.

Warning

BufferedRWPair does not attempt to synchronize accesses toits underlying raw streams. You should not pass it the same objectas reader and writer; useBufferedRandom instead.

16.2.3.4.Text I/O¶

classio.TextIOBase¶

Base class for text streams. This class provides a character and line basedinterface to stream I/O. There is noreadinto() method becausePython’s character strings are immutable. It inheritsIOBase.There is no public constructor.

TextIOBase provides or overrides these data attributes andmethods in addition to those fromIOBase:

encoding¶

The name of the encoding used to decode the stream’s bytes intostrings, and to encode strings into bytes.

errors¶

The error setting of the decoder or encoder.

newlines¶

A string, a tuple of strings, orNone, indicating the newlinestranslated so far. Depending on the implementation and the initialconstructor flags, this may not be available.

buffer¶

The underlying binary buffer (aBufferedIOBase instance) thatTextIOBase deals with. This is not part of theTextIOBase API and may not exist in some implementations.

detach()¶

Separate the underlying binary buffer from theTextIOBase andreturn it.

After the underlying buffer has been detached, theTextIOBase isin an unusable state.

SomeTextIOBase implementations, likeStringIO, may nothave the concept of an underlying buffer and calling this method willraiseUnsupportedOperation.

New in version 3.1.

read(size=-1)¶

Read and return at mostsize characters from the stream as a singlestr. Ifsize is negative orNone, reads until EOF.

readline(size=-1)¶

Read until newline or EOF and return a singlestr. If the stream isalready at EOF, an empty string is returned.

Ifsize is specified, at mostsize characters will be read.

seek(offset[,whence])¶

Change the stream position to the givenoffset. Behaviour depends onthewhence parameter. The default value forwhence isSEEK_SET.

• SEEK_SET or0: seek from the start of the stream(the default);offset must either be a number returned byTextIOBase.tell(), or zero. Any otheroffset valueproduces undefined behaviour.

• SEEK_CUR or1: “seek” to the current position;offset must be zero, which is a no-operation (all other valuesare unsupported).

• SEEK_END or2: seek to the end of the stream;offset must be zero (all other values are unsupported).

Return the new absolute position as an opaque number.

New in version 3.1:TheSEEK_* constants.

tell()¶

Return the current stream position as an opaque number. The numberdoes not usually represent a number of bytes in the underlyingbinary storage.

write(s)¶

Write the strings to the stream and return the number of characterswritten.

classio.TextIOWrapper(buffer,encoding=None,errors=None,newline=None,line_buffering=False,write_through=False)¶

A buffered text stream over aBufferedIOBase binary stream.It inheritsTextIOBase.

encoding gives the name of the encoding that the stream will be decoded orencoded with. It defaults tolocale.getpreferredencoding(False).

errors is an optional string that specifies how encoding and decodingerrors are to be handled. Pass'strict' to raise aValueErrorexception if there is an encoding error (the default ofNone has the sameeffect), or pass'ignore' to ignore errors. (Note that ignoring encodingerrors can lead to data loss.)'replace' causes a replacement marker(such as'?') to be inserted where there is malformed data.'backslashreplace' causes malformed data to be replaced by abackslashed escape sequence. When writing,'xmlcharrefreplace'(replace with the appropriate XML character reference) or'namereplace'(replace with\N{...} escape sequences) can be used. Any other errorhandling name that has been registered withcodecs.register_error() is also valid.

newline controls how line endings are handled. It can beNone,'','\n','\r', and'\r\n'. It works as follows:

• When reading input from the stream, ifnewline isNone,universal newlines mode is enabled. Lines in the input can end in'\n','\r', or'\r\n', and these are translated into'\n'before being returned to the caller. If it is'', universal newlinesmode is enabled, but line endings are returned to the caller untranslated.If it has any of the other legal values, input lines are only terminatedby the given string, and the line ending is returned to the calleruntranslated.

• When writing output to the stream, ifnewline isNone, any'\n'characters written are translated to the system default line separator,os.linesep. Ifnewline is'' or'\n', no translationtakes place. Ifnewline is any of the other legal values, any'\n'characters written are translated to the given string.

Ifline_buffering isTrue,flush() is implied when a call towrite contains a newline character or a carriage return.

Ifwrite_through isTrue, calls towrite() are guaranteednot to be buffered: any data written on theTextIOWrapperobject is immediately handled to its underlying binarybuffer.

Changed in version 3.3:Thewrite_through argument has been added.

Changed in version 3.3:The defaultencoding is nowlocale.getpreferredencoding(False)instead oflocale.getpreferredencoding(). Don’t change temporary thelocale encoding usinglocale.setlocale(), use the current localeencoding instead of the user preferred encoding.

TextIOWrapper provides one attribute in addition to those ofTextIOBase and its parents:

line_buffering¶

Whether line buffering is enabled.

classio.StringIO(initial_value='',newline='\n')¶

An in-memory stream for text I/O. The text buffer is discarded when theclose() method is called.

The initial value of the buffer can be set by providinginitial_value.If newline translation is enabled, newlines will be encoded as if bywrite(). The stream is positioned at the start ofthe buffer.

Thenewline argument works like that ofTextIOWrapper.The default is to consider only\n characters as ends of lines andto do no newline translation. Ifnewline is set toNone,newlines are written as\n on all platforms, but universalnewline decoding is still performed when reading.

StringIO provides this method in addition to those fromTextIOBase and its parents:

getvalue()¶

Return astr containing the entire contents of the buffer.Newlines are decoded as if byread(), althoughthe stream position is not changed.

Example usage:

importiooutput=io.StringIO()output.write('First line.\n')print('Second line.',file=output)# Retrieve file contents -- this will be# 'First line.\nSecond line.\n'contents=output.getvalue()# Close object and discard memory buffer --# .getvalue() will now raise an exception.output.close()

classio.IncrementalNewlineDecoder¶

A helper codec that decodes newlines foruniversal newlines mode.It inheritscodecs.IncrementalDecoder.

16.2.4.1.Binary I/O¶

By reading and writing only large chunks of data even when the user asks for asingle byte, buffered I/O hides any inefficiency in calling and executing theoperating system’s unbuffered I/O routines. The gain depends on the OS and thekind of I/O which is performed. For example, on some modern OSes such as Linux,unbuffered disk I/O can be as fast as buffered I/O. The bottom line, however,is that buffered I/O offers predictable performance regardless of the platformand the backing device. Therefore, it is almost always preferable to usebuffered I/O rather than unbuffered I/O for binary data.

16.2.4.2.Text I/O¶

Text I/O over a binary storage (such as a file) is significantly slower thanbinary I/O over the same storage, because it requires conversions betweenunicode and binary data using a character codec. This can become noticeablehandling huge amounts of text data like large log files. Also,TextIOWrapper.tell() andTextIOWrapper.seek() are both quite slowdue to the reconstruction algorithm used.

StringIO, however, is a native in-memory unicode container and willexhibit similar speed toBytesIO.

16.2.4.3.Multi-threading¶

FileIO objects are thread-safe to the extent that the operating systemcalls (such asread(2) under Unix) they wrap are thread-safe too.

Binary buffered objects (instances ofBufferedReader,BufferedWriter,BufferedRandom andBufferedRWPair)protect their internal structures using a lock; it is therefore safe to callthem from multiple threads at once.

TextIOWrapper objects are not thread-safe.

16.2.4.4.Reentrancy¶

Binary buffered objects (instances ofBufferedReader,BufferedWriter,BufferedRandom andBufferedRWPair)are not reentrant. While reentrant calls will not happen in normal situations,they can arise from doing I/O in asignal handler. If a thread tries tore-enter a buffered object which it is already accessing, aRuntimeErroris raised. Note this doesn’t prohibit a different thread from entering thebuffered object.

The above implicitly extends to text files, since theopen() functionwill wrap a buffered object inside aTextIOWrapper. This includesstandard streams and therefore affects the built-in functionprint() aswell.