Theio module provides Python’s main facilities for dealing with varioustypes of I/O. There are three main types of I/O:text I/O,binary I/Oandraw I/O. These are generic categories, and various backing stores canbe used for each of them. A concrete object belonging to any of thesecategories is called afile object. Other common terms arestreamandfile-like object.
Independently of its category, each concrete stream object will also havevarious capabilities: it can be read-only, write-only, or read-write. It canalso allow arbitrary random access (seeking forwards or backwards to anylocation), or only sequential access (for example in the case of a socket orpipe).
All streams are careful about the type of data you give to them. For examplegiving astr object to thewrite() method of a binary streamwill raise aTypeError. So will giving abytes object to thewrite() method of a text stream.
Changed in version 3.3:Operations that used to raiseIOError now raiseOSError, sinceIOError is now an alias ofOSError.
Text I/O expects and producesstr objects. This means that wheneverthe backing store is natively made of bytes (such as in the case of a file),encoding and decoding of data is made transparently as well as optionaltranslation of platform-specific newline characters.
The easiest way to create a text stream is withopen(), optionallyspecifying an encoding:
f=open("myfile.txt","r",encoding="utf-8")
In-memory text streams are also available asStringIO objects:
f=io.StringIO("some initial text data")
The text stream API is described in detail in the documentation ofTextIOBase.
Binary I/O (also calledbuffered I/O) expects and producesbytesobjects. No encoding, decoding, or newline translation is performed. Thiscategory of streams can be used for all kinds of non-text data, and also whenmanual control over the handling of text data is desired.
The easiest way to create a binary stream is withopen() with'b' inthe mode string:
f=open("myfile.jpg","rb")
In-memory binary streams are also available asBytesIO objects:
f=io.BytesIO(b"some initial binary data:\x00\x01")
The binary stream API is described in detail in the docs ofBufferedIOBase.
Other library modules may provide additional ways to create text or binarystreams. Seesocket.socket.makefile() for example.
Raw I/O (also calledunbuffered I/O) is generally used as a low-levelbuilding-block for binary and text streams; it is rarely useful to directlymanipulate a raw stream from user code. Nevertheless, you can create a rawstream by opening a file in binary mode with buffering disabled:
f=open("myfile.jpg","rb",buffering=0)
The raw stream API is described in detail in the docs ofRawIOBase.
An int containing the default buffer size used by the module’s buffered I/Oclasses.open() uses the file’s blksize (as obtained byos.stat()) if possible.
This is an alias for the builtinopen() function.
This is a compatibility alias for the builtinBlockingIOErrorexception.
An exception inheritingOSError andValueError that is raisedwhen an unsupported operation is called on a stream.
It is also possible to use astr orbytes-like object as afile for both reading and writing. For stringsStringIO can be usedlike a file opened in text mode.BytesIO can be used like a fileopened in binary mode. Both provide full read-write capabilities with randomaccess.
See also
The implementation of I/O streams is organized as a hierarchy of classes. Firstabstract base classes (ABCs), which are used tospecify the various categories of streams, then concrete classes providing thestandard stream implementations.
Note
The abstract base classes also provide default implementations of somemethods in order to help implementation of concrete stream classes. Forexample,BufferedIOBase provides unoptimized implementations ofreadinto() andreadline().
At the top of the I/O hierarchy is the abstract base classIOBase. Itdefines the basic interface to a stream. Note, however, that there is noseparation between reading and writing to streams; implementations are allowedto raiseUnsupportedOperation if they do not support a given operation.
TheRawIOBase ABC extendsIOBase. It deals with the readingand writing of bytes to a stream.FileIO subclassesRawIOBaseto provide an interface to files in the machine’s file system.
TheBufferedIOBase ABC deals with buffering on a raw byte stream(RawIOBase). Its subclasses,BufferedWriter,BufferedReader, andBufferedRWPair buffer streams that arereadable, writable, and both readable and writable.BufferedRandomprovides a buffered interface to random access streams. AnotherBufferedIOBase subclass,BytesIO, is a stream of in-memorybytes.
TheTextIOBase ABC, another subclass ofIOBase, deals withstreams whose bytes represent text, and handles encoding and decoding to andfrom strings.TextIOWrapper, which extends it, is a buffered textinterface to a buffered raw stream (BufferedIOBase). Finally,StringIO is an in-memory stream for text.
Argument names are not part of the specification, and only the arguments ofopen() are intended to be used as keyword arguments.
The following table summarizes the ABCs provided by theio module:
| ABC | Inherits | Stub Methods | Mixin Methods and Properties |
|---|---|---|---|
| IOBase | fileno,seek,andtruncate | close,closed,__enter__,__exit__,flush,isatty,__iter__,__next__,readable,readline,readlines,seekable,tell,writable, andwritelines | |
| RawIOBase | IOBase | readinto andwrite | InheritedIOBase methods,read,andreadall |
| BufferedIOBase | IOBase | detach,read,read1, andwrite | InheritedIOBase methods,readinto |
| TextIOBase | IOBase | detach,read,readline, andwrite | InheritedIOBase methods,encoding,errors, andnewlines |
The abstract base class for all I/O classes, acting on streams of bytes.There is no public constructor.
This class provides empty abstract implementations for many methodsthat derived classes can override selectively; the defaultimplementations represent a file that cannot be read, written orseeked.
Even thoughIOBase does not declareread(),readinto(),orwrite() because their signatures will vary, implementations andclients should consider those methods part of the interface. Also,implementations may raise aValueError (orUnsupportedOperation)when operations they do not support are called.
The basic type used for binary data read from or written to a file isbytes.bytearrays are accepted too, and in some cases(such asreadinto()) required. Text I/O classes work withstr data.
Note that calling any method (even inquiries) on a closed stream isundefined. Implementations may raiseValueError in this case.
IOBase (and its subclasses) supports the iterator protocol, meaningthat anIOBase object can be iterated over yielding the lines in astream. Lines are defined slightly differently depending on whether thestream is a binary stream (yielding bytes), or a text stream (yieldingcharacter strings). Seereadline() below.
IOBase is also a context manager and therefore supports thewith statement. In this example,file is closed after thewith statement’s suite is finished—even if an exception occurs:
withopen('spam.txt','w')asfile:file.write('Spam and eggs!')
IOBase provides these data attributes and methods:
Flush and close this stream. This method has no effect if the file isalready closed. Once the file is closed, any operation on the file(e.g. reading or writing) will raise aValueError.
As a convenience, it is allowed to call this method more than once;only the first call, however, will have an effect.
True if the stream is closed.
Return the underlying file descriptor (an integer) of the stream if itexists. AnOSError is raised if the IO object does not use a filedescriptor.
Flush the write buffers of the stream if applicable. This does nothingfor read-only and non-blocking streams.
ReturnTrue if the stream is interactive (i.e., connected toa terminal/tty device).
Read and return one line from the stream. Iflimit is specified, atmostlimit bytes will be read.
The line terminator is alwaysb'\n' for binary files; for text files,thenewlines argument toopen() can be used to select the lineterminator(s) recognized.
Read and return a list of lines from the stream.hint can be specifiedto control the number of lines read: no more lines will be read if thetotal size (in bytes/characters) of all lines so far exceedshint.
Note that it’s already possible to iterate on file objects usingforlineinfile:... without callingfile.readlines().
Change the stream position to the given byteoffset.offset isinterpreted relative to the position indicated bywhence. Values forwhence are:
Return the new absolute position.
New in version 3.1:TheSEEK_* constants.
New in version 3.3:Some operating systems could support additional values, likeos.SEEK_HOLE oros.SEEK_DATA. The valid valuesfor a file could depend on it being open in text or binary mode.
ReturnTrue if the stream supports random access. IfFalse,seek(),tell() andtruncate() will raiseOSError.
Return the current stream position.
Resize the stream to the givensize in bytes (or the current positionifsize is not specified). The current stream position isn’t changed.This resizing can extend or reduce the current file size. In case ofextension, the contents of the new file area depend on the platform(on most systems, additional bytes are zero-filled, on Windows they’reundetermined). The new file size is returned.
ReturnTrue if the stream supports writing. IfFalse,write() andtruncate() will raiseOSError.
Write a list of lines to the stream. Line separators are not added, so itis usual for each of the lines provided to have a line separator at theend.
Base class for raw binary I/O. It inheritsIOBase. There is nopublic constructor.
Raw binary I/O typically provides low-level access to an underlying OSdevice or API, and does not try to encapsulate it in high-level primitives(this is left to Buffered I/O and Text I/O, described later in this page).
In addition to the attributes and methods fromIOBase,RawIOBase provides the following methods:
Read up ton bytes from the object and return them. As a convenience,ifn is unspecified or -1,readall() is called. Otherwise,only one system call is ever made. Fewer thann bytes may bereturned if the operating system call returns fewer thann bytes.
If 0 bytes are returned, andn was not 0, this indicates end of file.If the object is in non-blocking mode and no bytes are available,None is returned.
Read and return all the bytes from the stream until EOF, using multiplecalls to the stream if necessary.
Read up tolen(b) bytes intobytearrayb and return thenumber of bytes read. If the object is in non-blocking mode and nobytes are available,None is returned.
Write the givenbytes orbytearray object,b, to theunderlying raw stream and return the number of bytes written. This canbe less thanlen(b), depending on specifics of the underlying rawstream, and especially if it is in non-blocking mode.None isreturned if the raw stream is set not to block and no single byte couldbe readily written to it.
Base class for binary streams that support some kind of buffering.It inheritsIOBase. There is no public constructor.
The main difference withRawIOBase is that methodsread(),readinto() andwrite() will try (respectively) to read as muchinput as requested or to consume all given output, at the expense ofmaking perhaps more than one system call.
In addition, those methods can raiseBlockingIOError if theunderlying raw stream is in non-blocking mode and cannot take or giveenough data; unlike theirRawIOBase counterparts, they willnever returnNone.
Besides, theread() method does not have a defaultimplementation that defers toreadinto().
A typicalBufferedIOBase implementation should not inherit from aRawIOBase implementation, but wrap one, likeBufferedWriter andBufferedReader do.
BufferedIOBase provides or overrides these methods and attribute inaddition to those fromIOBase:
The underlying raw stream (aRawIOBase instance) thatBufferedIOBase deals with. This is not part of theBufferedIOBase API and may not exist on some implementations.
Separate the underlying raw stream from the buffer and return it.
After the raw stream has been detached, the buffer is in an unusablestate.
Some buffers, likeBytesIO, do not have the concept of a singleraw stream to return from this method. They raiseUnsupportedOperation.
New in version 3.1.
Read and return up ton bytes. If the argument is omitted,None, ornegative, data is read and returned until EOF is reached. An emptybytes object is returned if the stream is already at EOF.
If the argument is positive, and the underlying raw stream is notinteractive, multiple raw reads may be issued to satisfy the byte count(unless EOF is reached first). But for interactive raw streams, at mostone raw read will be issued, and a short result does not imply that EOF isimminent.
ABlockingIOError is raised if the underlying raw stream is innon blocking-mode, and has no data available at the moment.
Read and return up ton bytes, with at most one call to the underlyingraw stream’sread() method. This can be useful if youare implementing your own buffering on top of aBufferedIOBaseobject.
Read up tolen(b) bytes into bytearrayb and return the number ofbytes read.
Likeread(), multiple reads may be issued to the underlying rawstream, unless the latter is interactive.
ABlockingIOError is raised if the underlying raw stream is innon blocking-mode, and has no data available at the moment.
Write the givenbytes orbytearray object,b andreturn the number of bytes written (never less thanlen(b), since ifthe write fails anOSError will be raised). Depending on theactual implementation, these bytes may be readily written to theunderlying stream, or held in a buffer for performance and latencyreasons.
When in non-blocking mode, aBlockingIOError is raised if thedata needed to be written to the raw stream but it couldn’t acceptall the data without blocking.
FileIO represents an OS-level file containing bytes data.It implements theRawIOBase interface (and therefore theIOBase interface, too).
Thename can be one of two things:
Themode can be'r','w','x' or'a' for reading(default), writing, exclusive creation or appending. The file will becreated if it doesn’t exist when opened for writing or appending; it will betruncated when opened for writing.FileExistsError will be raised ifit already exists when opened for creating. Opening a file for creatingimplies writing, so this mode behaves in a similar way to'w'. Add a'+' to the mode to allow simultaneous reading and writing.
Theread() (when called with a positive argument),readinto()andwrite() methods on this class will only make one system call.
A custom opener can be used by passing a callable asopener. The underlyingfile descriptor for the file object is then obtained by callingopener with(name,flags).opener must return an open file descriptor (passingos.open asopener results in functionality similar to passingNone).
See theopen() built-in function for examples on using theopenerparameter.
Changed in version 3.3:Theopener parameter was added.The'x' mode was added.
In addition to the attributes and methods fromIOBase andRawIOBase,FileIO provides the following dataattributes:
The mode as given in the constructor.
The file name. This is the file descriptor of the file when no name isgiven in the constructor.
Buffered I/O streams provide a higher-level interface to an I/O devicethan raw I/O does.
A stream implementation using an in-memory bytes buffer. It inheritsBufferedIOBase.
The argumentinitial_bytes contains optional initialbytes data.
BytesIO provides or overrides these methods in addition to thosefromBufferedIOBase andIOBase:
Return a readable and writable view over the contents of the bufferwithout copying them. Also, mutating the view will transparentlyupdate the contents of the buffer:
>>>b=io.BytesIO(b"abcdef")>>>view=b.getbuffer()>>>view[2:4]=b"56">>>b.getvalue()b'ab56ef'
Note
As long as the view exists, theBytesIO object cannot beresized.
New in version 3.2.
A buffer providing higher-level access to a readable, sequentialRawIOBase object. It inheritsBufferedIOBase.When reading data from this object, a larger amount of data may berequested from the underlying raw stream, and kept in an internal buffer.The buffered data can then be returned directly on subsequent reads.
The constructor creates aBufferedReader for the given readableraw stream andbuffer_size. Ifbuffer_size is omitted,DEFAULT_BUFFER_SIZE is used.
BufferedReader provides or overrides these methods in addition tothose fromBufferedIOBase andIOBase:
Return bytes from the stream without advancing the position. At most onesingle read on the raw stream is done to satisfy the call. The number ofbytes returned may be less or more than requested.
Read and returnn bytes, or ifn is not given or negative, until EOFor if the read call would block in non-blocking mode.
Read and return up ton bytes with only one call on the raw stream. Ifat least one byte is buffered, only buffered bytes are returned.Otherwise, one raw stream read call is made.
A buffer providing higher-level access to a writeable, sequentialRawIOBase object. It inheritsBufferedIOBase.When writing to this object, data is normally placed into an internalbuffer. The buffer will be written out to the underlyingRawIOBaseobject under various conditions, including:
The constructor creates aBufferedWriter for the given writeableraw stream. If thebuffer_size is not given, it defaults toDEFAULT_BUFFER_SIZE.
BufferedWriter provides or overrides these methods in addition tothose fromBufferedIOBase andIOBase:
Force bytes held in the buffer into the raw stream. ABlockingIOError should be raised if the raw stream blocks.
Write thebytes orbytearray object,b and return thenumber of bytes written. When in non-blocking mode, aBlockingIOError is raised if the buffer needs to be written out butthe raw stream blocks.
A buffered interface to random access streams. It inheritsBufferedReader andBufferedWriter, and further supportsseek() andtell() functionality.
The constructor creates a reader and writer for a seekable raw stream, givenin the first argument. If thebuffer_size is omitted it defaults toDEFAULT_BUFFER_SIZE.
BufferedRandom is capable of anythingBufferedReader orBufferedWriter can do.
A buffered I/O object combining two unidirectionalRawIOBaseobjects – one readable, the other writeable – into a single bidirectionalendpoint. It inheritsBufferedIOBase.
reader andwriter areRawIOBase objects that are readable andwriteable respectively. If thebuffer_size is omitted it defaults toDEFAULT_BUFFER_SIZE.
BufferedRWPair implements all ofBufferedIOBase‘s methodsexcept fordetach(), which raisesUnsupportedOperation.
Warning
BufferedRWPair does not attempt to synchronize accesses toits underlying raw streams. You should not pass it the same objectas reader and writer; useBufferedRandom instead.
Base class for text streams. This class provides a character and line basedinterface to stream I/O. There is noreadinto() method becausePython’s character strings are immutable. It inheritsIOBase.There is no public constructor.
TextIOBase provides or overrides these data attributes andmethods in addition to those fromIOBase:
The name of the encoding used to decode the stream’s bytes intostrings, and to encode strings into bytes.
The error setting of the decoder or encoder.
A string, a tuple of strings, orNone, indicating the newlinestranslated so far. Depending on the implementation and the initialconstructor flags, this may not be available.
The underlying binary buffer (aBufferedIOBase instance) thatTextIOBase deals with. This is not part of theTextIOBase API and may not exist in some implementations.
Separate the underlying binary buffer from theTextIOBase andreturn it.
After the underlying buffer has been detached, theTextIOBase isin an unusable state.
SomeTextIOBase implementations, likeStringIO, may nothave the concept of an underlying buffer and calling this method willraiseUnsupportedOperation.
New in version 3.1.
Read and return at mostn characters from the stream as a singlestr. Ifn is negative orNone, reads until EOF.
Read until newline or EOF and return a singlestr. If the stream isalready at EOF, an empty string is returned.
Iflimit is specified, at mostlimit characters will be read.
Change the stream position to the givenoffset. Behaviour dependson thewhence parameter:
Return the new absolute position as an opaque number.
New in version 3.1:TheSEEK_* constants.
Return the current stream position as an opaque number. The numberdoes not usually represent a number of bytes in the underlyingbinary storage.
Write the strings to the stream and return the number of characterswritten.
A buffered text stream over aBufferedIOBase binary stream.It inheritsTextIOBase.
encoding gives the name of the encoding that the stream will be decoded orencoded with. It defaults tolocale.getpreferredencoding(False).
errors is an optional string that specifies how encoding and decodingerrors are to be handled. Pass'strict' to raise aValueErrorexception if there is an encoding error (the default ofNone has the sameeffect), or pass'ignore' to ignore errors. (Note that ignoring encodingerrors can lead to data loss.)'replace' causes a replacement marker(such as'?') to be inserted where there is malformed data. Whenwriting,'xmlcharrefreplace' (replace with the appropriate XML characterreference) or'backslashreplace' (replace with backslashed escapesequences) can be used. Any other error handling name that has beenregistered withcodecs.register_error() is also valid.
newline controls how line endings are handled. It can beNone,'','\n','\r', and'\r\n'. It works as follows:
Ifline_buffering isTrue,flush() is implied when a call towrite contains a newline character.
Ifwrite_through isTrue, calls towrite() are guaranteednot to be buffered: any data written on theTextIOWrapperobject is immediately handled to its underlying binarybuffer.
Changed in version 3.3:Thewrite_through argument has been added.
Changed in version 3.3:The defaultencoding is nowlocale.getpreferredencoding(False)instead oflocale.getpreferredencoding(). Don’t change temporary thelocale encoding usinglocale.setlocale(), use the current localeencoding instead of the user preferred encoding.
TextIOWrapper provides one attribute in addition to those ofTextIOBase and its parents:
Whether line buffering is enabled.
An in-memory stream for text I/O.
The initial value of the buffer (an empty string by default) can be set byprovidinginitial_value. Thenewline argument works like that ofTextIOWrapper. The default is to consider only\n charactersas end of lines and to do no newline translation.
StringIO provides this method in addition to those fromTextIOBase and its parents:
Return astr containing the entire contents of the buffer at anytime before theStringIO object’sclose() method iscalled.
Example usage:
importiooutput=io.StringIO()output.write('First line.\n')print('Second line.',file=output)# Retrieve file contents -- this will be# 'First line.\nSecond line.\n'contents=output.getvalue()# Close object and discard memory buffer --# .getvalue() will now raise an exception.output.close()
A helper codec that decodes newlines foruniversal newlines mode.It inheritscodecs.IncrementalDecoder.
This section discusses the performance of the provided concrete I/Oimplementations.
By reading and writing only large chunks of data even when the user asks for asingle byte, buffered I/O hides any inefficiency in calling and executing theoperating system’s unbuffered I/O routines. The gain depends on the OS and thekind of I/O which is performed. For example, on some modern OSes such as Linux,unbuffered disk I/O can be as fast as buffered I/O. The bottom line, however,is that buffered I/O offers predictable performance regardless of the platformand the backing device. Therefore, it is almost always preferable to usebuffered I/O rather than unbuffered I/O for binary data.
Text I/O over a binary storage (such as a file) is significantly slower thanbinary I/O over the same storage, because it requires conversions betweenunicode and binary data using a character codec. This can become noticeablehandling huge amounts of text data like large log files. Also,TextIOWrapper.tell() andTextIOWrapper.seek() are both quite slowdue to the reconstruction algorithm used.
StringIO, however, is a native in-memory unicode container and willexhibit similar speed toBytesIO.
FileIO objects are thread-safe to the extent that the operating systemcalls (such asread(2) under Unix) they wrap are thread-safe too.
Binary buffered objects (instances ofBufferedReader,BufferedWriter,BufferedRandom andBufferedRWPair)protect their internal structures using a lock; it is therefore safe to callthem from multiple threads at once.
TextIOWrapper objects are not thread-safe.
Binary buffered objects (instances ofBufferedReader,BufferedWriter,BufferedRandom andBufferedRWPair)are not reentrant. While reentrant calls will not happen in normal situations,they can arise from doing I/O in asignal handler. If a thread tries tore-enter a buffered object which it is already accessing, aRuntimeErroris raised. Note this doesn’t prohibit a different thread from entering thebuffered object.
The above implicitly extends to text files, since theopen() functionwill wrap a buffered object inside aTextIOWrapper. This includesstandard streams and therefore affects the built-in functionprint() aswell.
16.1.os — Miscellaneous operating system interfaces
16.3.time — Time access and conversions
Enter search terms or a module, class or function name.
