pandas: powerful Python data analysis toolkit¶

PDF Version

Zipped HTML

Date: Nov 03, 2016Version: 0.19.1

Binary Installers:http://pypi.python.org/pypi/pandas

Source Repository:http://github.com/pydata/pandas

Issues & Ideas:https://github.com/pydata/pandas/issues

Q&A Support:http://stackoverflow.com/questions/tagged/pandas

Developer Mailing List:http://groups.google.com/group/pydata

pandas is aPython package providing fast,flexible, and expressive data structures designed to make working with“relational” or “labeled” data both easy and intuitive. It aims to be thefundamental high-level building block for doing practical,real world dataanalysis in Python. Additionally, it has the broader goal of becomingthemost powerful and flexible open source data analysis / manipulation toolavailable in any language. It is already well on its way toward this goal.

pandas is well suited for many different kinds of data:

• Tabular data with heterogeneously-typed columns, as in an SQL table orExcel spreadsheet
• Ordered and unordered (not necessarily fixed-frequency) time series data.
• Arbitrary matrix data (homogeneously typed or heterogeneous) with row andcolumn labels
• Any other form of observational / statistical data sets. The data actuallyneed not be labeled at all to be placed into a pandas data structure

The two primary data structures of pandas,Series (1-dimensional)andDataFrame (2-dimensional), handle the vast majority of typical usecases in finance, statistics, social science, and many areas ofengineering. For R users,DataFrame provides everything that R’sdata.frame provides and much more. pandas is built on top ofNumPy and is intended to integrate well within a scientificcomputing environment with many other 3rd party libraries.

Here are just a few of the things that pandas does well:

• Easy handling ofmissing data (represented as NaN) in floating point aswell as non-floating point data
• Size mutability: columns can beinserted and deleted from DataFrame andhigher dimensional objects
• Automatic and explicitdata alignment: objects can be explicitlyaligned to a set of labels, or the user can simply ignore the labels andletSeries,DataFrame, etc. automatically align the data for you incomputations
• Powerful, flexiblegroup by functionality to performsplit-apply-combine operations on data sets, for both aggregating andtransforming data
• Make iteasy to convert ragged, differently-indexed data in otherPython and NumPy data structures into DataFrame objects
• Intelligent label-basedslicing,fancy indexing, andsubsettingof large data sets
• Intuitivemerging andjoining data sets
• Flexiblereshaping and pivoting of data sets
• Hierarchical labeling of axes (possible to have multiple labels pertick)
• Robust IO tools for loading data fromflat files (CSV and delimited),Excel files, databases, and saving / loading data from the ultrafastHDF5format
• Time series-specific functionality: date range generation and frequencyconversion, moving window statistics, moving window linear regressions,date shifting and lagging, etc.

Many of these principles are here to address the shortcomings frequentlyexperienced using other languages / scientific research environments. For datascientists, working with data is typically divided into multiple stages:munging and cleaning data, analyzing / modeling it, then organizing the resultsof the analysis into a form suitable for plotting or tabular display. pandasis the ideal tool for all of these tasks.

Some other notes

• pandas isfast. Many of the low-level algorithmic bits have beenextensively tweaked inCython code. However, as withanything else generalization usually sacrifices performance. So if you focuson one feature for your application you may be able to create a fasterspecialized tool.
• pandas is a dependency ofstatsmodels, making it an important part of thestatistical computing ecosystem in Python.
• pandas has been used extensively in production in financial applications.

See the package overview for more detail about what’s in the library.