A May 2017 Economist cover declared data to be the world’s most valuable resource. It’s none truer than in algorithmic trading. As algorithmic traders, it’s our job to acquire and make sense of billions of rows of market data for use in trading algorithms. In this context, it’s crucial to gather high-quality, reliable data that can adequately support trading algorithms and market research. Luckily for us, it’s possible to acquire high-quality data for free (ornearly free).

This chapter offers recipes for a series of different Python libraries—including the cutting-edge OpenBB Platform—to acquire free financial market data using Python. One of the primary challenges most non-professional traders face is getting all the data required for analysis together in one place. The OpenBB Platform addresses this issue. We’ll dive into acquiring data for a variety of assets, including stocks, options, futures (both continuous and individual contracts), andFama-French factors.

One crucial point to remember is that data can vary across different sources. For instance, prices from two sources might differ due to distinct data sourcing methods or different adjustment methods for corporate actions. Some of the libraries we’ll cover might download data for the same asset from the same source. However, libraries vary in how they return that data based on options that help you preprocess the data in preparationfor research.

Lastly, while we’ll focus heavily on mainstream financial data in this chapter, financial data is not limited to prices. The concept of “alternative data,” which includes non-traditional data sources such as satellite images, web traffic data, or customer reviews, can be an important source of information for developing trading strategies. The Python tools to acquire and process this type of data are outside the scope of this book. We’ve intentionally left out the methods of acquiring and processing this type of data since it’s covered in other resources dedicated tothe topic.

In this chapter, we’ll cover thefollowing recipes:

• Working with stock market data with theOpenBB Platform
• Fetching historic futures data with theOpenBB Platform
• Navigating options market data with theOpenBB Platform
• Harnessing factor datausingpandas_datareader

This book relies on the Anaconda distribution of Python. We’ll use Jupyter Notebook and Python script files to write the code. Unless specified otherwise, all the code can be written inJupyter Notebooks.

Download and install the Anaconda distribution of Python. You can do this by going tohttps://www.anaconda.com/download. Depending on your operating system, the instructions for downloading and installing will vary. Please refer to the Anaconda documentation fordetailed instructions.

Anaconda ships with a package manager calledconda. Package managers make it easy to install, remove, and update Python packages. There’s a great cheat sheet for theconda package manager that you can downloadfromhttps://docs.conda.io/projects/conda/en/4.6.0/_downloads/52a95608c49671267e40c689e0bc00ca/conda-cheatsheet.pdf.

Once you’ve installed the Anaconda distribution, open your Terminal on Mac or Linux or the Anaconda Prompt on Windows. If you’re a Windows user, make sure to use the Command Prompt instead of the Powershell prompt. Then followthese steps:

1. Update thecondapackage manager:

   conda update -n base conda -y

2. Create avirtual environment:

   conda create -n my-quant-stack python=3.10 -y

3. After the installation process is complete, activatethe environment:

   conda activate my-quant-stack

4. Install Jupyter Notebook using the package manager that ships withPython,pip:

   pip install notebook matplotlib

This will set up a virtual environment using Python 3.10 and installJupyter Notebook.

This chapter will use Two Python libraries to acquire financial market data: the OpenBB Platform andpandas_datareader. The good news is that installing the OpenBB Platform installs many of the libraries you will need to acquire financial market data, includingpandas_datareader. As such, there is no need to install thelibraries separately.

Install the OpenBB Platform with all extensions and providers (both officially supported and community-maintained ones)usingpip:

pip install openbb[all]

This is the easiest way to set up the OpenBB Platform forthis book.

Important note

In a macOS zsh Terminal shell, add quotation marks around the libraryname:"openbb[all]"

To install asingle extension:

pip install openbb[charting]pip install openbb[ta]

Or install asingle provider:

pip install openbb[yfinance]

To install the Nightly distribution (this installs all extrasby default):

pip install openbb-nightly

Important note

At the time of writing, installing the OpenBB Platform usingpip isn’t compatible with environments such as Google Colab and Kaggle since they come with preinstalled packages that can conflict with the ones used with the OpenBB Platform. If you run into trouble installing the OpenBB Platform, please check the online documentation for the mostup-to-date instructions.

You may remember thememe stock hysteria that sent GameStop’s stock up 1,744% in January 2021. One of the good things that came from that episode was the GameStonk terminal, now rebranded as OpenBB. OpenBB is the most popular open-source finance projects on GitHub for good reason: it provides a single interface to access hundreds of data feeds from one place in a standard way. OpenBB has a command-line interface that is great for manual investment research. But when it’s time to get data into Python, you want the OpenBB Platform. This recipe will guide you through the process of using the OpenBB Platform to fetch stockmarket data.

Getting ready…

By now, you should have the OpenBB Platform installed in your virtual environment. If not, go back to the beginning of this chapter and get it set up. The OpenBB Platform is free to use and offers a web-based UI to manage your configuration files, store API keys, and get code walkthroughs and examples. Sign up for a free Hub account at https://my.openbb.co/login. The popular course,Getting Started with Python for Quant Finance, uses OpenBB exclusively for all the code. Check outhttps://www.pyquantnews.com/getting-started-with-python-for-quant-finance for information on howto join.

How to do it…

Using the OpenBB Platform involvesone import:

1. Import theOpenBB Platform:

   from openbb import obbobb.user.preferences.output_type = "dataframe"

2. Use thehistorical method todownload price data for theSPY ETF:

   data = obb.equity.price.historical("SPY", provider="yfinance")

3. Inspect theresulting DataFrame:

   print(data)

   Running the preceding code generates a pandas DataFrame and prints the data tothe screen:

Figure 1.1: Historic price data for SPY

How it works…

The OpenBB Platform follows an easy-to-understand namespace convention. All the methods for acquiring stock price data aremethodsonopenbb.equity.

Thehistorical method accepts a ticker symbol and returns the open, high, low, close, adjusted close, volume, dividend, and split adjustments in a pandas DataFrame. The additional parameters you can specify areas follows:

• start_date: Start date to get datafrom with
• interval: Interval (in minutes) to get data—that is, 1, 5, 15, 30, 60,or 1,440
• end_date: End date to get datafrom with
• provider: Source ofdata extracted

There’s more…

An important benefit of using the OpenBB Platform is choosing your data source. By default, the OpenBB Platform will attempt to download data from free sources such as Yahoo! Finance. In most OpenBB Platform calls, you can indicate a different source. To use a source that requires an API key (either free or paid), you can configure it in theOpenBB Hub.

Tip

Check out the OpenBB Platform documentation forthe latestfunctionality:https://docs.openbb.co.

Let’s look at some more of the functions of theOpenBB Platform.

Comparison of fundamental data

Not only can the OpenBB Platform download fundamental data in an organized and usable way, but it can also concatenate it in a single Pandas DataFrame forfurther analysis.

We can use the following code to see the balance sheet metrics fromAAPLandMSFT:

obb.equity.fundamental.metrics(    "AAPL,MSFT",    provider="yfinance")

The output of the preceding snippet is a pandas DataFrame with fundamental data for each ticker thatwas passed:

Figure 1.2: Balance sheet data for MSFT and AAPL

Building stock screeners

One of the most powerful features ofthe OpenBB Platform is the custom stock screener. It uses theFinviz stock screener under the hood and surfaces metrics across a range of stocks based on either pre-built or custom criteria. See the documentation for more on how to use the OpenBB screenerfunctions (https://docs.openbb.co/platform/reference/equity/screener):

1. Create an overview screener based on a list of stocks using thedefault view:

   obb.equity.compare.groups(    group="industry",    metric="valuation",    provider="finviz")

   The output of the preceding snippet is the followingpandas DataFrame:

Figure 1.3: Results of a comparison screener between F, GE, and TSLA

2. Create a screener that returns the top gainers from the technology sector based ona preset:

   obb.equity.compare.groups(    group="technology",    metric="performance",    provider="finviz")

   The output of the preceding snippet is the followingpandas DataFrame:

Figure 1.4: Results of a screener showing the day’s top-gaining stocks

3. Create a screener that presents an overview groupedby sector:

   obb.equity.compare.groups(    group="sector",    metric="overview",    provider="finviz")

   The output of the preceding snippet is the followingpandas DataFrame:

Figure 1.5: Results of a screener grouped by sector

See also

For more on OpenBB and theFinviz stock screener, check out thefollowing resources:

• The OpenBB Platform documentation, which contains details on dozens of functions to acquire stock market data forfree:https://docs.openbb.co/platform/reference/equity
• TheFinviz home page, which includes free access to the web-basedscreenerhttps://finviz.com/?a=2548677
• The top-selling cohort-based course to help beginners get up and running with Python for quant finance, algorithmic trading, and dataanalysis:https://www.pyquantnews.com/getting-started-with-python-for-quant-finance

Traders use continuous futures data for backtesting trading strategies. Futures traders use the roll from one contract to another as a potential opportunity for profit. Some traders simply pick a date beforeexpiration to roll to the next contract, while others use sophisticated techniques involving open interest. Thisbasis trade is persistently one of the most popular trading strategies for futures traders. These traders wantcontrol over the data that’s used to compute the basis trade, so acquiring individual contract data is important. This recipe will guide you through the process of using the OpenBB Platform to fetch individual futurescontract data.

Getting ready…

By now, you should have the OpenBB Platform installed in your virtual environment. If not, go back to the beginning of this chapter and get itset up.

How to do it…

We’ll use the futures functionality in the OpenBB Platform to download individual futures datafor free:

1. Import pandas and theOpenBB Platform:

   import pandas as pdfrom openbb import obbobb.user.preferences.output_type = "dataframe"

2. Download thecurrent futures curve for the VIX futures contract from theChicago Board OptionsExchange (CBOE):

   data = obb.derivatives.futures.curve(symbol="VX")

3. Inspect theresulting DataFrame:

   print(data)

   Running the preceding code generates the futures curve for the VIXfutures contract:

Figure 1.6: Settlement prices for the forward Eurodollar futures contracts

4. Update the DataFrame index to the expiration dates and plot thesettlement prices:

   data.index = pd.to_datetime(data.expiration)data.plot()

   By running theproceeding code, we plot the VIXfutures curve:

Figure 1.7: VIX futures curve

There’s more…

You can use theobb.derivatives.futures.historical method to get historical data for an individual expiration. Stitching together data across a range of years can provide insight into the market’s expectation of supply anddemand of theunderlying commodity:

1. First, create a list containing the year and month expirations you’reinterested in:

   expirations = [    "2024-12",    "2025-12",    "2026-12",    "2027-12",    "2028-12",    "2029-12",    "2030-12",]

2. The preceding code createsa Python list of expiration years and dates in string format. Now, loop through each of theexpirations to downloadthe data:

   contracts = []for expiration in expirations:    df = (        obb        .derivatives        .futures        .historical(            symbol="CL",            expiration=expiration,            start_date="2020-01-01",            end_date="2022-12-31"        )    ).rename(columns={        "close": expiration    })    contracts.append(df[expiration])

3. For each of the contracts, use the OpenBB Platform to download historical futures data for the CL contract between January 1, 2020, and 31 December 31, 2022. Using the pandasrename method, change the column name from"close" to the expiration date. Finally, append the newly created pandas DataFrame to a listof DataFrames:

   historical = (    pd    .DataFrame(contracts)    .transpose()    .dropna())

4. Concatenate the DataFrames together, swap the columns and rows using thetranspose method, and drop any recordswith no data using thedropna method. Inspect theresulting DataFrame:

   print(historical)

   By printing the DataFrame, we will see the historicalsettlement prices:

Figure 1.8: Historic settlement prices for the December CL futures contract

The result is the historicaldata between January 2020 and December 2022 for each of the December expirationsbetween 2023and 2030:

5. To visualize the market’s expectation of the future supply and demand of the December contract, you can plot thelast price:

   historical.iloc[-1].plot()

   Here'sthe output:

Figure 1.9: Futures curve for the December CL contract

See also

For more on the OpenBBPlatform futures functionality, youcan browse thefollowing documentation:

• Documentation on the OpenBB Platform’s futurescurvemethod:https://docs.openbb.co/platform/reference/derivatives/futures/curve
• Documentation on the OpenBB Platform’s historicalcurvemethod:https://docs.openbb.co/platform/reference/derivatives/futures/historical

Options are exchange-listed derivative contracts that convey the right (but not the obligation) to buy or sell the underlying stock at a certain price on or before a certain expiration date. Options are among the mostversatile financial instruments in the market. They allow traders to define their risk profiles before entering trades andexpress market views not only on the direction of the underlying but the volatility. While options offer a high degree of flexibility for trading, this feature complicates data collection for researchand backtesting.

A single underlying stock can have an array of options contracts with different combinations of strike prices and expiration dates. Collecting and manipulating this data is a challenge. The combination of options contracts for all strikes and expiration dates is commonly referred to as an options chain. There can be thousands of individual options contracts at a given time for a single underlying stock. Not only does the number of individual contracts pose a challenge, but getting price data has historically been expensive. With the introduction of the OpenBB Platform, it is now only a few lines of Python code to download options chains into a pandas DataFrame. This recipe will walk you through acquiring options data using theOpenBB Platform.

Getting ready…

By now, you should have the OpenBB Platform installed in your virtual environment. If not, go back to the beginning of this chapter and get itset up.

How to do it…

Similar to how we used the OpenBB Platform for futures data, we can use it for optionsdata too:

1. Import the OpenBB Platform and Matplotlibfor visualization:

   from openbb import obbobb.user.preferences.output_type = "dataframe"

2. Use thechains method to download the entireoptions chain:

   chains = obb.derivatives.options.chains(symbol="SPY")

3. Inspect theresulting DataFrame:

   chains.info()

   By running the precedingcode, we’ll see the details of the optionschains data:

Figure 1.10: Preview of the data downloaded for the SPY options chains

Note that there are 8,518 options contracts for theSPY Exchange Traded Fund (ETF) that can be downloaded from CBOE (for free).

How it works…

Theobb.derivatives.options.chains method downloads the entire options chain and stores it in a pandas DataFrame. Theobb.derivatives.options.chains has an additionaloptional parameter:

• provider: The source from which the data should be downloaded. The default is CBOE. You can also select Tradier, Intrinio, or TMX. Note that for Tradier, Intrinio, and TMX, you need to provide your API key, which can be configured in theOpenBB Hub.

There’s more…

You can use the OpenBB Platform to download historical options data for a single contract. To do this, you need theoption symbol.

We’ll use theobb.equity.price.historical method to get the historical options data for an SPY call option with a strike price of $550 expiring on December20, 2024:

data = obb.equity.price.historical(    symbol="SPY241220C00550000",    provider="yfinance")[["close", "volume"]]

The result is a pandas DataFrame with the closing price and volume of theoptions contract.

Figure 1.11: Closing prices and volume of the SPY options contract

Options Greeks

Options Greeks measure how options prices change given a change in one of the inputs to an options pricing model. For example,delta measures how an options pricechanges given a change in the underlyingstock price.

Usingobb.derivatives.options.chains, the OpenBB Platform returns the most used Greeks including Delta, Gamma, Theta, Vega,and Rho.

See also

Options are a fascinating and deep topic that is rich with opportunities for trading. You can learn more about options, volatility, and how to analyze both via theOpenBB Platform:

• Free articles, code, and other resources for optionstrading:https://pyquantnews.com/free-python-resources/options-trading-with-python/
• Learn more about financial options if you’re unfamiliar withthem:https://en.wikipedia.org/wiki/Option_(finance)
• Documentation on the OpenBB Platform’s optionsmethods:https://docs.openbb.co/platform/reference/derivatives/options

Diversification is great until the entire market declines in value. That’s because the overall market influences all assets. Factors can offset some of these risks by targeting drivers of return not influenced by the market. Common factors are size (large-cap versus small-cap) and style (value versus growth). If you think small-cap stocks will outperform large-cap stocks, then you might want exposure to small-cap stocks. If you think value stocks will outperform growth stocks, then you might want exposure to value stocks. In either case, you want to measure the risk contribution of the factor. Eugene Fama and Kenneth French built the Fama-French three-factor model in 1992. The three Fama-French factors are constructed using six value-weight portfolios formed on capitalizationand book-to-market.

The three factors areas follows:

• Small Minus Big, which represents the differential between the average returns of three small-cap portfolios and threelarge-cap portfolios.
• High Minus Low, which quantifies the difference in average returns between two value-oriented portfolios and twogrowth-oriented portfolios.
• Rm-Rf, which denotes themarket’s excess return over therisk-free rate.

We’ll explore how to measure and isolate alpha inChapter 5,Build Alpha Factors for Stock Portfolios. This recipe will guide you through the process of usingpandas_datareader to fetch historic factor data for use inyour analysis.

Getting ready…

By now, you should have theOpenBB Platform installed in your virtual environment. If not, go back to the beginning of this chapter and get it set up. By installing the OpenBB Platform,pandas_datareader will be installed and readyto use.

How to do it…

Using thepandas_datareader library, we have access to dozens of investmentresearch factors:

1. Importpandas_datareader:

   import pandas_datareader as pdr

2. Download the monthly factor data starting inJanuary 2000:

   factors = pdr.get_data_famafrench("F-F_Research_Data_Factors")

3. Get a description of the researchdata factors:

   print(factors[“DESCR”])

   The result is an explanation of the data included inthe DataFrame:

Figure 1.12: Preview of the description that is downloaded with factor data

4. Inspect the monthlyfactor data:

   print(factors[0].head())

   By running the preceding code, we get a DataFrame containing monthlyfactor data:

Figure 1.13: Preview of the monthly data downloaded from the Fama-French Data Library

5. Inspect the annualfactor data:

   print(factors[1].head())

   By running the preceding code, we get a DataFrame containing annualfactor data:

Figure 1.14: Preview of the annual data downloaded from the Fama-French Data Library

How it works…

Under the hood,pandas_datareader fetches data from the Fama-French Data Library by downloading a compressed CSV file, uncompressing it, and creating apandas DataFrame.

There are 297 different datasets with different factor data available from the Fama-French Data Library. Here are some popular versions of the Fama-French 3-factor model fordifferent regions:

• Developed_3_Factors
• Developed_ex_US_3_Factors
• Europe_3_Factors
• Japan_3_Factors
• Asia_Pacific_ex_Japan_3_Factors

You can use these in theget_data_famafrench method, justlikeF-F_Research_Data_Factors.

Some datasets return a dictionary with more than one DataFrame representing data for different time frames, portfolio weighting methodologies, and aggregate statistics. Data for these portfolios can be accessed using numerical keys. For example, the5_Industry_Portfolios dataset returns eight DataFrames in the dictionary. The first can be accessed using the0 key, the second using the1 key, and so on. Each dictionary includes a descriptionof the dataset, which can be accessed using theDESCR key.

There’s more…

pandas_datareader can be used to access data from many remote online sources. These include Tiingo, IEX, Alpha Vantage, FRED, Eurostat, and many more. Review the full list of data sources on the documentationpage:https://pandas-datareader.readthedocs.io/en/latest/remote_data.html.

See also

For more details on the factors available in the investment factor research library, take a look at the following resources. For another example of using the Fama-French 3-factor model, see the resources on the PyQuantNews website:

• Documentation for all the Fama-French factordata:https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/data_library.html
• Details on the Fama-French 3-factormodel:https://mba.tuck.dartmouth.edu/pages/faculty/ken.french/Data_Library/f-f_factors.html
• Code walkthrough for using the Fama-French 3-factormodel:https://www.pyquantnews.com/past-pyquant-newsletter-issues