Structuring a Python Application (Overview)
Python, though opinionated on syntax and style, is surprisingly flexible when it comes tostructuring your applications. On the one hand, this flexibility is great: it allows different use cases to use structures that are necessary for those use cases. On the other hand, though, it can be very confusing to the new developer. The Internet isn’t a lot of help either. There are as many opinions as there are Python blogs!
In this course, you’ll:
- Walk through a dependablePython application layout reference guide that you can refer to for most use cases
- See examples of common Python application structures, includingcommand-line applications (CLI apps), one-offscripts, installablepackages, and web application layouts withpopular frameworks likeFlask andDjango
00:00Welcome to Python Application Layouts.My name is Chris and I will be your guide.This series of videos will show you how to organize your Python code and thefiles associated therein. Each lesson in the videowill cover different kinds of configurations: simple one-off scripts, installablesingle packages, larger applications, a Django web application,and finally, a Flask application. First off,I’m going to assume that you have some familiarity with Python modules andpackages. If you need a refresher,there’s a course on Real Python that can help you with the details.
00:32So, let’s get started.Python is very flexible when it comes to how you structure your applications.You can more or less get away with almost anything you want.
00:39That’s very powerful,but it can also be very daunting if you’re just getting going. Unfortunately,the internet is no help. If you start googling for how to do this,there’s lots and lots of opinions on how it should be done. That being said,the content in this video is based on a Real Python article from Kyle.
00:56You can see the original article here.This means that it in itself is also opinionated.Now, I happen to agree with Kyle’s opinions and he’s outlining things that arebest practices,but you will be able to find contradictory information out there.
01:10The intent of this course is to give you good places to start.Once you’re comfortable, you can adjust things to your own style.Accompanying this video is a selection of sample code.
01:19It’s actually executable. It doesn’t just show the file layout;it actually works. This means you can package it,run the tests, and see how things change from a small application to a largeapplication. Hopefully, this is helpful as you’re following along.
01:35I’ll start with the simplest situation:a program consisting of a single Python file. This kind of situation worksif you don’t have any dependencies and everything’s just pure Python or ifyou’re using a virtual env with tools likepip or Pipenv to manage externaldependencies. Of course, single-file programsoften aren’t just single files. You’re writing automated tests, right?
01:57You should be. Even if you’re not, there’s also things that go along with it—for example, configuration for yourgit setup,setup.py for organizing how your system is packagedif you are trying to do packaging,requirements.txt that specifies yourdependencies, and licensing andREADME information is always usefulif you’re trying to hand this script off to somebody else.
02:19This course comes with sample code that actually executes.I’m going to give you a quick tour of what it does.It’s a little Hello World so that you can see as I build on it in the differentlessonshow the pieces fit together, and what changes as things become more complicated.
02:33So, here it is:helloworld.py.Normally, a Hello World example is nothing butprint("Hello, World!").In order to show you how to interact with external libraries and how thesepieces fit together, my Hello World is a little bit more complicated,but not by much.
02:48All it’s doing in line 12 is using the very popularrequests library.If you haven’t seen it before,requests wraps HTTP code.
02:57It’s the easiest way of grabbing the contents ofa web page. Line 8 shows the URL that I’m grabbing. The URL I’m grabbing is aWikipedia article on Hello World programs—how very meta. And then on line 13, I’m printing it—finding the<title> tag inside of it and printing out just the result of that.
03:17So, as I said, pretty simple—essentially, it just prints"Hello, World!"but uses therequests library to do it.
03:25The other thing to notice here is I’ve embedded a version number.
03:29There are multiple ways of specifying version numbers;this is my preferred mechanism because you can actually get at itprogrammatically. Later on, when this goes from a single file to multiple files,I’ll move this out of the script and into the__init__.
03:43And here it is in action.
03:46That’s the<title> tag from the Wikipedia page on HelloWorld programs, as expected.
03:54So, this is some test code to go with my Hello World program.Interestingly enough,there’s actually more code in the tests than there is in the program itself.
04:01That’s because I’m avoiding the call to the internet inside ofrequests.The meat of the test is lines 15 through 19. Line 18 callsthe actual code.requests is mocked out so that the internet call isn’t actuallymade. I generally try to make a difference between my unit tests,which are self-contained and don’t require the dependencies, versus myintegration tests, which would include going off to the world.
04:28This allows me to write much faster unit tests, and I’m not dependent on theenvironment I’m running them in.If you haven’t seen unit tests or mocks before,there’s plenty of good content on Real Python to help you out with that. For now,just understand that this is the code that tests it, and I’ll show you quicklyhow that works.
04:46The Pythonunittest module is executable. On the command line,I can specify-m to executeunittest, passing in my tests file. Python’sunittest library will then execute the tests in mytests.py file.
05:01Here it goes. You see the results of the program:"Hello, World!" program - Wikipedia, becausedo_hello() prints something to the screen,and then you see the output from theunittest module specifying that it ran onetest successfully and everything’s okay.
05:17So, you’ve seen the code and the tests that go with it. Now,I want to show you the accompanying filesthat would probably also be in the directory,particularly if you’re going to package this up for other people. For starters,.gitignore. This is a configuration fileforgit to tell it to ignore certain file types.
05:32It stops it from trying to check in things like backup files and swap files thatyou’re not interested in. Here’s a really, really simple example.If you’re writing code on a Mac, the.DS_Store directory shows up everywhere.
05:44It’s where meta information is stored. You probably don’t want that checked in.If you’re writing Python 3,__pycache__/will have the contents of the compiled Python files. Any Pythonc, Pythono, orPythond files—so that’s.pyc,.pyo,or.pyd—are all object files and don’t need to be in your repository.
06:05These are examples of things you probably don’t want to store inside ofgit. Acomplete version of this file is much longer than this. You can see sample.gitignore files off at GitHub itself.
06:15It has good.gitignore files for almost every programming language you can thinkof. My Hello World program uses therequests library.
06:23That means I need to be able topip install it to a virtual env.Therequirements.txt file shows this dependency as well as anything elsethat I might use when I’m developing. A sample file includesrequests 2.22,which is the library I was using, as well ascoverage andpudb—coveragebeing a program to help me see what my tests are covering and aren’t covering,andpudb is my favorite command-linedebugger.requirements.txt generally storeseverything you need to run your tests and develop.
06:57This is often more than what you would include in a shipped application.So,setup.py won’t have thecoverage andpudb items in it—it’ll only haverequests. Just the strict dependencies. Speaking ofsetup.py—what is it? It’s a way of specifying what is packaged in your application.
07:16Unfortunately, Pythonisn’t really strong when it comes to packaging. Or an alternative way of lookingat that is there’s lots and lots of choice;it isn’t really baked into the language and there’s tons of third-party ways ofdoing it. The Python community is trying to tackle this,but it’s changed a lot over the yearsand so you’re going to bump into all sorts of ways of doing things.
07:37setup.py isn’t the only way, but it is one of the more commonpopular ways, and a lot of tools interact with it. For example,if you’ve heard oftox,it’s a test harness for testing your program under multiple versions of Python.
07:49twine is a tool for uploading a package to PyPIso you can share it with other people.The packages thattwine uploads are the result ofsetup.pybeing executed and creating bundles. For more information onsetup.py and howto use it with Pipenv, go to the Real Python article on this topic.
08:07Now, I’ll show you an example. This is the bottom chunk of asetup.py file.It’s kind of funny; not only were the tests twice as long as the code, nowthe configuration file is four times as long.
08:20The heart ofsetup.py is down at the bottom here in these final four lines—calling thesetup() library and passing in a dictionary that describes yourpackage. I’ll move up a little bitnow. Lines 16 through 38—this is the dictionary specifying information. Really,this is just attributes about the programhelloworld.
08:40It includes things likename, theversion,description, a longer description,where to find it, who the author is, the license you’ve picked,what kind of program it is, what modules it uses, and what it requires.
08:5435 through 37 is important—this is what makes sure dependencies get installed when your package isinstalled by someone else.Remember how this is different fromrequirements.txt:requirements.txtincludes everything that you would put in your development virtual environment—and that includes things like your debugging tools—whereasinstall_requires is only what someone using your program would need.
09:18So.install_requires—therequestslibrary;requirements.txt only hasrequests as well as things likecoverageandpudb inside of it. I’m going to scroll up to the top now.
09:30Lines 1 through 13 aren’t required forsetup.py to work;as long as you fill in your dictionary args, you’re good to go.These 13 lines are things that I often do inside of mysetup.pyso that I’m reusing information from the outside world. For example,I have a regular expression looking for"__version__" inside of theprogram file. It reads it and puts it inside of the dictionary.
09:53This means I don’t have to remember to update the version number in two places:the code and thesetup.py file. Similarly,because I have aREADME file,I’m reading that in and putting it inside of thelong_description. You canof coursejust set these things up inside of the dictionary—or in some cases, not eveninclude them—but I wanted to show you what I often include inside of my own applicationconfiguration. Now, you’ve seenrequirements.txt andsetup.py.
10:18Just a couple of housekeeping files left. First off, theLICENSE.There’s a lot of misinformation out there about copyrights,particularly when it comes to software.
10:28The thing you have to know is if you do not include a license file,the code is fully copyrighted by default, in most jurisdictions.What that means is no one has any right to use it or copy it in any fashion.
10:42The open-source world would not exist if everyone did this. Licenses are whatgrant other people the ability to use your software,and you’re probably using an awful lot of opensource products right now,so in my view, it’s only fair to share some of what you’re using yourself.
10:59There’s plenty of variations on licenses. You can pick which one you like.If you’re using GitHub, when you create a new repositoryit will even give you a dropdown to choose which license to include in yourrepository. That massive text there is the MIT License.
11:14It’s one of the simpler licenses and it’s very, very popular in the Python world.This is the one I tend to go to myself unless I’m doing something that’sinteracting with other projects that don’t allow it.
11:26There’s lots of good information off at ChooseALicense.comif you want to see what the differences are and help you pick a license.Just remember: if you don’t include a licenseyou’re not actually sharing your code, regardless of whatever your intent was.
11:40Finally, aREADME file is also important. If you’re sharing your code,you need to be able to tell people what it is.This should be a short description of your project.
11:48Two very common formats for this are Markdown and reStructuredText.If you’re using GitHub,README.md orREADME.rst will automatically show as the project’s homepage.
12:00Writing these things sometimes can be a bit of a challenge. Us programmers—not so goodly in English.If you’re having a little trouble and you want to figure out how to do it,Dan’s written a great article on how to describe your project and how to tellothers about it. So, that’s a simple single-file program.
12:16Next up,I’ll make things a little more complicated and show you how to use a singlepackage which has multiple files in it.
drewmullen onApril 29, 2020
What do you gain from including the version inside your helloworld.py file?ive only been maintaining in my setup.py and it has seemed to work OK for me.
Denis Roy onApril 30, 2020
Here are clickable links of the references made in the video:
Modules & packages refresher:
realpython.com/python-modules-packages
Article from Kyle Stratis:
realpython.com/python-application-layouts
Lots of .gitignore files:
RealPython article on Pipenv:
Choose a license:
Hints on writing a good README:
dbader.org/blog/write-a-great-readme-for-your-github-project
blackray onMay 2, 2020
hi drewmullen, i think Chris is trying to convey that if you have the code itself manage the version, and your setup.py file reads it from there, you don’t have to worry about aligning them. (09:55)
plus, he has the readme file pointing to the same variable. So when his version got updated, readme file get updated too.
Hope that helps.
andersgs onMay 3, 2020
hi drewmullen, in addition to blackray’s comment, I would add that it is considered good practice to keep your version string with the code and import it in to your setup.py because then you can access the version string programmatically (e.g., import pandas; print(pandas.version).
That said, I am not sure you need to use the regex expression. I would just have added an import line:
fromhelloworldimport__version__asversionprint(version)drewmullen onMay 4, 2020
great point @andersgs. now that you mention it, i have actually ran into problems with apps where the version isnt available. it can be helpful if you use the code downstream and want to say “if version < 0.x behave one way or another”
Become a Member to join the conversation.
Course Contents
