Reviewing MongoDB’s Features

So far, you’ve learned what MongoDB is and what its main goals are. In this section, you’ll learn about some of MongoDB’s more important features. As for the database management side, MongoDB offers the following features:

• Query support: You can use many standard query types, such as matching (==), comparison (<,>), andregular expressions.
• Data accommodation: You can store virtually any kind of data, be it structured, partially structured, or even polymorphic.
• Scalability: It handles more queries just by adding more machines to the server cluster.
• Flexibility and agility: You can develop applications with it quickly.
• Document orientation and schemalessness: You can store all the information regarding a data model in a single document.
• Adjustable schema: You can change the schema of the database on the fly, which reduces the time needed to provide new features or fix existing problems.
• Relational database functionalities: You can perform actions common to relational databases, like indexing.

As for the operations side, MongoDB provides a few tools and features that you won’t find in other database systems:

• Scalability: Whether you need a stand-alone server or complete clusters of independent servers, you can scale MongoDB to whatever size you need it to be.
• Load-balancing support: MongoDB will automatically move data across various shards.
• Automatic failover support: If your primary server goes down, then a new primary will be up and running automatically.
• Management tools: You can track your machines using the cloud-based MongoDB Management Service (MMS).
• Memory efficiency: Thanks to the memory-mapped files, MongoDB is often more efficient than relational databases.

All these features are quite useful. For example, if you take advantage of the indexing feature, then much of your data will be kept in memory for quick retrieval. Even without indexing specific document keys, MongoDB caches quite a bit of data using theleast recently used technique.

Installing and Running MongoDB

Now that you’re familiar with MongoDB, it’s time to get your hands dirty and start using it. But first, you need to install it on your machine. MongoDB’s official site provides two editions of the database server:

1. The community edition offers the flexible document model along with ad hoc queries, indexing, and real-time aggregation to provide powerful ways to access and analyze your data. This edition is freely available.
2. The enterprise edition offers the same features as the community edition, plus other advanced features related to security and monitoring. This is the commercial edition, but you can use it free of charge for an unlimited time for evaluation and development purposes.

If you’re on Windows, then you can read through theinstallation tutorial for complete instructions. In general, you can go to thedownload page, select the Windows platform in the Available Downloads box, choose the.msi installer that fits your current system, and clickDownload.

Run the installer and follow the on-screen instructions on the installation wizard. This page also provides information on how torun MongoDB as a Windows service.

If you’re on macOS, then you can useHomebrew to install MongoDB on your system. See theinstallation tutorial to get the complete guide. Also, make sure to follow the instruction torun MongoDB as a macOS service.

If you’re on Linux, then the installation process will depend on your specific distribution. For a detailed guide on how to install MongoDB on different Linux systems, go to theinstallation tutorial page and select the tutorial that matches your current operating system. Make sure you run the MongoDB daemon,mongod, at the end of the installation.

Finally, you can also install MongoDB usingDocker. This is handy if you don’t want to clutter your system with another installation. If you prefer this installation option, then you can read through theofficial tutorial and follow its directions. Note that previous knowledge ofhow to use Docker would be required in this case.

With the MongoDB database installed and running on your system, you can start working with real databases using themongo shell.

Running themongo Shell

Themongo shell is an interactiveJavaScript interface to MongoDB. You can use this tool to query and manipulate your data as well as to perform administrative operations. Since it’s a JavaScript interface, you won’t use the familiar SQL language to query the database. Instead, you’ll use JavaScript code.

To launch themongo shell, open yourterminal or command line and run the following command:

$mongo

This command takes you to themongo shell. At this point, you’ll probably see a bunch of messages with information on the shell’s version and on the sever address and port. Finally, you’ll be presented with the shell prompt (>) to enter queries and commands.

You can pass the database address as an argument to themongo command. You can also use several options, such as specifying the host and port to access a remote database, and so on. For more details on how to use themongo command, you can runmongo --help.

Establishing a Connection

When you run themongo command without arguments, it launches the shell and connects to the default local server provided by themongod process atmongod://127.0.0.1:27017. This means you’re connected to the local host through port27017.

By default, themongo shell starts the session by establishing a connection to thetest database. You can access the current database through thedb object:

>dbtest>

In this case,db holds a reference totest, which is the default database. To switch databases, issue the commanduse, providing a database name as an argument.

For example, say you want to create a website to publish Python content, and you’re planning to use MongoDB to store your tutorials and articles. In that case, you can switch to the site’s database with the following command:

>userptutorialsswitchedtodbrptutorials

This command switches your connection to therptutorials database. MongoDB doesn’t create the physical database file on the file system until you insert real data into the database. So in this case,rptutorials won’t show up in your current database list:

>showdbsadmin0.000GBconfig0.000GBlocal0.000GB>

Themongo shell provides a lot of features and options. It allows you to query and manipulate your data and also to manage the database server itself.

Instead of using a standardized query language such as SQL, themongo shell uses the JavaScript programming language and a user-friendlyAPI. This API allows you to play around with your data, which is the topic for the next section.

Creating Collections and Documents

A MongoDBdatabase is a physical container forcollections ofdocuments. Each database gets its own set of files on the file system. These files are managed by the MongoDB server, which can handle several databases.

In MongoDB, acollection is a group ofdocuments. Collections are somewhat analogous to tables in a traditional RDBMS, but without imposing a rigid schema. In theory, each document in a collection can have a completely different structure or set of fields.

In practice, documents in a collection commonly share a similar structure to allow uniform retrieval, insertion, and update processes. You can enforce a uniform document structure by usingdocument validation rules during updates and insertions.

Allowing different document structures is a key feature of MongoDB collections. This feature provides flexibility and allows adding new fields to documents without the need for modifying a formal table schema.

To create a collection using themongo shell, you need to pointdb to your target database and then create the collections using thedot notation:

>userptutorialsswitchedtodbrptutorials>dbrptutorials>db.tutorialrptutorials.tutorial

In this example, you use the dot notation to createtutorial as a collection inrptutorials, which is your current database. It’s important to note that MongoDB creates databases and collectionslazily. In other words, they’re physically created only after you insert the first document.

Once you have a database and a collection, you can start inserting documents. Documents are the unit of storage in MongoDB. In an RDBMS, this would be equivalent to a table row. However, MongoDB’s documents are way more versatile than rows because they can store complex information, such asarrays, embedded documents, and even arrays of documents.

MongoDB stores documents in a format calledBinary JSON (BSON), which is a binary representation of JSON. MongoDB’s documents are composed of field-and-value pairs and have the following structure:

{   field1 → value1,   field2 → value2,   field3 → value3,   ...   fieldN → valueN}

The value of a field can be any BSONdata type, including other documents, arrays, and arrays of documents. In practice, you’ll specify your documents using the JSON format.

When you’re building a MongoDB database application, probably your most important decision is about the structure of documents. In other words, you’ll have to decide which fields and values your documents will have.

In the case of the tutorials for your Python site, your documents might be structured like this:

{"title":"Reading and Writing CSV Files in Python","author":"Jon","contributors":["Aldren","Geir Arne","Joanna","Jason"],"url":"https://realpython.com/python-csv/"}

A document is essentially a set of property names and their values. The values can be simple data types, such as strings and numbers, but they can also be arrays such ascontributors in the above example.

MongoDB’s document-oriented data model naturally represents complex data as a single object. This allows you to work with data objects holistically, without the need for looking at several places or tables.

If you were using a traditional RDBMS to store your tutorials, then you would probably have a table to store your tutorials and another table to store your contributors. Then you’d have to set up a relationship between both tables so you could retrieve the data later on.

Working With Collections and Documents

So far, you know the basics of how to run and use themongo shell. You also know how to create your own documents using the JSON format. Now it’s time to learn how toinsert documents into your MongoDB database.

To insert a document into a database using themongo shell, you first need to choose a collection and then call.insertOne() on the collection with your document as an argument:

>userptutorialsswitchedtodbrptutorials>db.tutorial.insertOne({..."title":"Reading and Writing CSV Files in Python",..."author":"Jon",..."contributors":[..."Aldren",..."Geir Arne",..."Joanna",..."Jason"...],..."url":"https://realpython.com/python-csv/"...}){"acknowledged":true,"insertedId":ObjectId("600747355e6ea8d224f754ba")}

With the first command, you switch to the database you want to use. The second command is a JavaScript method call that inserts a simple document into the selected collection,tutorial. Once you hitEnter, you get a message on your screen that informs you about the newly inserted document and itsinsertedId.

Just like relational databases need a primary key to uniquely identify every row in a table, MongoDB documents need to have an_id field that uniquely identifies the document. MongoDB allows you to enter a custom_id as long as you guarantee its uniqueness. However, a widely accepted practice is to allow MongoDB to automatically insert an_id for you.

Similarly, you can add several documents in one go using.insertMany():

>tutorial1={..."title":"How to Iterate Through a Dictionary in Python",..."author":"Leodanis",..."contributors":[..."Aldren",..."Jim",..."Joanna"...],..."url":"https://realpython.com/iterate-through-dictionary-python/"...}>tutorial2={..."title":"Python 3's f-Strings: An Improved String Formatting Syntax",..."author":"Joanna",..."contributors":[..."Adriana",..."David",..."Dan",..."Jim",..."Pavel"...],..."url":"https://realpython.com/python-f-strings/"...}>db.tutorial.insertMany([tutorial1,tutorial2]){"acknowledged":true,"insertedIds":[ObjectId("60074ff05e6ea8d224f754bb"),ObjectId("60074ff05e6ea8d224f754bc")]}

Here, the call to.insertMany() takes a list of tutorials and inserts them into the database. Again, the shell output shows information about the newly inserted documents and their automatically added_id fields.

Themongo shell also provides methods to performread,update, anddelete operations on the database. For example, you can use.find() to retrieve the documents in a collection:

>db.tutorial.find(){"_id":ObjectId("600747355e6ea8d224f754ba"),"title":"Reading and Writing CSV Files in Python","author":"Jon","contributors":["Aldren","Geir Arne","Joanna","Jason"],"url":"https://realpython.com/python-csv/"}...>db.tutorial.find({author:"Joanna"}){"_id":ObjectId("60074ff05e6ea8d224f754bc"),"title":"Python 3's f-Strings: An Improved String Formatting Syntax (Guide)","author":"Joanna","contributors":["Adriana","David","Dan","Jim","Pavel"],"url":"https://realpython.com/python-f-strings/"}

The first call to.find() retrieves all the documents in thetutorial collection. On the other hand, the second call to.find() retrieves those tutorials that are authored byJoanna.

With this background knowledge on how to use MongoDB through itsmongo shell, you’re ready to start using MongoDB with Python. The next few sections will walk you through different options for using MongoDB databases in your Python applications.

Installing PyMongo

To start using PyMongo, you first need to install it in your Python environment. You can use avirtual environment, or you can use your system-wide Python installation, although the first option is preferred.PyMongo is available onPyPI, so the quickest way to install it is withpip. Fire up your terminal and run the following command:

$pipinstallpymongo==3.11.2

After a few downloads and other related steps, this command installs PyMongo on your Python environment. Note that if you don’t supply a specific version number, thenpip will install the latest available version.

Once you’re done with the installation, you can start aPython interactive session and run the followingimport:

>>>importpymongo

If this runs without raising an exception in the Python shell, then your installation works just fine. If not, then carefully perform the steps again.

Establishing a Connection

To establish a connection to a database, you need to create aMongoClient instance. This class provides a client for a MongoDB instance or server. Each client object has abuilt-in connection pool, which by default handles up to a hundred connections to the server.

Get back to your Python interactive session and importMongoClient frompymongo. Then create a client object to communicate with your currently running MongoDB instance:

>>>frompymongoimportMongoClient>>>client=MongoClient()>>>clientMongoClient(host=['localhost:27017'], ..., connect=True)

The code above establishes a connection to the default host (localhost) and port (27017).MongoClient takes a set of arguments that allows you to specify custom host, port, and other connection parameters. For example, to provide a custom host and port, you can use the following code:

>>>client=MongoClient(host="localhost",port=27017)

This is handy when you need to provide ahost andport that differ from MongoDB’s default setup. You can also use theMongoDB URI format:

>>>client=MongoClient("mongodb://localhost:27017")

All these instances ofMongoClient provide the same client setup to connect your current MongoDB instance. Which one you should use just depends on how explicit you want to be in your code.

Once you’ve instantiatedMongoClient, you can use its instance to refer to that specific database connection, just like you did with themongo shell’sdb object in the above section.

Working With Databases, Collections, and Documents

Once you have a connected instance ofMongoClient, you can access any database managed by the specified MongoDB server. To define which database you want to use, you can use the dot notation just like you did in themongo shell:

>>>db=client.rptutorials>>>dbDatabase(MongoClient(host=['localhost:27017'], ..., connect=True), 'rptutorials')

In this case,rptutorials is the name of the database you’ll be working with. If the database doesn’t exist, then MongoDB creates it for you, but only when you perform the first operation on the database.

You can also usedictionary-style access if the name of the database isn’t a valid Python identifier:

>>>db=client["rptutorials"]

This statement is handy when the name of your database isn’t avalid Python identifier. For example, if your database is calledrp-tutorials, then you need to use dictionary-style access.

Storing data in your database using PyMongo is similar to what you did with themongo shell in the above sections. But first, you need to create your documents. In Python, you usedictionaries to create documents:

>>>tutorial1={..."title":"Working With JSON Data in Python",..."author":"Lucas",..."contributors":[..."Aldren",..."Dan",..."Joanna"...],..."url":"https://realpython.com/python-json/"...}

Once you’ve created the document as a dictionary, you need to specify which collection you want to use. To do that, you can use the dot notation on the database object:

>>>tutorial=db.tutorial>>>tutorialCollection(Database(..., connect=True), 'rptutorials'), 'tutorial')

In this case,tutorial is an instance ofCollection and represents a physical collection of documents in your database. You can insert documents intotutorial by calling.insert_one() on it with a document as an argument:

>>>result=tutorial.insert_one(tutorial1)>>>result<pymongo.results.InsertOneResult object at 0x7fa854f506c0>>>>print(f"One tutorial:{result.inserted_id}")One tutorial: 60084b7d87eb0fbf73dbf71d

Here,.insert_one() takestutorial1, inserts it into thetutorial collection and returns anInsertOneResult object. This object provides feedback on the inserted document. Note that since MongoDB generates theObjectId dynamically, your output won’t match theObjectId shown above.

If you have many documents to add to the database, then you can use.insert_many() to insert them in one go:

>>>tutorial2={..."title":"Python's Requests Library (Guide)",..."author":"Alex",..."contributors":[..."Aldren",..."Brad",..."Joanna"...],..."url":"https://realpython.com/python-requests/"...}>>>tutorial3={..."title":"Object-Oriented Programming (OOP) in Python 3",..."author":"David",..."contributors":[..."Aldren",..."Joanna",..."Jacob"...],..."url":"https://realpython.com/python3-object-oriented-programming/"...}>>>new_result=tutorial.insert_many([tutorial2,tutorial3])>>>print(f"Multiple tutorials:{new_result.inserted_ids}")Multiple tutorials: [   ObjectId('6008511c87eb0fbf73dbf71e'),   ObjectId('6008511c87eb0fbf73dbf71f')]

This is faster and more straightforward than calling.insert_one() multiple times. The call to.insert_many() takes an iterable of documents and inserts them into thetutorial collection in yourrptutorials database. The method returns an instance ofInsertManyResult, which provides information on the inserted documents.

To retrieve documents from a collection, you can use.find(). Without arguments,.find() returns aCursor object thatyields the documents in the collection on demand:

>>>importpprint>>>fordocintutorial.find():...pprint.pprint(doc)...{'_id': ObjectId('600747355e6ea8d224f754ba'), 'author': 'Jon', 'contributors': ['Aldren', 'Geir Arne', 'Joanna', 'Jason'], 'title': 'Reading and Writing CSV Files in Python', 'url': 'https://realpython.com/python-csv/'}    ...{'_id': ObjectId('6008511c87eb0fbf73dbf71f'), 'author': 'David', 'contributors': ['Aldren', 'Joanna', 'Jacob'], 'title': 'Object-Oriented Programming (OOP) in Python 3', 'url': 'https://realpython.com/python3-object-oriented-programming/'}

Here, you run a loop on the object that.find() returns and print successive results, usingpprint.pprint() to provide a user-friendly output format.

You can also use.find_one() to retrieve a single document. In this case, you can use a dictionary that contains fields to match. For example, if you want to retrieve the first tutorial byJon, then you can do something like this:

>>>importpprint>>>jon_tutorial=tutorial.find_one({"author":"Jon"})>>>pprint.pprint(jon_tutorial){'_id': ObjectId('600747355e6ea8d224f754ba'), 'author': 'Jon', 'contributors': ['Aldren', 'Geir Arne', 'Joanna', 'Jason'], 'title': 'Reading and Writing CSV Files in Python', 'url': 'https://realpython.com/python-csv/'}

Note that the tutorial’sObjectId is set under the_id key, which is the unique document identifier that MongoDB automatically adds when you insert a document into your database.

PyMongo also provides methods toreplace,update, anddelete documents from a database. If you want to dive deeper into these features, then take a look at thedocumentation forCollection.

Closing Connections

Establishing a connection to a MongoDB database is typically an expensive operation. If you have an application that constantly retrieves and manipulates data in a MongoDB database, then you probably don’t want to be opening and closing the connection all the time since this might affect your application’s performance.

In this kind of situation, you should keep your connection alive and only close it before exiting the application to clear all the acquired resources. You can close the connection by calling.close() on theMongoClient instance:

>>>client.close()

Another situation is when you have an application that occasionally uses a MongoDB database. In this case, you might want to open the connection when needed and close it immediately after use for freeing the acquired resources. A consistent approach to this problem would be to use thewith statement. Yes,MongoClient implements thecontext manager protocol:

>>>importpprint>>>frompymongoimportMongoClient>>>withMongoClient()asclient:...db=client.rptutorials...fordocindb.tutorial.find():...pprint.pprint(doc)...{'_id': ObjectId('600747355e6ea8d224f754ba'), 'author': 'Jon', 'contributors': ['Aldren', 'Geir Arne', 'Joanna', 'Jason'], 'title': 'Reading and Writing CSV Files in Python', 'url': 'https://realpython.com/python-csv/'}    ...{'_id': ObjectId('6008511c87eb0fbf73dbf71f'), 'author': 'David', 'contributors': ['Aldren', 'Joanna', 'Jacob'], 'title': 'Object-Oriented Programming (OOP) in Python 3', 'url': 'https://realpython.com/python3-object-oriented-programming/'}

If you use thewith statement to handle your MongoDB client, then at the end of thewith code block, the client’s.__exit__() method gets called, which at the same time closes the connection bycalling.close().

Installing MongoEngine

There are a handful of Python libraries to help you work with MongoDB. MongoEngine, however, is a popular one that provides a nice set of features, flexibility, andcommunity support. MongoEngine isavailable on PyPI. You can install it using the followingpip command:

$pipinstallmongoengine==0.22.1

Once you’ve installed MongoEngine into your Python environment, you’re ready to start working with MongoDB databases using Python’sobject-oriented features. The next step is to connect to your running MongoDB instance.

Establishing a Connection

To establish a connection with your database, you need to usemongoengine.connect(). This function takes several arguments. However, in this tutorial, you’ll use only three of them. Within your Python interactive session, type the following code:

>>>frommongoengineimportconnect>>>connect(db="rptutorials",host="localhost",port=27017)MongoClient(host=['localhost:27017'], ..., read_preference=Primary())

Here, you first set the database namedb to"rptutorials", which is the name of the database you want to work in. Then you provide ahost and aport to connect to your current MongoDB instance. Since you’re using the defaulthost andport, you can omit these two parameters and just useconnect("rptutorials").

Working With Collections and Documents

To create documents with MongoEngine, you first need to define what data you want the documents to have. In other words, you need to define adocument schema. MongoEngine encourages you to define a document schema to help you reduce coding errors and to allow you to define utility or helper methods.

Similar to ORMs, ODMs like MongoEngine provide a base or model class for you to define a document schema. In ORMs, that class is equivalent to a table, and its instances are equivalent to rows. In MongoEngine, the class is equivalent to a collection, and its instances are equivalent to documents.

To create a model, you need to subclassDocument and provide the required fields asclass attributes. To continue with the blog example, here’s how you can create a model for your tutorials:

>>>frommongoengineimportDocument,ListField,StringField,URLField>>>classTutorial(Document):...title=StringField(required=True,max_length=70)...author=StringField(required=True,max_length=20)...contributors=ListField(StringField(max_length=20))...url=URLField(required=True)

With this model, you tell MongoEngine that you expect aTutorial document to have a.title, an.author, a list of.contributors, and a.url. The base class,Document, uses that information along with the field types to validate the input data for you.

For example, if you try to save aTutorial object without a.title, then your model throws an exception and lets you know. You can take this even further and add more restrictions, such as the length of the.title, and so on.

There are a few general parameters that you can use to validate fields. Here are some of the more commonly used parameters:

• db_field specifies a different field name.
• required ensures that the field is provided.
• default provides a default value for a given field if no value is given.
• unique ensures that no other document in the collection has the same value for this field.

Each specific field type also has its own set of parameters. You can check thedocumentation for a complete guide to the available field types.

To save a document to your database, you need to call.save() on a document object. If the document already exists, then all the changes will be applied to the existing document. If the document doesn’t exist, then it’ll be created.

Here’s an example of creating and saving a tutorial into your sample tutorials database:

>>>tutorial1=Tutorial(...title="Beautiful Soup: Build a Web Scraper With Python",...author="Martin",...contributors=["Aldren","Geir Arne","Jaya","Joanna","Mike"],...url="https://realpython.com/beautiful-soup-web-scraper-python/"...)>>>tutorial1.save()# Insert the new tutorial<Tutorial: Tutorial object>

By default,.save() inserts the new document into a collection named after the model class,Tutorial, except using lowercase letters. In this case, the collection name istutorial, which matches the collection you’ve been using to save your tutorials.

PyMongo performsdata validation when you call.save(). This means that it checks the input data against the schema you declared in theTutorial model class. If the input data violates the schema or any of its constraints, then you get an exception, and the data isn’t saved into the database.

For example, here’s what happens if you try to save a tutorial without providing a.title:

>>>tutorial2=Tutorial()>>>tutorial2.author="Alex">>>tutorial2.contributors=["Aldren","Jon","Joanna"]>>>tutorial2.url="https://realpython.com/convert-python-string-to-int/">>>tutorial2.save()Traceback (most recent call last):...mongoengine.errors.ValidationError:... (Field is required: ['title'])

In this example, first note that you can also build aTutorial object by assigning values to its attributes. Second, since you don’t provide a.title for the new tutorial,.save() raises aValidationError telling you that the.title field is required. Having automatic data validation is a great feature that will save you some headaches.

EachDocument subclass has an.objects attribute that you can use to access the documents in the associated collection. For example, here’s how you canprint the.title of all your current tutorials:

>>>fordocinTutorial.objects:...print(doc.title)...Reading and Writing CSV Files in PythonHow to Iterate Through a Dictionary in PythonPython 3's f-Strings: An Improved String Formatting Syntax (Guide)Working With JSON Data in PythonPython's Requests Library (Guide)Object-Oriented Programming (OOP) in Python 3Beautiful Soup: Build a Web Scraper With Python

Thefor loop iterates over all your tutorials and prints their.title data to the screen. You can also use.objects to filter your documents. For example, say you want to retrieve the tutorials authored byAlex. In that case, you can do something like this:

>>>fordocinTutorial.objects(author="Alex"):...print(doc.title)...Python's Requests Library (Guide)

MongoEngine is well suited to manage your MongoDB databases for just about any type of application. Its features make it ideal for creating efficient and scalable programs using a high-level approach. If you’re looking for more information about MongoEngine, be sure to check out itsuser guide.