Note:Developers building new applications arestrongly encouraged to use theNDB Client Library, which has several benefitscompared to this client library, such as automatic entity caching via the MemcacheAPI. If you are currently using the older DB Client Library, read theDB to NDB Migration Guide

Overview

A datastore entity has a key and a set of properties. An application uses the datastore API to define data models, and create instances of those models to be stored as entities. Models provide a common structure to the entities created by the API, and can define rules for validating property values.

Model Classes

The Model Class

An application describes the kinds of data it uses withmodels. A model is a Python class that inherits from theModel class. The model class defines a new Kind of datastore entity and the properties the Kind is expected to take. The Kind name is defined by the instantiated class name that inherits fromdb.Model.

Model properties are defined using class attributes on the model class. Each class attribute is an instance of a subclass of theProperty class, usually one ofthe provided property classes. A property instance holds configuration for the property, such as whether or not the property is required for the instance to be valid, or a default value to use for the instance if none is provided.

fromgoogle.appengine.extimportdbclassPet(db.Model):name=db.StringProperty(required=True)type=db.StringProperty(required=True,choices=set(["cat","dog","bird"]))birthdate=db.DateProperty()weight_in_pounds=db.IntegerProperty()spayed_or_neutered=db.BooleanProperty()

An entity of one of the defined entity kinds is represented in the API by an instance of the corresponding model class. The application can create a new entity by calling the constructor of the class. The application accesses and manipulates properties of the entity using attributes of the instance. The model instance constructor accepts initial values for properties as keyword arguments.

fromgoogle.appengine.apiimportuserspet=Pet(name="Fluffy",type="cat")pet.weight_in_pounds=24

Note: The attributes of the modelclass are configuration for the model properties, whose values areProperty instances. The attributes of the modelinstance are the actual property values, whose values are of the type accepted by the Property class.

The Model class uses the Property instances to validate values assigned to the model instance attributes. Property value validation occurs when a model instance is first constructed, and when an instance attribute is assigned a new value. This ensures that a property can never have an invalid value.

Because validation occurs when the instance is constructed, any property that is configured to be required must be initialized in the constructor. In this example,name andtype are required values, so their initial values are specified in the constructor.weight_in_pounds is not required by the model, so it starts out unassigned, then is assigned a value later.

An instance of a model created using the constructor does not exist in the datastore until it is "put" for the first time.

Note: As with all Python class attributes, model property configuration is initialized when the script or module is first imported. Because App Engine caches imported modules between requests, module configuration may be initialized during a request for one user, and re-used during a request for another. Do not initialize model property configuration, such as default values, with data specific to the request or the current user. SeeApp Caching for more information.

The Expando Class

A model defined using the Model class establishes a fixed set of properties that every instance of the class must have (perhaps with default values). This is a useful way to model data objects, but the datastore does not require that every entity of a given kind have the same set of properties.

Sometimes it is useful for an entity to have properties that aren't necessarily like the properties of other entities of the same kind. Such an entity is represented in the datastore API by an "expando" model. An expando model class subclasses theExpando superclass. Any value assigned to an attribute of an instance of an expando model becomes a property of the datastore entity, using the name of the attribute. These properties are known asdynamic properties. Properties defined using Property class instances in class attributes arefixed properties.

An expando model can have both fixed and dynamic properties. The model class simply sets class attributes with Property configuration objects for the fixed properties. The application creates dynamic properties when it assigns them values.

classPerson(db.Expando):first_name=db.StringProperty()last_name=db.StringProperty()hobbies=db.StringListProperty()p=Person(first_name="Albert",last_name="Johnson")p.hobbies=["chess","travel"]p.chess_elo_rating=1350p.travel_countries_visited=["Spain","Italy","USA","Brazil"]p.travel_trip_count=13

Because dynamic properties do not have model property definitions, dynamic properties are not validated. Any dynamic property can have a value of any of thedatastore base types, includingNone. Two entities of the same kind can have different types of values for the same dynamic property, and one can leave a property unset that the other sets.

Unlike fixed properties, dynamic properties need not exist. A dynamic property with a value ofNone is different from a non-existent dynamic property. If an expando model instance does not have an attribute for a property, the corresponding data entity does not have that property. You can delete a dynamic property by deleting the attribute.

Attributes whose names begin with an underscore (_) are not saved to the datastore entity. This allows you to store values on the model instance for temporary internal use without affecting the data saved with the entity.

Note:Static properties will always be saved to the datastore entity regardless of whether it is Expando, Model, or begins with an underscore (_).

delp.chess_elo_rating

A query that uses a dynamic property in a filter returns only entities whose value for the property is of the same type as the value used in the query. Similarly, the query returns only entities with that property set.

p1=Person()p1.favorite=42p1.put()p2=Person()p2.favorite="blue"p2.put()p3=Person()p3.put()people=db.GqlQuery("SELECT * FROM Person WHERE favorite< :1",50)# people has p1, but not p2 or p3people=db.GqlQuery("SELECT * FROM Person WHERE favorite > :1",50)# people has no results

Note: The example above uses queries across entity groups, which may return stale results. For strongly consistent results, useancestor queries within entity groups.

TheExpando class is a subclass of theModel class, and inherits all of its methods.

The PolyModel Class

The Python API includes another class for data modeling that allows you to define hierarchies of classes, and perform queries that can return entities of a given class or any of its subclasses. Such models and queries are called "polymorphic," because they allow instances of one class to be results for a query of a parent class.

The following example defines aContact class, with the subclassesPerson andCompany:

fromgoogle.appengine.extimportdbfromgoogle.appengine.ext.dbimportpolymodelclassContact(polymodel.PolyModel):phone_number=db.PhoneNumberProperty()address=db.PostalAddressProperty()classPerson(Contact):first_name=db.StringProperty()last_name=db.StringProperty()mobile_number=db.PhoneNumberProperty()classCompany(Contact):name=db.StringProperty()fax_number=db.PhoneNumberProperty()

This model ensures that allPerson entities and allCompany entities havephone_number andaddress properties, and queries forContact entities can return eitherPerson orCompany entities. OnlyPerson entities havemobile_number properties.

The subclasses can be instantiated just like any other model class:

p=Person(phone_number='1-206-555-9234',address='123 First Ave., Seattle, WA, 98101',first_name='Alfred',last_name='Smith',mobile_number='1-206-555-0117')p.put()c=Company(phone_number='1-503-555-9123',address='P.O. Box 98765, Salem, OR, 97301',name='Data Solutions, LLC',fax_number='1-503-555-6622')c.put()

A query forContact entities can return instances of eitherContact,Person, orCompany. The following code prints information for both entities created above:

forcontactinContact.all():print'Phone:%s\nAddress:%s\n\n'%(contact.phone_number,contact.address)

A query forCompany entities returns only instances ofCompany:

forcompanyinCompany.all()# ...

For now, polymorphic models should not passed to theQuery class constructor directly. Instead, use theall() method, as in the example above.

For more information on how to use polymorphic models, and how they are implemented, seeThe PolyModel Class.

Property Classes and Types

The datastore supports a fixed set of value types for entity properties, including Unicode strings, integers, floating point numbers, dates, entity keys, byte strings (blobs), and various GData types. Each of the datastore value types has a corresponding Property class provided by thegoogle.appengine.ext.db module.

Types and Property Classes describes all of the supported value types and their corresponding Property classes. Several special value types are described below.

Strings and Blobs

The datastore supports two value types for storing text: short text strings up to 1500 bytes in length, and long text strings up to one megabyte in length. Short strings are indexed and can be used in query filter conditions and sort orders. Long strings are not indexed and cannot be used in filter conditions or sort orders.

A short string value can be either aunicode value or astr value. If the value is astr, an encoding of'ascii' is assumed. To specify a different encoding for astr value, you can convert it to aunicode value with theunicode() type constructor, which takes thestr and the name of the encoding as arguments. Short strings can be modeled using theStringProperty class.

classMyModel(db.Model):string=db.StringProperty()obj=MyModel()# Python Unicode literal syntax fully describes characters in a text string.obj.string=u"kittens"# unicode() converts a byte string to a Unicode string using the named codec.obj.string=unicode("kittens","latin-1")# A byte string is assumed to be text encoded as ASCII (the 'ascii' codec).obj.string="kittens"# Short string properties can be used in query filters.results=db.GqlQuery("SELECT * FROM MyModel WHERE string = :1",u"kittens")

A long string value is represented by adb.Text instance. Its constructor takes either aunicode value, or astr value and optionally the name of the encoding used in thestr. Long strings can be modeled using theTextProperty class.

classMyModel(db.Model):text=db.TextProperty()obj=MyModel()# Text() can take a Unicode string.obj.text=u"lots of kittens"# Text() can take a byte string and the name of an encoding.obj.text=db.Text("lots of kittens","latin-1")# If no encoding is specified, a byte string is assumed to be ASCII text.obj.text="lots of kittens"# Text properties can store large values.obj.text=db.Text(open("a_tale_of_two_cities.txt").read(),"utf-8")

The datastore also supports two similar types for non-text byte strings:db.ByteString anddb.Blob. These values are strings of raw bytes, and are not treated as encoded text (such as UTF-8).

Likedb.StringProperty values,db.ByteString values are indexed. Likedb.TextProperty properties,db.ByteString values are limited to 1500 bytes. AByteString instance represents a short string of bytes, and takes astr value as an argument to its constructor. Byte strings are modeled using theByteStringProperty class.

Likedb.Text, adb.Blob value can be as large as one megabyte, but is not indexed, and cannot be used in query filters or sort orders. Thedb.Blob class takes astr value as an argument to its constructor, or you can assign the value directly. Blobs are modeled using theBlobProperty class.

classMyModel(db.Model):blob=db.BlobProperty()obj=MyModel()obj.blob=open("image.png").read()

Lists

A property can have multiple values, represented in the datastore API as a Pythonlist. The list can contain values of any of the value types supported by the datastore. A single list property may even have values of different types.

Order is generally preserved, so when entities are returned by queries andget(), the list properties values are in the same order as when they were stored. There's one exception to this:Blob andText values are moved to the end of the list; however, they retain their original order relative to each other.

TheListProperty class models a list, and enforces that all values in the list are of a given type. For convenience, the library also providesStringListProperty, similar toListProperty(basestring).

classMyModel(db.Model):numbers=db.ListProperty(long)obj=MyModel()obj.numbers=[2,4,6,8,10]obj.numbers=["hello"]# ERROR: MyModel.numbers must be a list of longs.

A query with filters on a list property tests each value in the listindividually. The entity will match the query only if some value in the listpassesall of the filters on that property.See theDatastore Queries page for more information.

# Get all entities where numbers contains a 6.results=db.GqlQuery("SELECT * FROM MyModel WHERE numbers = 6")# Get all entities where numbers contains at least one element less than 10.results=db.GqlQuery("SELECT * FROM MyModel WHERE numbers < 10")

Query filters only operate on list members. There is no way to test two lists for similarity in a query filter.

Internally, the datastore represents a list property value as multiple values for the property. If a list property value is the empty list, then the property has no representation in the datastore. The datastore API treats this situation differently for static properties (with ListProperty) and dynamic properties:

• A static ListProperty can be assigned the empty list as a value. The property does not exist in the datastore, but the model instance behaves as if the value is the empty list. A static ListPropertycannot have a value ofNone.
• A dynamic property with alist valuecannot be assigned an empty list value. However, it can have a value ofNone, and can be deleted (usingdel).

The ListProperty model tests that a value added to the list is of the correct type, and throws a BadValueError if it isn't. This test occurs (and potentially fails) even when a previously stored entity is retrieved and loaded into the model. Becausestr values are converted tounicode values (as ASCII text) prior to storage,ListProperty(str) is treated asListProperty(basestring), the Python data type which accepts bothstr andunicode values. You can also useStringListProperty() for this purpose.

For storing non-text byte strings, usedb.Blob values. The bytes of a blob string are preserved when they are stored and retrieved. You can declare a property that is a list of blobs asListProperty(db.Blob).

List properties can interact with sort orders in unusual ways; see theDatastore Queries page for details.

References

A property value can contain the key of another entity. The value is aKey instance.

TheReferenceProperty class models a key value, and enforces that all values refer to entities of a given kind. For convenience, the library also providesSelfReferenceProperty, equivalent to a ReferenceProperty that refers to the same kind as the entity with the property.

Assigning a model instance to a ReferenceProperty property automatically uses its key as the value.

classFirstModel(db.Model):prop=db.IntegerProperty()classSecondModel(db.Model):reference=db.ReferenceProperty(FirstModel)obj1=FirstModel()obj1.prop=42obj1.put()obj2=SecondModel()# A reference value is the key of another entity.obj2.reference=obj1.key()# Assigning a model instance to a property uses the entity's key as the value.obj2.reference=obj1obj2.put()

A ReferenceProperty property value can be used as if it were the model instance of the referenced entity. If the referenced entity is not in memory, using the property as an instance automatically fetches the entity from the datastore. A ReferenceProperty also stores a key, but using the property causes the related entity to be loaded.

obj2.reference.prop=999obj2.reference.put()results=db.GqlQuery("SELECT * FROM SecondModel")another_obj=results.fetch(1)[0]v=another_obj.reference.prop

If a key points to a non-existent entity, then accessing the property raises an error. If an application expects that a reference could be invalid, it can test for the existence of the object using atry/except block:

try:obj1=obj2.referenceexceptdb.ReferencePropertyResolveError:# Referenced entity was deleted or never existed.

ReferenceProperty has another handy feature: back-references. When a model has a ReferenceProperty to another model, each referenced entity gets a property whose value is aQuery that returns all of the entities of the first model that refer to it.

# To fetch and iterate over every SecondModel entity that refers to the# FirstModel instance obj1:forobjinobj1.secondmodel_set:# ...

The name of the back-reference property defaults tomodelname_set (with the name of the model class in lowercase letters, and "_set" added to the end), and can be adjusted using thecollection_name argument to the ReferenceProperty constructor.

If you have multiple ReferenceProperty values that refer to the same model class, the default construction of the back-reference property raises an error:

classFirstModel(db.Model):prop=db.IntegerProperty()# This class raises a DuplicatePropertyError with the message# "Class Firstmodel already has property secondmodel_set"classSecondModel(db.Model):reference_one=db.ReferenceProperty(FirstModel)reference_two=db.ReferenceProperty(FirstModel)

To avoid this error, you must explicitly set thecollection_name argument:

classFirstModel(db.Model):prop=db.IntegerProperty()# This class runs fineclassSecondModel(db.Model):reference_one=db.ReferenceProperty(FirstModel,collection_name="secondmodel_reference_one_set")reference_two=db.ReferenceProperty(FirstModel,collection_name="secondmodel_reference_two_set")

Automatic referencing and dereferencing of model instances, type checking and back-references are only available using the ReferenceProperty model property class. Keys stored as values of Expando dynamic properties or ListProperty values do not have these features.