Repository files navigation

DataLoaderPHP is a generic utility to be used as part of your application's datafetching layer to provide a simplified and consistent API over various remotedata sources such as databases or web services via batching and caching.

This library requires PHP >= 7.3 to work.

First, install DataLoaderPHP using composer.

composer require"overblog/dataloader-php"

To get started, create aDataLoader object.

Batching is not an advanced feature, it's DataLoader's primary feature.Create loaders by providing a batch loading function.

useOverblog\DataLoader\DataLoader;$myBatchGetUsers =function ($keys) {/* ... */ };$promiseAdapter =newMyPromiseAdapter();$userLoader =newDataLoader($myBatchGetUsers,$promiseAdapter);

A batch loading callable / callback accepts an Array of keys, and returns a Promise whichresolves to an Array of values.

Then load individual values from the loader. DataLoaderPHP will coalesce allindividual loads which occur within a single frame of execution (usingawait method)and then call your batch function with all requested keys.

$userLoader->load(1)  ->then(function ($user)use ($userLoader) {return$userLoader->load($user->invitedByID); })  ->then(function ($invitedBy) {echo"User 1 was invited by$invitedBy"; });// Elsewhere in your application$userLoader->load(2)  ->then(function ($user)use ($userLoader) {return$userLoader->load($user->invitedByID); })  ->then(function ($invitedBy) {echo"User 2 was invited by$invitedBy"; });// Synchronously waits on the promise to complete, if not using EventLoop.$userLoader->await();// or `DataLoader::await()`

A naive application may have issued four round-trips to a backend for therequired information, but with DataLoaderPHP this application will make at mosttwo.

DataLoaderPHP allows you to decouple unrelated parts of your application withoutsacrificing the performance of batch data-loading. While the loader presents anAPI that loads individual values, all concurrent requests will be coalesced andpresented to your batch loading function. This allows your application to safelydistribute data fetching requirements throughout your application and maintainminimal outgoing data requests.

A batch loading function accepts an Array of keys, and returns a Promise whichresolves to an Array of values. There are a few constraints that must be upheld:

• The Array of values must be the same length as the Array of keys.
• Each index in the Array of values must correspond to the same index in the Array of keys.

For example, if your batch function was provided the Array of keys:[ 2, 9, 6, 1 ],and loading from a back-end service returned the values:

[  ['id' =>9,'name' =>'Chicago'],  ['id' =>1,'name' =>'New York'],  ['id' =>2,'name' =>'San Francisco']  ]

Our back-end service returned results in a different order than we requested, likelybecause it was more efficient for it to do so. Also, it omitted a result for key6,which we can interpret as no value existing for that key.

To uphold the constraints of the batch function, it must return an Array of valuesthe same length as the Array of keys, and re-order them to ensure each index alignswith the original keys[ 2, 9, 6, 1 ]:

[  ['id' =>2,'name' =>'San Francisco'],  ['id' =>9,'name' =>'Chicago'],null,  ['id' =>1,'name' =>'New York']]

DataLoader provides a memoization cache for all loads which occur in a singlerequest to your application. After->load() is called once with a given key,the resulting value is cached to eliminate redundant loads.

In addition to relieving pressure on your data storage, caching results per-requestalso creates fewer objects which may relieve memory pressure on your application:

$userLoader =newDataLoader(...);$promise1A =$userLoader->load(1);$promise1B =$userLoader->load(1);var_dump($promise1A ===$promise1B);// bool(true)

In certain uncommon cases, clearing the request cache may be necessary.

The most common example when clearing the loader's cache is necessary is aftera mutation or update within the same request, when a cached value could be out ofdate and future loads should not use any possibly cached value.

Here's a simple example using SQL UPDATE to illustrate.

useOverblog\DataLoader\DataLoader;// Request begins...$userLoader =newDataLoader(...);// And a value happens to be loaded (and cached).$userLoader->load(4)->then(...);// A mutation occurs, invalidating what might be in cache.$sql ='UPDATE users WHERE id=4 SET username="zuck"';if (true ===$conn->query($sql)) {$userLoader->clear(4);}// Later the value load is loaded again so the mutated data appears.$userLoader->load(4)->then(...);// Request completes.

If a batch load fails (that is, a batch function throws or returns a rejectedPromise), then the requested values will not be cached. However if a batchfunction returns anError instance for an individual value, thatError willbe cached to avoid frequently loading the sameError.

In some circumstances you may wish to clear the cache for these individual Errors:

$userLoader->load(1)->then(null,function ($exception) {if (/* determine if error is transient */) {$userLoader->clear(1);  }throw$exception;});

In certain uncommon cases, a DataLoader whichdoes not cache may be desirable.Callingnew DataLoader(myBatchFn, new Option(['cache' => false ])) will ensure that everycall to->load() will produce anew Promise, and requested keys will not besaved in memory.

However, when the memoization cache is disabled, your batch function willreceive an array of keys which may contain duplicates! Each key will beassociated with each call to->load(). Your batch loader should provide a valuefor each instance of the requested key.

For example:

$myLoader =newDataLoader(function ($keys) {echojson_encode($keys);returnsomeBatchLoadFn($keys);},$promiseAdapter,newOption(['cache' =>false ]));$myLoader->load('A');$myLoader->load('B');$myLoader->load('A');// [ 'A', 'B', 'A' ]

More complex cache behavior can be achieved by calling->clear() or->clearAll()rather than disabling the cache completely. For example, this DataLoader willprovide unique keys to a batch function due to the memoization cache beingenabled, but will immediately clear its cache when the batch function is calledso later requests will load new values.

$myLoader =newDataLoader(function($keys)use ($identityLoader) {$identityLoader->clearAll();returnsomeBatchLoadFn($keys);},$promiseAdapter);

DataLoaderPHP creates a public API for loading data from a particulardata back-end with unique keys such as theid column of a SQL table ordocument name in a MongoDB database, given a batch loading function.

EachDataLoaderPHP instance contains a unique memoized cache. Use caution whenused in long-lived applications or those which serve many users with differentaccess permissions and consider creating a new instance per web request.

Create a newDataLoaderPHP given a batch loading instance and options.

• $batchLoadFn: A callable / callback which accepts an Array of keys, and returns a Promise which resolves to an Array of values.

• $promiseAdapter: Any object that implementsOverblog\PromiseAdapter\PromiseAdapterInterface. (seeOverblog/Promise-Adapter)

• $options: An optional object of options:

  • batch: Defaulttrue. Set tofalse to disable batching, insteadimmediately invokingbatchLoadFn with a single load key.

  • maxBatchSize: DefaultInfinity. Limits the number of items that getpassed in to thebatchLoadFn.

  • cache: Defaulttrue. Set tofalse to disable caching, insteadcreating a new Promise and new key in thebatchLoadFn for every load.

  • cacheKeyFn: A function to produce a cache key for a given load key.Defaults tokey. Useful to provide when an objects are keysand two similarly shaped objects should be considered equivalent.

  • cacheMap: An instance ofCacheMap to beused as the underlying cache for this loader. Defaultnew CacheMap().

Loads a key, returning aPromise for the value represented by that key.

• $key: An key value to load.

Loads multiple keys, promising an array of values:

list($a,$b) = DataLoader::await($myLoader->loadMany(['a','b']));

This is equivalent to the more verbose:

list($a,$b) = DataLoader::await(\React\Promise\all([$myLoader->load('a'),$myLoader->load('b')]));

• $keys: An array of key values to load.

Clears the value at$key from the cache, if it exists. Returns itself formethod chaining.

• $key: An key value to clear.

Clears the entire cache. To be used when some event results in unknowninvalidations across this particularDataLoaderPHP. Returns itself formethod chaining.

Primes the cache with the provided key and value. If the key already exists, nochange is made. (To forcefully prime the cache, clear the key first with$loader->clear($key)->prime($key, $value). Returns itself for method chaining.

You can synchronously force promises to complete using DataLoaderPHP's await method.When an await function is invoked it is expected to deliver a value to the promise or reject the promise.Await method process all waiting promise in all dataLoaderPHP instances.

• $promise: Optional promise to complete.

• $unwrap: controls whether or not the value of the promise is returned for a fulfilled promiseor if an exception is thrown if the promise is rejected. Defaulttrue.

DataLoader pairs nicely well withWebonyx/GraphQL. GraphQL fields aredesigned to be stand-alone functions. Without a caching or batching mechanism,it's easy for a naive GraphQL server to issue new database requests each time afield is resolved.

Consider the following GraphQL request:

{me {namebestFriend {name    }friends(first:5) {namebestFriend {name      }    }  }}

Naively, ifme,bestFriend andfriends each need to request the backend,there could be at most 13 database requests!

When using DataLoader, we could define theUser typeat most 4 database requests,and possibly fewer if there are cache hits.

<?phpuseGraphQL\GraphQL;useGraphQL\Type\Definition\ObjectType;useGraphQL\Type\Definition\Type;useOverblog\DataLoader\DataLoader;useOverblog\DataLoader\Promise\Adapter\Webonyx\GraphQL\SyncPromiseAdapter;useOverblog\PromiseAdapter\Adapter\WebonyxGraphQLSyncPromiseAdapter;/*** @var \PDO $dbh*/// ...$graphQLPromiseAdapter =newSyncPromiseAdapter();$dataLoaderPromiseAdapter =newWebonyxGraphQLSyncPromiseAdapter($graphQLPromiseAdapter);$userLoader =newDataLoader(function ($keys) {/*...*/ },$dataLoaderPromiseAdapter);GraphQL::setPromiseAdapter($graphQLPromiseAdapter);$userType =newObjectType(['name' =>'User','fields' =>function ()use (&$userType,$userLoader,$dbh) {return ['name' => ['type' => Type::string()],'bestFriend' => ['type' =>$userType,'resolve' =>function ($user)use ($userLoader) {$userLoader->load($user['bestFriendID']);                }            ],'friends' => ['args' => ['first' => ['type' => Type::int() ],                ],'type' => Type::listOf($userType),'resolve' =>function ($user,$args)use ($userLoader,$dbh) {$sth =$dbh->prepare('SELECT toID FROM friends WHERE fromID=:userID LIMIT :first');$sth->bindParam(':userID',$user['id'],PDO::PARAM_INT);$sth->bindParam(':first',$args['first'],PDO::PARAM_INT);$friendIDs =$sth->execute();return$userLoader->loadMany($friendIDs);                }            ]        ];    }]);

You can also seean example.

See thebundle.

Overblog/DataLoaderPHP is a port ofdataLoader NodeJS versionbyFacebook.

Also, large parts of the documentation have been ported from the dataLoader NodeJS versionDocs.

Overblog/DataLoaderPHP is released under theMIT license.