Share data between handles

Sometimes applications need to share data between transfers. All easy handlesadded to the same multi handle automatically get a lot of sharing done betweenthe handles in that same multi handle, but sometimes that is not exactly whatyou want.

Multi handle

All easy handles added to the same multi handle automatically shareconnection cache anddns cache.

Sharing between easy handles

libcurl has a generic "sharing interface", where the application creates a"share object" that then holds data that can be shared by any number of easyhandles. The data is then stored and read from the shared object instead ofkept within the handles that are sharing it.

CURLSH *share = curl_share_init();

The shared object can be set to share all or any of cookies, connection cache,dns cache and SSL session id cache.

For example, setting up the share to hold cookies and dns cache:

curl_share_setopt(share, CURLSHOPT_SHARE, CURL_LOCK_DATA_COOKIE);curl_share_setopt(share, CURLSHOPT_SHARE, CURL_LOCK_DATA_DNS);

You then set up the corresponding transfer to use this share object:

curl_easy_setopt(curl, CURLOPT_SHARE, share);

Transfers done with thiscurl handle uses and stores its cookie and dnsinformation in theshare handle. You can set several easy handles to sharethe same share object.

What to share

CURL_LOCK_DATA_COOKIE - set this bit to share cookie jar. Note that eacheasy handle still needs to get its cookie "engine" started properly to startusing cookies.

CURL_LOCK_DATA_DNS - the DNS cache is where libcurl stores addresses forresolved hostnames for a while to make subsequent lookups faster.

CURL_LOCK_DATA_SSL_SESSION - the SSL session ID cache is where libcurl storeresume information for SSL connections to be able to resume a previousconnection faster.

CURL_LOCK_DATA_CONNECT - when set, this handle uses a shared connectioncache and thus is more likely to find existing connections to re-use etc,which may result in faster performance when doing multiple transfers to thesame host in a serial manner.

Locking

If you want have the share object shared by transfers in a multi-threadedenvironment. Perhaps you have a CPU with many cores and you want each core torun its own thread and transfer data, but you still want the differenttransfers to share data. Then you need to set the mutex callbacks.

If you do not use threading and youknow you access the shared object in aserial one-at-a-time manner you do not need to set any locks. If there is evermore than one transfer that access share object at a time, it needs to getmutex callbacks setup to prevent data destruction and possibly even crashes.

Since libcurl itself does not know how to lock things or even what threadingmodel you are using, you must make sure to do mutex locks that only allows oneaccess at a time. A lock callback for a pthreads-using application could looksimilar to:

static void lock_cb(CURL *handle, curl_lock_data data,                    curl_lock_access access, void *userptr){  pthread_mutex_lock(&lock[data]); /* uses a global lock array */}curl_share_setopt(share, CURLSHOPT_LOCKFUNC, lock_cb);

With the corresponding unlock callback could look like:

static void unlock_cb(CURL *handle, curl_lock_data data,                      void *userptr){  pthread_mutex_unlock(&lock[data]); /* uses a global lock array */}curl_share_setopt(share, CURLSHOPT_UNLOCKFUNC, unlock_cb);

Unshare

A transfer uses the share object during its transfer and share what thatobject has been specified to share with other handles sharing the same object.

In a subsequent transfer,CURLOPT_SHARE can be set to NULL to prevent atransfer from continuing to share. It that case, the handle may start the nexttransfer with empty caches for the data that was previously shared.

Between two transfers, a share object can also get updated to share adifferent set of properties so that the handles that share that object sharesa different set of data next time. You remove an item to share from a sharedobject with the curl_share_setopt()'sCURLSHOPT_UNSHARE option like thiswhen unsharing DNS data:

curl_share_setopt(share, CURLSHOPT_UNSHARE, CURL_LOCK_DATA_DNS);