Performance

This section collects general advice on what you can do as an applicationauthor to get the maximum performance out of libcurl.

libcurl is designed and intended to run as fast as possible by default. Youare expected to get top performance already without doing anything extra inparticular. There are however some common things to look at or perhapsmistakes to avoid.

reuse handles

This is a general mantra whenever libcurl is discussed. If you use the easyinterface, theprimary key to high performance is to reuse the handles whendoing subsequent transfers. That lets libcurl reuse connections, reuse TLSsessions, use its DNS cache as much as possible and more.

buffer sizes

If you download data, set theCURLOPT_BUFFERSIZE to a suitable size. It ison the smaller size from start and especially on high speed transfers, youmight be able to get more out of libcurl by increasing its size. We encourageyou to try out a few sizes in a benchmark with your use case.

Similarly, if you upload data you might want to adjust theCURLOPT_UPLOAD_BUFFERSIZE for the same reasons.

pool size

The number of live connections kept in the connection pool that you set withCURLOPT_MAXCONNECTS can be interesting to tweak. Depending of course howyour application uses connections, but if it for example iterates over Nhostnames in a short period of time, it could make sense for you to make surethat libcurl can keep all those connections alive.

CA store caching

For every new connection done using TLS, libcurl needs to access the CA storeto verify the remote server's certificate. If your applications do manyconnections, make sure you take advantage of libcurl's CA caching so thatlibcurl can avoid loading and parsing several hundred kilobytes from disk overand over. This is however not yet supported by all TLS backends.

make callbacks as fast as possible

In high speed data downloads, the write callback is called many times. If thisfunction is not written to execute the fastest possible way, there is a riskthat this function alone makesall transfers slower than they otherwisecould be.

The same of course goes for the read callback for uploads.

Avoid doing complicated logic or use locks/mutexes in your libcurl callbacks.

share data

If you use multiple easy handles, you can still share data and caches betweenthem in order to increase performance. Take a closer look atthe share API.

threads

If your transfer thread ends up consuming 100% CPU, then you might benefitfrom distributing the load onto multiple threads to increase bandwidth.

Normally then, you want to make each thread do transfers as independently aspossible to avoid them interfering with each other's performance or riskgetting into thread-safe problems due to shared handles. Try to make the samehostnames get transferred on the same thread so that connection reuse can beoptimized.

curl_multi_socket_action

If your application performs many parallel transfers, like more than a hundredconcurrent ones or so, then youmust consider switching to thecurl_multi_socket_action() and the event based API instead of the "regular"multi API. That allows and pushes you to use an event based approach whichlets your application avoid bothpoll() andselect(), which is key to highperformance combined with a high degree of parallelism.