5x performance - switching from LWP to Furl & Net::DNS::Lite

Recently I rewrote some of our code that used LWP::UserAgent to use Furl instead, and have been observing more than 5x increase in performance (the CPU time spent for each HTTP request in average has dropped 82%).

The fact clearly shows that if you are having performance issues with LWP::UserAgent it is a good idea to switch to Furl.  And here are my recommendations when doing so:

use the low-level interface (Furl::HTTP)

The OO-interface provided by Furl.pm is not as fast as the low level interface, though it is still about twice as fast as LWP::UserAgent.

use Net::DNS::Lite for accurate timeouts

The timeout parameter of LWP::UserAgent is not accurate.  The module may wait longer than the specified timeout while looking up hostnames.  Furl provides a callback to use non-default hostname lookup functions with support for timeouts, and Net::DNS::Lite can be used for the purpose.

use Cache::LRU to cache DNS queries

In general DNS queries are lightweight but caching the responses can still have positive effect (if the cache is very fast).  Cache::LRU is suitable for such an usecase.

The below code snippet illustrates how to setup a Furl::HTTP object with the described configuration.

use Cache::LRU;
use Furl;
use Net::DNS::Lite;

# setup cache for Net::DNS::Lite
$Net::DNS::Lite::CACHE = Cache::LRU->new(
    size => 256,
);

# create HTTP object...
my $furl = Furl::HTTP->new(
    inet_aton => \&Net::DNS::Lite::inet_aton,
    timeout   => $timeout_in_seconds,
    ...