Friday, February 4, 2011

5x performance - switching from LWP to Furl & Net::DNS::Lite

Recently I rewrote some of our code that used LWP::UserAgent to use Furl instead, and have been observing more than 5x increase in performance (the CPU time spent for each HTTP request in average has dropped 82%).

The fact clearly shows that if you are having performance issues with LWP::UserAgent it is a good idea to switch to Furl.  And here are my recommendations when doing so:

use the low-level interface (Furl::HTTP)

The OO-interface provided by is not as fast as the low level interface, though it is still about twice as fast as LWP::UserAgent.

use Net::DNS::Lite for accurate timeouts

The timeout parameter of LWP::UserAgent is not accurate.  The module may wait longer than the specified timeout while looking up hostnames.  Furl provides a callback to use non-default hostname lookup functions with support for timeouts, and Net::DNS::Lite can be used for the purpose.

use Cache::LRU to cache DNS queries

In general DNS queries are lightweight but caching the responses can still have positive effect (if the cache is very fast).  Cache::LRU is suitable for such an usecase.

The below code snippet illustrates how to setup a Furl::HTTP object with the described configuration.
use Cache::LRU;
use Furl;
use Net::DNS::Lite;

# setup cache for Net::DNS::Lite
$Net::DNS::Lite::CACHE = Cache::LRU->new(
    size => 256,

# create HTTP object...
my $furl = Furl::HTTP->new(
    inet_aton => \&Net::DNS::Lite::inet_aton,
    timeout   => $timeout_in_seconds,


  1. Nice work, you might want to label this entry as English also

  2. In your blog I was happy to see your article, better than last time, and have made great progress, I am very pleased. I am looking forward to your article will become better and better.
    hotmail login | red ball |

  3. Thanks for your page! Your share information it helped me alot! I pray for you to be happy and successful in your life.

    - - - - Choi Minzi - - - -
    Sorry Not Sorry lyrics

  4. This comment has been removed by the author.

  5. Showbox for PC Windows 10, 8.1, 8, 7, XP & Mac OS X Computer here
    Hulu for PC Stream TV and Movies Live and Online link
    Xender for PC Windows 10, 8.1, 8, 7, XP & Mac OS X Computer website

  6. It sounds good, but it is true. It is a direct link between employee health and higher productivity to get a better handle on this, we talked with Bodmer Berg manager of employee health and wellness of paycheck FLex Login.

  7. is a protection software solution that communicates with the cloud avoiding the hassle to manage the signature updates to deploy. for office setup visit

  8. Thank you, the article is very petrifying, hopefully it can be useful for everyone.

    Pengobatan Hamil Anggur
    Obat Kanker Nasofaring
    Obat Penyakit Addison