Friday, February 4, 2011

5x performance - switching from LWP to Furl & Net::DNS::Lite

Recently I rewrote some of our code that used LWP::UserAgent to use Furl instead, and have been observing more than 5x increase in performance (the CPU time spent for each HTTP request in average has dropped 82%).

The fact clearly shows that if you are having performance issues with LWP::UserAgent it is a good idea to switch to Furl.  And here are my recommendations when doing so:

use the low-level interface (Furl::HTTP)

The OO-interface provided by Furl.pm is not as fast as the low level interface, though it is still about twice as fast as LWP::UserAgent.

use Net::DNS::Lite for accurate timeouts

The timeout parameter of LWP::UserAgent is not accurate.  The module may wait longer than the specified timeout while looking up hostnames.  Furl provides a callback to use non-default hostname lookup functions with support for timeouts, and Net::DNS::Lite can be used for the purpose.

use Cache::LRU to cache DNS queries

In general DNS queries are lightweight but caching the responses can still have positive effect (if the cache is very fast).  Cache::LRU is suitable for such an usecase.

The below code snippet illustrates how to setup a Furl::HTTP object with the described configuration.
use Cache::LRU;
use Furl;
use Net::DNS::Lite;

# setup cache for Net::DNS::Lite
$Net::DNS::Lite::CACHE = Cache::LRU->new(
    size => 256,
);

# create HTTP object...
my $furl = Furl::HTTP->new(
    inet_aton => \&Net::DNS::Lite::inet_aton,
    timeout   => $timeout_in_seconds,
    ...

41 comments:

  1. Nice work, you might want to label this entry as English also

    ReplyDelete
  2. In your blog I was happy to see your article, better than last time, and have made great progress, I am very pleased. I am looking forward to your article will become better and better.
    hotmail login | red ball | 192.168.1.1

    ReplyDelete
  3. Thanks for your page! Your share information it helped me alot! I pray for you to be happy and successful in your life.

    - - - - Choi Minzi - - - -
    Sorry Not Sorry lyrics

    ReplyDelete
  4. This comment has been removed by the author.

    ReplyDelete
  5. Showbox for PC Windows 10, 8.1, 8, 7, XP & Mac OS X Computer here
    Hulu for PC Stream TV and Movies Live and Online link
    Xender for PC Windows 10, 8.1, 8, 7, XP & Mac OS X Computer website

    ReplyDelete
  6. It sounds good, but it is true. It is a direct link between employee health and higher productivity to get a better handle on this, we talked with Bodmer Berg manager of employee health and wellness of paycheck FLex Login.

    ReplyDelete
  7. Webroot.com/safe is a protection software solution that communicates with the cloud avoiding the hassle to manage the signature updates to deploy. for office setup visit office.com/setup

    ReplyDelete
  8. Thank you, the article is very petrifying, hopefully it can be useful for everyone.

    Pengobatan Hamil Anggur
    Obat Kanker Nasofaring
    Obat Penyakit Addison

    ReplyDelete


  9. flipkart emi,
    flipkart debit card emi,
    flipkart emi for debit card,
    flipkart emi on debit card,
    flipkart emi with debit card,
    flipkart emi sbi debit card,
    flipkart no emi cost,
    flipkart emi,
    flipkart with emi,
    flipkart email id,
    flipkart emi mobile,
    flipkart emi without credit card,
    flipkart mobile emi on debit card,
    flipkart emi options,
    flipkart emi card,
    flipkart emi using debit card,
    flipkart emi bajaj finserv
    zestmoney flipkart emi,
    flipkart emi laptop,
    flipkart emi process,
    flipkart emi phone,
    flipkart no cost emi laptops,
    flipkart emi in debit card,
    flipkart emi debit card icici,
    flipkart emi on credit card,
    flipkart emi credit card,
    flipkart emi policy,
    flipkart emi rules,
    flipkart emi cancellation,
    flipkart 0 emi,
    flipkart emi eligibility check,
    flipkart easy emi on debit card,
    flipkart zero emi,
    flipkart emi procedure,
    flipkart emi pay,
    flipkart emi eligibility check sms,
    flipkart emi calculator,
    flipkart emi customer care number,
    flipkart emi process debit card,,
    flipkart give emi on debit card,
    flipkart emi sms,
    flipkart emi icici debit card,,
    flipkart no emi cost ,
    flipkart emi app,
    flipkart emi shopping,
    flipkart no cost emi led tv,
    flipkart easy emi option,
    flipkart emi loan,
    flipkart emi for mobile,
    flipkart emi service,
    flipkart emi with credit card,
    flipkart in emi,
    flipkart debit card emi laptop,
    flipkart cardless emi link,
    flipkart free emi,
    flipkart emi hdfc,
    flipkart emi credit,,
    flipkart emi hdfc debit card,
    flipkart emi message,
    flipkart emi information,
    flipkart emi contact number,
    flipkart emi msg,
    flipkart new emi scheme,
    flipkart emi on debit card hdfc,
    flipkart emi number,
    flipkart debit card emi kaise kare,
    flipkart emi on debit card axis bank,
    flipkart emi registration,
    flipkart emi on icici debit card,
    flipkart emi on debit card quora,
    flipkart emi on axis debit card,
    flipkart debit card emi eligibility icici,
    flipkart debit card emi yes bank,
    flipkart emi on debit card sms,
    flipkart emi bank list,
    flipkart emi process in hindi,
    flipkart emi return policy
    flipkart debit card emi youtube,
    flipkart pay later emi,
    flipkart hdfc emi offer,
    flipkart emi how to pay,
    flipkart hdfc debit card emi eligibility,
    flipkart debit card emi eligibility sbi,
    flipkart debit card emi 2018,
    flipkart emi sbi,
    flipkart emi conditions,
    flipkart emi review,
    flipkart emi payment date,
    flipkart emi helpline number,
    flipkart emi how it works,
    flipkart se emi kaise kare,
    flipkart emi quora,
    flipkart debit card emi quora,
    flipkart emi faq,
    flipkart emi interest rate,
    flipkart emi 60000,,
    flipkart emi without interest,
    flipkart no cost emi quora

    ,

    ReplyDelete