Monday, September 8, 2014

The reasons I stopped using libuv for H2O

Libuv is a great cross-platform library that abstracts various types of I/O by using callbacks.

So when I started writing H2O - a high-performance HTTP server / library implementation with support for HTTP1, HTTP2 and websocket, using libuv seemed like a good idea. But recently, I have stopped using it for sereval reasons. This blog post explains them.

■No Support for TLS

Although libuv provides an unified interface for various types of streams, it does not provide access to a TLS stream though the interface, nor does provide a way for application developers to implement their own types of streams.

The fact has been a disappointment to me. Initially I had created a tiny library called uvwslay that binds libuv streams to wslay - a websocket protocol implementation. But since libuv does not provide support for TLS, it was impossible to support SSL within the tiny library.

To support wss protocol (i.e. websocket over TLS) I needed to reimplement the websocket binding, to stop using libuv as its stream abstraction layer and switch to an abstraction layer defined atop of libuv that abstracts the differences between libuv streams and the I/O API provided by the SSL library being used (in my case it is OpenSSL).

It would be great if libuv could provide direct support for TLS in future (or provide a way to implement their own type of libuv streams), so that code directly calling the libuv API can support various types of streams.

■Memory Usage is not Optimal

Libuv provides a callback-based API for writes and I love the idea. It simplifies the complicatedness of implementing non-blocking I/O operations. However the problem of libuv is that it does not call the completion callback right after write succeeds. Instead, it is only until all I/Os are performed that the callbacks are called. This means that if you have 1,000 connections sending 64KB of data, your code first allocates 64KB of memory 1000 times (64MB in total) and then call free for all of those memory chunks, even when the network operation does not block. IMO, libuv should better call the completion callback immediately after the application returns the control back to the library after calling uv_write, so that the memory allocation pattern could be a repetition of malloc-and-then-free for 1000 times.

■No Support for Delayed Tasks

Libuv does not provide an interface to schedule actions to be run just before the I/O polling method is being called.

Such interface is necessary if you need to aggregate I/O operations and send the results to a single client. In case of HTTP2, a certain number of HTTP requests need to be multiplexed over a single TCP (or TLS) connection.

With lack of support for delayed tasks, you would need to use uv_timer with 0 second timeout to implement delayed tasks.

This works fine if the intention of the application developer is to aggregate read operations, but does not if the intension was to aggregate the result of write operations, since in libuv the write completion callbacks are called after the timers are called.

■Synchronous File I/O is Slow

Synchronous file I/O seems to be slow when compared to directly calling the POSIX API directly.

■Network I/O has Some Overhead

After I found out the issues mentioned above, I started to wonder how much overhead libuv imposes for network I/O.

So I stripped the parts that depended on libuv off from the stream abstraction layer of H2O (that hides the difference between TCP and TLS as described above) and replaced it with direct calls to POSIX APIs and various polling methods (i.e. select, epoll, queue).

The chart below shows the benchmark results taken on my development environment running Ubuntu 14.04 on VMware. H2O become 14% to 29% faster by not depending on libuv.


■The Decision

Considering these facts altogether, I have decided to stop using libuv in H2O, and use the direct binding that I have written and used for the benchmark, at least until the performance penalty vanishes and the TLS support gets integrated.

OTOH I will continue to use libuv for other projects I have as well as recommending it to others, since I still think that it does an excellent job in hiding the messy details of platform-specific asynchronous APIs with adequate performance.

PS. Please do not blame me for not trying to feedback the issues as pull requests to libuv. I might do so in the future, but such task is too heavy for me right now. I am writing this post in the hope that it would help people (including myself) understand more about libuv, and as an answer to @saghul.

62 comments:

  1. Thanks for the detailed analysis. But according to https://github.com/h2o/h2o.git , it seems that h2o is still using libuv, doesn't it? Has libuv fixed the issues you mentioned here, or you are still in the progress to remove it?

    Thanks again!

    ReplyDelete
  2. But the libuv supports TLS:
    http://docs.libuv.org/en/v1.x/threading.html#thread-local-storage

    ReplyDelete
    Replies
    1. TLS = Transport Layer Security

      Delete
    2. TLS = Transport Layer Security

      Delete
  3. What lib are you using or is yours?
    If yours, can it be used as socket lib - for example for RPC?

    ReplyDelete
  4. What alternative do you find to libuv? I am also developing TCP/SSL server that should be handling hundreds of clients. Preferable language are c/c++. Which library can you recommend in order to handle such scenarios? Thanks.

    ReplyDelete
    Replies

    1. I have read your blog its very attractive and impressive. I like it your blog.

      Guaranteed SEO services Guaranteed SEO

      Delete
  5. Hi Admin,
    Your website is totally awesome and loaded with various informative posts on web design and development. Keep on updating your blog. Web Design Training | Web designing course in Chennai

    ReplyDelete
  6. Responsive desing can yield a very good revenue to a business. It has been discovered since the usage of multiple devices increase. The content furnished above too tells the same. Thanks for sharing this information in here. Please keep bloging content like this.

    Web designing course in chennai | Web design training in chennai | PHP Training in Chennai

    ReplyDelete
  7. Hi author I actually teach web designing, and after I read this article I was able to clarify a doubt and this helped me understanding a certain concept better and so I could teach my students well. Thank you.
    web designing course in chennai|web designing training in chennai

    ReplyDelete
  8. Really awesome blog. Your blog is really useful for me.
    Thanks for sharing this informative blog. Keep update your blog.
    Oracle Training In Chennai

    ReplyDelete
  9. I am reading ur post from the beginning, it was so interesting to read & i feel thanks to you for posting such a good blog, keep updates regularly.Best Hadoop Training Institute In Chennai

    ReplyDelete
  10. Nice post you have done here. I am really very happy to read this. This is a very informative topic that you are chosen. keep it up Nimble Messaging Servicing

    ReplyDelete
  11. We are offering website design, Application and other much more.....

    Buzz Application

    ReplyDelete
  12. The main thing which i like about web designing is that it needs creativity and we need to work differently acccording to our clients need this needs a creativity and innovation.
    web designing course in chennai|web designing training in chennai|web designing courses in chennai

    ReplyDelete
  13. All the best blogs that is very useful for keeping me share the ideas
    of the future as well this is really what I was looking for, and I am
    very happy to come here. Thank you very much
    earn to die play
    earn to die
    earn to die 3
    Hi! I’ve been reading your blog for a while now and finally got the
    earn to die 4
    courage to go ahead and give youu a shout out from
    earn to die 6
    Austin Texas! Just wanted to tell
    earn to die 5
    you keep up the fantastic work!my weblog
    age of war
    Hi! I’ve been reading your blog for a while now and finally got the
    happy wheels
    tank trouble 3
    slither io
    slitherio
    good game empire

    ReplyDelete
  14. To keep ourselves up to date with the current trend is not an easy task in IT. But we can, through quality and worth able content like this. Thanks for sharing this web page. Please write more articles like this in future.
    PHP training in Chennai | PHP course in Chennai

    ReplyDelete
  15. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it. The angular js programming language is very popular which are most widely used.
    AngularJS Training in Chennai | Angularjs training Chennai

    ReplyDelete
  16. Awesome Post! I like writing style, how you describing the topics throughout the post. I hope many web reader will keep reading your post at the end, Thanks for sharing your view.
    Regards,
    PHP Training in Chennai|PHP Course in Chennai|PHP Training Institute in Chennai

    ReplyDelete
  17. Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    Ethical hacking Course in Chennai | Ethical hacking Training in Chennai

    ReplyDelete
  18. Wonderful post. happy to visit your blog. Thanks for sharing.

    php training institute in chennai

    ReplyDelete
  19. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…
    Regards,
    DOT NET Training in Chennai|DOT NET Course in Chennai|DOT NET Training Institute in Chennai

    ReplyDelete
  20. Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    Hadoop Training Chennai | PHP Training in Chennai

    ReplyDelete
  21. Wonderful blog.. Thanks for sharing informative blog.. its very useful to me..

    iOS Training in Chennai

    ReplyDelete
  22. Super Bowl LI will be the 51st Super Bowl and the 47th modern-era National Football League. The Touchdown Tour is a way to excite the Houston community about the 2016-2017 football season culminating in Super Bowl LI.
    Super Bowl 51
    Super Bowl 51 Live
    Super Bowl 51 Live Stream
    Watch Super Bowl 51
    Watch Super Bowl 51 Live
    Super Bowl 51 Live Online

    ReplyDelete
  23. Thanks for this great post, i find it very interesting and very well thought out and put together. I look forward to reading your work in the future
    شركة نقل اثاث بالخبر
    ارخص شركة نقل اثاث بالدمام
    شركة نقل الاثاث بالدمام
    شركة نقل عفش بالدمام
    شركات نقل العفش والاثاث بالدمام والخبر على وجه الخصوص تعددت كثيرا ولكن القليل من شركات نقل الاثاث داخل الدمام والخبر ما زال يقدم الخدمة المنوط بها للعميل فنحن فى شركة نقل اثاث بالدمام وبالخبر افضل شركات نقل اثاث بالدمام والخبر والقطيف والجبيل على مستوي المملكه العربية السعودية وخاصة منها المنطقة الشرقية ونحن ارخص شركة نقل اثاث بالدمام وبالخبر

    ReplyDelete
  24. This is the right blog for anyone who wants to find out about this topic. You realize so much its almost hard to argue with you (not that I actually would want…HaHa). You definitely put a new spin on a topic thats been written about for years.
    شركة نقل اثاث بالقطيف
    شركة نقل اثاث بالدمام
    شركة نقل اثاث بالدمام
    شركة نقل عفش بالدمام
    شركة نقل اثاث بالجبيل
    شركة نقل العفش بالخبر

    ReplyDelete
  25. Pretty good post. I just stumbled upon your blog and wanted to say that I have really enjoyed reading your blog posts. Any way I’ll be subscribing to your feed and I hope you post again soon.
    شركة نقل اثاث بالقطيف
    نقل اثاث بالدمام رخيص

    ReplyDelete
  26. Hey – great blog, just looking around some blogs, seems a really nice platform you are using. I’m currently using WordPress for a few of my blogs but looking to change one of them over to a platform similar to yours as a trial run. Anything in particular you would recommend about it?
    ارخص شركة نقل اثاث بالدمام
    شركة نقل عفش بالدمام
    شركة نقل اثاث بالخبر
    شركة نقل اثاث بالدمام
    وفر شركة شامكو –0593757728 – ارخص شركة نقل اثاث بالجبيل وكذلك نقل عفش بالخبر ونقل اثاث بالدمام والقطيف العمالة المدربة على فك وتركيب العفش والاثاث بكل احترافية دون أي اخطاء لضمان عدم اتلاف أي قطعة اثاث او الحاق الضرر بها, بالإضافة الى توفير العمالة الخاصة بعملية نقل العفش والاثاث

    ReplyDelete
  27. thank you so much for this very usefull information. It's just a very effectively written article. It will likely be useful to anyone who makes use of it
    I was very pleased to find this site.I wanted to thank you for this great read!! I definitely enjoying every little bit of it and I have you bookmarked to check out new stuff you post
    شركة تنظيف بالخبر
    شركة تنظيف بالجبيل
    شركة تنظيف بالدمام
    شركة تنظيف بالقطيف

    ReplyDelete
  28. I really enjoyed reading this post, big fan. Keep up the good work andplease tell me when can you publish more articles or where can I read more on the subject?
    شركة تنظيف بالجبيل
    شركة تنظيف بالدمام
    شركة تنظيف بالخبر
    شركة تنظيف بالقطيف
    شركة تنظيف بالخبر لتقديم المستوي الراقي جدا فاننا نقوم بخدمات التنظيف الداخلي والخارجي باي منزل فداخليا نهتم بتنظيف الأبواب بكل ما يتعلق بها من أتربه وملوثات واتساخات ونزيل كل هذا علي الفور  ونقوم أيضا بأعمال تنظيف الشبابيك وما يتعلق بها من أتربه ونقوم أيضا بأعمال تنظيف شامله للأرضيات والسيراميك والرخام والبلاط ونقدم دائما الخدمة بأسلوب راقي اما خارجيا نقوم بأعمال تنظيف المسابح وتنقيه المياه بداخلها ونقوم أيضا بتنظيف الخزانات بما يضمن لك الراحة وعدم التفكير فى الاستعانه بشركات النظافة بالخبر والدمام

    ReplyDelete
  29. I really appreciate the kind of topics you post here. Thanks for sharing us a great information that is actually helpful
    Its my great pleasure to visit your blog and to enjoy your great posts here. I like it a lot. I can feel that you paid much attention for those articles, as all of them make sense and are very useful
    شركة مكافحة الحشرات بالقطيف
    شركة مكافحة النمل الابيض بالقطيف
    شركة مكافحة الحشرات بالدمام
    شركة مكافحة الحشرات بالخبر
    شركة مكافحة البق بالقطيف

    ReplyDelete
  30. شركة مكافحة البق بالدمام
    شركة مكافحة النمل الابيض بالدمام
    شركة مكافحة النمل الابيض بالخبر
    هتم شركة مكافحة الحشرات بالدمام باعمال المكافحة للصراصير بالدمام فالصراصير من اكثر انواع الحشرات التى تسبب الضيق والالم وتسبب الشعور بالاشمئزاز والقرف فى المكان فاذا كنت تعانى من وجود الصراصير وتعانى من وجودها وانتشارها فى اى مكان فى منزلك وبالاخص فى المطابخ والصراصير الصغيره المتواجدة فى المكان
    فعليك ان تعلم ان شركة مكافحة الحشرات بالدمام  هى افضل الشركات التى تقوم باعمال المكافحة والقضاء على الصراصير تماما من خلال احدث الاجهزة والمبيدات والايدى العامله المميزه التى تساعد فى اعمال المكافحة والرش للصراصير فشركة مكافحة حشرات بالدمام هى الحل الامثل فى لقضاء على الصراصير

    ReplyDelete
  31. Biggest Awards Oscars 2017 Live Stream Online Watch Full Show Coverage ,Time, Date, Red Carpet HD Video In Any Device.Get the latest news about the 2017 Oscars, including nominations, winners, predictions and red carpet fashion at 89th Academy Awards Oscar.com.

    Oscar 2017
    Oscars 2017
    Oscars 2017 Live
    Oscars 2017 Live Stream
    Watch Oscars 2017

    ReplyDelete
  32. Idaho: Despite the fact that this Tar High heel sandals lost inside the ACC contest semifinals, on the list of Duke relating to the ropes for high of the first half. Justin Jackson, with produced your experienced circumference sport, and then a foul-free Fran Berry II include the recommendations.
    March Madness

    March Madness Live

    March Madness Live Stream

    March Madness 2017

    Without a doubt, we all know Amount Number of hasn't constructed one final 3. That organization, stocked with proficient scorers that can as well lck straight down competitors, can be your partner's primary
    March Madness 2017 Live

    ncaa March Madness

    ncaa march madness live

    ncaa tournament

    March Madness Bracket

    ncaa final four

    Irrespective of whether you’re nonetheless over the containment system about that 8-9 gameplay, and are trying to find a total impression on your Finalized Several randomly chooses, we’ve got everyone included using a standing of all 68 matchups inside competition. You can now confidently get your choice upon those mid-major darlings you’ve recently been eyeing with regard to disappointed probable.
    March Madness

    March Madness Live

    March Madness Live Stream

    March Madness 2017

    March Madness 2017 Live

    ncaa March Madness

    ncaa march madness live

    ncaa tournament

    March Madness Bracket

    ncaa final four

    Villanova: The very best all around seedling. Quick grown timbers . team’s location inside most ruthless region (the Far east, using matchups next to Duke together with SMU looming), Villanova’s three-man a particular problem key associated with Jalen Brunson, Josh Hart along with Kris Jenkins offers the potential to repeat like domestic champs.

    ReplyDelete
  33. This time, we head to Birmingham, Alabama to check out UFC 10 from July 1996, which brings back the eight-man one-night tournament format. Barely a decade ago, the Ultimate Fighting Championship was looked at as little more than a freak show a human cockfight that drew the attention.
    UFC 210
    UFC 210 Live , UFC 210 Fight Card , UFC 210 Fight , UFC 210 Card , UFC 210 Live Stream , UFC 210 PPV



    UFC 10 Review: Mark Coleman Enters The Octagon For. Joel AbrahamContributor IIIDecember 24, 2010. I am still having nightmares about UFC 10. FS1 presents the Ultimate Fighting Championship's top pound-for-pound fighters across two weeks of prime time action with the UFC 10 Day.
    UFC 210
    UFC 210 Live , UFC 210 Fight Card , UFC 210 Fight , UFC 210 Card , UFC 210 Live Stream , UFC 210 PPV

    ReplyDelete
  34. Thanks To Watch The Masters 2017 Live Stream, one of the most prestigious tournaments in golf, is set to take place this week at Augusta National Golf Club from April 6-9.

    Masters 2017

    The Masters 2017

    Masters 2017

    The Masters 2017

    Masters Golf

    UFC 210

    UFC 210

    We already know Tiger Woods won’t be Masters 2017 Live participating in the 2017 Masters, but there are still intriguing questions entering this year’s tournament.

    ReplyDelete
  35. A week after Chavez vs Canelo went toe-to-toe in an instant classic, boxing fans get another star-studded fight when Saul “Canelo” Alvarez (48-1-1, 34 KOs) and Julio Cesar Chavez Jr. (50-2-1, 32 KOs) meet at the T-Mobile Arena in Las Vegas.
    Canelo vs Chavez Jr Live

    Chavez Jr vs Canelo Live

    Chavez Jr vs Canelo

    Canelo vs Chavez Jr Live

    ReplyDelete
  36. Are you going to watch the Kentucky Derby 2017 live online via app, on TV or in person? You have the Kentucky Derby games? Do not hesitate to leave your comments, reflections and opinions in the next section.

    Kentucky Derby 2017

    2017 Kentucky Derby

    Kentucky Derby 2017 Live

    Ky Derby 2017

    Canelo vs Chavez

    Chef Bobby Flay will not be the only one who promotes delicious food at the Derby. Tasty BuzzFeed and have partnered with NBC. BuzzFeed publishing live and tasty Derby stories has created a series of videos for those looking for the perfect Derby party dish. Watch the video for the tasty chocolate bourbon balls perfect for any part of the Kentucky Derby.

    ReplyDelete