Monday, September 8, 2014

The reasons I stopped using libuv for H2O

Libuv is a great cross-platform library that abstracts various types of I/O by using callbacks.

So when I started writing H2O - a high-performance HTTP server / library implementation with support for HTTP1, HTTP2 and websocket, using libuv seemed like a good idea. But recently, I have stopped using it for sereval reasons. This blog post explains them.

■No Support for TLS

Although libuv provides an unified interface for various types of streams, it does not provide access to a TLS stream though the interface, nor does provide a way for application developers to implement their own types of streams.

The fact has been a disappointment to me. Initially I had created a tiny library called uvwslay that binds libuv streams to wslay - a websocket protocol implementation. But since libuv does not provide support for TLS, it was impossible to support SSL within the tiny library.

To support wss protocol (i.e. websocket over TLS) I needed to reimplement the websocket binding, to stop using libuv as its stream abstraction layer and switch to an abstraction layer defined atop of libuv that abstracts the differences between libuv streams and the I/O API provided by the SSL library being used (in my case it is OpenSSL).

It would be great if libuv could provide direct support for TLS in future (or provide a way to implement their own type of libuv streams), so that code directly calling the libuv API can support various types of streams.

■Memory Usage is not Optimal

Libuv provides a callback-based API for writes and I love the idea. It simplifies the complicatedness of implementing non-blocking I/O operations. However the problem of libuv is that it does not call the completion callback right after write succeeds. Instead, it is only until all I/Os are performed that the callbacks are called. This means that if you have 1,000 connections sending 64KB of data, your code first allocates 64KB of memory 1000 times (64MB in total) and then call free for all of those memory chunks, even when the network operation does not block. IMO, libuv should better call the completion callback immediately after the application returns the control back to the library after calling uv_write, so that the memory allocation pattern could be a repetition of malloc-and-then-free for 1000 times.

■No Support for Delayed Tasks

Libuv does not provide an interface to schedule actions to be run just before the I/O polling method is being called.

Such interface is necessary if you need to aggregate I/O operations and send the results to a single client. In case of HTTP2, a certain number of HTTP requests need to be multiplexed over a single TCP (or TLS) connection.

With lack of support for delayed tasks, you would need to use uv_timer with 0 second timeout to implement delayed tasks.

This works fine if the intention of the application developer is to aggregate read operations, but does not if the intension was to aggregate the result of write operations, since in libuv the write completion callbacks are called after the timers are called.

■Synchronous File I/O is Slow

Synchronous file I/O seems to be slow when compared to directly calling the POSIX API directly.

■Network I/O has Some Overhead

After I found out the issues mentioned above, I started to wonder how much overhead libuv imposes for network I/O.

So I stripped the parts that depended on libuv off from the stream abstraction layer of H2O (that hides the difference between TCP and TLS as described above) and replaced it with direct calls to POSIX APIs and various polling methods (i.e. select, epoll, queue).

The chart below shows the benchmark results taken on my development environment running Ubuntu 14.04 on VMware. H2O become 14% to 29% faster by not depending on libuv.


■The Decision

Considering these facts altogether, I have decided to stop using libuv in H2O, and use the direct binding that I have written and used for the benchmark, at least until the performance penalty vanishes and the TLS support gets integrated.

OTOH I will continue to use libuv for other projects I have as well as recommending it to others, since I still think that it does an excellent job in hiding the messy details of platform-specific asynchronous APIs with adequate performance.

PS. Please do not blame me for not trying to feedback the issues as pull requests to libuv. I might do so in the future, but such task is too heavy for me right now. I am writing this post in the hope that it would help people (including myself) understand more about libuv, and as an answer to @saghul.

29 comments:

  1. Thanks for the detailed analysis. But according to https://github.com/h2o/h2o.git , it seems that h2o is still using libuv, doesn't it? Has libuv fixed the issues you mentioned here, or you are still in the progress to remove it?

    Thanks again!

    ReplyDelete
  2. But the libuv supports TLS:
    http://docs.libuv.org/en/v1.x/threading.html#thread-local-storage

    ReplyDelete
    Replies
    1. TLS = Transport Layer Security

      Delete
    2. TLS = Transport Layer Security

      Delete
  3. What lib are you using or is yours?
    If yours, can it be used as socket lib - for example for RPC?

    ReplyDelete
  4. What alternative do you find to libuv? I am also developing TCP/SSL server that should be handling hundreds of clients. Preferable language are c/c++. Which library can you recommend in order to handle such scenarios? Thanks.

    ReplyDelete
  5. Hi Admin,
    Your website is totally awesome and loaded with various informative posts on web design and development. Keep on updating your blog. Web Design Training | Web designing course in Chennai

    ReplyDelete
  6. Responsive desing can yield a very good revenue to a business. It has been discovered since the usage of multiple devices increase. The content furnished above too tells the same. Thanks for sharing this information in here. Please keep bloging content like this.

    Web designing course in chennai | Web design training in chennai | PHP Training in Chennai

    ReplyDelete
  7. Hi author I actually teach web designing, and after I read this article I was able to clarify a doubt and this helped me understanding a certain concept better and so I could teach my students well. Thank you.
    web designing course in chennai|web designing training in chennai

    ReplyDelete
  8. Really awesome blog. Your blog is really useful for me.
    Thanks for sharing this informative blog. Keep update your blog.
    Oracle Training In Chennai

    ReplyDelete
  9. I am reading ur post from the beginning, it was so interesting to read & i feel thanks to you for posting such a good blog, keep updates regularly.Best Hadoop Training Institute In Chennai

    ReplyDelete
  10. Nice post you have done here. I am really very happy to read this. This is a very informative topic that you are chosen. keep it up Nimble Messaging Servicing

    ReplyDelete
  11. We are offering website design, Application and other much more.....

    Buzz Application

    ReplyDelete
  12. The main thing which i like about web designing is that it needs creativity and we need to work differently acccording to our clients need this needs a creativity and innovation.
    web designing course in chennai|web designing training in chennai|web designing courses in chennai

    ReplyDelete
  13. All the best blogs that is very useful for keeping me share the ideas
    of the future as well this is really what I was looking for, and I am
    very happy to come here. Thank you very much
    earn to die play
    earn to die
    earn to die 3
    Hi! I’ve been reading your blog for a while now and finally got the
    earn to die 4
    courage to go ahead and give youu a shout out from
    earn to die 6
    Austin Texas! Just wanted to tell
    earn to die 5
    you keep up the fantastic work!my weblog
    age of war
    Hi! I’ve been reading your blog for a while now and finally got the
    happy wheels
    tank trouble 3
    slither io
    slitherio
    good game empire

    ReplyDelete
  14. To keep ourselves up to date with the current trend is not an easy task in IT. But we can, through quality and worth able content like this. Thanks for sharing this web page. Please write more articles like this in future.
    PHP training in Chennai | PHP course in Chennai

    ReplyDelete
  15. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it. The angular js programming language is very popular which are most widely used.
    AngularJS Training in Chennai | Angularjs training Chennai

    ReplyDelete
  16. Awesome Post! I like writing style, how you describing the topics throughout the post. I hope many web reader will keep reading your post at the end, Thanks for sharing your view.
    Regards,
    PHP Training in Chennai|PHP Course in Chennai|PHP Training Institute in Chennai

    ReplyDelete
  17. Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    Ethical hacking Course in Chennai | Ethical hacking Training in Chennai

    ReplyDelete
  18. Wonderful post. happy to visit your blog. Thanks for sharing.

    php training institute in chennai

    ReplyDelete
  19. Good Post! Thank you so much for sharing this pretty post, it was so good to read and useful to improve my knowledge as updated one, keep blogging…
    Regards,
    DOT NET Training in Chennai|DOT NET Course in Chennai|DOT NET Training Institute in Chennai

    ReplyDelete
  20. Excellent post!!!. The strategy you have posted on this technology helped me to get into the next level and had lot of information in it.
    Hadoop Training Chennai | PHP Training in Chennai

    ReplyDelete
  21. Wonderful blog.. Thanks for sharing informative blog.. its very useful to me..

    iOS Training in Chennai

    ReplyDelete