Monday, December 15, 2014

PicoHTTPParser now has a chunked-encoding decoder

Today I have added phr_decode_chunked - a function for decoding chunked-encoded input - to picohttpparser.

As suggested in the doc-comment of the function (shown below), the function is designed to decode the data in-place. In other words, it is not copy-less.
/* the function rewrites the buffer given as (buf, bufsz) removing the chunked-
 * encoding headers. When the function returns without an error, bufsz is
 * updated to the length of the decoded data available. Applications should
 * repeatedly call the function while it returns -2 (incomplete) every time
 * supplying newly arrived data. If the end of the chunked-encoded data is
 * found, the function returns a non-negative number indicating the number of
 * octets left undecoded at the tail of the supplied buffer. Returns -1 on
 * error.
 */
ssize_t phr_decode_chunked(struct phr_chunked_decoder *decoder, char *buf,
                           size_t *bufsz);
It is intentionally designed as such.

Consider a input like the following. The example is more than 2MB long even though it contains only 2 bytes of data. The input is conformant to the HTTP/1.1 specification since it does not define the maximum length of the chunked extensions, requires every conforming implementation to ignore unknown extensions.
1 very-very-veery long extension that lasts ...(snip) 1MB
a
1 very-very-veery long extension that lasts ...(snip) 1MB
a
To handle such input without getting the memory exhausted, a decoder should either a) only preserve the decoded data (requires a copy), or b) limit the size of the chunked-encoded data.

B might have been easier to implement, but such a feature might be difficult to administer. So I decided to take the route a, and for simplicity implemented the decoder to always adjust the position of the data in-place.

Always calling memmove for adjusting the position might induce some overhead, but I assume it to be negligible for two reasons: both the source and destination would exist in the CPU cache / the overhead of unaligned memory access is small on recent Intel CPU.

For ease-of-use, I have added examples to the README.

35 comments:

  1. I read your blog.I thought it was great.. Hope you have a great day. God bless.

    Rica
    www.imarksweb.org

    ReplyDelete
  2. I like this a lot. Thank you for sharing. I'm always looking for upcycles like this. In the end, you don't know it was a shipping pallet to begin with!
    ICC T20 World Cup 2016 Venues & Starting

    Date

    Australia vs India Prediction

    ReplyDelete

  3. شركة نقل عفش بالرياض وجدة والدمام والخبر والجبيل اولقطيف والاحساء والرياض وجدة ومكة المدينة المنورة والخرج والطائف وخميس مشيط وبجدة افضل شركة نقل عفش بجدة نعرضها مجموعة الفا لنقل العفش بمكة والخرج والقصيم والطائف وتبوك وخميس مشيط ونجران وجيزان وبريدة والمدينة المنورة وينبع افضل شركات نقل الاثاث بالجبيل والطائف وخميس مشيط وبريدة وعنيزو وابها ونجران المدينة وينبع تبوك والقصيم الخرج حفر الباطن والظهران
    شركة نقل عفش بجدة
    شركة نقل عفش بالمدينة المنورة
    شركة نقل اثاث بالرياض
    شركة نقل عفش بالدمام
    شركة نقل عفش بالطائف

    ReplyDelete
  4. This is very interesting, You’re a very skilled blogger. I have joined your feed and look forward to seeking more of your fantastic post. Also, I have shared your web site in my social networks!
    Regards - www.office.com/setup
    www.office.com/setup

    ReplyDelete
  5. This comment has been removed by the author.

    ReplyDelete
  6. I'm shakshi.I am working in India top most Escort service Mumbai Call girls.If u want to join the all facility of escorts Puri call girls
    pls call me & whatssap Cuttack call girls
    visit the sites. Bhubaneswar call girls or follow me. jaipur call girls

    ReplyDelete

  7. Hi friends ! I'm Archana Kumari. I like to have bed relationship with different persons rather than with a single man for the whole of life. Not only is this my notion , but also is this fact that the secret organs are made for enjoyment.
    Please follow me on the links given bellow.


    Mumbai Escorts
    Mumbai Call girls
    Escorts in Kolkata

    Escorts in Kolkata
    Escorts In kolkata
    Jaipur Call girls


    Kolkata Escorts
    Call girl in Kolkata
    House wife escorts in Kolkata
    If you are interested in me and my ideology ,then you are invited to my bed. Thanks.
    ========================================================

    ReplyDelete
  8. This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information.

    www.office.com/setup
    www.norton.com/setup
    office.com/setup

    ReplyDelete
  9. Hey Thanks for sharing this valuable information with us. I will come back to your site and keep sharing this information with us.
    Best Regards - www.office.com/setup
    www.office.com/setup
    www.office.com/setup

    ReplyDelete
  10. This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information.

    www.office.com/setup
    www.norton.com/setup
    office.com/setup

    ReplyDelete
  11. This is a great inspiring article.I am pretty much pleased with your good work.You put really very helpful information.

    www.office.com/setup
    www.norton.com/setup
    office.com/setup

    ReplyDelete
  12. Webroot.com/safe is a protection software solution that communicates with the cloud avoiding the hassle to manage the signature updates to deploy. for office setup visit office.com/setup

    ReplyDelete
  13. Sample Assignment is the leading brand in Australia and know for their work that offers complete online assignment services like contract law assignment, it assignment help, financial assignment help, project management assignment, electrical engineering help and more. Those students who looking for assignment help  Melbourne, Adelaide, Perth etc

    ReplyDelete

  14. La veilleuse coranique bluetooth avec sa télécommande pour offrir.
    Cadeau ramadam idéal
    La veilleuse coranique personnalisée pas cher
    Veilleuse coranique pas cher
    Veilleuse coranique pas cher

    Découvrez La veilleuse coranique Munawara
    Video de la Veilleuse coranique munawara
    Veilleuse coranique munawara


    Je travailles sur un projet de fabrication de cornes de gazelle personnalisée
    cornes de gazelle marocaine
    cornes de gazelle algerienne
    Merci de laisser ce lien c'est sympa...

    Le casque vapeur hair steamer permet de lutter contre la sécheresse, la chute des cheveux et leur mauvaise santé , dans le confort de votre domicile. Le hair steamer est un casque vapeur qui apporte une dose d'hydratation pour les cheveux crépus.
    Hair steamer vapohair
    Lee hair steamer casque vapeur est recommendé par fes femmes aux cheveux crépus
    Casque vapeur
    La casque vapeur hair steamer apporte beaucoup de bienfait au cheveux crépus de type afro Hair.
    hair steamer Casque vapeur hydratation cheveux crépus
    hair steamer casque vapeur

    ReplyDelete
  15. What a way to express things. I have found comfort in reading this post and i am sure other users may feel the same, because this post carries a social message in it. Keep it up ! Regards:-
    office.com/setup | norton.com/setup | mcafee.com/activate

    ReplyDelete
  16. Thanks for sharing the valuable information. In continuation of your excellent web post, I want to tell about a website (My Assignment Services) that provides assignment help services to students at cost effective prices. Whatever be your assignment, essay, reports, dissertation, their experts are sources of every problem that comes your way. They are not just limited to a single domain, instead they will guide you in every assignment related to nursing assignment help, management, engineering, nursing, law, and much more.
    My Assignment Services has been consistently delivering and fulfilling the demand of assignment across the world with their assignment help melbourne, Australia, UK, Malaysia, USA, Russia, Canada, UAE, South Africa, Vietnam, Singapore, etc. Whenever it comes to quality assignment help services at affordable prices with satisfied customer satisfaction rating, My Assignment Services would be a good name to call.

    ReplyDelete

Note: Only a member of this blog may post a comment.