Thursday, July 23, 2015

前方秘匿性 (forward secrecy) をもつウェブサイトの正しい設定方法

前方秘匿性(forward secrecy)とは、以下のような性質を指します。

公開鍵暗号の秘密鍵のように、比較的長期に渡って使われる鍵が漏えいしたときでも、それまで通信していた暗号文が解読されないという性質
鍵が漏れることも想定せよ――クラウド時代における「楕円曲線暗号」の必然性 - @IT

鍵が攻撃者や諜報機関など第三者の知るところとなった場合でも、それまで通信していた暗号文が解読されないようにしないといけない、という考え方とともに、最近 HTTPS を利用するウェブサイトにおいても導入が求められるようになってきた概念です。

前方秘匿性を満たすウェブサイトの設定方法については、TLSの暗号化方式をECDH_RSAあるいはECDHE_RSAに設定すれば良い、と述べている文献が多いです。

ですが、ほとんどのウェブサーバにおいて、それは誤りです。

なぜか。

通信を暗号化する鍵(セッション鍵)の確立にDHEあるいはECDHEを用いれば前方秘匿性が確保されるというのは、鍵交換に議論を限ってしまえば正しいです。

しかし、TLSプロトコルには、鍵交換以外にセッション鍵を保存し、再利用する仕組みが存在します。Session ResumptionとSession Ticketの2つがそれに該当します。

このうち、OpenSSLが標準でサポートし、ウェブブラウザが対応している場合(IE9以前とMobile Safari以外の全ての主要ウェブブラウザが該当します)にSession Resumptionより優先して利用されるSession Ticketは、セッション鍵をサーバのみが知るマスターキーで暗号化して、クライアントに送信します。

セッション鍵をマスターキーで暗号化…いやな響きがしませんか?

そうです。このSession Ticketの暗号化に使われるマスターキーを定期的に更新しない限り、マスターキーが流出してしまった場合、過去の通信内容がすべて解読されてしまうことになるのです。

しかし残念なことに、ほとんどのウェブサーバは、マスターキーを自動的に更新する仕組みをそなえていません

では、どのようにサーバを設定すればいいでしょうか。

解決策1. Session Ticket機能を無効化する

第一の解決策はウェブサーバの設定でSession Ticket機能を無効化することです。Apache の場合は SSLSessionTickets、nginx の場合は ssl_session_tickets という設定コマンドを使うことで、無効化することができます。

しかし、nginx (とevent mpmを使用するApache)は、複数サーバにまたがる session resumption 機能を実装していません。そのため、ロードバランサの背後に複数のHTTPSサーバを立てている場合は、session ticket をオフにすると、新規接続確立のたびにTLSハンドシェイクが発生し、ユーザの体感速度の低下やサーバの負荷増大につながってしまいます。

解決策2. サーバを定期的に再起動する

ApacheやNginxは、サーバを起動するタイミングで Session Ticket のマスターキーを再生成します。なので、たとえば1時間に1回サーバを再起動すれば、前方秘匿性を確保しつつ Session Ticket を使うことができます。簡単ですね!!!

なお、ロードバランサの背後に複数のHTTPSサーバを立てている場合は、それらのサーバを再起動する度に、新しいマスターキーが記述された設定ファイルを再配布する必要があります。このへんは Twitter さんの涙ぐましい努力の話があるので、それを参考にがんばりましょう。

解決策3. H2O を使う

これまで述べてきたように、これまでのウェブサーバの前方秘匿性のサポートはお粗末と言わざるをえません(他にも、マスターキーの強度が強力なセッションキーよりも弱いといった問題も存在します)。

そこで H2O ですよ!!!!

私たちが開発しているH2Oの最新バージョンは、これらの問題を全て解決しています。

具体的に言うと、

・Session Ticket のマスターキーは、他のサーバのように AES-128 ではなく AES-256 です
・Session Ticket は自動的に更新されます(デフォルト15分間隔)
・ロードバランサの背後に複数台のH2Oを設置する場合、Session Ticketをmemcachedで共有するように設定することが可能です
・ついでに、Memcached を用いた複数サーバにまたがるSession Resumptionにも対応しているので、Mobile Safari でのHTTPSアクセスも高速に処理することができます

というわけで、安全なウェブサイトを構築したい人は H2O 使えばいいんじゃないでしょうか。ちなみに、H2O 使うとユーザの体感速度も速くなるのでいいことづくめかと思います。

以上、ウェブサーバの設定に関する注意喚起、兼、H2Oの宣伝でした。

Wednesday, July 22, 2015

H2O version 1.4.0 released with outstanding support for forward secrecy and load balancing (and the experimental mruby handler)

Today I am happy to announce the release of the H2O HTTP/2 server version 1.4.0.

There have been a few changes and bug fixes from version 1.3.1 (that showed big performance improvements over the older generations of HTTP servers without support for request prioritization), but the most prominent ones are the following.

Support for the PROXY protocol

The PROXY protocol is a de-facto standard protocol used by L4 load balancers (such as AWS Elastic Load Balancing) to notify the web servers running behind them the IP addresses of the clients. Support for the protocol is essential for running a web server behind such load balancer; without the support it is impossible to log the address of the client or to work against attacks.

In version 1.4.0, we have added support for the protocol which makes H2O a good choice for large scale and/or highly-available web sites running multiple HTTP servers behind a load balancer.

Support for cache-based and ticket-based TLS session resumption using Memcached (and forward secrecy)

The PROXY protocol is not the only thing that now makes H2O a good choice for such websites.

When running a HTTPS server cluster behind a L4 load balancer, it is desirable that the server supports session resumption using a shared datastore such as memcached.

TLS Session Resumption: Full-speed and Secure is a good read for those who are interested in what session resumption is; in short, it reduces the time spent for establishing a TLS connection to about half, and also reduces the CPU time to below 10%!

However, until now, front-end HTTP servers have been not good at supporting session resumption using a shared datastore.

Among the two resumption methods, Nginx does not support the more-widely-deployed cache-based session resumption using a shared datastore.

It should also be noted that most web servers are not good at supporting ticket-based resumption; they use 128-bit AES for storing master secrets (even in cases when a more complex ciphersuite is used), and also do not automatically roll-over the secrets (which botches forward-secrecy).

As pointed out by Tim Taubert, the sad state of the server-side TLS session resumption implementations has been the headache of administrators trying to setup secure websites. Forward Secrecy at Twitter is an example that shows how difficult it is to configure a website supporting forward-secrecy.

Being the primary developer of H2O, I believe that web servers should be easily configurable to be secure; so in version 1.4.0 we have implemented the following features:

  • cache-based session resumption using memcached
  • automatic rollover of master secret used for ticket-based resumption
  • synchronization of master secrets that rollover, using memcached
  • directive to configure the cipher used for encrypting tickets (with default being aes-256-cbc)

Table 1.Supported Methods of Session Resumption
Resumption Method
Session ResumptionSession Ticket
Apache (prefork)yesno forward-secrecy (AES-128)
Apache (worker)yesno forward-secrecy (AES-128)
Apache (event)not sharableno forward-secrecy (AES-128)
Nginxnot sharableno forward-secrecy (AES-128)
Varnish (hitch)needs recompileno
H2Oyesyes (AES-256)

And with H2O, they are easy to use! A simple configuration like below activates all the features. The H2O server cluster will share information of both cache-based and ticket-based session resumption using memcached, with complex cipher used for protecting master secrets that are automatically rolled over.

listen:
  port: 443
  ssl:
    key-file: /path/to/key-file
    certificate-file: /path/to/certificate-file
  proxy-protocol: ON
ssl-session-resumption:
  method: all
  memcached:
    host: address.of.memcached.server
    port: 11211

Please refer to the documentation for the details of the configuration directive.

Experimental mruby Handler

We are also proud to announce that we now have a scripting engine running within the H2O standalone server that can be used to customize the behavior, and that the programming language is Ruby.

Developed by Yukihiro Matz (the father of the Ruby programming language) and others, mruby is an implementation of the language for embedded use. Thanks to MATSUMOTO Ryosuke the language runtime can now be used to script how the HTTP requests should be handled within H2O.

The handler is still in very early stages and considered unstable (therefore it is not turned on by default, you would need to pass -DWITH_MRUBY=ON as an argument to CMake to build H2O with support for the mruby handler), but nevertheless it is already a great addition to the H2O HTTP server; such a scripting engine gives you great flexibility to customize the behavior of the server depending on the tiny aspects of a HTTP request, or to mitigate attacks.

Please refer to Ryosuke's weblog for more information (in Japanese). In addition to topics related to H2O, you can find excellent entries about how to use a scripting engine within web servers to work against cyber attacks.

Conclusion

All in all, we are happy to provide a new release of the H2O server to the public, that is secure and easy to use (with flexibility), once again raising the bar of what people should expect on a HTTP server to provide.

I hope you enjoy using the new release of H2O.

Thursday, June 18, 2015

H2OとPHPを組み合わせるの、超簡単です(もしくはmod_rewriteが不要な理由)

FastCGI対応機能がH2Oにマージされたことを受けて、uzullaさんが「H2OでPHP(がちょっとだけ動くまで)」という記事を書いてくださっています。

ありがたやありがたや。

その中で、
http://hoge/entry/1 みたいなのをphpにマップする方法はまだよくわかってません。その内しらべます

github.comを読む限り
FastCGI (or PHP) applications should be as easily configurable as it is for the Apache HTTP server
ということで、やったぜ!ってなるんですけど、nginxはもとより、Apacheにおいても現状ルーターをつかっているようなアプリだとhtaccessをいちいちかかないといけないので、Apacheみたいなスタイルが楽なのか?というとちょっと疑問があります。

(たとえば以下みたいなの)
RewriteEngine On
RewriteCond %{REQUEST_FILENAME} !-f
RewriteRule ^ index.php [QSA,L]
ここらへんがスッっとかけないと、まあApache、nginx同様にH2Oの設定のタレが必要になるかなあ、という感じはあります。
いやはや、まったくそのとおりなわけです。

ちなみにこの手のことをNginxでやろうとするともっと難しくて、以下のようにtry_filesfastcgi_split_path_infoを使わないと脆弱性が発生するなんて話があるそうです(参照: Setting up PHP-FastCGI and nginx? Don’t trust the tutorials: check your configuration! » Neal Poole)。これ、みんなちゃんとできてるんですかね?
# Pass all .php files onto a php-fpm/php-fcgi server.
location ~ \.php$ {
   try_files $uri =404;

   fastcgi_split_path_info ^(.+\.php)(/.+)$;
   include fastcgi_params;
   fastcgi_index index.php;
   fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
   fastcgi_pass php;
}

こんな難しい設定、書けるわけないだろうが!!!!!!!

というわけで、H2Oの場合はもっと簡単に設定できるようになっています。以下のような感じ。
paths:
  "/":
    # /path/to/doc-root以下の静的ファイルを返す(存在した場合)
    file.dir: /path/to/doc-root
    # 存在しなければ、/index.php/ に内部転送
    redirect:                     
      url: /index.php/
      internal: YES
      status: 307

単純ですね。どうなってるかはコメントを見れば自明かとも思いますが、以下解説します。

実は、H2Oの設定ファイルにおいては、1つのパスに複数のハンドラを設定することができるのです。

複数のハンドラが設定されている場合、H2O は有効なレスポンスが生成できるまで、ハンドラを順次実行していきます。つまり、コメントにあるように、ファイルが存在すればそれを返すし、存在しなければ /index.php/ 以下にリクエストを内部的に転送し、その転送した結果の応答をクライアントに返すわけです。

わかりやすいですね。簡単ですね。


少し理屈っぽい話をすると、mod_rewriteのようなリクエストを書き換える黒魔術は、おおむね3種類の目的に使われてきました。第1の目的は、複数のローカルのディレクトリツリーを重ね合わせてひとつのhttpサイトとして表示すること。第2の目的は、uzullaさんが指摘したような、特定の条件をもつ以外のURLの処理を、FastCGIやアプリケーションサーバに委譲すること。第3の目的は、User-Agentや言語設定によって異なる応答を返すこと。

ですが、前2者を実現するためだけならば、正規表現を用いてURLを書き換えるような手法は過剰にすぎて管理がしづらかったわけです。H2OにおけるPHP対応においては、この点を意識し、複数のハンドラの連鎖を可能にすることで、設定が安全簡潔にできるようにしたのでした。

ちなみに、uzullaさんの例ではphp-fpmを使っていますが、H2Oはphp-cgiを自分で管理することができるので、小規模なウェブサイトにおいては、
file.custom-handler:          
  extension: .php
  fastcgi.spawn: "PHP_FCGI_CHILDREN=10 exec /usr/bin/php-cgi"
のように書くと、別途FastCGIデーモンを管理する必要がなくなって、より幸せになれるでしょう(名前はphp-cgiでも、この設定においてはFastCGIとして動作します)。


最後になりますが、ここで説明したFastCGI対応機能を盛り込んだH2Oバージョン1.3は今朝方リリースされたので、PHPerの皆様もそれ以外の皆様も、お試しいただければ幸甚に存じます。

リンク先のリリース告知エントリにも書いてあるけど、他のウェブサーバより表示速度(first-paint time)が、かなり速いですよ!

H2O HTTP/2 server version 1.3.0 released; provides faster response to user, adds support for FastCGI, range-request

Today we are happy to announce the release of H2O version 1.3.0. The new release includes many changes and bug fixes since 1.2.0, but the notable ones are as follows.

Faster response to user

It is known that providing faster response to users greatly improve their experience, and many web engineers work hard on optimizing the time spent until user starts seeing the contents. The theoretically easiest way to optimize the speed (without reducing the size of the data being transmitted) is to transfer essential contents first, before transmitting other data such as images.

With the finalization of HTTP/2, such approach has become practical thanks to it's dependency-based prioritization features. However, not all web browsers (and web servers) optimally prioritize the requests. The sad fact is that some of them do not prioritize the requests at all which actually leads to worse performance that HTTP/1.1 in some cases.

Since the release of version 1.2.0, we have conducted benchmark tests that measure first-paint time (time spent until the web browser starts rendering the new webpage), and have added a tuning parameter that can be turned on to optimize the first-paint time of web browsers that do not leverage the dependency-based prioritization, while not disturbing those that implement sophisticated prioritization logic.

The chart below shows the first-paint time measured using a virtual network with 100ms latency (typical for 4G mobile networks), rendering a web page containing jquery, CSS and multiple image files.

First-Paint Time (network latency: 100ms)

It is evident that the prioritization logic implemented in H2O and the web browsers together offer a huge reduction in first-paint time. As the developer of H2O, we believe that the prioritization logic implemented in H2O to be the best of class (if not the best among all), not only implementing the specification correctly but also for having practical tweaks to optimize against the existing web browsers.

In other words, web-site administrators can provide better (or the best) user-experience to the users by switching their web server to H2O. For more information regarding the topic, please read HTTP/2 (and H2O) improves user experience over HTTP/1.1 or SPDY.

Version 1.3.0 also supports TCP fast open, an extension to TCP/IP that reduces the time required for establishing a new connection. The extension is already implemented in Linux (and Android), and is also expected to be included in iOS 9. As of H2O version 1.3.0 the feature is turned on by default to provide even quicker user experience. Kudos go to Tatsuhiko Kubo for implementing the feature.

FastCGI support

Since the initial release of H2O many users have asked for the feature; it is finally available! And we are also proud that it is easy to use.

First, it can be configured either at path-level or extension-level. The latter means that for example you can simply map .php files to the FastCGI handler without writing regular expressions to extract PATH_INFO.

The second is the ability to launch FastCGI process manager under the control of H2O. You do not need to spawn an external FastCGI server and maintain it separately.

Using these features, for example Wordpress can be set-up just by writing few lines of configuration.
paths:
  "/":
    # serve static files if found
    file.dir: /path/to/doc-root
    # if not found, internally redirect to /index.php/...
    redirect:                     
      url: /index.php/
      internal: YES
      status: 307
    # handle PHP scripts using php-cgi (FastCGI mode)
    file.custom-handler:          
      extension: .php
      fastcgi.spawn: "PHP_FCGI_CHILDREN=10 exec /usr/bin/php-cgi"
Of course it is possible to configure H2O to connect to FastCGI applications externally using TCP/IP or unix sockets.

Support for range-requests

Support for range-requests (HTTP requests that request a portion of a file) is essential for serving audio/video files. Thanks to Justin Zhu it is now supported by H2O.

Conclusion

All in all, H2O has become a much better product in version 1.3 by improving end-user experience and by adding new features.

We plan to continue improving the product. Stay tuned!

Wednesday, June 3, 2015

HTTP/2 (and H2O) improves user experience over HTTP/1.1 or SPDY

HTTP/2 is expected to offer better user experience than HTTP/1.1, the unanswered question is how much the benefit is in practice.

Tonight I have given a presentation regarding the issue, showing HTTP/2 performance of H2O HTTP server at shibuya.pm, a popular technology meetup at Tokyo. This blog post is a summary of the presentation at the meetup, following my recent blog post about the prioritization logic of HTTP/2 and web browser implementations.

Test Scenario:
  • a modified version of http2rulez.com (<script> tags moved into <head>)
    • contains 5 CSS files
    • 8 script files in <head>, including a minified version of jquery.js
    • contains 18 not-blocking assets (e.g. images files), both small and large
  • client: Chrome/43 and Firefox/38
  • server: Nginx/1.9.1 and H2O/HEAD
  • network: dedicated virtual network to avoid noise; 100ms latency (typical for 4G mobile) added artificially using tc disc command

Benchmark Results:


Analysis:

When using H2O (and HTTP/2) with optimal settings, the first-paint times are reduced by about 30% when compared to HTTP/1.1 or SPDY/3.1.

First-paint time is a good index of user experience; it shows the timing when the user sees the page being rendered for the first time (or when all assets that block rendering arrives at the client side) since he/she triggered the navigation.

The improvements are due to the fact that Firefox (or Chrome with H2O's http2-reprioritize-blocking-assets configuration directive set to ON) downloads assets files that block rendering (e.g. CSS or JS files) before other asset files (e.g. images).

Conclusion:

With the benchmark showing 30% reduction in first-paint time, it is obvious that HTTP/2 (implemented by H2O) is offering superior performance against the protocols of previous generations. However, implementations might not be mature enough to offer such boost in performance (as discussed in recent blog post, Chrome has issues with its prioritization logic which is worked around by the configuration directive of H2O; Nginx does not prioritize the streams even though such feature is available in SPDY).

For the time being it is suggested that users conduct benchmarks to see if their applications actually become faster, or consult the developers of the HTTP servers for benchmark results.

Or, you can use H2O, which has been hereby shown to provide good performance under the described scenario :-)


PS. The presentation slides used at the meetup is available here:

Wednesday, May 27, 2015

C言語で「1時間以内に解けなければプログラマ失格となってしまう5つの問題が話題に」の5問目を解いてみた

Java8で「ソフトウェアエンジニアならば1時間以内に解けなければいけない5つの問題」の5問目を解いてみた」と「Perl6で「ソフトウェアエンジニアならば1時間以内に解けなければいけない5つの問題」の5問目を解いてみた」経由。

以下のような問題ですね。
1,2,…,9の数をこの順序で、”+”、”-“、またはななにもせず結果が100となるあらゆる組合せを出力するプログラムを記述せよ。例えば、1 + 2 + 34 – 5 + 67 – 8 + 9 = 100となる
とてもいい問題だと思うし、一方で上の回答例がeval的な手法を使っていたので、そういうズルをせずに解いたらどうなるだろう、ということでCで書いてみた。

正解が出るようになるまでの所要時間、約30分。なんとかプログラマ合格のようです。
#include <stdio.h>

#define MAX_POS 9
#define EXPECTED 100

static char buf[32];

static void doit(int pos, int sum, char *p, int sign)
{
    int i, n, s;

    *p++ = sign == 1 ? '+' : '-';
    for (i = pos, n = 0; i <= MAX_POS; ++i, n *= 10) {
        *p++ = '0' + i;
        n += i;
        s = sum + sign * n;
        if (i == MAX_POS) {
            if (s == EXPECTED) {
                *p = '\0';
                printf("%s = %d\n", buf + 1, s);
            }
        } else {
            doit(i + 1, s, p, 1);
            doit(i + 1, s, p, -1);
        }
    }
}   

int main(void)
{
    doit(1, 0, buf, 1);
    return 0;
}

Thursday, May 21, 2015

How to properly spawn an external command in C (or not use posix_spawn)

When spawning an external command, as a programmer, you would definitely want to determine if you have succeeded in doing so.

Unfortunately, posix_spawn (and posix_spawnp) does not provide such a feature. To be accurate, there is no guaranteed way to synchronously determine if the function has succeeded in spawning the command synchronously.

In case of Linux, the function returns zero (i.e. success) even if the external command does not exist.

The document suggests that if the function succeeded in spawning the command should be determined asynchronously by checking the exit status of waitpid. But such approach (that waits for the termination of the sub-process) cannot be used if your intension is to spawn a external command that is going to run continuously.

Recently I have faced the issue while working on H2O, and have come up with a solution; a function that spawns an external command that synchronously returns an error if it failed to do so.

What follows is the core logic I implemented. It is fairly simple; it uses the traditional approach of spawning an external command: fork and execvp. And at the same time uses a pipe with FD_CLOEXEC flag set to detect the success of execvp (the pipe gets closed), which is also used for returning errno in case the syscall fails.

pid_t safe_spawnp(const char *cmd, char **argv)
{
    int pipefds[2] = {-1, -1}, errnum;
    pid_t pid;
    ssize_t rret;

    /* create pipe, used for sending error codes */
    if (pipe2(pipefds, O_CLOEXEC) != 0)
        goto Error;

    /* fork */
    if ((pid = fork()) == -1)
        goto Error;

    if (pid == 0) {
        /* in child process */
        execvp(cmd, argv);
        errnum = errno;
        write(pipefds[1], &errnum, sizeof(errnum));
        _exit(127);
    }

    /* parent process */
    close(pipefds[1]);
    pipefds[1] = -1;
    errnum = 0;
    while ((rret = read(pipefds[0], &errnum, sizeof(errnum))) == -1
           && errno == EINTR)
        ;
    if (rret != 0) {
        /* spawn failed */
        while (waitpid(pid, NULL, 0) != pid)
            ;
        pid = -1;
        errno = errnum;
        goto Error;
    }

    /* spawn succeeded */
    close(pipefds[0]);
    return pid;

Error:
    errnum = errno;
    if (pipefds[0] != -1)
        close(pipefds[0]);
    if (pipefds[1] != -1)
        close(pipefds[1]);
    errno = errnum;
    return -1;
}

The actual implementation used in H2O does more; it has a feature to remap the file descriptors so that the caller can communicate with the spawned command via pipes. You can find the implementation here.

I am not sure if this kind of workaround is also needed for other languages, but I am afraid it might be the case.

Anyways I wrote this blogpost as a memo for myself and hopefully others. Happy hacking!