ベンチマーク

abによる比較
簡易HTTPサーバを作成してベンチマークを行いました。

ベンチ対象のサーバ
 ・libevent1…C言語
 ・libevent2…C言語
 ・libevent2…PHP言語
 ・node.js …JavaScript
 ・nginx
 ・shortfin


ベンチマークリスト

Requests/s min median max
libevent1 5555.34 47 87 125
libevent2 5207.06 51 91 113
PHP 5013.12 53 89 270
JavaScript 2042.88 2 47 912
nginx 4612.45 50 81 137
shortfin 5240.77 12 19 40


abのインストール

$ sudo aptitude install apache2-utils

[Apache Benchを使った負荷テストのやり方]


libevent1

$ ab -n 5000 -c 500 http://localhost:9090/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:
Server Hostname:        localhost
Server Port:            9090

Document Path:          /
Document Length:        87 bytes

Concurrency Level:      500
Time taken for tests:   0.900 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      755000 bytes
HTML transferred:       435000 bytes
Requests per second:    5555.34 [#/sec] (mean)
Time per request:       90.004 [ms] (mean)
Time per request:       0.180 [ms] (mean, across all concurrent requests)
Transfer rate:          819.20 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       31   41   2.8     42      48
Processing:     7   46   3.1     46      84
Waiting:        4   27  11.0     27      51
Total:         47   87   4.1     87     125

Percentage of the requests served within a certain time (ms)
  50%     87
  66%     89
  75%     89
  80%     90
  90%     91
  95%     92
  98%     92
  99%     92
 100%    125 (longest request)

httpd1.c
[c]
#include <stdio.h>
#include <stdlib.h>

#include <event.h>
#include <evhttp.h>

/**
* コールバック
*/
static void send_document_cb(struct evhttp_request *req, void *arg) {
struct evbuffer *evb = NULL;

evb = evbuffer_new();
if (!evb) {
fprintf(stderr, "evbufferの生成に失敗しました。\n");
exit(1);
}

evbuffer_add_printf(
evb,
"<html>\n"
"<head>\n"
"<title>libevent</title>\n"
"</head>\n"
"<body>\n"
"libeventテスト\n"
"</body>\n"
"</html>\n"
);

evhttp_send_reply(req, 200, "OK", evb);
evbuffer_free(evb);
}

/**
* メイン
*/
int main() {
struct evhttp *http;

unsigned short port = 9090;

event_init();

/* アドレス・ポート指定 */
http = evhttp_start("0.0.0.0", port);
if (!http) {
fprintf(stderr, "evhttpの生成に失敗しました。\n");
return 1;
}

// evhttp_set_cb(http, "/dump", dump_request_cb, NULL);
/* デフォルトアクセス */
evhttp_set_gencb(http, send_document_cb, NULL);

event_dispatch();
evhttp_free(http);

return 0;
}
[/c]

コンパイル

gcc -L/usr/lib -levent -o httpd1 httpd1.c

libevent2

$ ab -n 5000 -c 500 http://localhost:9090/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:
Server Hostname:        localhost
Server Port:            9090

Document Path:          /
Document Length:        87 bytes

Concurrency Level:      500
Time taken for tests:   0.960 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      755000 bytes
HTML transferred:       435000 bytes
Requests per second:    5207.06 [#/sec] (mean)
Time per request:       96.023 [ms] (mean)
Time per request:       0.192 [ms] (mean, across all concurrent requests)
Transfer rate:          767.84 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       27   42   4.6     43      51
Processing:     7   50  10.6     48      76
Waiting:        5   38   8.9     37      58
Total:         51   92   9.6     91     113

Percentage of the requests served within a certain time (ms)
  50%     91
  66%     94
  75%     99
  80%    102
  90%    104
  95%    106
  98%    109
  99%    110
 100%    113 (longest request)

httpd2.c
[c]
#include <stdio.h>
#include <stdlib.h>

#include <event2/event.h>
#include <event2/http.h>
#include <event2/buffer.h>

/**
* コールバック
*/
static void send_document_cb(struct evhttp_request *req, void *arg) {
struct evbuffer *evb = NULL;

evb = evbuffer_new();
if (!evb) {
fprintf(stderr, "evbufferの生成に失敗しました。\n");
exit(1);
}

evbuffer_add_printf(
evb,
"<html>\n"
"<head>\n"
"<title>libevent</title>\n"
"</head>\n"
"<body>\n"
"libeventテスト\n"
"</body>\n"
"</html>\n"
);

evhttp_send_reply(req, 200, "OK", evb);
evbuffer_free(evb);
}

/**
* メイン
*/
int main() {
struct event_base *base;
struct evhttp *http;
struct evhttp_bound_socket *handle;

unsigned short port = 9090;

base = event_base_new();
if (!base) {
fprintf(stderr, "event_baseの生成に失敗しました。\n");
return 1;
}

/* Create a new evhttp object to handle requests. */
http = evhttp_new(base);
if (!http) {
fprintf(stderr, "evhttpの生成に失敗しました。\n");
return 1;
}

// evhttp_set_cb(http, "/dump", dump_request_cb, NULL);
/* デフォルトアクセス */
evhttp_set_gencb(http, send_document_cb, NULL);

/* アドレス・ポート指定 */
handle = evhttp_bind_socket_with_handle(http, "0.0.0.0", port);
if (!handle) {
fprintf(stderr, "%d. ポートを確保出来ませんでした。\n", (int)port);
return 1;
}

event_base_dispatch(base);
event_base_free(base);

return 0;
}
[/c]

コンパイル

gcc -L/usr/local/lib -levent -o httpd2 httpd2.c

libevent2 + PHP

$ ab -n 5000 -c 500 http://localhost:9090/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:
Server Hostname:        localhost
Server Port:            9090

Document Path:          /
Document Length:        87 bytes

Concurrency Level:      500
Time taken for tests:   0.997 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      755000 bytes
HTML transferred:       435000 bytes
Requests per second:    5013.12 [#/sec] (mean)
Time per request:       99.738 [ms] (mean)
Time per request:       0.199 [ms] (mean, across all concurrent requests)
Transfer rate:          739.24 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        8   36   7.3     36      52
Processing:    15   53  13.2     53     262
Waiting:       10   41  12.2     41     243
Total:         53   89  10.4     89     270

Percentage of the requests served within a certain time (ms)
  50%     89
  66%     93
  75%     95
  80%     96
  90%     99
  95%    104
  98%    108
  99%    110
 100%    270 (longest request)

httpd.php
[php]
<?php
function _http_default($req) {
$buff = $req->getOutputBuffer();
$buff->add(
"<html>\n".
"<head>\n".
"<title>libevent</title>\n".
"</head>\n".
"<body>\n".
"libeventテスト\n".
"</body>\n".
"</html>\n"
);
$req->sendReply(200, "OK");
}

$base = new EventBase();
$http = new EventHttp($base);
$http->setAllowedMethods(EventHttpRequest::CMD_GET | EventHttpRequest::CMD_POST);

//$http->setCallback("/err400", "_http_400");
$http->setDefaultCallback("_http_default");

$http->bind("0.0.0.0", 9090);
$base->loop();
?>
[/php]


node.js

$ ab -n 5000 -c 500 http://localhost:9090/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:
Server Hostname:        localhost
Server Port:            9090

Document Path:          /
Document Length:        87 bytes

Concurrency Level:      500
Time taken for tests:   2.448 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      940000 bytes
HTML transferred:       435000 bytes
Requests per second:    2042.88 [#/sec] (mean)
Time per request:       244.752 [ms] (mean)
Time per request:       0.490 [ms] (mean, across all concurrent requests)
Transfer rate:          375.06 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        0    4  11.8      0      54
Processing:     2   46  29.7     45     912
Waiting:        2   45  29.8     45     912
Total:          2   50  33.2     47     912

Percentage of the requests served within a certain time (ms)
  50%     47
  66%     60
  75%     68
  80%     73
  90%     85
  95%    106
  98%    122
  99%    129
 100%    912 (longest request)

httpd.js
[javascript]
var http = require(‘http’);
var server = http.createServer(
function (request, response) {
response.writeHead(200, {‘Content-Type’: ‘text/plain’});
response.write(
"<html>\n" +
"<head>\n" +
"<title>libevent</title>\n" +
"</head>\n" +
"<body>\n" +
"libeventテスト\n" +
"</body>\n" +
"</html>\n"
);
response.end();
}
).listen(9090);
[/javascript]


nginx

$ ab -n 5000 -c 500 http://localhost:9090/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:        nginx/1.4.1
Server Hostname:        localhost
Server Port:            9090

Document Path:          /
Document Length:        168 bytes

Concurrency Level:      500
Time taken for tests:   1.084 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Non-2xx responses:      5000
Total transferred:      1585000 bytes
HTML transferred:       840000 bytes
Requests per second:    4612.45 [#/sec] (mean)
Time per request:       108.402 [ms] (mean)
Time per request:       0.217 [ms] (mean, across all concurrent requests)
Transfer rate:          1427.88 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:       22   38   4.9     39      53
Processing:    11   44  12.0     42     102
Waiting:        8   33   8.5     33      65
Total:         50   82  10.6     81     137

Percentage of the requests served within a certain time (ms)
  50%     81
  66%     83
  75%     84
  80%     84
  90%     91
  95%    102
  98%    123
  99%    130
 100%    137 (longest request)

shortfin

$ ab -n 5000 -c 500 http://localhost:9090/
This is ApacheBench, Version 2.3 <$Revision: 655654 $>
Copyright 1996 Adam Twiss, Zeus Technology Ltd, http://www.zeustech.net/
Licensed to The Apache Software Foundation, http://www.apache.org/

Benchmarking localhost (be patient)
Completed 500 requests
Completed 1000 requests
Completed 1500 requests
Completed 2000 requests
Completed 2500 requests
Completed 3000 requests
Completed 3500 requests
Completed 4000 requests
Completed 4500 requests
Completed 5000 requests
Finished 5000 requests


Server Software:        shortfin/0.9.5
Server Hostname:        localhost
Server Port:            9090

Document Path:          /
Document Length:        87 bytes

Concurrency Level:      500
Time taken for tests:   0.954 seconds
Complete requests:      5000
Failed requests:        0
Write errors:           0
Total transferred:      875000 bytes
HTML transferred:       435000 bytes
Requests per second:    5240.77 [#/sec] (mean)
Time per request:       95.406 [ms] (mean)
Time per request:       0.191 [ms] (mean, across all concurrent requests)
Transfer rate:          895.64 [Kbytes/sec] received

Connection Times (ms)
              min  mean[+/-sd] median   max
Connect:        5    9   2.3      9      23
Processing:     2    9   2.0      9      19
Waiting:        1    7   3.1      7      19
Total:         12   18   1.9     19      40

Percentage of the requests served within a certain time (ms)
  50%     19
  66%     19
  75%     19
  80%     19
  90%     19
  95%     20
  98%     28
  99%     28
 100%     40 (longest request)

単にHTMLを出力するだけなので、差が分からない結果でした。
Shortfinが抜きに出た速度がでています。

libevent2

libeventとは
Chromium、Memcachedなどで有名なC言語で作成されたイベント駆動型フレームワークです。

以下の機能などが提供されています。
 ・ネットワーク監視
 ・I/O監視
 ・クライアント・サーバ問い合わせ
 ・HTTPサーバ
 ・SSL
 ・DNS

libeventを使ったサンプルは結構あるのですが、PHPのサンプルが少ないようでした。
PHP5とlibeventを使った、WebSocketのチャットを作成しました。

[github:WebSocket]


libevent2のインストール

$ wget http://jaist.dl.sourceforge.net/project/levent/libevent/libevent-2.0/libevent-2.0.21-stable.tar.gz
$ tar xzvf libevent-2.0.21-stable.tar.gz
$ cd libevent-2.0.21-stable
$ ./configure
$ make
$ sudo make install

php eventモジュールのインストール

$ wget http://pecl.php.net/get/event-1.7.2.tgz
$ tar xzvf event-1.7.2.tgz
$ cd event-1.7.2
$ phpize
$ ./configure
$ make
$ sudo -s
# make install
# cd /etc/php5/mods-available
# echo extension=event.so > event.ini
# cd /etc/php5/conf.d
# ln -s ../mods-available/event.ini ./20-event.ini

phpイベントループ
[php]
<?php
/**
* イベントベースの生成
*/
$base = new EventBase();

/**
* クライアント接続の生成
*/
$listener = new EventListener(
$base,
‘eventCallback’,
$base,
EventListener::OPT_CLOSE_ON_FREE | EventListener::OPT_REUSEABLE,
-1,
‘0.0.0.0:8888’
);

/**
* ループスタート
*/
$base->dispatch();
?>
[/php]


クライアント接続イベント
[php]
<?php
/**
* クライアント接続時コールバック
*/
function eventCallback ($listener, $fd, $address, $base) {
/**
* 遠隔手続呼出処理
*/
$bev = new EventBufferEvent($base, $fd, EventBufferEvent::OPT_CLOSE_ON_FREE);
$bev->setCallbacks(
‘readCallback’,
NULL,
NULL,
NULL
);
}

/**
* クライアント接続時コールバック
*/
function readCallback ($bev, $address) {
static $handshake;

if (empty($handshake)) {
/**
* ハンドシェイクの確立
*/
$buff = $bev->read(4096);
if (preg_match(‘/Sec-WebSocket-Key: ([^\s]+)\r\n/’, $buff, $matches)) {
$key = $matches[1];
}

/* 認証キー生成 */
$accept = $key.’258EAFA5-E914-47DA-95CA-C5AB0DC85B11′;
$accept = base64_encode(sha1($accept, true));

$header = "HTTP/1.1 101 Switching Protocols\r\n".
"Upgrade: WebSocket\r\n".
"Connection: Upgrade\r\n".
"Sec-WebSocket-Accept: {$accept}\r\n".
"\r\n";
$bev->write($header);
$handshake = true;
} else {
$frame = $bev->read(4096);
$msg = ws_decode($frame);
$bev->write(ws_encode($msg));
}
}
?>
[/php]


PHPはC言語のLibraryを簡単に試すことができて便利です。
libeventは大量の作業を効率的に処理できる素晴らしいLibraryです。

あのnode.jsもlibeventを採用しようとしたそうですが、当時はまだlibevent2が
正式版ではなく速度の出るlibevとlibeio採用しているそうです。

※libevent1からlibevent2になり速度が大幅にアップしています。


WebSocketについての解説
uuwan Lab:ハンドシェイクなどの説明
uuwan Lab:Framing Protocolなどの説明
上記のページを参考にさせて頂きました。