Feature:Advantage/Performance

From Kiwiphp

Contents 1 Preface 2 Methodology 3 Environment 3.1 Software 3.2 Hardware 4 Framework Version 5 Benchmark Step 5.1 HTML Baseline 5.2 PHP Baseline 5.3 Kiwiphp 5.4 CodeIgniter 5.5 Zend Framework 5.6 Symfony 5.7 CakePHP 5.8 Solar 5.9 Measure 6 Benchmark Result

Preface

Many popular php frameworks may cause more than 90 percents performance loss. There are some performance benchmark testing:

• PHP framework comparison benchmarks
  
• Performance bench of PHP/Perl/Python/Ruby frameworks
• Additional test of zend framework, cakephp
• Another test of zend framework, cakephp, solar, symfony

Methodology

Write a minimum action to "return true" without any output, then measure its QPS through http_load.

The methodology is samilar with Paul M. Jones'(Paul is the lead developer of Solar PHP framework), the only difference is his test print "Hello World" but mine don't.

Environment

Software

• OS: RedHat Enterprise Linux Advanced Server 32bit
• Web Server: Apache 1.3.37
• PHP 5.2.5 with APC enabled

Hardware

/bin/hwconfig
Summary: HP DL140 G3, 2 x Xeon E5310 1.60GHz, 4GB
System: HP DL140 G3
Processors: 2 x Xeon E5310 1.60GHz (8 cores) - Clovertown G0, 64-bit, quad-core, 65nm, L2: 8MB
Memory: 4GB

 

Framework Version

Framework Name	Version
Cakephp	1.1.20.7692
CodeIgniter	1.7.0, 1.5.4
Kiwiphp	0.1.1, 0.1.12
SolarPHP	1.0.0 alpha2
Symfony	1.1.4
Zend Framework	1.7
Yii	1.0.3
QeePHP	2.1

Benchmark Step

HTML Baseline

I use a zero bytes html file for html baseline.

PHP Baseline

Simply return a boolean using native php:

<?php
class HelloWorldAction
{
    public function execute()
    {
        return true;
    }
}
$obj = new HelloWorldAction();
$obj->execute();

Kiwiphp

Create a "Hello World" application, but without any output, just "return true".

• Bootstrap File

<?php
/**
 * Please replace the relative path(s) with absolute ones
 * It will take some performance benefit
 */
include("/path/to/kiwiphp/trunk/runtime/kiwi.php");
define("ENV_MODE", "product");
Kiwi::run(array(
    "proj_dir" => "/path/to/kiwiphp/trunk/example/share/project1/",
    "app_name" => "tool"
));

• Action File

<?php
class HelloWorldAction extends Action
{
    public function execute()
    {
        return true;
    }
}

CodeIgniter

Create a controller "hello" with a method "world":

<?php
class Hello extends Controller
{
    function Hello()
    {
        parent::Controller();
    }

    function world()
    {
        return true;
    }
}

Replace require_once with require in bootstrap file (index.php):

<?php
//chang
require_once BASEPATH.'codeigniter/CodeIgniter'.EXT;
//to
require BASEPATH.'codeigniter/CodeIgniter'.EXT;

Zend Framework

• Bootstrap File

<?php
include "/absolute_path_to/Zend/Loader.php";
Zend_Loader::registerAutoload();
$frontController = Zend_Controller_Front::getInstance();
$frontController->throwExceptions(true);
$frontController->setControllerDirectory('application/controllers');
$frontController->setParam('useDefaultControllerAlways', true);
$frontController->dispatch();

• Controller File

<?php
class HelloController extends Zend_Controller_Action
{
    public function worldAction()
    {
        exit;
    }
}

Symfony

• Bootstrap File

Use the default bootstrap file (index.php) of symfony sandbox archive.

<?php
require_once(dirname(__FILE__).'/../config/ProjectConfiguration.class.php');
$configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'prod', false);
sfContext::createInstance($configuration)->dispatch();

• Action File

<?php
class helloActions extends sfActions
{
    public function executeWorld($request)
    {
        exit;
    }
}

• Other Change
  • Disable the session module, because my server does not support session:

//file:lib/symfony/storage/sfSessionStorage.class.php
public function initialize($options = null)
{return true;
//...

CakePHP

• Bootstrap File

Use the default bootstrap file (index.php) of cake archive.

• Controller File

<?php
class HelloController extends AppController
{
    var $layout = null;
    var $autoLayout = false;
    var $uses = array();
    var $helpers = null; function world()
    {
        exit;
    }
}

• Other Change
  • Remove config file inclusion, because this test does not need any config file:

//file: index.php
require APP_PATH . 'config' . DS . 'core.php';

//file: cake/libs/configure.php
require APP_PATH . 'config' . DS . 'bootstrap.php';

• • Disable the session module, because my server does not support session:

//file:cake/libs/session.php
function __startSession() {return true;
//...

Solar

• Bootstrap File

<?php
require '/absolute_path_to/Solar.php';
$config['Solar_Uri_Action']['path'] = '/fx_bench/solar-1.0a2/index.php/';
Solar::start($config);
$front = Solar::factory('Solar_Controller_Front');
$front->display();
Solar::stop();

• Controller File

<?php
class Solar_App_Hello extends Solar_Controller_Page
{
    public function actionWorld()
    {
        exit;
    }
}

Measure

Measure their QPS with http_load:

http_load -parallel 100 -seconds 10 native_html
http_load -parallel 100 -seconds 10 native_php
http_load -parallel 100 -seconds 10 kiwi-0.1
http_load -parallel 100 -seconds 10 ci-1.7.0
http_load -parallel 100 -seconds 10 ci-1.5.4
http_load -parallel 100 -seconds 10 zf-1.7
http_load -parallel 100 -seconds 10 solar-1.0a2
http_load -parallel 100 -seconds 10 cake_1.1.20

Each url file contains 100 same lines:

native_html http://localhost/fx_bench/native_html.htm
native_php http://localhost/fx_bench/native_php.php
kiwi-0.1 http://localhost/fx_bench/kiwi-0.1.php?module=hello&action=HelloWorld
ci-1.7.0 http://localhost/fx_bench/ci-1.7.0/index.php/hello/world
ci-1.5.1 http://localhost/fx_bench/ci-1.5.1/index.php/hello/world
zf-1.7 http://localhost/fx_bench/zf-1.7/index.php
solar-1.0a2 http://localhost/fx_bench/solar-1.0a2/index.php/hello/world
cake_1.1.20 http://localhost/fx_bench/cake_1.1.20/index.php/hello/world

Benchmark Result

In this task, kiwiphp reach 60% of native php.

That means, with kiwiphp, the application will cost extra 0.01 microseconds per http request.

Framework	Queries Per Second	Percentage of native php
native html	12359.4
native php	7364.23	100.00%
kiwiphp 0.1.12	4517.63
CodeIgniter 1.7.0	748.997
CodeIgniter 1.5.4	771.799
Symfony 1.1.4
Zend Framework 1.7	38.2999
Solar 1.0.0 alpha2	246.07
Cakephp 1.1.20.7692	297.39
Yii 1.0.3	2416.69
QeePHP 2.1	3319.2

Comments

• CodeIgniter is a high-performance framework, In the test I did in Janary, 2008, it has reached 27% of native php (but native php only gets 733 qps). So I download the 1.5.4 version to test again.
  
• Kiwiphp is 60%-66% of native php, this test result is a little higher to kiwi, I will test it again soon.
• Zend Framework is the slowest one, but you can tune it by replace <code>require_once</code> with <code>require</code>, and use absolute path for file inclusion.