Feature:Advantage/Performance

From Kiwiphp

Contents 1 Preface 2 Methodology 3 Environment 3.1 Software 3.2 Hardware 4 Framework Version 5 Benchmark Step 5.1 HTML Baseline 5.2 PHP Baseline 5.3 Kiwiphp 5.4 CodeIgniter 5.5 Zend Framework 5.6 Symfony 5.7 CakePHP 5.8 Solar 5.9 Measure 6 Benchmark Result

Preface

Many popular php frameworks may cause more than 90 percents performance loss. There are some performance benchmark testing:

• PHP framework comparison benchmarks
  
• Performance bench of PHP/Perl/Python/Ruby frameworks
• Additional test of zend framework, cakephp
• Another test of zend framework, cakephp, solar, symfony

Methodology

Write a minimum action to "return true" without any output, then measure its QPS through http_load.

The methodology is samilar with Paul M. Jones'(Paul is the lead developer of Solar PHP framework), the only difference is his test print "Hello World" but mine don't.

Environment

Software

• OS: RedHat Enterprise Linux Advanced Server 32bit
• Web Server: Apache 1.3.37
• PHP 5.2.5 with APC enabled

Hardware

/bin/hwconfig
Summary: HP DL140 G3, 2 x Xeon E5310 1.60GHz, 4GB
System: HP DL140 G3
Processors: 2 x Xeon E5310 1.60GHz (8 cores) - Clovertown G0, 64-bit, quad-core, 65nm, L2: 8MB
Memory: 4GB

 

Framework Version

 

Benchmark Step

HTML Baseline

I use a zero bytes html file for html baseline.

PHP Baseline

Simply return a boolean using native php:

<?php
class HelloWorldAction
{
 public function execute()
 {
 return true;
 }
}
$obj = new HelloWorldAction();
$obj->execute();

Kiwiphp

Kiwiphp version: release 0.1

Create a "Hello World" application, but without any output, just "return true".

• Bootstrap File

<?php
/**
 * Please replace the relative path(s) with absolute ones
 * It will take some performance benefit
 */
include("/path/to/kiwiphp/trunk/runtime/kiwi.php");
define("ENV_MODE", "product");
Kiwi::run(array(
 "proj_dir" => "/path/to/kiwiphp/trunk/example/share/project1/",
 "app_name" => "tool"
)); 

• Action File

<?php
class HelloWorldAction extends Action
{
 public function execute()
 {
 return true;
 }
} 

Kiwiphp will load all files in kiwiphp runtime

It's url is http://localhost/kiwiphp_tool.php?module=hello&action=HelloWorld

CodeIgniter

CodeIgniter version: 1.7.0, 1.5.1

Create a controller "hello" with a method "world":

<?php
class Hello extends Controller
{
 function Hello()
 {
 parent::Controller();
 }

 function world()
 {
 return true;
 }
}

Replace require_once with require in bootstrap file (index.php):

<?php
//chang
require_once BASEPATH.'codeigniter/CodeIgniter'.EXT;
//to
require BASEPATH.'codeigniter/CodeIgniter'.EXT;

Zend Framework

Zend framework version: 1.7 preview release

• Bootstrap File

<?php
include "/absolute_path_to/Zend/Loader.php";
Zend_Loader::registerAutoload();
$frontController = Zend_Controller_Front::getInstance();
$frontController->throwExceptions(true);
$frontController->setControllerDirectory('application/controllers');
$frontController->setParam('useDefaultControllerAlways', true);
$frontController->dispatch();

• Controller File

<?php
class HelloController extends Zend_Controller_Action
{
 public function worldAction()
 {
 exit;
 }
}

Symfony

Symfony version: 1.1.4 sandbox

• Bootstrap File

Use the default bootstrap file (index.php) of symfony sandbox archive.

<?php
require_once(dirname(__FILE__).'/../config/ProjectConfiguration.class.php');
$configuration = ProjectConfiguration::getApplicationConfiguration('frontend', 'prod', false);
sfContext::createInstance($configuration)->dispatch();

• Action File

<?php
class helloActions extends sfActions
{
 public function executeWorld($request)
 {
 exit;
 }
}

• Other Change
  • Disable the session module, because my server does not support session:

//file:lib/symfony/storage/sfSessionStorage.class.php
public function initialize($options = null)
{return true;
//...

CakePHP

CakePHP version: 1.1.20.7692

• Bootstrap File

Use the default bootstrap file (index.php) of cake archive.

• Controller File

<?php
class HelloController extends AppController
{
 var $layout = null;
 var $autoLayout = false;
 var $uses = array();
 var $helpers = null; function world()
 {
 exit;
 }
}

• Other Change
  • Remove config file inclusion, because this test does not need any config file:

//file: index.php
require APP_PATH . 'config' . DS . 'core.php';

//file: cake/libs/configure.php
require APP_PATH . 'config' . DS . 'bootstrap.php';

• • Disable the session module, because my server does not support session:

//file:cake/libs/session.php
function __startSession() {return true;
//...

Solar

• Bootstrap File

<?php
require '/absolute_path_to/Solar.php';
$config['Solar_Uri_Action']['path'] = '/fx_bench/solar-1.0a2/index.php/';
Solar::start($config);
$front = Solar::factory('Solar_Controller_Front');
$front->display();
Solar::stop();

• Controller File

<?php
class Solar_App_Hello extends Solar_Controller_Page
{
 public function actionWorld()
 {
 exit;
 }
}

Measure

Measure their QPS with http_load:

http_load -parallel 100 -seconds 10 native_html
http_load -parallel 100 -seconds 10 native_php
http_load -parallel 100 -seconds 10 kiwi-0.1
http_load -parallel 100 -seconds 10 ci-1.7.0
http_load -parallel 100 -seconds 10 ci-1.5.4
http_load -parallel 100 -seconds 10 zf-1.7
http_load -parallel 100 -seconds 10 solar-1.0a2
http_load -parallel 100 -seconds 10 cake_1.1.20

Each url file contains 100 same lines:

native_html http://localhost/fx_bench/native_html.htm
native_php http://localhost/fx_bench/native_php.php
kiwi-0.1 http://localhost/fx_bench/kiwi-0.1.php?module=hello&action=HelloWorld
ci-1.7.0 http://localhost/fx_bench/ci-1.7.0/index.php/hello/world
ci-1.5.1 http://localhost/fx_bench/ci-1.5.1/index.php/hello/world
zf-1.7 http://localhost/fx_bench/zf-1.7/index.php
solar-1.0a2 http://localhost/fx_bench/solar-1.0a2/index.php/hello/world
cake_1.1.20 http://localhost/fx_bench/cake_1.1.20/index.php/hello/world

Benchmark Result

In this task, kiwiphp reach 60% of native php.

That means, with kiwiphp, the application will cost extra 0.01 microseconds per http request.

Framework	Queries Per Second	Percentage of native php
native html	6787.67	208.08%
native php	3262.17	100.00%
kiwiphp 0.1	2448.94	75.01%
CodeIgniter 1.7.0	301.963	9.27%
CodeIgniter 1.5.4	336.973	10.45%
Symfony 1.1.4
Zend Framework 1.7	64.0558	1.97%
Solar 1.0.0 alpha2	365.488	11.24%
Cakephp 1.1.20.7692	343.7	10.62%

Comments

• CodeIgniter is a high-performance framework, In the test I did in Janary, 2008, it has reached 27% of native php (but native php only gets 733 qps). So I download the 1.5.4 version to test again.
  
• Kiwiphp is 60%-66% of native php, this test result is a little higher to kiwi, I will test it again soon.
• Zend Framework is the slowest one, but you can tune it by replace <code>require_once</code> with <code>require</code>, and use absolute path for file inclusion.