SQL 基准测试：PHP ActiveRecord ORM 与 MySQL 与 CodeIgniter Active Record 与标准 PHP

Question

Tests updated to be more readable; all done within a 100x foreach loop.

The test query is SELECT * FROM school_courses;

Can anyone provide "outside-the-box-thinking" feedback as to:

a) Why PHP ActiveRecord ORM takes 4 seconds to perform the same query per the below results?

b) Whether this is a practical benchmark or more of a hypothetical one for comparing methods of querying?

c) Are there other methods (test cases) I should try (or revise these ones) to get a clearer picture?

Results (with PDO & MySQLi)

Iterations: 100

PHP (config file)
Base Time: 5.793571472168E-5
Gross Time: 0.055607080459595
Net Time: 0.055549144744873

PHP ActiveRecord ORM
Base Time: 5.2213668823242E-5
Gross Time: 4.1013090610504
Net Time: 4.1012568473816

MySQL (standard)
Base Time: 5.1975250244141E-5
Gross Time: 0.32771301269531
Net Time: 0.32766103744507

CodeIgniter (Active Record)
Base Time: 5.1975250244141E-5
Gross Time: 0.28282189369202
Net Time: 0.28276991844177

MySQLi
Base Time: 5.1975250244141E-5
Gross Time: 0.20240592956543
Net Time: 0.20235395431519

PDO
Base Time: 5.2928924560547E-5
Gross Time: 0.17662906646729
Net Time: 0.17657613754272

Tests

// Benchmark tests
$runs = 100;

// PHP (config file)
for ($i = 0; $i < $runs; $i++) {
    $this->view_data['courses'] = course_info();
}

// PHP ActiveRecord ORM
for ($i = 0; $i < $runs; $i++) {
    $this->view_data['courses'] = Course::all();
}

// mysql_* (MySQL standard; deprecated)
for ($i = 0; $i < $runs; $i++) {
    $sql = mysql_query('SELECT * FROM school_courses') or die(mysql_error());
    while ($row = mysql_fetch_object($sql)) {
        array_push($this->view_data['courses'], $row);
    }
}

// CodeIgniter (Active Record)
for ($i = 0; $i < $runs; $i++) {
    $this->view_data['courses'] = $this->db->get('school_courses');
}

// mysqli_* (MySQLi)
for ($i = 0; $i < $runs; $i++) {
    $res = $mysqli->query('SELECT * FROM school_courses');
    while ($row = $res->fetch_object()) {
        array_push($this->view_data['courses'], $row);
    }
}

// PDO
for ($i = 0; $i < $runs; $i++) {
    foreach($conn->query('SELECT * FROM school_courses') as $row) {
        array_push($this->view_data['courses'], $row);
    }
}

Answer 1

So the reason why PHP ActiveRecord ORM introduces so much overhead when benchmarking concurrent connections is due to the fact that each result that is returned instantiates a new Model object. This being integral to the usage of this ORM library, I don't see any reasonable way of making changes without overhauling the entire library.

Here is what I found:

Inside the find_by_sql() method in the Table class, you have:

    $sth = $this->conn->query($sql,$this->process_data($values));

    while (($row = $sth->fetch()))
    {
        $model = new $this->class->name($row,false,true,false);

        if ($readonly)
            $model->readonly();

        if ($collect_attrs_for_includes)
            $attrs[] = $model->attributes();

        $list[] = $model;
    }

Specifically, the dynamic model instantiation new $this->class->name() is responsible for the overhead, weighing in somewhere around 0.004 per result fetched, let's say.

You take this and multiply it times the number of records now, (10 records = 0.04). Now multiply that by the number of concurrent connections, let's say 100, and you have an foreseeable bottleneck problem.

Four (4) seconds for 100 users (hypothetically speaking) accessing a table at the same time containing 10 records.

Should I be concerned at this point that the number of records being fetched could potentially cause bottlenecking issues due to the way this library is instantiating a model class for every record?

Again, this all could be hypothetical speech at this point that may never exist or present a problem in the real world assuming proper use of an ORM. And unless these tests or conclusions are inaccurate, what I'm trying to simulate here is traffic load for say 100, 1,000 and 10,000 active on-site visitors.

In other words, if I don't ever add another course (limit 10), will 10,000 visitors browsing the courses page, for example, lead to a 400 second (6.67 minute) wait time for others to move off the page? And if that's the case, then I will have discovered my own answer (hence this post) and will look into finding another ORM or resort to refactoring on a case-by-case basis.

Is this this most appropriate way of benchmarking and simulating traffic load?

Additional Resources

How to Apache Stress Test With ab Tool https://wiki.appnexus.com/display/documentation/How+to+Apache+Stress+Test+With+ab+Tool

Answer 2

Rewrite recommendation:

I don't want to sound brutal but you can save a lot of headache in the future as well as keep up with current practices if you forget everything you know about mysql_(). By today's standards it is trash honestly. Look into mysqli_ or PDO as your db interfaces.

mysqli_ : http://us2.php.net/manual/en/book.mysqli.php

PDO: http://us2.php.net/manual/en/book.pdo.php

Report back benchmarks then...

Answer 3

Your simple query isn't really a fair test. ORMs are fine and fairly competitive for simple queries that like that. It's more complex ones (ie LEFT JOIN) that ORMs create inefficient queries on and you end having to bypass them. ORMs will always be slower than raw SQL written by someone who knows SQL. Of course, knowing SQL is the key.

If you are considering ORMs, you really should try Doctrine. I am not a fan ORMs (at all), but that is the most popular PHP ORM out there.

Bulk inserts are also another area where some ORMs and DB abstraction layers trip up. Instead of recognizing that a bulk insert can be used, they do single inserts in a loop. That is going to cause table locking issues on MyISAM in addition to being slow. Perhaps add a bulk insert test, letting each DB layer generate the insert query if possible.

What your testing method does reveal is that over many iterations the overhead of each DB access method adds up. I would suggest eliminating query overhead altogether and just use "SELECT VERSION()" instead.