Femtosecond optimizations

In 1974, Donald Knuth, which we can safely say is one of the founding fathers of modern Computer Science, said:

We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil.

The PHP ecosystem is plagued with premature optimizers. Not that they really intend to do evil on purpose, but they probably have lost some perspective.

Some PHP developers seem so convinced that calling functions is evil, they avoid it at all costs and program in big chunks of unimaginable cyclomatic complexity instead. Hence, I dared to measure this factor by implementing the same functionality in two different manners. Implementation (1) is written using many functions and implementation (2) is written in one big chunk (13 user land function calls versus 0).

Implementation 1:

function calculateDistanceMatrix(array $strings) {
	$distances = array();
	for ($l1 = 0; $l1 < 10; $l1++)
	{
		$distances[$l1] = array();
		for ($l2 = 0; $l2 < 10; $l2++)
		{
			$distances[$l1][$l2] = levenshtein($strings[$l1], $strings[$l2]);
		}
	}
	return $distances;
}

function generateArrayOfRandomStrings($length, $num) {
	$strings = array();
	for ($i = 0; $i < $num; $i++)  {
		$strings[] = generateRandomString($length);
	}
	return $strings;
}

function generateRandomString($length) {
	$str = '';
	for ($e = 0; $e < $length; $e++) {
		$chr = chr(mt_rand(97, 122));
		if (mt_rand(0, 1)) {
			$chr = strtoupper($chr);
		}
		$str{$e} = $chr;
	}
	return implode('', $str);
}

function printDistanceMatrix(array $distances) {
	foreach ($distances as $distancesRow)
	{
		foreach ($distancesRow as $distance)
		{
			printf("%02d ", $distance);
		}
		echo PHP_EOL;
	}
}

$strings = generateArrayOfRandomStrings(20, 10);
$distances = calculateDistanceMatrix($strings);
printDistanceMatrix($distances);

Implementation 2:

$strings = array();
for ($i = 0; $i < 10; $i++)  {
	$str = '';
	for ($e = 0; $e < 20; $e++) {
		$chr = chr(mt_rand(97, 122));
		if (mt_rand(0, 1)) {
			$chr = strtoupper($chr);
		}
		$str{$e} = $chr;
	}
	$strings[] = implode('', $str);
}

$distances = array();
for ($l1 = 0; $l1 < 10; $l1++)
{
	$distances[$l1] = array();
	for ($l2 = 0; $l2 < 10; $l2++)
	{
		$distances[$l1][$l2] = levenshtein($strings[$l1], $strings[$l2]);
		printf("%02d ", $distances[$l1][$l2]);
	}
	echo PHP_EOL;
}

To create a conclusive benchmark, I used real HTTP requests on an Apache server running PHP5.3 with APC. The two implementations were benchmarked using the “Apache Benchmark” tool, with 50 000 requests and a concurrency of 50 connections. The IS implementation is just a static page, containing the same output data as one of the two implementations, and is used for reference.

Impl. Avg. Throughput (#/s) Avg. Response Time (ms) Response Time 99th percentile (ms) Highest Response Time (ms)
IS 5659.09 8.835 12 16
I1 450.20 111.063 171 247
I2 466.68 107.139 153 229

What does this amount to ? In terms of response time, (2) does 3.66% better than (1). In terms of throughput, (2) also does 3.66% better than (1). Then comes the usual victory speech:

Don’t get me wrong, I’d prefer to maintain the first implementation but when it comes to performance, a 4% gain really warrants creating maintainability problems.

The difference is significant, but is it that important ? Probably not. If we were talking about an operating system or an embedded software to direct missiles on Soviet Russia (no harm intended to my fellow humans from this part of the world), my opinion would be slightly different, but a web site is quite a different beast.

Usually, the cost of operating a web site is not hardware renting. In this fast-paced world, business needs change all the freaking time. How much it costs to adapt to these changes is the real cost driver. Hardware is cheap. Good programmers are not. Saving 4% on hardware costs but paying a developer thousands of dollars to fix issues is a bad business decision.

These kinds of operations are what I call femtosecond optimizations. Of course, the formulation is a bit hyperbolic since we can’t really go as far as saving femtoseconds, but you get the idea: they are useless, premature optimizations which sole accomplishment is to diminish the quality of the code and increase maintenance cost. They are toxic, costly and difficult to manage. They are the root of (almost) all evil.

Trust good design. Make it work. Make it work right. And then, make it work fast. Not the other way around.

Advertisements


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s