Adventures in PHP web asset minimization

When I've had time for 'side projects' lately I've mostly been working on preparing the ground for ownCloud 8 in Fedora, trying to get out ahead of upstream changes and package new library dependencies.

ownCloud's web asset minification

Today I wound up looking at OC's new web asset minification stuff. A few releases back OC delivered CSS and JavaScript itself and minified them in real time, using minifiers ripped out of the Mediawiki source code. That was pretty ugly.

In ownCloud 7 they switched to using Assetic for asset management, which is a lot better. It didn't do any minification at that point, though.

With the PR linked above, OC 8 does minification using Assetic filters. This means it grows some new dependencies on the minimization libraries that back the Assetic filters.

Not surprisingly, this being the world of PHP, there's some fun craziness here. Craziness follows - but if you're interested in a performance comparison of available PHP web asset minifiers, skip it and look down a bit further.

The filters ownCloud currently uses are Assetic's CssMinFilter and JSMinFilter. To back those, ownCloud's composer.json now pulls in mrclay/minify and natxet/CssMin.

natxet/CssMin

I started out by taking a look at natxet/CssMin, which isn't too horrible. It loses some points for not being sanely laid out (there's one source file with a whole bunch of classes in it and it just uses classmap autoloading...), but basically it minifies CSS and that's it. It was fairly simple to build a package. It claims to be just a github mirror of this 'cssmin' project, but in fact it's clearly being actively maintained and has developed on from that project; I'm treating it as its own project, now. It doesn't appear to be a port or rewrite of any other minifier.

mrclay/minify and its exciting assortment of minifiers

Then I looked at the JS minifier, though, and started to encounter the crazy. mrclay/minify looks, to my monkey eyes at least, like a fairly hairy ball of olde-worlde PHP craziness. It's not just a minification library, or even several minification libraries. In its own words, it's "an HTTP content server", i.e. it's sort of trying to do the same thing as Assetic, only it's a rather older and smaller implementation. But because choice is oh so tasty, it includes at least three CSS minifiers and two JS minifiers, and lets you pick whichever you like!

• It has its own CSS minifier, Minify_CSS_Compressor.
• It includes the github project YUI-CSS-compressor-PHP-port - which is, as the name suggests, a PHP port of the CSS bit of Yahoo's YUI Compressor, which is written in Java. This is named CSSmin.php.
• It also, for no goddamn reason I can see, ships a different but far less complete port of the YUI Compressor, as lib/Minify/YUI/CssCompressor.php.

For JavaScript:

• It includes JSMin.php. This (so far as I can figure out) originated as a port of Douglas Crockford's JSMin (which is written in C), by Ryan Grove. It lived at Google Code until it was moved for license reasons (see long bit below!) to GitHub, where it is now very definitely unmaintained. mrclay has continued to work on it in his tree since it was abandoned by Mr. Grove, so the mrclay/minify version is now a fork. Again see below for details, but briefly, this code is clearly non-free by Debian and Fedora standards, as it imposes a 'field-of-use' restriction.
• It includes JSMinPlus.php. This is a copy of JSMin+, which is by all appearances abandoned by its author (though there were two years between the releases of 1.3 and 1.4, so who knows). Despite the name, JSMin+ is a completely different project from (anything else at all called) JSMin, it is not an unofficial successor nor is it based on JSMin (again, whatever you consider to be 'JSMin' exactly - the original code, or any of its various ports) in any way.
• It includes lib/Minify/ClosureCompiler.php, which is a wrapper around the offline Java version of Google's Closure Compiler - you're expected to provide it with a copy of the CC .jar file.
• Not at all confusingly it also provides lib/Minify/JS/ClosureCompiler.php, which is an entirely different way to use the Closure Compiler - this time via Google's public REST API for it.

When I say it 'includes' or 'has' something, of course, I don't mean it sensibly expresses a dependency on a copy of it, or anything. In classic fashion for the PHP 'Wild West' interregnum between the reigns of PEAR and Packagist/Composer, copies of the libraries are just dumped willy-nilly into minify's tree, not in any particularly obvious layout (in fact most of them appear to be PSR-0 compliant relative to lib/, but you have to poke about a bit to figure that out), without any kind of manifest to explain clearly what they all are, never mind what they're for.

Interlude: Adam is sad

So I was, obviously, rather reluctant to just package up the mrclay 'minify' for Fedora simply in order to satisfy ownCloud's need for a JavaScript minimizer. This kind of haphazardly bundled source tree is a goddamn nightmare to package in a guideline-compliant way just in general, and the redundancy was too absurd for me to swallow - if you're not keeping count, so far we've found that minify has three CSS minimizers, two JavaScript minimizers, and two different ways you can use yet a third JavaScript minifier it doesn't directly include. And don't forget that natxet\CssMin is not the same thing as minify's CSSmin.php (which, if you've forgotten, is tubalmartin's port of the YUI compressor). If I'd just gone ahead and packaged minify, I'd have been packaging six minifier libraries and eight minifier approaches in order to provide ownCloud with two minifiers. No. Not happening.

My next thought was to just package the actual minifier ownCloud uses, via Assetic's JSMinFilter, which is JSMin.php - not any of the other bits of minify. It looks like it'd pretty much lift right out (not surprisingly, as it was initially a third party lib that minify dumped into its tree, see above). At this point, though, I run into another rather larger problem which I mentioned briefly above.

JSMin, freedom, and the JSON License

JSMin.php is not free. This is because the original, Douglas Crockford-written JSMin is not free; no port or derivative of it is free either. It's licensed under the MIT license, but with an added clause:

"The Software shall be used for Good, not Evil."

There's a page where the story of JSMin being kicked off Google Code for this is related, which includes an excerpt from a talk the author gave where he plays the whole situation for laughs and talks about how ha ha funny it is that Google and IBM want him to allow them to use his software for evil, and that's all nice and cute, but the best legal advice available to Debian, Red Hat and (to my knowledge) all other authorities on software licensing is that such restrictions clearly make the software non-free. License wonks refer to this license as 'the JSON license' and it is explicitly listed as non-free by the FSF, Fedora, and Debian (it's hard to find a good reference for DFSG determinations, that's the best I could get).

So, I can't possibly have Fedora's OC package use that code regardless of how or where it comes from. Darn.

Yet more minifiers (no, of course we're not done yet)

So I started looking into alternatives, which (me being me) turned into a bit of a review of PHP web asset minifiers. Here's the ones I found beyond those listed above:

• JSqueeze, a native PHP JS minifier
• JShrink, a different native PHP JS minifier
• matthiasmullie/minify, a native PHP JS and CSS minifier (not at all the same thing as mrclay/minify)

There may have been others, but I'm sort of losing the will to live at this point. As I was planning to suggest ownCloud switch away from JSMin, it seemed prudent to check out all the alternatives, so I hacked up a script for doing some quick and dirty benchmarking on minifiers. (If you want to try it, you'll have to download all the minifier libs and dump them in the directory with the script, dump some .css and .js files in the same directory, and edit CSSmin.php to rename the class to CSSmin1 so it doesn't conflict with CssMin.php - I told you it was dirty).

Benchmarking minifiers

I dumped all ownCloud's CSS and JS assets (minus translations) into a directory and wrote a thing which just calls each minifier on all the files for its asset class, saves the output with a unique extension, and tells me how long it took. Then I compared the sizes of the minified files. The contenders are:

JS

• mrclay/minify's JSMin.php, (today's git master)
• mrclay/minify's JSMinPlus.php, which I believe is still identical to upstream JSMin+ 1.4
• JSqueeze (today's git master) - tested as 'strong', with its global var, method and property renaming enabled, and 'safe', with it disabled
• JShrink (today's git master)
• matthiasmullie/minify (today's git master)

CSS

• mrclay/minify's CSSmin.php (today's git master), which is tubalmartin/YUI-CSS-compressor-PHP-port
• mrclay/minify's Minify/CSS/Compressor.php (today's git master)
• natxet/CssMin (today's git master)
• matthiasmullie/minify (today's git master)

My CSS results are probably a bit questionable as OC has fairly little CSS, but here they are:

[adamw@adam jstest]$ php ./mintest.php 
CSSmin (tubal) time: 0.15763902664185
CSS Compressor time: 0.099493980407715
natxet time: 0.51680493354797
MM minifiy (CSS) time: 0.090332984924316
[adamw@adam jstest]$ for i in css compressor cssmin natxet mmmcss; do echo $i; du -c *.$i | grep -i total; done
css (original)
360 total
compressor (CSS Compressor)
344 total
cssmin (CSSmin tubal)
340 total
natxet (CssMin natxet)
344 total
mmmcss (MM minify)
344 total

Compressor and Matthias Mullie's minify are the fastest, tubalmartin's CSSmin is a bit slower, and natxet/CssMin is quite a lot slower...but in practice, for OC's purposes, they'd all run pretty much instantly, the 'slowest' takes half a second. They're all almost the same in file size - tubalmartin's 'wins' by 4KB but that's probably in the margin of error (which I don't know how to calculate cos I'm not a scientist), and they only save about 5% on the original size, not really worth crowing about.

The JS results are rather more interesting, as OC has a lot of JS.

JSMin time: 13.471320867538
JSMinPlus time: 15.368160009384
JSqueeze (strong) time: 6.8297760486603
JSqueeze (safe) time: 6.8371610641479
JShrink time: 11.497622013092
MM minifiy (JS) time: 19.345649003983

js
3364 total
jsmin
2468 total
jsminplus
2408 total
jsqueeze.strong
2192 total
jsqueeze.safe
2220 total
jshrink
2468 total
mmmjs
2464 total

I noticed that JSqueeze's global renaming feature seems to cause quite a lot of issues - half the bug reports in its github repo seem to be related to it - so it seemed prudent to test it both ways: the 'safe' version is simply $jsqueeze->squeeze(file_get_contents($filename), $specialVarRx=false) instead of $jsqueeze->squeeze(file_get_contents($filename)) (the 'strong' version).

JSqueeze rather beats the pants off of everything else, there - it's nearly twice as fast as the closest competitor, and does substantially better on file size, even in its 'safe' version.

edit 2014-12-30: After initially writing this post I managed to get a test instance of ownCloud git master up and running and tried plunking JSqueeze in in place of JSMin and, lo and behold...it had a bug.

edit 2015-01-01: p@tchwork fixed the bug mentioned in the previous edit; JSqueeze 2.0.1 works well with ownCloud in my testing. 2.0.1 also namespaces the class (which makes the layout a PSR-4 one), and disables the global renaming stuff by default (i.e. 'safe' not 'strong' is now the default). Also, Matthias Mullie substantially sped up his minifier: after that commit his is the very fastest for me, at 5.8328921794891 seconds. Its compression ratio still ranks in the group at the back of the pack, though.

What did we learn on the show tonight, Adam?

PHP developers, for the love of all that is freaking holy, can you all please just goddamn well sit down together and decide on one implementation of trivial things like asset minifiers, and just work on that? Yeeeeeeeeeeeeeeesh, people. And please don't dump your code's third party dependencies into its tree by hand, without an apparent plan or a manifest. At least use Composer, that gives us a fighting chance at figuring out what the hell you've got in there and splitting it out. And, you know, sit down and really think about whether your minifier blob needs seven different goddamned minifier implementations in different places, half of which are variants of each other...

All things considered for JS I'd recommend using JSqueeze, JShrink or Matthias' 'minify', in about that order - those three all seem to be actively maintained and following good current practices. Avoid JSMin due to the licensing issue and because it's not very actively maintained, and JSMinPlus as it doesn't seem to be maintained at all.