Peter-Paul Koch warns on automated tests and WebKit compatibility variance
Late in 2012, mobile platform strategist Peter-Paul Koch received sponsorship for QuirksMode.org, which he said would enable him to spend more time researching web standards and working on compatibility tables that would be shared with webplatform.org.
We spoke to Koch about his work, how his tests are written and why developers should be more cautious regarding testing on a per-engine basis.
.net: You’re putting a lot of effort into your mobile tables. Is this something that wasn't being done to this extent elsewhere?
PPK: No, it's not really being done. The tables that come closest to mine are those by Max Firtman and they focus on HTML5 APIs.
I don’t believe in automating browser testing or scores, and so I don’t really count tests such as The HTML5 test. Then we have Can I Use…, which is useful but sometimes doesn’t give correct browser information.
As far as I know, I'm still the one that does the most detailed tests — and the only one to publish test pages as well as results.
.net: How do you go about writing the tests?
PPK: Slowly! Sometimes it's not easy to figure out what is meant in a specification, especially when there are only two implementations that are subtly (or wildly) different. Fortunately, I have a lot of experience in browser-test writing, and so I know how to prevent common pitfalls.
For example, at first it looked as if Opera Mini did not support CSS classes, but that is obviously nonsense. The problem turned out to be that I test for the support of classes by giving a test element font-style: italic. Many Opera Minis don’t support that style. Because I’d encountered this before, I knew I had to change test styles. And the MeeGo browser does not support font-variant: small-caps. Same story.
.net: In your recent article about selectors and columns, you state browsers using the same WebKit build have varying compatibility. Does this further highlight how devs should be careful regarding cutting-edge techniques and more rigorously test across devices?
PPK: Yup. There is no WebKit on mobile. There are at least twelve different browsers (excluding versions that use WebKit as their rendering engine), but they do not necessarily resemble each other.
The best example is the -webkit-column-span declaration. It turns out WebKit dropped support recently. This is the weirdest compatibility issue I have ever encountered because it's not possible to tie in support with specific WebKit versions. So support for this declaration is a mess, and the fact that a browser uses WebKit says exactly nothing.