Extension testing

MediaWiki provides a rich environment to its many users. Because of its open source nature, many people develop functionality that is not provided out of the box.

Wikiation does wiki hosting for organisations. When creating a new extension changes has to be made. Quality within organisation is of very high interest. So if a extension has been modified in one detail, the complete extension has to be tested again. Because this change may create an other bug. That is not acceptable. So testing must be intensive. So when there are changes, testing is required again.

Brion Vibber, the CTO of the Wikimedia Foundation, releases several times a year new stable versions of the software. Its recommended to upgrade installations because improved functionality, security and localisations become available. This does results, for each new MediaWiki version, changes to existing extension which may not work any more as they did in the previous version.
So if there are external changes extension testing is required.

There are more then 1000 MediaWiki extensions If a customer of Wikiation does have an interest in a specific extension, Wikiation would like to make it available. But does the exension work as expected? Does the extension make modifications in the core of MediaWiki which might influence the behaviour of the rest of the system? So when a extension written by others is to be used, extension testing is required.

All tree cases are about quality. Does it work as it supposed to work? Is it reproducible. Will my current wikipage, with a specific extension, produce a identical result with a new version of the extension. People are very good in creative tasks, but quite bad in working accurately. Proper testing requires accurate work. And if you want really quality each change requires that everything is tested again.

This task is so huge that a human hardly can do it. The most often used solution is, less testing. That does work but it has to be paid by less quality. Wikiation did search if automated MediaWiki extension testing did exists. It did not. Because of the quality requirements, Wikiation decided to develop its own extension testing system. Each extension on any MW version has to be tested in an automated way. With predefined test sets for specific extensions in specific situation a predefined result has to be generated by the extension. If it does, the status is Ok otherwise it fails. A failure may be an error, but it also may be the wished for improvement. Only a human can be the judge of this.

Wikiation does use MediaWiki software as a hosting company, it could keep the results of this extension testing all to it self. But I do feel it is good to return something to the MediaWiki community. So this extension testing is free to use. The developed software is provided as open source under the GNU 3 license. The documentation and test results at http://extensiontesting.wikiation.nl is based on the Creative Commons Licence. And if you need a CleanMachine for testing, for a very short period, you can use it free of charge. The only thing within extention testing were all rights are reserved is the extension testing logo. I hope you can live with that.

I am not really a blogger. I did start the extension testing. GerardM took the initiative to start this blog. I do thank GerardM also for his help to localise the extensions Wikiation has build.

There is still much to do. Please do your testing, and use the test results others have made. If quality is important to you, join.