With the Aikau framework for Share construction we have created a large number of widgets with some underlying code from Dojo’s UI Library, Dijit. It makes sense to automatically test the Aikau widgets in isolation and therefore we have created a test framework using Intern (http://theintern.io/) to do just that. The tests run quite fast and with predictable results. It is now easy to see if a recent addition or update to an existing widget has caused a bug or regression.
To perform tests we have written a number of test web scripts that contain widget models. The widget models are usually quite simple but occasionally have a more complex structure to expose edge case behaviours. Some test web scripts also contain intentionally broken models to test the catching of errors. For testing purposes the test web scripts are run using Jetty as we do not need any of the more complex overhead from Share.
To run a suite of tests we do the following:
Launch an instance of an Ubuntu virtual machine using Vagrant
The virtual machine provides a set of test browsers, drivers for those browsers and a running instance of Selenium that Intern can target
Run up the test web scripts using Jetty
We do not need all of Share so a basic Jetty server is fine here
Run tests against the test web scripts using Intern
Each of the tests will be run according to the Intern configuration and against the browsers defined and requested therein
Note: One of the tools we install when setting up testing is Grunt (http://gruntjs.com/). Much of the process of running tests is handled for us in commands we have already added using Grunt. In fact, once we have set up the testing framework for the first time, we really only need two commands to start a test run - firstly launch the virtual machine and then secondly run the tests.
Step 1 - Prerequisites
The testing framework for Aikau makes use of a number of technologies and you will need the following free downloads to proceed. Please install with the default settings:
I am assuming that you can use a command line and have a vague idea of what a virtual machine and Vagrant are. If not, please read about it first: http://www.vagrantup.com/
Step 2 - Installation
Having downloaded and installed a current version of the Alfresco code base, open a command prompt and navigate to the following Alfresco directory:
Run the following command to install Node dependencies:
>> npm install
If you’re interested to know what is being installed you can look at file package.json in the directory above which contains a list of the dependencies. Once you have installed all of the components required for the test framework, you should be able to launch a virtual machine with vagrant for the first time. The first time you run this process it may be slow as it has a lot to download:
>> g vup
Note: This command is ‘g’ for grunt and ‘vup’ for vagrant up.
When the ‘g vup’ command has completed, if it has been successful you should be able to observe the Selenium instance running here:
The selenium console should look something like this:
Step 3 - Running the suite of tests
With an instance of Selenium available on the virtual machine, you should now be able to run Intern by issuing the following command:
>> g test
Note: This command is ‘g’ for grunt and ‘test’ for run intern test suite (virtual machine).
This command first checks if the test application server is running and launches it if not. Once it is happy that the server has started completely it will proceed to launch the Intern test runner. You should see the test suite run through a large number of tests (about 110 tests run twice for two different browsers at time of writing) and log them to the console. Hopefully they will all pass.
This is the sort of output you should expect to see once the initialisation steps have been performed:
>> Starting 'AccessibilityMenuTest' on chrome
>> Test page for 'AccessibilityMenuTest' loaded successfully
>> AccessibilityMenuTest: Find the menu element
>> AccessibilityMenuTest: Find the heading text
>> AccessibilityMenuTest: Find the menu items
>> AccessibilityMenuTest: Find the first target
>> AccessibilityMenuTest: Find the second target
>> AccessibilityMenuTest: Find the first menu link - which links the first target
>> AccessibilityMenuTest: Hit the browser with a sequence of different accesskey combinations and the letter 's' for a nav skip
>> Starting 'SemanticWrapperMixinTest' on chrome
>> Test page for 'SemanticWrapperMixinTest' loaded successfully
>> SemanticWrapperMixinTest: Check NO_WRAPPER dom is correct
>> SemanticWrapperMixinTest: Check GOOD_WRAPPER dom is correct
>> SemanticWrapperMixinTest: Check BAD_WRAPPER dom is correct
>> SemanticWrapperMixinTest: Check LEFT_AND_RIGHT_WRAPPER dom is correct
>> Starting 'Pie Chart Test' on chrome
>> Test page for 'Pie Chart Test' loaded successfully
>> Starting 'PublishPayloadMixinTest' on chrome
>> Test page for 'PublishPayloadMixinTest' loaded successfully
Note: When a test run is complete the Jetty server is left running. If there are any failures it is possible to immediately investigate if something catastrophic has occurred by simply observing the test web script in a browser.
There are several grunt commands that can be used with Vagrant, the test application server and Intern. Here are the ones we've already seen and a few more, all of which should be run from the directory shown above:
Short for ‘vagrant up’ this will launch a virtual machine instance with Vagrant
Short for ‘vagrant down’ this will stop a running instance of a Vagrant virtual machine
Short for ‘vagrant clean’ this will delete an existing instance of a Vagrant virtual machine
Run up an instance of the test application server and run the Intern test suite against it
Start the test application server
Stop the test application server if it is running
Short for ‘new test’ this command will restart the Vagrant virtual machine, restart the test application server and finally run the Intern test suite against them
Short for ‘update test deployment’ this command will bring down the test application server, rebuild slingshot and then relaunch the test application server with any file modifications that have been made
Adding a test
If you wanted to add a test of your own there are three steps to the process:
Create a test web script
Create an Intern test
Add the test to the Intern Suites.js file
Let's investigate those steps individually:
Create a test web script
Test web scripts all live in this location or a sub-directory of it:
The specific way in which an Intern test addresses a test web script can be seen if any of the existing tests is viewed. Pay close attention to this part of most tests which is the point at which the test web script is loaded:
The testing framework supports testing against a local instance of Selenium rather than a VM instance as described above. It is also possible to run a coverage report on the test framework to indicate the amount of the code base being interrogated. The details of these alternative scenarios will follow in another post.
You must be a registered user to add a comment. If you've already registered, sign in. Otherwise, register and sign in.