What is this pull

This pull contains a first set of System tests based in Codeception Framework

They automate a Firefox browser and execute a set of tasks to ensure that all works as expected.

This first set of tests includes:

• basic joomla installation
• Administrator login
• set up Error reporting as "development" in Joomla administrator

See it in action

If you want to see this tests being executed you can check the following video: https://www.youtube.com/watch?v=-GYou8kWpYI

Running the tests in your machine

This pull request includes a Markdown file called README.md at tests/codeception/ with step by step instructions for running the tests:

The first step to execute the System tests at Joomla-CMS a Joomla website. To do it automatically you can execute the following commands:

cd tests/codeception
composer install
# The following comand uses a Joomla Framework App that downloads the latests Joomla
php cli/getjoomlacli.php

note: to execute the previous commands you will need Composer in your system. See https://docs.joomla.org/Testing_Joomla_Extensions_with_Codeception#Testing_with_Codeception.

Rename tests/acceptance.suite.dist.yml to tests/acceptance.suite.yml

Modify the configuration at tests/acceptance.suite.yml to fit your server details. Find the instructions in the same file: https://github.com/joomla/joomla-cms/tests/codeception/acceptance.suite.dist.yml#L3

Run Selenium server (is the software that drives your Firefox browser):

# Download Selenium Server
curl -O http://selenium-release.storage.googleapis.com/2.41/selenium-server-standalone-2.41.0.jar

# Go to the folder were you have downloaded the file and start the Selenium Server
java -Xms40m -Xmx256m -jar ./selenium-server-standalone-2.41.0.jar

Execute the tests:

cd tests/codeception
php vendor/bin/codecept build
php vendor/bin/codecept run tests/acceptance/installation/ --steps 
php vendor/bin/codecept run tests/acceptance/administrator/ --steps 

You can also execute the tests using runsystemtests.sh file

What is this pull about?

This pull is not intended to replace the current set of Webdriver-Nearsoft Selenium tests (see https://docs.joomla.org/Running_Automated_Tests_for_the_Joomla_CMS). Is just proposing a new base for System Testing.

The main idea is to create a set of tests that can be reused by external Extensions for example https://github.com/joomla-extensions/weblinks. So Weblinks can have it's own tests but when it needs to install joomla or set Error Reporting in Joomla as "Development" it can reuse the current tests. See: http://codeception.com/docs/07-AdvancedUsage#Running-from-different-folders

acknowledgment

This test are result of the work of the Automated Testing Working Group https://docs.joomla.org/Automated_Tests_Working_Group

Specially thanks to @puneet0191, the present work is based on his http://mustached.org/ project. Also thanks to @ronnikc for his support.