Test Runner

Purpose

  • Performs package testing on a given image.

Future

Expands upon phase 0 by allowing a set of PPAs and additional packages to be added before running the tests, support using autopilot on a touch device, perform a custom test (TBD), support executing tests on bare metal.

API for submitting test run requests

The ‘Test Runner’ exchanges requests/responses with the ‘Lander’.

test_package

This run the DEP8 tests for each package in ‘package_list’ on ‘image_id’ and is sent by the ‘Lander’.

test_image(ticket_id, image_id, package_list, progress_trigger)

The caller is responsible for providing a unique ‘progress_trigger’ identifier. This unique identifier is used to define the rabbit queue and the data store container.

This is composed of:
  • boots the image from ‘image_id’,
  • install each package from ‘package_list’ monitoring its execution and emitting progress messages (see below),
  • collect the test results,
  • send a ‘done’ message.

The ‘progress’ and ‘done’ messages are sent to a rabbit queue to communicate with the ‘Lander’.

done

This ends the processing request and is sent to the ‘Lander’.

done(progress_trigger, status=[FAIL, SUCCESS], test_result_urls)

‘test_result_urls’ is a list of subunit streams uploaded to the data store.

progress

This is sent to the ‘Lander’ to provide feedback during the test run.

progress(progress_trigger, message='% or ETA', current, total)

Based on test execution feedback, a progress message is sent at regular intervals until completion. ‘total’ is the number of tests to execute, ‘current’ is the rank of the test currently running. Both can be empty if this info is not available.

Deployment

  • This can be deployed as a juju service, but the actual test runners may be on bare metal (vms only for phase 0).
  • Needs relationship to the Lander and the Data Store.
  • Operations are transient, no need to save state.

Interactions

  • Lander - Supplies test requests and waits for results.

Development Plan

Phase 0

  • Setup an instance from the image builder id
  • Run the dep8 tests for a list of packages
  • Collect the test results
  • Send test results to the lander (via the data store).

Future

  • Collect a set of artifacts (defined by the ‘Lander’)
  • Execute test on bare metal (including touch device)
  • Execute autopilot (requires an emulator or a graphic card)
  • Execute arbitrary test (may require additional packages)