Lander

Purpose

Phase 0

Coordinates building of source packages into a full image, testing and publication of those packages and image upon successful completion of the tests.

Future

Expands upon Phase 0 by adding MPs as a build source and adds the ability to test packages without building a full image. It then coordinates the building and testing of MPs or source packages and publication into the archive. The Lander merges the MP after a successful publication while ensuring that MP’s target branch has remained unchanged.

Deployment

Phase 0

  • Can run as a Juju service.
  • Needs relationship to Ticket System, PPA Assigner, Branch/Source Builder, Image Builder and Test Runner.
  • No public access needed.

Future

  • Incoming requests are queued and played back in the event of a restart. On restart, checks are made to determine the state of each request.

Interactions

Phase 0

  • Ticket System - Provides build requests.
  • Branch/Source Builder - Source packages are dispatched to the builder.
  • PPA - Location where the package build will take place and published.
  • PPA Assigner - Provides a single PPA to perform package builds.
  • Image Builder - Builds a complete image from the PPAs and packages.
  • Test Runner - Runs the specified test (if any) on the produced image.
  • Data Store - The logs and artifacts are archived to the data store.

Future

  • Ticket System - Provides meta information regarding the project owning a branch (i.e. what trunks to process, which tests to run, etc.). Status updates are sent to the project manager.
  • LP Merge Proposal - MPs are the unit of work managed by the trunk delivering system, if the necessary criteria is in place, the MP will be merged to trunk.
  • Branch/Source Builder - MP branches are dispatched to the builder.
  • PPA - Location where the package build will take place.
  • PPA Assigner - Provides a single PPA to perform package builds.
  • Test Runner - Runs the specified test (if any) on the packages from the PPA.
  • Image Builder - Builds a complete image from the PPAs and packages.
  • Data Store - The packages are copied to the archive from the PPA on successful completion of testing.

Development Plan

Phase 0

  • Define APIs.
  • Deploy an instance.
  • Allocate a PPA (interact with PPA Assigner).
  • Build a source package in the PPA.
  • Monitor a PPA for build status.
  • Report status (interact with Ticket System).
  • Build an image (interact with Image Builder).
  • Initiate integration tests (interact with Test Runner).
  • Publish packages from one PPA to another PPA.
  • Handle binary build failures.
  • Handle image build failures.
  • Handle integration test failures.

Future

  • Spawn an MP build (interact with Branch/Source Builder).
  • Ensure that trunk has not changed during execution.
  • Merge an MP to trunk.
  • Binary copy a package from PPA to the archive.
  • Monitor archive for status of package.
  • Full end-to-end “Direct merge proposal to trunk” success case.
  • Handle source build failures.
  • Handle trunk changing during execution failures.
  • Handle failure to merge.
  • Handle push to archive failures.
  • Handle component failure recovery/restart.

Design

Phase 0

A Lander service handles the workflow by using Jenkins to schedule individual tasks. When a request is received to the Lander from the Ticket System, it triggers the master Jenkins job with the request parameters. The master job then triggers a series of child jobs to execute the workflow.

The child jobs themselves execute a service handler. The service handler is responsible for setting up the progress queue and triggering the service via its REST API. As the service handler runs, it outputs the progress updates it receives to the console (to be viewed via Jenkins) and pushes the progress update to the Ticket System.

When the service completes, the service handler closes with a return code matching the status of the service itself. The result data from the service is archived as a Jenkins artifact. That data is combined with the existing set of job parameters to be used as input to the next Jenkins child job.

Only one master job may execute at a time. If additional build requests are received, they will be queued by Jenkins.

The Lander supplies regular notification events to the Ticket System while a job is executing. There are no notifications sent when the service is idle.

The Lander archives the Jenkins console logs and the archived results of each job to the data store.

Future

The Lander will support concurrent build requests as long as they don’t have conflicting sources. Execution priority is given to first class ticket requests.

API

Phase 0

execute_request

Schedules a new request for building source packages and creation and test of an image.

URL Pattern

Parameters

  • ticket: The identification handle for a ticket request.
  • source_packages: an array of data-store URLs to the source package files.
  • binary_packages: an array of binary package names to use when constructing the image.
  • series: the Ubuntu series to base the image (i.e. ‘trusty’)
  • base_image: the image to use as source for the new image
  • progress_trigger: the amqp exchange to use for sending progress events.

Example

curl --dump-header - -H "Content-Type: application/json" -X POST --data  '{"ticket": "1234", "source_packages": ["http://data_store-url/autopilot.tar.gz", "http://data_store_url/ubuntu-ui-toolkit.tar.gz"], "binary_packages": ["python-autopilot", "ubuntu-ui-toolkit-autopilot", "libunity8"], "series": "trusty", "base_image": "trusty-desktop-amd64.iso", "progress_trigger": "lander-1234"}' http://lander-url:8080/api/v1/execute_request

status

Returns the current status of the lander service and state of the currently running jobs for debugging purposes.

Url Pattern

Example

curl --dump-header - http://lander-url:8080/api/v1/status