Branch/Source Builder

Purpose

Accepts one or more source package(s) and dispatches it to the provided PPA for building It then monitors the PPA for completion of the build and collection of results.

Deployment

  • Standard Juju service with a juju-deployer config
  • A service providing the REST interface
  • A rabbitmq system where the service will send messages to a work queue. A configurable set of workers will then be able to process the messages. The queue should provide fault-tolerance so that requests will always be handled.
  • No public access needed.

Interactions

  • Lander sends a “build_source” request to the service (this includes swift URLs from the data-store on what to build)
  • A request is placed in the queue (at each step below, the progress_trigger will be called to notify of status changes). * The source packages are dput to the PPA * The PPA is monitored for the completion of the build
  • The Branch Source Builder sends progress updates back to the Lander. When the build is complete, the progress message will indicate completion and provide links to the build logs.

Service Design

The main web-service will be a stripped down REST-ful server providing:

REST APIs

build_source

Request a PPA build of the provided sources.

URL Pattern

Parameters
  • ticket_id
  • cancel_url(can be null): a link to json url return {‘building’: true|false}
  • source_packages: An array of data-store URLs containing all of the source package files.
  • series
  • ppa: The PPA allocated by the ppa-creator for this operation.
  • archive
  • progress_trigger: A string used to create a dedicated message queue between the Branch/Source builder and the build_source caller.

Example

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"ticket_id": 1, "cancel_url": null, "source_packages": ["https://swift.canonistack.canonical.com/v1/AUTH_bucket_id/ticket.1/autopilot_1.5~dev.1.diff.gz", "https://swift.canonistack.canonical.com/v1/AUTH_bucket_id/ticket.1/autopilot_1.5~dev.1.dsc", "https://swift.canonistack.canonical.com/v1/AUTH_bucket_id/ticket.1/autopilot_1.5~dev.1_source.changes"], "series": "saucy", "ppa": "ppa_build_1", "progress_trigger": "lander_master-1-bsbuilder"}' http://bsbuilder-url:8080/api/v1/build_source

Progress is communicated at regular intervals using the following messages:

Waiting

The build has not yet started.

message = {
    "state": "WAITING",
    "source_packages": source_packages,
    "ppa": ppa,
    "progress_trigger": progress_trigger
}

In progress

The source package(s) has been accepted by the PPA and are building.

message = {
    "state": "STATUS",
    "source_packages": source_packages,
    "ppa": ppa,
    "progress_trigger": progress_trigger
}

Completed

The build completed successfully.

message = {
    "state": "COMPLETED",
    "exit": True
    "source_packages": source_packages,
    "ppa": ppa,
    "progress_trigger": progress_trigger,
    "logs": ["https://swift.canonistack.canonical.com/v1/AUTH_bucket_id/ticket.1/autopilot_1.5~dev.1.build.log"]
}

Failed

The build failed.

message = {
    "state": "FAILED",
    "exit": True
    "source_packages": source_packages,
    "ppa": ppa,
    "progress_trigger": progress_trigger,
    "logs": ["https://swift.canonistack.canonical.com/v1/AUTH_bucket_id/ticket.1/autopilot_1.5~dev.1.build.log"]
}

status

Useful for debug and monitoring. This method will return information like the number of worker queues and if they are busy or not.

URL Pattern

Example

curl --dump-header - http://bsbuilder-url:8080/api/v1/status

Worker Design

Steal logic from the daisy charm to set up a worker node. Then create a small python service using py-amqplib to pull messages off the queue. The service needs to respond to the caller’s progress_trigger so it can track the state. The messages we should send are:

  • STARTED - the message was pulled off the queue
  • INPROGRESS - the source packages were dput to the PPA and its now building
  • COMPLETE - the package has been built
  • FAILURE - an error occurred at any step in the process