Ticket System

Purpose

The Ticket System contains project-centric knowledge such as location of trunk names of integration tests, point of contact, etc. as well as the ticketing management for the CI Engine which includes the status of current MPs being built an

It maintains the following sets of data:

  • The history and status of tickets (the ticket tracker).
  • The definition and configuration of projects, their branches, source packages and produced binary packages (the project manager).
  • The test database. (This will not be part of phase 0)

Deployment

  • Can run as a juju service.
  • Runs as a persistent service, when it goes down, the engine halts.
  • State needs to serialized whenever it is updated, so that status can be regenerated on a restart.

Development Plan

Phase 0

  • Define API.
  • Define backend ticket storage.
  • Define backend project specification storage.
  • Handle status updates from the Landing Manager.
  • Provide read API for the Web Server.

TODO:

  • Bundle the ticket creation API
    • For speed, I have left the different APIs to fully create a ticket as separate. If/when there is time, it would be much improved to bundle this together into one single ‘Bundle API’

Future

  • Integrate with the Landing Manager, supply metadata to build and test a single project and generate an automatic low-cost ticket (phase 0 will accept only source package uploads directly, the branch polling for new merges and creation of
  • Define backend test database storage, and if it should be a separated system component.

Interactions

Ticket System

  • Provide API for Web UI
  • Provide API for Landing Manager to update tickets and subtickets. Also provide API for any system component that needs information about a ticket or subticket.

Project Manager

  • Provide API for any system component that needs information about a project/branch/merge proposal/source package/binary package.

Design

Project Manager

In phase 0 the project manager will store and manage information about:

  • Source package upload: component that will be tracked by a subticket. Is a direct source package upload.
  • Source package: a source package as found in the archive.

We’ll be provided a list of binary packages that should be monitored and tested, and a script outside project manager will generate the respective sources list.

This data will be updated by submitting a file to project manager, in JSON format, that should contain the binary name, source package name and version per entry.

For next phases, we’ll extend it to store info about:

  • Binary package: a binary package as found in the archive, produced by a known source package.
  • Upstream project: a launchpad project, that might have branches.
  • Bazaar branches: launchpad bazaar branches, might be associated with a source package. A branch can produce only one source package at a time.
  • Merge proposal: a launchpad merge proposal, that will be tracked by a subticket.

Ticket System

Tickets and subtickets

A ticket is a way of grouping parts of a bug fix or feature. In the ticket you’ll describe the objective, add relevant information like blueprints or other references and the ticket status will reflect the status of its subtickets. Other system components will rely on the ticket status to take actions.

A subticket refers to a source package upload or a merge proposal.

When a developer submits the package upload request, the ticket system creates a subticket for each uploaded package and/or merge proposal, and a ticket that contains all of them.

In phase 0, it won’t be possible to add more subtickets to a ticket after ticket is created (no editing). To create a ticket with several parts, the developer needs to submit all of them at once. If a subticket fails, the developer needs to submit another request that will generate another ticket with a subticket.

The ticket and subtickets will keep only the current step of the workflow they are and their status, as defined in ticket and subticket models.

In phase 1, if the subticket processing fails, the subticket is closed as failed, and the developer can submit another source package upload to the same ticket. It’ll also be possible to add merge proposal subtickets.

Ticket System features

Create, monitor and manage ticket related activities

  • Define and create a ticket including:
  • describe the targeted feature
  • Links to any bugs/BPs/other design docs that are related
  • People/teams working/owning on the ticket
  • Components involved (packages/PPAs/feature branches/infrastructure required to code/test/build/deploy)
  • PPA(s) dedicated to the ticket
  • Describe the test suites that need to be run (smarts can be added in the long run to assist users in choosing the Right test to run). In phase 0 it consists on the dep8 tests in all source packages that are relevant.
  • Knowing what image is supposed to be ‘frozen’ for developing against
  • Provides test and status of a ticket:
  • Built images in the image store
  • Status of the ticket as it progresses through the CI engine
  • Clear indication of merge/build/test failures and where the failure occurred, make it obvious to developers what a corrective course of action should be
  • Provide a means to execute a landing when all criteria have passed
  • Provides ticket management:
  • list and status of all tickets

REST APIs

Tickets

List tickets

List all open tickets.

curl --dump-header - http://localhost:8000/api/v1/ticket/

Create ticket

Create a ticket from one or more source package uploads.

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"owner": "owner@example.com", "title": "My first ticket", "description": "This if my first ticket. See what it can do", "bug_id": "12345"}' http://localhost:8000/api/v1/ticket/

Get ticket

Full ticket

Returns ALL information about the given ticket.

curl --dump-header - http://localhost:8000/api/v1/fullticket/1/

Model info

Return the ticket model info only for the given ticket.

curl --dump-header - http://localhost:8000/api/v1/ticket/1/

Get open tickets

Show a list of all open tickets.

curl --dump-header - http://localhost:8000/api/v1/opentickets/

Get ticket status

Get the status of all tickets or tickets for a specific status. (Mostly for the WebUI)

All

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/

Queued

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=100

Package Building

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=200

Image Building

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=300

Image Testing

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=400

Package Publishing

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=500

Failed

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=999

Complete

curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=1000

Update ticket status

To be used by the CLI and the lander

curl --dump-header - -H "Content-Type: application/json" -X PATCH --data '{"current_workflow_step": "100", "status": "000"}' http://localhost:8000/api/v1/updateticketstatus/1/

Mark ticket complete

To be used by the lander.

curl --dump-header - -H "Content-Type: application/json" -X PATCH --data '{"current_workflow_step": "1000", "status": "1000"}' http://localhost:8000/api/v1/updateticketstatus/1/

Update subticket status

To be used by the CLI and the lander

curl --dump-header - -H "Content-Type: application/json" -X PATCH --data '{"current_workflow_step": "100", "status": "000"}' http://localhost:8000/api/v1/updatesubticketstatus/1/

Create source package upload

To be used by the CLI

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"sourcepackage": "/api/v1/sourcepackage/X/", "version": "1.0"}' http://localhost:8000/api/v1/spu/

Create artifact

Ticket

Valid types are: “RESULTS”, “LOGS”, “IMAGE”

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "my_artifact", "ticket": "/api/v1/ticket/X/", "reference": "http://path.to/artifact/", "type": "IMAGE"}' http://localhost:8000/api/v1/ticketartifact/

Subticket

Valid types are: “SPU”, “RESULTS”, “LOGS”

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "my_artifact", "subticket": "/api/v1/subticket/X/", "reference": "http://path.to/artifact/", "type": "SPU"}' http://localhost:8000/api/v1/subticketartifact/

Create subticket

To be used by the CLI

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"source_package_upload": "/api/v1/spu/X/, "ticket": "/api/v1/ticket/X/, "assignee": "test@example.com"}' http://localhost:8000/api/v1/subticket/

Project

Get source package

return all source packages

curl --dump-header - http://localhost:8000/api/v1/sourcepackage/

Add source package

This action would be completed by the CLI when it encounters a new source package that the ticket system hasn’t had before.

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "my-package"}' http://localhost:8000/api/v1/sourcepackage/

Get binary package

return all binary packages

curl --dump-header - http://localhost:8000/api/v1/binarypackage/

Models

Ticket System will be django + REST + Postgres. The phase 0 models are defined below with the proposed future models coded out below that.

Phase 0

sourcepackage

A source package is an existing source package in the Ubuntu archives. A source package record is created when a source package has changes submitted to the ticket system for the first time.

sourcepackageupload

A ‘Source Package Upload’ is a file uploaded by the user that is related to the changes being made. It will have a related source package and artifacts.

ticket

A ticket is created to get a change (or set of changes) into the Ubuntu archive. Creating a ticket allows the changes to be tracked through the processes of package building, image building, image testing and publishing as well as the results to be seen by the user.

subticket

Each source package upload that is added to a ticket will have its own subticket. A ticket can contain one of more subtickets. The subticket allows the user to track the progress and results of the package building for each source package upload.

artifact

Artifacts can be multiple things. When a ticket is initially created, it will have artifacts attached to it which are the source package upload files. Artifacts can also be test results and log files. They can be assigned to a relevant ticket or subticket based on the step of the process.

Future Apps/Models

person

class Person(models.Model):
   name = models.CharField(max_length=4096)
   email = models.EmailField(max_length=200)
   is_team = models.BooleanField(default=False)

testsuites

class TestSuites(models.Model):
   pass

project

class Project(models.Model):
   # Class that defines an upstream project.
   name = models.CharField(max_length=4096)
   display_name = models.CharField(max_length=4096)
   maintainer = models.ForeignKey("Person")
   contact = models.ForeignKey("Person")
   description = models.TextField()

branch

class Branch(models.Model):
   unique_name = models.CharField(max_length=4096)
   owner = models.ForeignKey("Person")
   type = models.CharField(choice=["trunk", "development", "maintenance", "regular"])
   project = models.ForeignKey("Project", null=True, blank=True)

binarypackage

class BinaryPackage(models.Model):
   name = models.CharField(max_length=4096)
   sourcepackage = models.ForeignKey("SourcePackage")
   seeded = models.BooleanField(default=False)

mergeproposal

class MergeProposal(models.Model):
    branch = models.ForeignKey("Branch")
    submitter = models.ForeignKey("Person")
    status = models.CharField(choices=["work_in_progress", "needs_review", "approved", "rejected", "merged"])
    lp_weblink = models.CharField(max_length=4096)

Person API’s - TODO

add_person

Add a person (or a team) to the database.

person

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "Chris Johnston", "email": "user@example.com", "is_team": "False"}' http://localhost:8000/api/v1/person/

team

curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "Canonical CI Engineering", "email": "team@lists.example", "is_team": "True"}' http://localhost:8000/api/v1/person/

get_person

Return all persons

curl http://localhost:8000/api/v1/person/

search by name

curl --dump-header - http://localhost:8000/api/v1/person/?name__exact=My%20Name
curl --dump-header - http://localhost:8000/api/v1/person/?name__iexact=my%20name
curl --dump-header - http://localhost:8000/api/v1/person/?name__startswith=My
curl --dump-header - http://localhost:8000/api/v1/person/?name__istartswith=my

search by email

curl --dump-header - http://localhost:8000/api/v1/person/?email__exact=user@example.com
curl --dump-header - http://localhost:8000/api/v1/person/?email__iexact=User@example.com
curl --dump-header - http://localhost:8000/api/v1/person/?email__startswith=user
curl --dump-header - http://localhost:8000/api/v1/person/?email__istartswith=User

show/don’t show teams

curl --dump-header - http://localhost:8000/api/v1/person/?is_team=True
curl --dump-header - http://localhost:8000/api/v1/person/?is_team=False