Ticket System¶
Purpose¶
The Ticket System contains project-centric knowledge such as location of trunk names of integration tests, point of contact, etc. as well as the ticketing management for the CI Engine which includes the status of current MPs being built an
It maintains the following sets of data:
- The history and status of tickets (the ticket tracker).
- The definition and configuration of projects, their branches, source packages and produced binary packages (the project manager).
- The test database. (This will not be part of phase 0)
Deployment¶
- Can run as a juju service.
- Runs as a persistent service, when it goes down, the engine halts.
- State needs to serialized whenever it is updated, so that status can be regenerated on a restart.
Development Plan¶
Phase 0¶
- Define API.
- Define backend ticket storage.
- Define backend project specification storage.
- Handle status updates from the Landing Manager.
- Provide read API for the Web Server.
TODO:¶
- Bundle the ticket creation API
- For speed, I have left the different APIs to fully create a ticket as separate. If/when there is time, it would be much improved to bundle this together into one single ‘Bundle API’
Future¶
- Integrate with the Landing Manager, supply metadata to build and test a single project and generate an automatic low-cost ticket (phase 0 will accept only source package uploads directly, the branch polling for new merges and creation of
- Define backend test database storage, and if it should be a separated system component.
Interactions¶
Ticket System¶
- Provide API for Web UI
- Provide API for Landing Manager to update tickets and subtickets. Also provide API for any system component that needs information about a ticket or subticket.
Project Manager¶
- Provide API for any system component that needs information about a project/branch/merge proposal/source package/binary package.
Design¶
Project Manager¶
In phase 0 the project manager will store and manage information about:
- Source package upload: component that will be tracked by a subticket. Is a direct source package upload.
- Source package: a source package as found in the archive.
We’ll be provided a list of binary packages that should be monitored and tested, and a script outside project manager will generate the respective sources list.
This data will be updated by submitting a file to project manager, in JSON format, that should contain the binary name, source package name and version per entry.
For next phases, we’ll extend it to store info about:
- Binary package: a binary package as found in the archive, produced by a known source package.
- Upstream project: a launchpad project, that might have branches.
- Bazaar branches: launchpad bazaar branches, might be associated with a source package. A branch can produce only one source package at a time.
- Merge proposal: a launchpad merge proposal, that will be tracked by a subticket.
Ticket System¶
Tickets and subtickets¶
A ticket is a way of grouping parts of a bug fix or feature. In the ticket you’ll describe the objective, add relevant information like blueprints or other references and the ticket status will reflect the status of its subtickets. Other system components will rely on the ticket status to take actions.
A subticket refers to a source package upload or a merge proposal.
When a developer submits the package upload request, the ticket system creates a subticket for each uploaded package and/or merge proposal, and a ticket that contains all of them.
In phase 0, it won’t be possible to add more subtickets to a ticket after ticket is created (no editing). To create a ticket with several parts, the developer needs to submit all of them at once. If a subticket fails, the developer needs to submit another request that will generate another ticket with a subticket.
The ticket and subtickets will keep only the current step of the workflow they are and their status, as defined in ticket and subticket models.
In phase 1, if the subticket processing fails, the subticket is closed as failed, and the developer can submit another source package upload to the same ticket. It’ll also be possible to add merge proposal subtickets.
Ticket System features¶
Create, monitor and manage ticket related activities
- Define and create a ticket including:
- describe the targeted feature
- Links to any bugs/BPs/other design docs that are related
- People/teams working/owning on the ticket
- Components involved (packages/PPAs/feature branches/infrastructure required to code/test/build/deploy)
- PPA(s) dedicated to the ticket
- Describe the test suites that need to be run (smarts can be added in the long run to assist users in choosing the Right test to run). In phase 0 it consists on the dep8 tests in all source packages that are relevant.
- Knowing what image is supposed to be ‘frozen’ for developing against
- Provides test and status of a ticket:
- Built images in the image store
- Status of the ticket as it progresses through the CI engine
- Clear indication of merge/build/test failures and where the failure occurred, make it obvious to developers what a corrective course of action should be
- Provide a means to execute a landing when all criteria have passed
- Provides ticket management:
- list and status of all tickets
REST APIs¶
Tickets¶
Create ticket¶
Create a ticket from one or more source package uploads.
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"owner": "owner@example.com", "title": "My first ticket", "description": "This if my first ticket. See what it can do", "bug_id": "12345"}' http://localhost:8000/api/v1/ticket/
Get ticket¶
Full ticket
Returns ALL information about the given ticket.
curl --dump-header - http://localhost:8000/api/v1/fullticket/1/
Model info
Return the ticket model info only for the given ticket.
curl --dump-header - http://localhost:8000/api/v1/ticket/1/
Get open tickets¶
Show a list of all open tickets.
curl --dump-header - http://localhost:8000/api/v1/opentickets/
Get ticket status¶
Get the status of all tickets or tickets for a specific status. (Mostly for the WebUI)
All
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/
Queued
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=100
Package Building
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=200
Image Building
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=300
Image Testing
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=400
Package Publishing
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=500
Failed
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=999
Complete
curl --dump-header - http://localhost:8000/api/v1/ticketstatus/?current_workflow_step=1000
Update ticket status¶
To be used by the CLI and the lander
curl --dump-header - -H "Content-Type: application/json" -X PATCH --data '{"current_workflow_step": "100", "status": "000"}' http://localhost:8000/api/v1/updateticketstatus/1/
Mark ticket complete¶
To be used by the lander.
curl --dump-header - -H "Content-Type: application/json" -X PATCH --data '{"current_workflow_step": "1000", "status": "1000"}' http://localhost:8000/api/v1/updateticketstatus/1/
Update subticket status¶
To be used by the CLI and the lander
curl --dump-header - -H "Content-Type: application/json" -X PATCH --data '{"current_workflow_step": "100", "status": "000"}' http://localhost:8000/api/v1/updatesubticketstatus/1/
Create source package upload¶
To be used by the CLI
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"sourcepackage": "/api/v1/sourcepackage/X/", "version": "1.0"}' http://localhost:8000/api/v1/spu/
Create artifact¶
Ticket
Valid types are: “RESULTS”, “LOGS”, “IMAGE”
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "my_artifact", "ticket": "/api/v1/ticket/X/", "reference": "http://path.to/artifact/", "type": "IMAGE"}' http://localhost:8000/api/v1/ticketartifact/
Subticket
Valid types are: “SPU”, “RESULTS”, “LOGS”
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "my_artifact", "subticket": "/api/v1/subticket/X/", "reference": "http://path.to/artifact/", "type": "SPU"}' http://localhost:8000/api/v1/subticketartifact/
Create subticket¶
To be used by the CLI
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"source_package_upload": "/api/v1/spu/X/, "ticket": "/api/v1/ticket/X/, "assignee": "test@example.com"}' http://localhost:8000/api/v1/subticket/
Project¶
Get source package¶
return all source packages
curl --dump-header - http://localhost:8000/api/v1/sourcepackage/
Add source package¶
This action would be completed by the CLI when it encounters a new source package that the ticket system hasn’t had before.
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "my-package"}' http://localhost:8000/api/v1/sourcepackage/
Get binary package¶
return all binary packages
curl --dump-header - http://localhost:8000/api/v1/binarypackage/
Models¶
Ticket System will be django + REST + Postgres. The phase 0 models are defined below with the proposed future models coded out below that.
Phase 0¶
sourcepackage¶
A source package is an existing source package in the Ubuntu archives. A source package record is created when a source package has changes submitted to the ticket system for the first time.
sourcepackageupload¶
A ‘Source Package Upload’ is a file uploaded by the user that is related to the changes being made. It will have a related source package and artifacts.
ticket¶
A ticket is created to get a change (or set of changes) into the Ubuntu archive. Creating a ticket allows the changes to be tracked through the processes of package building, image building, image testing and publishing as well as the results to be seen by the user.
subticket¶
Each source package upload that is added to a ticket will have its own subticket. A ticket can contain one of more subtickets. The subticket allows the user to track the progress and results of the package building for each source package upload.
artifact¶
Artifacts can be multiple things. When a ticket is initially created, it will have artifacts attached to it which are the source package upload files. Artifacts can also be test results and log files. They can be assigned to a relevant ticket or subticket based on the step of the process.
Future Apps/Models¶
person¶
class Person(models.Model):
name = models.CharField(max_length=4096)
email = models.EmailField(max_length=200)
is_team = models.BooleanField(default=False)
testsuites¶
class TestSuites(models.Model):
pass
project¶
class Project(models.Model):
# Class that defines an upstream project.
name = models.CharField(max_length=4096)
display_name = models.CharField(max_length=4096)
maintainer = models.ForeignKey("Person")
contact = models.ForeignKey("Person")
description = models.TextField()
branch¶
class Branch(models.Model):
unique_name = models.CharField(max_length=4096)
owner = models.ForeignKey("Person")
type = models.CharField(choice=["trunk", "development", "maintenance", "regular"])
project = models.ForeignKey("Project", null=True, blank=True)
binarypackage¶
class BinaryPackage(models.Model):
name = models.CharField(max_length=4096)
sourcepackage = models.ForeignKey("SourcePackage")
seeded = models.BooleanField(default=False)
mergeproposal¶
class MergeProposal(models.Model):
branch = models.ForeignKey("Branch")
submitter = models.ForeignKey("Person")
status = models.CharField(choices=["work_in_progress", "needs_review", "approved", "rejected", "merged"])
lp_weblink = models.CharField(max_length=4096)
Person API’s - TODO¶
add_person¶
Add a person (or a team) to the database.
person
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "Chris Johnston", "email": "user@example.com", "is_team": "False"}' http://localhost:8000/api/v1/person/
team
curl --dump-header - -H "Content-Type: application/json" -X POST --data '{"name": "Canonical CI Engineering", "email": "team@lists.example", "is_team": "True"}' http://localhost:8000/api/v1/person/
get_person¶
Return all persons
curl http://localhost:8000/api/v1/person/
search by name
curl --dump-header - http://localhost:8000/api/v1/person/?name__exact=My%20Name
curl --dump-header - http://localhost:8000/api/v1/person/?name__iexact=my%20name
curl --dump-header - http://localhost:8000/api/v1/person/?name__startswith=My
curl --dump-header - http://localhost:8000/api/v1/person/?name__istartswith=my
search by email
curl --dump-header - http://localhost:8000/api/v1/person/?email__exact=user@example.com
curl --dump-header - http://localhost:8000/api/v1/person/?email__iexact=User@example.com
curl --dump-header - http://localhost:8000/api/v1/person/?email__startswith=user
curl --dump-header - http://localhost:8000/api/v1/person/?email__istartswith=User
show/don’t show teams
curl --dump-header - http://localhost:8000/api/v1/person/?is_team=True
curl --dump-header - http://localhost:8000/api/v1/person/?is_team=False