This week, we crossed an interesting milestone in operations – the creation of our 500,000 ticket. A long time ago, we ran things by email. Moving to a ticket based system for tracking work was not trivial. What was a ticket? An email seemed to be easy to understand – whatever was in the email was in the email. But what should a ticket be? We decided to go with the loose idea that a ticket tracked a unit of work. The definition wasn’t made more specific than that. After our people got used to using tickets, we started hooking up our software to the ticketing system. We integrated monitoring first and then came status updates from various software jobs. We stumbled here a bit because the ticket was not the same as logging from a process. If we treated the ticket as a log, we could have tickets with 20K entries in them. That wasn’t making the tickets more useful, just more noisy. So we came up with a different idea – the ticket tool. The ticket tool is a very simple PHP application that accepts a ticket number, a task, and a note. It appends to a text file. It was written a long time ago, so it does things we probably wouldn’t do now like it returns status codes in the HTML body instead of using status codes in the HTTP header. It’s also old enough to have been started in CVS. (Redacted source at the end of this post).
With the invention of Ticket Tool, the view of the ticket changed subtly. Instead of being the place to track the details of a unit of work, it became the hub to find all of the details. The secret was simply recording URL links to the ticket tool inside the ticket. Now it’s not uncommon for our tickets to have five to ten different tools recording details in ticket tool and posting links in our ticket.
We have integration with tickets baked in everywhere in operations. We have hooks for mail, bash, python, Windows, and probably everything else too. We use the API from the ticketing system, but we also have written our own that does more things than the original API. We have a system that extracts the records from the ticket database and converts them to XML to be loaded into a full text system that gives us powerful searching of the ticket history. Our use of tickets will likely continue to grow.
Here’s the monthly count of tickets created since we started.
Our ticketing system is provided by UserScape’s HelpSpot. We’ve had great success with Ian and his team.