Server deployment
Configuration management Continuous integration Log analysis
Organizing for Success: Kanban
Traditional task management for operations teams usually involves some sort of ticket system. Trouble tickets are great for keeping track of problems and for historical analysis after the problem is resolved. The problem with trouble ticket systems is that they can present a challenge to correlate tasks that should be related. Another challenge is to identify root cause of bottlenecks in production delivery. Further, workers may not have visibility into what trouble tickets are being handled by others to whom they can lend their expertise. Last, what is the best way to manage work-in-progress so that operations team members are not overallocated? That is to say, if the operations team is constantly focused on assigned tasks that keep piling up, when will they have time to make improvements to systems and pay down technical debt? How do we prioritize work properly and account for dependencies between tasks?
The Kanban (translated literally “signal card”) system can help address these and other concerns. The practice was developed by Taiichi Ohno during his work with Toyota
Production Systems to help achieve just-in-time (JIT) production goals. This methodology examines flows of work through different steps of a manufacturing process and identifies bottlenecks that need to be remedied for the entire system to be more efficient. The idea is that as bottlenecks are mitigated, this will pull work tasks from a work-in-progress state to completion. Work-in-progress is limited to allow for slack time for workers to make
improvements to the manufacturing process (for example, identify and eliminate new bottlenecks as old ones are mitigated). Figure 2-1 shows an example of a manufacturing process where estimated times to completion are not being met.
Figure 2-1 Manufacturing process
As the workflow proceeds from the raw material inputs through to the fabrication process and finally to the assembly process, we can see that the estimated times to completion are not matching the reality. There seems to be a problem with the assembly process. On
further investigation, factory workers may see that the process to apply a special coating to the sheet metal may not be efficient. For example, the inputs for the server panel
manufacturing process could show signs of not having the coating applied properly. So, rework is done, and the whole process is less efficient. Perhaps if the tray that holds the sheet metal for the coating process isn’t overloaded, the outputs can be produced properly
the first time, and there is little to no need for any rework to be done.
In the previous example, if the workers are heads down and obsessed with just fixing the incorrectly coated sheet metal instead of identifying the true root cause of the problem, wasted effort will continue. We’ve never seen this kind of problem in our IT operations workflows, right?
You may be wondering why we’re talking about a manufacturing workflow organization methodology in a chapter about DevOps tools. However, before we can have a successful change in the way we work, we need to have an organized way of assigning tasks and identifying trouble in the system.
Even though Kanban is a manufacturing discipline, it has become popular in IT through the work of David J. Anderson and others who combined the concepts of lean with Toyota’s Kanban methods. We will not cover the entirety of Kanban, but we will discuss the key points that can be helpful to IT operations teams.
The most important aspect of the Kanban system is the management of work-in-progress.
It may seem counterintuitive that keeping IT operations staff 100% allocated on new work requests can lead to inefficiencies. However, operations teams face similar issues to
manufacturing teams. For example, if a team is using gold images from prebuilt ISOs to build servers, there are opportunities for missteps when manual updates and patch
applications are required if the gold images have not been updated in a while. If the team’s manager’s focus is primarily on satisfying server requests to the point that workers are 100% engaged in doing those workflows instead of finding ways to make the process more efficient, the work may continue. However, human error can contribute to setbacks.
Important security updates may be missed or vulnerable ports left open, and remediation will be necessary to make sure that servers are not vulnerable. The teams may be so used to these inefficiencies that management is pleased with the amount of output. However, a lot of time and effort is wasted with this rework, a condition known as technical debt.
Technical debt is any unplanned work that arises due to mistakes or inefficiencies during planned work. Operations teams need to be given sufficient time to make system
improvements even if this work is not tied to a specific deliverable. I am not advocating that operations teams be given “goof off” time. (Managers may think otherwise.) The operations team may seem to be efficiently servicing the current customer demands.
However, what happens when the company wants to increase the scale of operations at a rapid pace? Existing workflows just won’t cut it. Make the improvements to your systems now before you encounter scale problems that you’re not equipped to address. If you’re encountering some of those scale problems now, actively investigating and mitigating bottlenecks can help yield efficiencies.
Another important aspect of Kanban is the visualization of the workflows from start to completion. The most popular manifestation of this is the Kanban board. It can either be physical or digital. Figure 2-2 shows a Kanban board example that might be used by the DevWidgets, Inc. company introduced in Chapter 1.
Figure 2-2 Online Kanban board
The idea is that each task is represented by an index card or Post-it note (also referred to as a kanban) and is queued on the left side of the board under the Backlog category. The columns that exist between Backlog and Done represent work-in-progress. Work-in-progress (WIP) limits (for example, 4/4 and 5/5) are placed on the work tasks as they are moved out of the Backlog. The Working column represents tasks that are actively being worked on, and the Waiting column represents tasks that are assigned but have
dependencies on tasks in the Working column. Until one of the kanban moves on to the next column in the workflow, no additional work can be placed in the WIP columns.
Figure 2-2 shows different rows of tasks, also known as swim lanes. These swim lanes correspond to different types of work (for example, New Work, Maintenance, and Defects). Classifying the work correctly will help with prioritization.
When a team first starts working with Kanban, the WIP limits may be set based on past experiences. Afterward, as bottlenecks are mitigated and team efficiency improves, these limits may change so that the team is constantly seeking to improve the flow of work over time. Remember, the goal is to continuously improve while managing to keep the team from being 100% engaged on new tasks. If you are interested in introducing Kanban to your team, it would be worthwhile to have a consulting engagement with a lead
practitioner in the discipline like Dominica DeGrandis or Lean Kanban University–
accredited practitioners. Your Kanban board may not necessarily look like what is shown in Figure 2-2, and that is okay. The goal is to develop a clear and useful system that makes sense to your team.