Lean Behind a Green Machine
It was April 1968. A cold and grey day at an F2 race in Hockenheim Germany. The crowd had been stunned into silence by the announcement that there had been a fatal accident. The driver involved had defined motor racing for a generation – that man was Jim Clark.
Clark and his green Lotus F1 car had become the epitome of cool, daring and skill; an international icon. In 1965 he won the F1 world championship whilst simultaneously winning the Indianapolis 500. The first non American winner of that race in nearly fifty years and he won it by two whole laps.
But what made him consistently the most efficient round a race track? Let’s identify his winning traits and compare them with another car related efficiency world beater – that of the Toyota production system, origin of the philosophy of Lean. Finally let’s also compare it with modern software development DevOps principles which is inspired by that same Lean thinking.
Maximal output
The fastest wins surely? It depends what you mean by fastest. It is not the fastest at any point in time but the fastest averaging across the whole race. For speed is not a constant. Rather it is continual change of speed (not to mention direction which is a whole other topic) in response to the challenges of a race. Right from the starter’s flag, initial acceleration, rapid adjustments to the jostling of other drivers, entry speed into the corner, exit speed out of the corner. Then over the longer course that speed in the context of changing weather conditions, tyre temperatures, wear and mechanical failures. All of these challenges can create perturbations, a deviation from the goal which in this case is to win the race.
Likewise in car production, efficiency is not the sheer quantity of what you produce. It is the rate at which you sell what you produce.
As Taiichi Ohno, mastermind of the Toyota production system said:
“Efficiency is never a function of quantity and speed. (Instead it is) producing in response to the needs of the marketplace”
Trying to predict what to build creates many challenges. There are fluctuations in market demand across the year. On top of this, it is not just predicting overall quantity but each and every consumer will choose from a large set of permutations across a diverse and various set of models, stylings, add-ons and colours. So it is predicting for every single permutation. Any unsold cars would be a perturbation – a deviation from our prediction of continually seeing return on our investment.
Another major source of perturbation is the impact that this continuously changing demand and variety has upon a complex production process. This can lead to imbalances of inventory up and down the production line leading to waste due to starvation or excess buffering of stock.
In software development the equivalent of speed or quantity is keeping people busy – as long as everyone is busy we must be efficient. But software development has even more sources of perturbation than manufacturing.
Whereas manufacturing deals with variety at least it is finite. For software development the variety is effectively infinite as every software change is novel rather than a mere permutation of a specific model. Rather than trying to predict how many of each permutation will sell, the question is more will it sell at all? In this sense software development is more like car design than manufacturing. Day to day manufacturing has at least established a market demand for its product.
Also manufacturing has a complex production process, but in software, the complexity is ever expanding as each feature adds to the code. This dynamism combined with increasing complexity leads to increasing perturbations in the form of unanticipated impacts of change and software bugs.
So speed, quantity and busyness can be symptomatic of efficiency but they are not at the heart of it. They are often used as the goal but in reality the goal is not to go the fastest, it is to win the race. It is not to produce the most, but to sell the most of what you produce. It is not just to value hard work but more specifically to work hard to optimise value created.
Maximal output is a crude and oversimplified representation of the desired outcome. It also blindsides many from one of the main sources of inefficiency – the many perturbations caused within the system due to interactions internally, and externally with the wider environment. So how to avoid perturbations?
Smoothness
The most common description of Jim Clark’s driving style was silky smooth. He once took an executive from Ford for a spin around the race track and afterwards the man thanked him for taking it easy on him. Then that executive found out they had just set a new lap record.
There are many reasons for this smoothness but a prerequisite is definitely fast reactions. This determines the fineness in which adjustments can be made. Perturbations typically worsen over time so if you can respond quickly you will minimise their impact.
In Lean manufacturing, smoothness is embodied in the concept of Heijunka which means production levelling or smoothing. Once again, the key to this is enabling fine adjustments. In this case it was something called SMED which drastically reduced the changeover time of producing different variants of cars. This allowed smaller batch production runs and therefore better ability to adapt to changing customer demand, thus reducing any perturbation of unsold cars.
In DevOps, automation of test and deployment means releases can be made regularly – multiple times a day. This is in sharp contrast to a more traditional project approach where releases can take months or even years. The fineness in release regularity in DevOps allows much more flexibility to emerging needs or change in priorities.
So fineness or granularity of adjustments are an enabler for smoothness. But before we can adjust we need to understand the nature of that adjustment.
Adaptability
Jim Clark was incredibly adaptive to any car or situation. When he was not winning in F1 or Indianapolis, he did rallying, touring cars, even NASCAR and Le Mans 24 hour. He was brilliant in different weather conditions like when he won the Belgian Grand Prix at Spa in the rain in 1965. He could adjust his driving to compensate a mechanical problem during a race or to a car in race preparation so that he did not need to tinker with car setup as much as other drivers. So why was this?
Clark had great vision and spatial awareness as many drivers do. But what really cut him apart from most others was an innate understanding of the feel and noise of the car. He would come off the track as and tell his mechanic the car had a problem. The mechanic would check the car over and say it was fine. Clark would say check again and on subsequent inspection the mechanic would find some problem, like a worn wheel bearing.
This sensitivity allowed him to adapt quicker to a car than others as he was getting feedback that others either did not get or know how to interpret. This was combined with an intense concentration which allowed him to make the most of each and every signal he received through the car. Acting rapidly on every minute level of detail meant he would prevent all sorts of perturbations, some that others just could not.
“I don’t drive any faster, I just concentrate harder” Jim Clark
Jim Clark faced many challenges on the race track but one significant advantage he had over car manufacturing was that all feedback on current events, decisions and actions were centralised in one person. In a large and complex production process such as car manufacturing there are thousands of different parts, hundreds of employees and subcontractors, dozens of processes, all coordinating into a continually varying output of cars. Amongst this maelstrom of activities, widely distributed knowledge and considerable interdependence there is vast scope for perturbations.
Taiichi Ohno recognised this and set out to create a system to prevent emergent perturbations – what he described as the “absolute elimination of waste”. He understood that in order to prevent a perturbation, he would need feedback and adjustment the moment an abnormal situation occurred on the factory floor. The people with this feedback were not management but those working on the production line. This meant instilling a problem solving mindset in factory workers, empowering them to stop the production line immediately, deep diving into the cause with techniques such as five whys and then taking corrective action. This zero tolerance culture to issues was known as Jidoka. It helped to create a smooth and efficient flow of production.
“The larger a business, the better reflexes it needs” Taiichi Ohno
Equally in software development, the potential perturbations are many and at many scales. It starts with the syntax error as the code is typed. Then there is the unexpected side effect or consequence during execution. The conflict as the code is merged back from a branch. The misunderstanding of the intent of a feature. The realisation that no one is using a feature.
In the early days of software, technological limitations and inferior software practices meant irregular feedback and slow adjustment. This built up large amounts of pain – compilations with hundreds of compiler errors, long lived branches leading to merge hell, big bang deployments causing mayhem in production.
As the discipline has matured into modern DevOps principles, this has evolved into tight feedback loops at every level. Starting with the IDE highlighting syntax error as the code is typed. TDD giving feedback on side effects and unintended consequences. Continuous integration and daily standups to coordinate across teams. Most importantly regular deployments into production to measure the value delivered. All these rapid cycles allow continual adjustment based on critical data to detect and eradicate any perturbations.
“Because work is a process rather than an individual operation, it needs a built-in control. It needs a feedback mechanism to sense unexpected deviations that call for change in the process and to maintain the process at the level needed to obtain the desired results.” Peter Drucker
So smoothness is the product of putting in place fast feedback loops that instantly detect deviation from the goal, and then making rapid adjustment to course correct.
There is one more key aspect to this smoothness and it is embodied in yet another feature of Clark’s driving.
Unerring
When news filtered through to the motorsport community of Jim Clark’s death there was widespread astonishment. The immediate reaction of many of his peers was that it had to have been a mechanical failure as Clark just did not make mistakes. So what was it about him that made mistakes such a rarity?
It is evident in his fairness in exchanges with other drivers and the fact he was universally liked and respected by all his peers. This is not always the case in a sport where intense competition often spills over into all out aggression, unsporting behaviour and development of bitter rivalries. This apparent restraint was at odds though with how successful he was – if he finished a race he generally finished first.
The contradiction was because Jim Clark was a master at finding the limits of the race track and critically not exceeding them. This made him incredibly efficient. Partly because of always moving at an optimal speed. But just as importantly, it prevented the antithesis of silky smooth – that jarring interruption to a continuous and steady motion that results from a mistake.
“The supreme attraction of motor racing to me is driving a car as near the physical limit as possible without stepping over it.” Jim Clark
Another benefit of limiting his driving was he did not induce unnecessary stress or wear. Clark’s mechanics claimed you could always tell on sight which engine was his after a race – it was the one with the least wear.
The Toyota production system also recognised the dangers of over stressing with the concept of muri. This is where the system excessively utilises staff or machines leading to failures.
There is also a very powerful limiting mechanism called kanban. In mass production, work orders are pushed into the production process based on forecasts and then passed downstream. This can often lead to stress if there are imbalances in the sequence of processes, or if the forecast is incorrect. Kanban completely turns this on its head and instead pulls actual sales orders from the end of the production process and this then ripples upstream with a sequence of signals. In this case the work is pulled when the production process is ready thus reducing overloading between processes.
This combined with the fast changeovers due to SMED allow the work to flow according to actual customer demand. Work is carried out at each production process when it is needed, and exactly no more than needed. Incredibly efficient just like Jim Clark.
The traditional busyness culture of software development can lead to mistakes, demoralisation and burnout of staff. It is well established that knowledge workers such as software engineers are particularly effected when they overwork as creativity suffers more significantly than manual work when people are tired.
DevOps uses the same limiting principles in Kanban (WIP limit) or Scrum (sprint) to limit work in process and let the team pull work when they are ready rather than work being externally pushed into the team and leading to overloading.
It is also about learning to prioritise business needs rather than everything being deemed important. Or even just empowering the team to say no.
All these methods help to establish operating limits which are critical for a sustainable pace. After all what is the point of going fast if you cannot sustain it to achieve the goal?
Summary
Jim Clark was one of the most successful racing drivers of all time. Like the Toyota production system and DevOps, his success is not about fanatical focus on output such as speed, quantity or busyness.
Partly it is dealing with the constant challenges of a complex and volatile environment within the context of the desired outcome. By establishing fast feedback at multiple scales and using all of the data meticulously, continual adjustments can be made to stay on track with the goal. This will minimise perturbations, each of which left unattended can represent growing, sometimes compounding inefficiency.
The other aspect is pushing hard but always within limits. In other words striking a perfect balance between the ying and yang of a world of scarcity – that of efficiency on the one hand and sustainability on the other.
Do these well and you can be just like Jim Clark – pure and silky smooth.
References
Jim Clark: Tribute to a Champion – Eric Dymock
Toyota Production System: Beyond Large-Scale Production – Taiichi Ohno
The Devops Handbook: How to Create World-Class Agility, Reliability, and Security in Technology Organizations – Gene Kim, Patrick Debois, John Willis
