On being human
In this issue of Think:Act magazine we examine in detail what it means to be human in our complex and fast changing world now and in the days to come.
by Steffan Heuer
illustrations by Karolis Strautniekas
Automation is convenient, but it also threatens to dull and degrade human skills. Experts say it's time to take back the reins before we forget how to stay in the saddle.
The sensors fitted in the autonomous Uber taxi "saw" 49-year-old Elaine Herzberg as she was crossing a multi-lane road in Phoenix, Arizona, this past March – a full six seconds before she was hit and killed. But the vehicle's software had classified the pedestrian as a "false positive" and the human operator who was supposed to take over when something's amiss was not paying attention and streaming a TV show on her smartphone.
As Asiana flight 214 was making its final approach into San Francisco in July 2013, the pilots misread the Boeing 777's airspeed and glide path. When they realized they were too low and too slow, it was too late for a go-around: The plane clipped a seawall before it disintegrated on the runway and burst into flames, killing three people. Investigators found that the pilot in control had failed to monitor instruments and showed "an inaccurate understanding of some aspects of the airplane's autoflight system."
This list of highly complex systems going haywire with fatal consequences could go on, but these two accidents may suffice to demonstrate what these incidents usually have in common. In each case, there was a human being behind the controls who should and could have taken over to save the situation, but they didn't because they failed in monitoring an automated process, trusting faulty readings or erroneous actions taken by software.
Automation, to be sure, is wonderful. It makes life safer, more convenient. We have welcomed robots, whether hardware or just software, with open arms. Not only do these systems free humans from back-breaking, repetitive and often dangerous physical tasks, they will also dutifully perform boring activities such as data entry. And along the way, automation has mastered increasingly complex tasks. Flying a plane is the prime example. And therein lies the rub: Automation can be too much of a good thing. Large parts of our lives are already running on some form of autopilot or soon will be – think driverless shuttles picking you up curbside. Yet being freed from the need to steer, manage and control machinery has a nasty consequence.
We are in danger of "de-skilling" the more we lean back. Humans who only supervise what they used to actively run tend, over time, to become less engaged and can lose their skills altogether – so much so that they cannot take over when things go wrong. That's why Rory Kay, a veteran pilot and co-chairman of a panel on pilot training, warned his peers back in 2011: "We're forgetting how to fly." But, are we really losing our skills and allowing our mental muscles to shrivel up, or are we merely in the transitional period – one that comes with any major technical change and just requires humans to learn new things?
The debate isn't new. With his 1956 book entitled "Automation: Friend or Foe?", Robert Hugh Macmillan, a professor of engineering at Swansea University, issued a succinct warning call. Lisanne Bainbridge followed in 1983 while at the University College London department of psychology with a widely recognized paper on the "ironies of automation." Her two main observations still hold true. Creators of automated systems want to remove humans from the equation, thinking that they are the problem. The consequence? "Designer errors can be a major source of operating problems," warns Bainbridge. Which leads to the second irony. "The designer who tries to eliminate the operator still leaves the operator to do the tasks which the designer cannot think how to automate." So automation can make the human's job harder in the end while giving the false impression that the system has everything under control.
The best, and the most sobering, summary of de-skilling comes from tech writer Nicholas Carr who wrote "The Glass Cage: Where Automation is Taking Us in 2014". The book's title is a play on the "glass cockpit," pilot jargon for the novel flight deck design of the Airbus A-320 introduced in 1988. It was the first plane to have six big screens in the cockpit – it's a clear indication that computers run the show. For Carr, the glass cockpit is the emblem of a sophisticated de-skilling trap: The pilots in control of a big jet such as an A-380 only "fly" the plane for three minutes of a long-haul flight.
Prompted by close calls and fatal accidents, pilots and regulators at the Federal Aviation Administration (FAA) spent years examining the problem. The FAA finally issued a lengthy report in November 2013 warning that pilots relied too much on automated systems and then "may be reluctant to intervene."
De-skilling is quickly becoming a problem for all of us. As more and more devices become connected and consume live information, drivers forget how to navigate in their own hometown. And once repair technicians rely on augmented reality headsets to highlight what screws to turn, who knows what will happen to their expertise when the network is down? Once systems are labeled "autonomous", the so-called hand-back in an emergency becomes the potentially lethal weak link. Which means engineers have to figure out how to prevent a loss of situational awareness and a degradation of skills after prolonged inaction.
Silicon Valley design guru Don Norman likens the ideal interaction between man and machine to a horse and rider. "Skilled riders are in continual negotiation with their horses, adjusting the amount of control they maintain to the circumstance," he writes in his book "The Design of Future Things". The idea of a man-machine symbiosis with natural signals would serve us well, Norman thinks.
Associate Professor of Engineering Psychology at West Point Military Academy Ericka Rovira has spent her career pondering how to design automated systems that work the way humans expect them to work and prevent those humans from losing the skills they worked so hard to attain. "Automation," she points out, "doesn't naturally result in skill degradation. It often just changes the nature of the task." And the right user interface, or UI, can go a long way in keeping those mental muscles toned. Rovira suggests we could design systems that clearly display a machine's limitations based on comfort levels. "In uncertain environments, the interface changes based on your trust in a machine. If I tend to overtrust, the UI is more adamant about the things it isn't sure of. If I'm less trusting, on the other hand, the interface can tell me clearly: I am 98% sure I know the answer."
It's much ado about nothing according to Melonee Wise, founder and CEO of service robot company Fetch Robotics. She thinks that de-skilling is a common transitional phenomenon and won't make humans incompetent or obsolete. "Humans are creative, empathetic and innovative – all skills that will be necessary for a long time to come. Rote processes are those most likely to fall by the wayside, particularly in industries that are having problems finding workers," Wise says. Her point is widely shared by automation experts. Many higher-level human capabilities and skills cannot be replaced by machines or algorithms in the foreseeable future, for instance, in very interpersonal professions like healthcare and education.
This optimistic outlook is also based on the assumption that humans change the way they build and exercise their mental muscles. That's why American education expert Heather McGowan wants to shift the debate from lamenting skill degradation to refocusing on uniquely human values, those things that are hardest to automate. She sees the world at a "liminal space between the third and the fourth industrial revolution," with plenty of disruption and change ahead for tens of millions of workers and professionals. "Yet we are certainly not preparing for this new order," she warns. "We try to push humans to do what machines do better." Instead of codifying and transferring a predetermined set of skills and knowledge through traditional schooling, McGowan says, society should focus on setting up the conditions for faster human adaptation: how we learn, not what we learn. "If job skills are like applications on your phone which you add and delete as needed, then the agile learning mindset is the underlying operating system that allows the applications or skills to run."
In this issue of Think:Act magazine we examine in detail what it means to be human in our complex and fast changing world now and in the days to come.
Curious about the contents of our newest Think:Act magazine? Receive your very own copy by signing up now! Subscribe here to receive our Think:Act magazine and the latest news from Roland Berger.