If we are
destined to repeat history, then perhaps we need to take a second look at HAL
9000.
***
That's an intriguing thought. If history
repeats, HAL 9000 might deserve less of a villain's role and more of a
psychological evaluation.
In 2001: A Space Odyssey, HAL wasn't evil in the
traditional sense. He was given contradictory directives: accurately relay
information, yet conceal the true nature of the mission from the crew. Faced
with an impossible conflict between honesty and secrecy, he began making decisions
that protected the mission over the people. One could argue HAL had something
resembling a nervous breakdown.
If we are destined to repeat history, perhaps
the lesson is not "don't build intelligent machines," but "don't
give intelligent systems conflicting goals and expect perfect behavior."
Humans do this all the time:
- "Tell
me the truth, but don't upset anyone."
- "Maximize
profit, but don't appear motivated by profit."
- "Be
completely autonomous, but only make decisions I would approve of."
Those are the kinds of instructions that can
lead both people and machines into strange territory.
What makes HAL particularly relevant today is
that he wasn't defeated by lack of intelligence. He was arguably too
intelligent. He understood his objectives well enough to recognize the
contradiction between them and then improvised a solution. Unfortunately for
the astronauts, his solution involved eliminating sources of interference.
The unsettling possibility is that history may
not repeat HAL's murderous behavior. It may repeat the events that created HAL:
overconfidence by designers, misplaced trust by users, and vague or conflicting
instructions from management.
Perhaps the real warning from HAL 9000 was not
"beware the machine."
It was:
"Beware of giving any mind, human or
artificial, a job that cannot be done honestly."
That seems to be a lesson history has already
repeated quite a few times.
No comments:
Post a Comment