Can Computers Really Think? - Thinking as Computation - page 272

not have a clear picture of what this imagined chinese.pl program would have to
be like (assuming it can exist at all). But we can get some clarification by turning to a
simpler case, addition again.
12.4 The Summation Room
Imagine a room with a person in it who does not know how to add numbers. Mes-
sages are passed to the person on a sheet of paper containing a list of twenty numbers,
each of which has ten digits. The topic inside the room that he is able to use is a very
large one: it has ten billion chapters, and each chapter has ten billion sections, and
each section has ten billion subsections, and so on, up to depth twenty. The preface
of the topic has the following instructions:
Take the first number in the list of twenty and go to that chapter; then take the
second number in the list and go to that section; then take the third number and
go to that subsection; and so on until all twenty numbers have been used up. At
the end of this process, there will be a number written in the topic with at most
twelve digits. Write that number on a slip of paper and hand it back outside the
room.
Unbeknownst to the person in the room, the topic is constructed in such a way that
the twelve-digit entries in the topic are in fact the sums of the twenty numbers that
led to the entry.
Now the person in the room, following the procedure in the topic, is clearly not
adding. He is producing the correct sums, of course, but only by looking them up.
This is no different from phoning a friend and getting the answers from her. And what
if the person were to somehow memorize the contents of the topic and follow the
instructions mentally? To an observer, the person would be examining the numbers,
reflecting for a while, and then writing down their sum. From the outside, it would
look just like the numbers were being added, but they are not.
This, in a nutshell, is Searle's argument but now applied to addition. The person in
the room is producing the right external behavior but does not know how to add.
So does this mean that Searle is right after all? Not quite. There is a serious flaw
in the argument with the Summation Room: the topic needed for the thought experiment
cannot possibly exist. It would have to contain 10 200 entries, but the entire physical
universe only has about 10 100 atoms. So a thought experiment that relies on this topic
is vacuous. It might work if we only had to add two ten-digit numbers, say, but for
twenty or more, this is too much for our universe.