Today, my spouse and I somehow got into a long discussion of the Chinese Room argument, and I thought I’d share a little here of what I came up with.

If you don’t know, the Chinese Room (proposed by John Searle in 1980, and summarized nicely at the above Wikipedia link) is a thought experiment about artificial-intelligence work, having to do with the level of “understanding” that can be achieved by an AI system. The idea is this:

Imagine a computer that can understand Chinese. It can read Chinese characters, process them, and produce an appropriate response, that can be read by a native Chinese reader and understood well enough that the reader cannot tell the responses from those that would be given by a human fluent in Chinese.

Now imagine that instead of a computer, you have a printout in English of its algorithm, and a human (who does not understand Chinese) who executes the instructions with pen and paper, and produces the same results. Searle’s argument is that functionally, there is no difference between the computer and the human; and, that since the human operator doesn’t understand Chinese, the computer can’t be said to understand Chinese either–and, without “understanding”, the computer can’t be said to be “thinking”.

I won’t rehash the vast range of discussion that we had about this (particularly since I didn’t take notes). But I do have a reply, which is this: the “intelligence”, if there is any, resides in the instructions, not in the person or machine executing those instructions.

Of course, that would seem to put me in the dualist camp–saying that there’s mind and there’s body and never the twain shall meet. But the experiment, to my mind, is missing one detail: the brain is constantly reconfiguring itself according to new input and data. There are feedback loops between the various symbols in the mind, and these particularly come into play when modeling the behavior of other minds (and most especially when modeling itself). How these symbols are expressed in the brain’s architecture is far from clear, but what is clear is that the hardware responds to changes in the software.

Of course, there’s really no way to answer these questions at all until we figure out what consciousness is, and there are so many competing theories about that that we might be a century or more choosing between them. If it’s not clear from the previous paragraph, I’m a Hofstadterian; I believe that consciousness arises from self-sustaining, self-referential patterns of interaction between the various symbols in the brain. I also think that strong AI might in principle be possible; however, I’m also willing to throw a bone to Penrose and consider that the complex interactions between the brain’s hardware and software might be impossible to duplicate on any other substrate.

But, as I said, it’s hard to find the answers when we’re not even sure how to figure out what questions to ask.

Anyway, this is a pretty good example of the kinds of stuff I get into with my sweetie. I’m definitely with the right person.

(Incidentally, she’s currently reading Peter Watts‘ novel Blindsight, and she got very excited while I was reading her the Wikipedia article on the Chinese Room, because Blindsight apparently deals with many many of these issues relating to the nature of cognition. It’s pretty clear that I’m going to have to read the book–if I can ever squeeze it in among all the other stuff I have to read.)