ON SELF-AWARE MACHINE INTELLIGENCE.

Self-aware machines 01Self-aware machines 02Self-aware machines 03Self-aware machines 04

 

Typewriter: Hermes 3000.

The article is transcribed in self-aware-machine-readable form below.

Contrary to what seems to be the orthodox futurological opinion, self-aware machine intelligence will not take over the world and destroy us all.

Why does Dr. Boli say that?

For a very simple reason: it is not necessary for machines to be self-aware in order to take over the world, and quite possibly destroy us all. Machines will doubtless take over the world long before they have become self-aware—whatever that elusive and nebulous term actually means.

Dr. Boli will tell you a story told to him by his secretary. The man had been sent to the post office to ship a package of books. There were five people ahead of him in line when the clerk made a mistake: a customer had wanted four stamps, but the clerk rang up four books of stamps.

Understandably, the customer did not want to pay for four books of stamps. And with that the entire post office ground to a halt. The manager had gone home for the day; the clerk was the only employee left in the little postal substation. The cash register would not let him void the sale without the manager’s authorization. It would not let him begin a new transaction until he had completed the current one.

He had crashed the post office.

Now, this particular story has a happy ending: someone else in the line said she had come in to buy books of stamps, and was willing to pay for the four books so that life could continue.

But that was just a bit of luck. If no one had been willing to adopt those orphan stamps, business would have come to an end for the day. There were seven intelligent human beings in that post office, but the machine beat them all. It controlled them. And it did so in a remarkably stupid and obtuse way. Surely, an intelligent retail system would think of a way of allowing a sale to be voided now, while keeping an indelible record and leaving an alert for the manager to prevent fraud. But this machine was both stupid and intransigent. There can be no clearer demonstration of Dr. Boli’s assertion: It is not necessary for machines to be self-aware in order to take over the world, and quite possibly destroy us all.

Comments

  1. Ben Ieghn says:

    Isn’t this precisely how World War I began?

    PS. Dr Boli, How does a stamp know when it’s been licked?…ah…,wait a minute, it’s right on the tip of my tongue…

  2. Coincidentally, I literally just woke up from a nightmare.

    I dreamed I had been re-hired at an old job at the bookstore department of the Marshall Field and Company department store in downtown Chicago (which is now a Macy’s, and no longer runs their own book store, but my subconscious does not care). For some reason, they still had an ancient computerized POS register system running some version of MS-DOS and using monochrome-green-only monitor screens (which the store in question was well beyond when I worked there, having color-monitor screens and running Windows NT). I dreamed a couple of customers came in and wanted to place orders to be shipped, but this was my first day back on the job and I’d overstated how well I remembered the use of the point-of-sale system and thus hadn’t gotten much or any re-training on how to use it when I got the job back. So my nightmare consisted of me trying with increasing frustration and futility to print out a summary of their orders after they had paid and left, so as to include with the box of books when sent down to the shipping department to be actually shipped out.

    Stupid, non-self-aware machines may or may not control the future. But they apparently already rule my dream world.

    • Sean says:

      I always enjoy the alternate translation of “Point Of Sale”‘s acronym.

      The cash registers at our company cafeteria are made by “Future POS”. Considering the frequency with which they break down, this prediction is probably accurate.

  3. Something in this story doesn’t add up.

    Jeffery Hodges

    * * *

  4. Captain_DaFt says:

    So much like stupid people, stupid machines can totally FUBAR something, then?
    Looks like the rise of Artificial Stupidity.

Leave a Reply to Ben Ieghn Cancel reply

Your email address will not be published. Required fields are marked *