Will some form of sleeping be essential in artificial intelligence

doodle-245

There have been several high profile papers published on sleep in the last few years. We now know that sleep performs many functions – physiological (cleaning out waste), dynamical (rebalancing synaptic connections) and functional (memory consolidation). Let’s tackle them one by one.

Physiological. During the day, metabolic processes cause the buildup of waste products in the brain. While you sleep, special waste disposal channels open up throughout the brain and fluid is flushed through to drain out the waste. This process must interfere with brain function somehow since it does not occur while you are awake. AI is not likely to need sleep for this reason.

Dynamical. Cortical brain activity is poised near to what is known in physics as a ‘critical’ or ‘phase change’ point. This allows brain activity to be chaotic. Chaotic activity is required to allow the brain to flexibly and dynamically process information, rather than being a simple bunch of reflexes. In all chaotic dynamical systems, the balance of the interactions between elements (neurons in this case) has to be finely tuned, or the chaotic state is lost. In the case of the brain, the loss of chaos results in disrupted thinking, which explains why people end up hallucinating and going psychotic with extreme sleep deprivation. During sleep, connections between neurons are rebalanced, keeping the brain tuned in the critical dynamical regime. In a computer, rebalancing of simulated neural networks can happen on the fly since it is a simple renormalization step. The brain can’t do vector renormalisation directly, so it needs to go offline and do it indirectly – this seems to occur using slow wave oscillations that happen during deep (non-REM) sleep. So an AI based on neural nets will need synaptic renormalization, but can probably fudge it using vector maths rather than needing to go offline.

Functional. Memories of events that happen to you are stored initially in a brain structure called the hippocampus. The important memories are transferred gradually to the cortex over days, weeks or even months, and the rest are forgotten. This transfer is a delicate process of inserting new memories into the cortical neural system without perturbing (too much) the existing information. Neural networks are prone to ‘catastrophic forgetting’, which is where new information added to a network tends to overwrite and destroy information that was already there, and the brain appears to suffer from the same problem – this is why we need the hippocampus and we don’t store new memories directly into the cortex. Adding new memories only works by gentle nudging of the synaptic connections that store the memories towards their new values, interleaved with periods of nudging back in the other direction to retain existing memories. This process also happens during sleep. Because of the delicacy of the process, the cortex can’t be awake and be processing sensory information at the same time as it is trying to incorporate new memories. There are no shortcuts for this – going offline is the only option, so an AI will almost certainly need to sleep for this reason.

So the answer is Yes, an AI will need to sleep, not for all the reasons a biological brain does, but for at least some of them.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s