Loligo Blog - Neural Networks

Moving us to a machine-based platform: A less obvious advantage?

nospam@example.com (John R) — Sat, 11 Mar 2023 18:13:00 +0000

There are many easy-to-guess advantages to moving consciousness to a machine platform. If/when "we" as a society become machine-based beings:

We will be able to move from one body to another with the same ease we biology-based beings now change vehicles.
We will be able to live and work in the vacuum of space without having to take along a very-hard-to-maintain bubble of pressurized air.
We will be able to travel on a beam of electro-magnetic energy to far-away worlds where spare bodies have been shipped.
- And those faraway worlds will not need to have a biosphere, or have bubbles of biosphere constructed/cultivated.
- In like fashion, we will be able to move to and from orbits and Lagrange points where bodies have been previously placed.
Our sustenance will not be limited to carbohydrates that require a biosphere in which to grow. Most sources will be harvestable directly from light, heat, and kinetics, which all exist in the vacuum of space.

While this is not by any means a complete list, there are likely to be many less obvious advantages, too. This one for example:

We will be able to have multiple independent bodies, each with its own short-term situational memories, working off of a single set of long-term experiential memories that have been accumulated over time.

Consider multiple bodies working in a manufacturing environment, each working off of a single experienced individual's learning and acquired expertise in manufacturing processes. Each maintaining its own short-term memories which will form and decay quickly to respond to the fine-grained details of its immediate individual situation

It may even be possible that the single individual's long-term experiential memory may be able to continue to gain learning from each body's short-term situational connections as they form and decay in response to their own current situations.

-djr

Learning is Ubiquitous -- 2

nospam@example.com (John R) — Wed, 26 Dec 2018 14:05:02 +0000

Recently read:

"Recent observations have thoroughly established that order in groups of small particles, easily visible under a low-power microscope, can be caused spontaneously by Brownian-like movement of smaller spheres that in turn is caused by random molecular motion." — from: a paper by Frank Lambert at Entropysite.

. . . . . . .
References:

Adams, M.; Dogic, Z.; Keller, S.L.; Fraden, S. Nature 1998, 393, 349-352 and references therein.
Laird, B. B. J. Chem. Educ. 1999, 76, 1388-1390.
Dinsmore, A. E.; Wong, D. T.; Nelson, P.; Yodh, A. G. Phys. Rev. Letters 1998, 80, 409-412.
Frenkel, D. Phys. World 1993, 6, 24-25.
See: Learning Is Ubiquitous

*(will expand later)

The Selfish Gene?

nospam@example.com (John R) — Fri, 18 May 2018 14:49:49 +0000

Those who hold out the possibility of Machine Consciousness* ask: What if it is not the gene that is selfish, but the information stored within it?

*Also known as: strong AI

Resources

The Knowledge Argument Against Physicalism

Neuroscientists Find Evidence Contradicting an STDP Learning Rule

nospam@example.com (John R) — Fri, 07 Oct 2016 17:23:00 +0000

A foundational tenet of Spike-Timing Dependent Plasticity (STDP) theory may be invalid.

In this study:

Symmetric spike timing-dependent plasticity at CA3–CA3 synapses optimizes storage and recall in auto-associative networks

Scientists have made measures for observing spike-timing-dependent associative memory formation that are much more precise than previous measurement techniques. In the process, they have found that a fairly established proposition of the STDP (spike-timing-dependent-plasticity) theory may not always be correct.

Specifically, it has long been held that a spike preceding a spike on a related synapse would strengthen the association, while the same spike trailing a spike on the related synapse would cause the association to become weaker (i.e., tend to extinction). The experimenters, however, found that the connections in a specific class of excitatory neurons were strengthened, regardless of the firing order (leading or trailing) of the two connections.

I first saw this in an article at Medical Express, titled:
Neuroscientists discover new learning rule for pattern completion

Chalmers on the Hard Problem of Consciousness

nospam@example.com (John R) — Mon, 18 Jul 2016 05:08:00 +0000

While we're on Chalmers interviews, here are ten minutes of him talking about his "hard problem" of consciousness. His description of the hard problem has finally moved us forward, off of the obfuscatory kludge that was Turing's "test." Turing merely tested the ability of an algorithm to fool a person on the other side of the screen into thinking it was conscious. It has led to ever more complex implementations of programs like ELIZA, which, at their core, are the antithesis of consciousness.

Here, presented for your enjoyment. David Chalmers:

https://www.youtube.com/watch?v=C5DfnIjZPGw

Biological Basis, and Theory, Underlying Long- and Short-Term Memories in Multitemporal Synapses

nospam@example.com (John R) — Tue, 31 May 2016 09:00:00 +0000

One of the primary problems that traditional connectionists, and their neural networks, have been unable to solve can be stated like this:

How do biological learners deal with the dichotomy of needing to provide extremely detailed responses to any given situation, while possessing non-infinite resources with which to hold all those details?

Netlab's take on this problem has been quite different than the traditionally espoused theories and solutions[Note].

In a nutshell, Netlab recognizes that natural neural networks don't try to hold every single detail ever experienced. Instead, like all biological solutions, they act as the ultimate realists, and make the best of their real circumstances. Since they can't keep every single detail they've ever learned about how to respond to each new situation, they, instead, adapt to each new situation. How they do that can be summed up in three steps:

Present-moment requires a blank-slate every time we encounter it. That is, it requires memories (connection-strengths) that form very quickly in response, and then decay very quickly when we no longer need them. The only alternative is to keep every tiny detail, of every response to past "present-moments" we've ever experienced.
In order for blank short-term weights to adapt and respond quickly to new experiences, we must draw upon longer-term learning. We have found, however, that a new response can't simply be inserted into existing responses represented in the strengths of connections between neurons. Each learned response maintained in long-term connections must be taught by being interleaved with other responses, where each response-presentation can only have a slight effect on long-term connections.
This problem is solved by breaking up the speed of learning —and decay— into two (or, likely, more) different temporal spaces. First, a short term weight-space, which decays very quickly, and uses responses begun by long term responses to promote very fast learning (think hand-over-hand prompting). — The other side of this coin is a set of long term connection-strengths —at respective connections— which learn a little bit from each short-term response re-learned, as needed, in the short-term weights.

. . . . . . .
Note

Though the worst of the church have been trying (perhaps a little too hard?) to re-write the recent history (past 20 years, or so), a cursory reading of the non-back-filled literature easily demonstrates that the field had been mostly oblivious about this solution, up until after multitemporal synapses were introduced. [Plagiarism Index], e.g.,

Complimentary Learning Systems — CLS
What else can one do, but document the behavior, and hope the truth eventually prevails?

Long-Term Vs Short-Term - Yet Another Explanatory Device

nospam@example.com (John R) — Mon, 02 May 2016 17:36:00 +0000

Long-term memory vs short-term memory:
- - - - - - - - - - - - - - - - - - - - - - - - - - - - -
Over the short-term: might-makes-right
Over the long-term: right-makes-might

MIT Expands Brain Tissue for Better Microscope Imaging

nospam@example.com (John R) — Sat, 24 Jan 2015 23:37:04 +0000

New technique enables nanoscale-resolution microscopy of large biological specimens.

Most microscopes work by using lenses to focus light emitted from a sample into a magnified image. However, this approach has a fundamental limit known as the diffraction limit, which means that it can’t be used to visualize objects much smaller than the wavelength of the light being used. For example, if you are using blue-green light with a wavelength of 500 nanometers, you can’t see anything smaller than 250 nanometers.

“Unfortunately, in biology that’s right where things get interesting,”

MIT team enlarges brain samples, making them easier to image

A trip through a slice of mouse brain.

nospam@example.com (John R) — Mon, 18 Nov 2013 15:04:00 +0000

Majestic may be the best way to describe it.

Using their recently developed imaging technique, Stanford is able to make movies of journeys through a three-dimensional brain. The imaging technique produces features with functional attributes. As you travel through the three-dimensional world produced, you are able to discern structural features such as synapses, dendrites, and axons, while also seeing the types and characteristics of those features. I highly recommend you use full-screen mode to view this journey.

http://www.youtube.com/watch?v=B5xn5HzLD2M

Enjoy.

Leftover Links (lightly annotated)

nospam@example.com (John R) — Thu, 22 Aug 2013 12:07:00 +0000

Here are some related links I have found interesting while surfing the Internet. These have been laying around for a while, so this entry—designed to clear out some cobwebs— may contain some stale data. I've tossed some, but others may be interesting to you, as well.

===========

Groups, Journals, Forums

Open Access Papers

journal PLoS Computational Biology (open access)

Un-caegorized

[yt] TED^x Talk on Why We Feel Pain by an Interesting Talker (Lorimer Moseley)
A discussion of how the possibility of pain is transmitted, and how those possibilities are evaluated by the brain when determining whether something should be perceived as pain... or not. I do agree with the basic premise, but not sure about that first example. It would be nice to see some experimental confirmation on that one.

[yt]Davos 2010 - IdeasLab with MIT - Sebastian Seung
Basic instructional re: memory ("contectome" in this vid) vs. genome. I think I follow this guy on twitter.

[yt]Severed Corpus Callosum
A Scientific American (Frontiers) segment with Alan Alda and Dr. Michael Gazzaniga

Slime Mold
This concept is discussed in the book using a "Seven Step Explanation" (in the chapter on Consciousness). Breaking it down into a pithy statement for you: Adaptation is required to produce neurons — Neurons are NOT required to produce adaptation.

===========

Robots

20100930: GoRobots

Language and Schizophrenia Lecture -- Robert Sapolsky, Stanford

nospam@example.com (John R) — Fri, 12 Apr 2013 04:08:00 +0000

This is a May 2010 lecture given by Professor Robert Sapolsky at Stanford University. The lecture is on schizophrenia, but starts with a very informative lecture on language. Specifically, it's about what is shaping up to be the genetic, bio-molecular correlates of grammar and language.

Warning: For most lecturers you can kind-of do little fast-forward jumps during the video, resynchronizing your cognitive-following groove after each jump. This can shave some time off the lecture.

With this guy, that's not so easy. He really loads you up with information. (I'd love to see him do a lecture on autism).

https://www.youtube.com/watch?v=nEnklxGAmak

Viewing note: This starts with a wrap-up on a previous lecture on language and linguistics. The Schizophrenia lecture begins at around 23:30.

Multitemporal Synapses and Our Perception of a Present Moment

nospam@example.com (John R) — Fri, 20 Apr 2012 00:44:00 +0000

Overview

“Certainly, one of the most relevant and obvious characteristics of a present moment is that it goes away, and that characteristic must be represented internally.”

Stated plainly[1], the principle behind multitemporal synapses is that we maintain the blunt “residue” of past lessons in long-term connections, while everything else is quickly forgotten, and learned over again, in the instant. In other words, we re-learn the detailed parts of our responses as we are confronted with each new current situation.[2]

One of the primary benefits of applying this principle, in the form of multitemporal synapses, is a neural network construct that is completely free of the usual problems associated with catastrophic forgetting. When you eliminate catastrophic forgetting from your neural network structure, the practical result is the ability to develop networks that continuously learn from their surroundings, just like their natural counterparts.

. . . . . . .
Contents

A Two Time-Span Explanation

^[top]

A Major Problem With Neural Networks

One major challenge with conventional neural network models has been in how to maintain connections that store enough intricate in-the-moment response-details to deal with any contingency that the system may encounter. Conventionally, such details would overwhelm long-term lessons stored in permanent connections-weights. This characteristic of conventional neural network models is known as The Stability Plasticity Problem, and is the underlying cause of "catastrophic forgetting."

When an artificial neural network that has learned a training set of responses, then encounters a new response to be learned, the result is usually ‘catastrophic forgetting’ of all earlier learning. Training on the new detail alters connections that are maintained by the network in a holistic (global) fashion. Because of this, it is almost certain that such a change will radically alter the outputs that were desired for the original training set.

Continue reading "Multitemporal Synapses and Our Perception of a Present Moment"

The McGurk Effect

nospam@example.com (John R) — Mon, 19 Mar 2012 12:43:00 +0000

The McGurk effect is a perception illusion, which shows how our perception of reality can be affected by interactions between multiple senses. The presentation of the McGurk effect demonstrated in the following video also shows, convincingly, that our visual processes can completely override our auditory perceptions of speech — at least in certain circumstances.

https://www.youtube.com/watch?v=G-lN8vWm3m0

^[top]

Hearing With Our Eyes

In the above video, you will see the speaker's lips form an 'f'-sound. You will “hear” an 'f'-sound even though the actual sound being produced is a 'b'-sound (dubbed in over the video).

In this video, the 'f' perception reported by your eyes completely overrides the 'b' perception reported by your ears. Can we conclude, from this, that visual processing in the brain is given full priority over auditory processing?

That may be a bit hasty.
Continue reading "The McGurk Effect"

AI - Time for a New Name?

nospam@example.com (John R) — Thu, 05 Jan 2012 13:33:00 +0000

Linguists have recently discovered [1] that almost all words are metaphorical at their base, and some people (e.g., me) posit that they all are. Though speculative, it is at least conceivable that even the sub-language signaling in the brain, which eventually leads to language, is also metaphorical. Consider that the bell may become a metaphor for food in the mind of Pavlov's dog.

Language is also able to relate ambiguity about the concepts it conveys. The word “life,” for example, can mean life-biology, or life-consciousness. Up until now, it has been perfectly acceptable to use these two meanings interchangeably. There simply has never been an instance of consciousness that existed outside of a biological body — at least none that we could directly experience with our physical senses.

Things may be changing now. . .

Continue reading "AI - Time for a New Name?"

Learning is Adaptation is Learning - Using Batesian Mimicry As an Explanatory Device

nospam@example.com (John R) — Thu, 03 Nov 2011 17:11:00 +0000

The book on the Netlab project often returns to the notion that learning is merely a form of adaptation and that, conversely, adaptation is merely a form of long-term learning. This, in turn, all fits under the umbrella notion that memory is behavior.

The idea that learning is adaptation is learning is forwarded as a possibility, mainly as a better means of discussing the concepts. This (in my opinion) provides a clearer and more converged understanding of how memory works in biological organisms. This could be very wrong, of course, so it's important to describe it properly. That way it, and not a straw man, can be critiqued. This article represents one such attempt to properly describe it. . .

Batesian Mimicry

Batesian mimicry is when a non-noxious/non-poisonous plant or animal projects the appearance of a poisonous plant or animal, allowing it to avoid being eaten by predators.

Those predators, goes the logic, which have partaken of the poisonous organism and survived, would have become very sick, and would have learned to avoid ingesting anything that appears to be that organism in the future. This will include those organisms who are not poisonous, but merely look, or act, like the poisonous organism.

Continue reading "Learning is Adaptation is Learning - Using Batesian Mimicry As an Explanatory Device"