Loligo Blog

Language and Schizophrenia Lecture -- Robert Sapolsky, Stanford

nospam@example.com (John R) — Fri, 12 Apr 2013 00:08:00 -0700

This is a May 2010 lecture given by Professor Robert Sapolsky at Stanford University. The lecture is on schizophrenia, but starts with a very informative lecture on language. Specifically, it's about what is shaping up to be the genetic, bio-molecular correlates of grammar and language.

Warning: For most lecturers you can kind-of do little fast-forward jumps during the video, resynchronizing your cognitive-following groove after each jump. This can shave some time off the lecture.

With this guy, that's not so easy. He really loads you up with information. (I'd love to see him do a lecture on autism).

http://www.youtube.com/watch?v=nEnklxGAmak

Viewing note: This starts with a wrap-up on a previous lecture on language and linguistics. The Schizophrenia lecture begins at around 23:30.

Defining Fungible

nospam@example.com (John R) — Tue, 02 Apr 2013 09:42:00 -0700

Probably the most repeated synonym for the word Fungible, is "interchangeable," followed closely by "substitutable," but there are some subtle differences in connotations (I think). This entry is an attempt to clarify, at least a little bit, just what the word fungible means.

As always, with writing, it's important to emphasize that I'm NOT an expert by any stretch. I'm just a novice, trying to muddle through these issues, and to invite corrections or comments, both from "real" writers, as well as those who, like me, are just trying to get better.

Continue reading "Defining Fungible"

New Description and Keyword Fields for the Glossary

nospam@example.com (John R) — Thu, 21 Mar 2013 18:33:00 -0700

The glossary now has fields and template codes allowing for the addition of a description, and a list of keywords for each entry.

Of course, it is still necessary to go back and add them to the terms that are there.

Today I've added descriptions and keyword lists for about 80 entries. This is pretty close to my limit for one sitting.

When combined with earlier work, I'm happy to report that four letters have been filled so far. This leaves 23 letters still to do. . .

Aha! Did you notice that this adds up to 27 letters? Good eye! The glossary has a separate "letter'" for terms that start with numbers or symbols.

Phenomenal Consciousness vs AI -- Yet another explanatory device

nospam@example.com (John R) — Fri, 08 Mar 2013 12:38:00 -0700

Developing an A.I. robot?:

Program it to cry when it experiences something very bad or very good.

Developing a conscious robot?:

Program it to want to cry when it experiences something very bad or very good.

^[top]

Sources and Resources

Related Background
- What is it like to be a bat?
  This classic discussion by Thomas Nagel is one of the best known "explanatory devices." It begins to move us away from Turing's flawed (but necessary, at the time) kludge for explaining what consciousness is.
- [pdf] Facing Up to the Problem of Consciousness
  This is an introductory level explanation of consciousness and where we are in dealing with the hard problem of explaining it. It is authored by one of the premier thinkers in the field David Chalmers
- Flagging the Present Moment with Qualia
- Rhythm Perception Timeline - Important Durations and Interonset Intervals
- Time In Three Parts, Temporality
- "Through Other Eyes" — A short story from a collection of short stories called Nine Hundred Grandmothers by R. A. Lafferty, 1970
  This is a fictional treatment of the concept of first person knowledge (see below).

Related Glossary Entries

Recent Glossary Updates

nospam@example.com (John R) — Tue, 01 Jan 2013 10:28:00 -0700

Some recent glossary updates have included:

Connection Strength [Refreshed]
Banging away some more on the "basics" drum. Attempting to address some common confusion surrounding weight-values vs. the connection-strengths they represent.
Catastrophic Forgetting [New]
A definition of a fairly common term from neural network literature, which labels one of the major problems that have been encountered. The entry also includes a discussion of how the problem is fully resolved by Multitemporal Synapses.
Interference [New]
Definition and some resources regarding current understanding and ideas on interference in neuronal memory acquisition.
The Stability Plasticity Problem [New]
Definition of another term from neural network literature, which is used as a more general label for the problem that is responsible for catastrophic forgetting. This entry also explains how the problem it labels is resolved by Multitemporal Synapses.
Memory [Refreshed]
Added some pics to the section on non-neuronal biological learning, along with a terse discussion of things like herding, schooling, and flocking behaviors. These are used to demonstrate how learning can occur in biological systems via an extra-neuronal mechanism.
Multitemporal Synapses [Refreshed]
Better explanations and editing plus a diagram (5.1) from the book (originally from the patent application).
Kinetic Depth Effect [Refreshed]
Added content, plus reorganized.
Multilayer Perceptron (MLP)[New]
Quick add to support other entries.

Multitemporal Synapses and Our Perception of a Present Moment

nospam@example.com (John R) — Thu, 19 Apr 2012 20:44:00 -0700

Overview

“Certainly, one of the most relevant and obvious characteristics of a present moment is that it goes away, and that characteristic must be represented internally.”

Stated plainly[1], the principle behind multitemporal synapses is that we maintain the blunt “residue” of past lessons in long-term connections, while everything else is quickly forgotten, and learned over again, in the instant. In other words, we re-learn the detailed parts of our responses as we are confronted with each new current situation.[2]

One of the primary benefits of applying this principle, in the form of multitemporal synapses, is a neural network construct that is completely free of the usual problems associated with catastrophic forgetting. When you eliminate catastrophic forgetting from your neural network structure, the practical result is the ability to develop networks that continuously learn from their surroundings, just like their natural counterparts.

. . . . . . .
Contents

A Two Time-Span Explanation

^[top]

A Major Problem With Neural Networks

One major challenge with conventional neural network models has been in how to maintain connections that store enough intricate in-the-moment response-details to deal with any contingency that the system may encounter. Conventionally, such details would overwhelm long-term lessons stored in permanent weights. This characteristic of conventional neural network models is known as The Stability Plasticity Problem, and is the underlying cause of "catastrophic forgetting."

When an artificial neural network that has learned a training set of responses, then encounters a new response to be learned, the result is usually ‘catastrophic forgetting’ of all earlier learning. Training on the new detail alters connections that are maintained by the network in a holistic (global) fashion. Because of this, it is almost certain that such a change will radically alter the outputs that were desired for the original training set.

Continue reading "Multitemporal Synapses and Our Perception of a Present Moment"

The McGurk Effect

nospam@example.com (John R) — Mon, 19 Mar 2012 08:43:00 -0700

The McGurk effect is a perception illusion, which shows how our perception of reality can be affected by interactions between multiple senses. The presentation of the McGurk effect demonstrated in the following video also shows, convincingly, that our visual processes can completely override our auditory perceptions of speech — at least in certain circumstances.

http://www.youtube.com/watch?v=G-lN8vWm3m0

^[top]

Hearing With Our Eyes

In the above video, you will see the speaker's lips form an 'f'-sound. You will “hear” an 'f'-sound even though the actual sound being produced is a 'b'-sound (dubbed in over the video).

In this video, the 'f' perception reported by your eyes completely overrides the 'b' perception reported by your ears. Can we conclude, from this, that visual processing in the brain is given full priority over auditory processing?

That may be a bit hasty.

Continue reading "The McGurk Effect"

A New Hosting Provider for the Site

nospam@example.com (John R) — Fri, 02 Mar 2012 22:40:00 -0700

The site was switched to a new hosting service at the end of February. The blog and glossary were the pieces I was most anxious about, but they seem to have handled the move just fine.

So far, this host seems to be providing much faster responses. It should also provide better up-time.

Responses have gone from often taking 40-70 seconds, down to less than ten seconds. In fact, I haven't counted a single response greater than 12 yet.

The previous provider would regularly (about once a month) make changes that completely hid most, or all, of the site's content from the search-engines and in-links. Those down-times would typically last from two to six days. Many down-times, including the last one, only ended when I wrote some defensive code to work around their new server-settings.

Hoping this provider will do better in that department as well.

So far, I'm happy with it.

P.S. — BTW, I'm also new to something called TraceWatch, which is a stats package. So far, I'm totally addicted to it. It's like FarmVille for webmasters.

Netlab's Compatibility Mode

nospam@example.com (John R) — Thu, 19 Jan 2012 13:17:00 -0700

Spent some time today doing minor edits to glossary entries. Of all the small edits, the most significant change made was to add the following section to the entry for weights.

“

. . . . . . .
Netlab's Compatibility Mode

ANN models that use floating point signed-value weights in the conventional fashion are math-centric. That is, they typically are concerned only with the signed numeric weight-value, rather than with the connection-strength represented by its absolute value. In this case, for example, increasing the weight value will make it more positive, regardless of whether it is representing an excitatory or inhibitory connection.

Netlab's default behavior is to operate directly on connection-strength representations, regardless of how they are implemented internally. Netlab neurons facilitate the conventional practice, however, by allowing it to be specified in the learning method for each weight-layer.

The table below shows how Netlab facilitates compatibility with existing practices. The table documents how the translation is carried out between the traditional math-centric convention, and Netlab's connection-strength-centric convention.

Connection-Type->
v--Operation Excitatory Inhibitory

Increase Increase
Connection Strength Decrease
Connection Strength

Decrease Decrease
Connection Strength Increase
Connection Strength

Translations performed when conventional adjustment
practice is specified for a connection.

”

One possible analogy for the conventional, value-based, adjustment practice is that of adjusting for a specific water temperature from a faucet. If the water is too cold, for example, adjusting the weight value is comparable to simultaneously increasing the hot and reducing the cold (hot being the negative inhibitory weights, and cold being the positive excitatory weights in this analogy). Conversely, if the water is too hot, it is adjusted by simultaneously decreasing the hot, and increasing the cold.

In this way, Netlab is able to fully support the practice of working directly with the numeric value of a signed weight, but it also supports its own alternative strategy of adjusting connection strength representations. This strategy seems to be more representative of what has been learned about the cell, and molecular biology of neurons. The faucet analogy used above to describe the value-based adjustment is not sufficient to describe this strategy[1].

^[top]

Sources and Resources

Related glossary entries:

===========
Notes:

[1] - This is not to say the connection-strength adjustment strategy can't be related with an analogy, just that I have been too lazy, or too unfocused to come up with one that feels satisfyingly apt.

AI - Time for a New Name?

nospam@example.com (John R) — Thu, 05 Jan 2012 09:33:00 -0700

Linguists have recently discovered [1] that almost all words are metaphorical at their base, and some people (e.g., me) posit that they all are. Though speculative, it is at least conceivable that even the sub-language signaling in the brain, which eventually leads to language, is also metaphorical. Consider that the bell may become a metaphor for food in the mind of Pavlov's dog.

Language is also able to relate ambiguity about the concepts it conveys. The word “life,” for example, can mean life-biology, or life-consciousness. Up until now, it has been perfectly acceptable to use these two meanings interchangeably. There simply has never been an instance of consciousness that existed outside of a biological body — at least none that we could directly experience with our physical senses.

Things may be changing now. . .

Continue reading "AI - Time for a New Name?"

Unto Us A Child Is Born!

nospam@example.com (John R) — Fri, 16 Dec 2011 12:12:00 -0700

Merry Christmas, Happy Hanukkah, and happy new year. May your days be filled with happiness, love, and joy this Christmas season, and may your new year be a blessing to you and others.

http://www.youtube.com/watch?v=DKk9rv2hUfA

-djr

Site Glossary - A Better Way to Relate Ockham's Razor?

nospam@example.com (John R) — Mon, 14 Nov 2011 15:06:49 -0700

The Glossary entry for William of Ockham here at the site has a new section titled “In Other Words?”. This new section attempts to provide a nutshell explanation of William's original advice more accurately than the nutshell statement commonly used today. The advice in question is commonly referred to as Ockham's Razor. Here's the suggested new nutshell definition from the glossary entry.

"Always express things using the most general representation possible for the context in which the representation is being used."

The glossary entry goes on to clarify that this is just an attempted improvement over the current vague fashion statement, and it welcomes other suggestions.

-djr

Learning is Adaptation is Learning - Using Batesian Mimicry As an Explanatory Device

nospam@example.com (John R) — Thu, 03 Nov 2011 13:11:00 -0700

The book on the Netlab project often returns to the notion that learning is merely a form of adaptation and that, conversely, adaptation is merely a form of long-term learning. This, in turn, all fits under the umbrella notion that memory is behavior.

The idea that learning is adaptation is learning is forwarded as a possibility, mainly as a better means of discussing the concepts. This (in my opinion) provides a clearer and more converged understanding of how memory works in biological organisms. This could be very wrong, of course, so it's important to describe it properly. That way it, and not a straw man, can be critiqued. This article represents one such attempt to properly describe it. . .

Batesian Mimicry

Batesian mimicry is when a non-noxious/non-poisonous plant or animal projects the appearance of a poisonous plant or animal, allowing it to avoid being eaten by predators.

Those predators, goes the logic, which have partaken of the poisonous organism and survived, would have become very sick, and would have learned to avoid ingesting anything that appears to be that organism in the future. This will include those organisms who are not poisonous, but merely look, or act, like the poisonous organism.

Continue reading "Learning is Adaptation is Learning - Using Batesian Mimicry As an Explanatory Device"

Goodbye, World - Dennis Ritchie Creator of C Has Died

nospam@example.com (John R) — Wed, 12 Oct 2011 17:43:00 -0700

Dennis Ritchie, the creator of the C programming language, died on Saturday after battling a long illness. The C programming language, arguably, changed the world. It can be found at the heart of most modern computer applications, operating systems, and successor programming languages.

   Dennis Ritchie
   Creator of the C programming language

   9 September 1941 — 8 October 2011



There's an obituary, and a very well researched history, at the Guardian

From his book, "The C Programming Language"

main()
{
        printf("hello, world\n");
}

Neural Networks Backgrounder: Ce n'est pas une l'histoire

nospam@example.com (John R) — Mon, 03 Oct 2011 08:24:00 -0700

Overview

This article provides a layman's-level discussion of neural network technology within the framework of a sketchy historical sequence. Neural networks are described while presenting an overview of just one of the many routs taken by the field in the last half-century or so.

It is not for those interested in a full history of neural networks (i.e., connectionism). It is just a quick backgrounder, which should suffice to give readers a little bit of perspective into how we got from "there" to "here." The actual history of this field is storied, and sometimes even checkered and controversial. I highly recommend to anybody who is interested, that you get a good book or two on the subject.

This entry will also serve as a place to accumulate links to resources and information on the subject of neural networks and their history at this layman's level.

. . . . . . .
Contents

Continue reading "Neural Networks Backgrounder: Ce n'est pas une l'histoire"

Connection-Type-> v--Operation	Excitatory	Inhibitory
Increase	Increase Connection Strength	Decrease Connection Strength
Decrease	Decrease Connection Strength	Increase Connection Strength