Could this be a model for neurons in the neocortex?

Sebas, 03 May 2025

I emailed Dr. Sherif to ask for some papers to read before I join his lab this upcoming summer. Attached below is the first paper I have started reading.

Download PDF

So far, I see that the paper proposes that synapses on dendrites on neurons can act as pattern detectors. As the paper states, all artificial neural networks (ANNs) lack active dendrites, and it seems that previous to this paper, all ANNs did not have dendrites represented in their code. A quick google search shows that some papers claim having dendrites represented in ANNs can help in computer vision tasks by reducing the number of parameters. The paper at time of publishing (2016) stated that the functional and theoretical benefits of having active dendrites in ANNs was unclear, but now it seems like there might be. This is exciting stuff!

The paper proposes that dendrites with synapses act as pattern detectors in this way:

  1. Proximal synapses (ones that are close to the cell body) have a large effect on the cell body, and therefore define the receptive field.
  2. Basal synapses learn transitions in sequences. Basal dendrites only slightly depolarize the cell (the neuron), and they propose that this slight depolarization is important and acts as a source of prediction for the neuron.
  3. They propose that Apical synapses act as another form of prediction similarly to Basal synapses, but top-down. In other words, through feedback connections, apical dendrites predict multiple sequences simultaneously.

The paper goes into detail about how this is implemented in the model:

  • Learning in the model is accomplished by the growing and removing of synapses from a pool of ‘potential’ synapses.
  • For each dendritic segment, a set of potential synapses that could form is kept updated.
  • A scalar –‘permanence’– is assigned to each synapse to indicate to what extent the synapse is developed.
  • And a binary weight, with a set threshold of permance to decide it, to indicate if a synapse has formed

They fed the network random and repeated sequences, with setting the maximum accuracy to 50%. I am unsure why they decided to set it to 50%, also curious to how maximum accuracy is determined. I guess they ‘trick’ the model into predicting a sequence but randomly cut one of the characters to make the predictions wrong. I will have to ask Dr. Sherif for clarification on why they decided to do that.

They bring up a previous paper that suggested that a neuron with active dendrites could be modeled by a multilayer perceptron, which is interesting because I was thinking that the dendrites could be modeled by artificial neurons themselves as they offer predictions. But they also mention some benefits of the HTM model (their model neuron in the paper) as opposed to other models without active dendrites:

  • inhibitory effects of the network do not apply equally to different synaptic integration zones
  • it maintains longer lasting subthreshold depolarization (I suspect this is some ANN lingo that I am not privvy to)
  • the unsupervised learning rules are different and operate at different time scales

They also mention the benefits of the HTM model over other sequence memory techniques in 1:machine learning, and 2:other biologically motivated models

  1. HTM attempts to model biology.
  2. HTM is more robust, able to learn continuously and has ability to make multiple predictions simultaneously.
  3. Other biologically motivated models are limited to non high-order sequences and have not been applied to real world tasks
  4. Spiking neuron models are at a lower level of detail than HTM

The paper talks about its predictions based on their theory, (their model of neurons in the neocortex, HTM) most notably:

  1. They predict that it offers a reason for the experimental observation that cell activity gets sparser when a continuous predictable sensory stream is fed as input.
  2. They predict that current pattern of cell activity contains information about past stimuli which has been supported by early experimental work.
  3. They predict that all excitatory neurons are learning transitions or sequences.

From my basic understanding the HTM model really looks like a candidate for models of the neocortex, and I assume it becomes one in the later papers. I would really like to see how this can be applied to understand illnesses such as Schizophrenia. From my understanding a negative prediction bias is prevalent in Schizophrenia so I wonder how that would be encoded using this framework.

Tags: Brown, Computational-Neuroscience, Models