TM design¶
We will first talk about the design of TM and see how the Segmented memory was born out of it.
Below you can see a schematic picture of part of a cortical/macro column (CC/MC). The columns in the image represent mini-columns. Every circle represent a neuron.
In this case the input SDP:SDR comes from the bottom, the outputs goes out from the top.
The algorithm¶
Given that structure the process work like this :
Fetch the next SDP from a source and then pass it to the columns (vertically)
For every column for which the SDP have value of 1 If a cell in the column is in predictive state, change the state to active
If there is no predicted cell in a column, change the state of all cells in this column to active (bursting)
Propagate the signal from all active cells laterally.
Decide which cells should switch to predictive state based on the lateral flow of information.
Rinse and repeat
Simplify, simplify, simplify¶
We can implement the Cortical column (CC) by making a model of every neuron with segments and synapses, but this requires too many resources. So we have to simplify.
Every neuron in a mini-column have the same feed-forward input plus there are also a myriad of lateral connections between the neurons within the CC.
Because of the common feed-forward, we can think of every mini-column as a single CELL with combined count of all the neurons distal segments.
The goal of a mini column is to detect patterns we do that by storing an SDP or thinned SDP’s or union of SDP’s into a “segment”. Once stored we can compare the input data-SDP against those segments for a match. This mimics lateral neuron connections.
So to recap : A CELL can be implemented as 2D-array where every row represent a segment that store a iSDP or a UNION of iSDP’s. And a CC can be implemented as a 3D array with a shape ( num-of-segments, num-of-mCols, nbits-per-iSDP ).
CC Capacity¶
If we assume that all of the CC neurons are used for the single purpose of memorizing TRANSITIONS /this will make our calculations easier/.
We can do back of the envelope calculation :
mCol capacity = 100 mCol-neurons * 100 segments * 10 patterns-per-seg = 100 000 patterns
The number of CC in the cortex if we use 2000 bits SDP:SDR per CC is ~100 000, then :
CC capacity = patterns / sparsity = 100 000 / 0.02 = 5 Million transitions
the assumption is that to fill UNION with 2% sparse orthogonal SDP’s, you need 1/0.02 = 50 of them.
Cortex capacity = 5M * 100 000 CC = 500 Billion transitions