Sequence learning

The problem of sequence learning is major problem of intelligence. There are four major sequence problems that have to be solved.

Here are the definition of them :

from Ron Sun, C. Lee Giles : “Sequence Learning”

1. Sequence prediction

The goal is to predict the next element.

s_i, s_i+1, …., s_j → s_j+1, where 1 ≤ i ≤ j < ∞; that is, given s_i, s_i+1, …., s_j, we want to predict s_j+1. When i = 1, we make predictions based on all of the previously seen elements of the sequence. When i = j, we make predictions based only on the immediately preceding element.

1. Sequence generation

   The goal is to generate the next element.

s_i, s_i+1, …., s_j → s_j+1 where 1 ≤ i ≤ j < ∞; that is, given s_i, s_i+1, we want to generate s_j+1. (Put in this way, it is clear that sequence prediction and generation are essentially the same task.)

1. Sequence recognition

   The goal is to generate correct next element. This is akin to Classification.

s_i, s_i+1, …., s_j → yes or no, where 1 ≤ i ≤ j < ∞; that is, given s_i, s_i+1, …., s_j, we want to determine if this subsequence is legitimate or not. (There are alternative ways of formulating the sequence recognition problem, for example, as an one-shot recognition process, as opposed to an incremental step-by-step recognition process as formulated here.)

With this formulation, sequence recognition can be turned into sequence generation/prediction, by basing recognition on prediction; that is, s_i, s_i+1, …., s_j → yes (a recognition problem), if and only if s_i, s_i+1, …., s_j-1 → s^p_j (a prediction problem) and s^p_j = s^a_j, where s^p_j is the prediction and s^a_j is the actual element.

So recognition can be interpreted either as comparing the sequence elements one by one OR by the condition that the element before-last will predict the last element (this assumes that the last two elemrnts characterize the sequince i.e. that is what makes it unique).

1. Sequential decision making

That is, sequence generation through actions.

there are several possible variations. In the goal oriented case, we have s_i, s_i+1, …., s_j; s_G → a_j, where 1 ≤ i ≤ j < ∞; that is, given s_i, s_i+1, …., s_j and the goal state s_G , we want to choose an action a_j at time step j that will likely lead to s_G in the future. In the trajectory oriented case, we have s_i, s_i+1, …., s_j; s_j+1 → a_j , where 1 ≤ i ≤ j < ∞; that is, given s_i, s_i+1, …., s_j and the desired next state s_j+1, we want to choose an action a j at time step j that will likely lead to s_j+1 in the next step. In the reinforcement maximizing case, we have s_i, s_i+1, …., s_j → a_j , where 1 ≤ i ≤ j < ∞; that is, given s_i, s_i+1, …., s_j we want to choose an action a_j at time step j that will likely lead to receiving maximum total reinforcement in the future. The calculation of total reinforcement can be in terms of discounted or undiscounted cumulative reinforcement, in terms of average reinforcement, or in terms of some other functions of reinforcement

The problems described so far are closed-loop i.e. find the next element. They can also be redefined as open-loop where we are interested the next N elements, not just the next one.

There is a fifth problem when we want to recognize or make a decision for a group of similiar sequences. In this case we need a function that makes them invariant and compare the result.

Lexicon HTM Theory : short intro