Neural Code and Neural Encoding
Week 2 - Computational Neuroscience Coursera
To put it simply, neural code is how information about a stimulus is expressed via neurons by their response to these stimuli.
Encoding Problem: How does a stimulus cause a pattern of responses
Decoding Problem: What does these responses tell us about the stimulus
Because neural systems are very noisy, the models we use are probabilistic...
So when we are looking at neural encoding, we are looking at the probability of a certain response given a certain stimulus . . . . p(response|stimulus)
And vice versa for neural decoding; the probability of a certain stimulus being present given a certain response. . . . p(stimulus|response)
How do we quantify the stimulus and the response?
In computational neuroscience it is important to quantify the stimulus and the response so that we can measure them and we can find their relationship.
We often hope that a given stimulus causes a given response in a neuron (though this is not always the case - sometimes a stimulus and a response have no relationship).
Therefore, the stimulus is the independent variable and the response is the dependent variable. So we often visualize it as such, where the stimulus is on the x-axis and the neural response is on the y-axis.
Often the neural response is quantified as the average firing rate of a neuron.
The stimulus parameter varies. Some examples include:
The average amplitude of an audio recording of someone speaking a certain syllable
The angle orientation of a bar of light
The luminance of an array of pixels
Visualizing Stimulus-Response Relationships:
In the below tuning curve a V1 cortical neuron. These V1 cortical neurons often respond to orientated bars of light. In this case the neuron fires more frequently when the bar of light is almost vertical. When it is horizontal it almost doesn’t respond at all.
This tuning curve is more cosine because the motor neuron still fires despite the angle of the arm. It just fires more and more frequently as the arm approaches 150 degrees.
Stimulus Representations:
A stimulus does not have to be a specific sound or picture - it can also be a concept in general.
In an experiment published by Quiroga et al. in Nature(2005), 80 images were shown to epilepsy patients undergoing surgery. A neuron’s firing rate was recorded for each picture and the following tuning curve was made:
The images with the highest responses were actually pictures of Brad Pitt and Jennifer Aniston. This neuron would not fire at such a high frequency at images of Brad Pitt or Jennifer Aniston alone.
In that same experiment, the patients were shown images and another neuron was recorded. A tuning curve (not shown here) indicated that the stimuli which produced the highest firing rate was the concept of Pamela Anderson. The neuron would fire frequently if an image, drawing or voice recording of Pamela Anderson was shown. It even fired frequently if her name written out was shown!
This indicates that neurons can have varying complexity in stimulus representations. Some stimuli are more geometric (such as in the Lateral Geniculate Nucleus) and some are more semantic (such as in the cortex).
Thus far we have talked about the 2 variables we mostly deal with: the stimulus and its neural response. We understand how we could quantify these variables individually and we know how to visualize them.
But we do not understand their relationship. As mentioned earlier there is encoding and decoding. Let’s first look at neural encoding models - which focus on how a neuron responds to a stimulus: P(response|stimulus).
This is the simplest response coding model (i.e. relationship between the response and the stimulus). It assumes that the response is linearly dependent on the stimulus at that time (or at some time recorded in the past).
This means that our response function, r(t), will look like our stimulus function, s(t), but scaled by some factor number. It may also be delayed a little bit because the response is not immediate after a stimulus is present.
Con: It does not account that there may be more than one input other than the stimulus that the response could be dependent on.
It is also known as delayed amplification
Temporal filtering involves looking back over k time points on s(t). At each of these time points we weight the stimulus at that time by some factor function f(k).
The function f(k) is used to change the weight in noisy signals, such that stimuli signals which are not resulting in the response are “filtered out”. This makes the response function look much less noisy than the stimulus one.
Mathematically, this means that the response function r(t) is a convolution of the stimulus function s(t) and the filtering factor function f(t):
The running average is an example of temporal filtering, where the filtering factor function is equal to 1/N :
f(t) = 1/N, where N is the number of time steps back.
So basically we find the value of s(t) at each of the N time steps and then we weight it with 1/N.
The larger N becomes, the more the noise in the stimulus becomes cancelled out, therefore smoothing out s(t). If N=0, it will just be r(t) but slightly delayed.
The leaky average is another example of a temporal filtering system. This system assumes that r(t) still depends on s(t); however, the memory fades over time. This means that time points which are more recent will be weighted larger than time points which are older. In fact, as we go back in time t by N time steps, the weight for older time steps will exponentially decrease.
This filtering factor function is called a leaky integrator - a filtering function that’s strength exponentially decreases into the past.
This filtering system is important in computational neuroscience because the neuronal cell membrane also behaves like this - i.e. older signals decay.
In spatial filtering the goal is very similar to temporal filtering except now it is in 2 dimensions. Instead of taking time points and weighting them by some factor, now we are taking spatial points (x,y) and then weighting them by some factor.
Spatial filtering is important for understanding responses in neurons which respond to light patterns in space - i.e the receptive filed in the visual cortex (very important).
Let’s start by defining a visual space s(x,y) as shown below:
We have a point (x,y) which is varied by x’ and y’. The x’ and y’ are similar to the k variable in temporal filtering. Whereas, k defines the time interval we look at, x’ and y’ define the area in the space that we are going to look. And at each point defined between x’ and y’ we filter it by f(x’,y’) which is a filtering function of x’ and y’, so that we can get our response function r(x,y):
Just like in temporal filtering, the response at a specific point is dependent on how similar the original point is to the point on the filtering function. The filter will often be a shape - if this is ringing any bells it should because this is a receptive field. Therefore, this is the simplest way receptive fields are modelled!
Spatial Filtering and Retinal Receptive Fields:
Let’s say we have a neuron with the following on-centre and off-surround receptive field:
This is the cartoon representation of a receptive field that we have used thus far. f(x’,y’) is roughly represented as being either negative or positive depending on its shading. Realistically, it actually looks more like this:
As you can see f(x’,y’) is very positive at (0,0,0) and then it has a ring around where f(x’,y’) is negative. Now let’s see how we can apply such a receptive field to an image.
Let’s say we have the following image of the Taj Mahal:
We define our point (x,y) on the image and then create our coordinate frame with x’ and y’ as we did before. Then, we multiply them by f(x’,y’) at those points and sum them together.
If there is a bright point in the centre on the image, then there is going to be a positive response. However, if there is a bright point in the surround on the image, then there is going to be a decreased response (NOT NEGATIVE THOUGH - a neuron cannot fire negatively) because the positive image point will be multiplied by a negative filter point. However, a dark spot on the surround will produce an increase in r(x,y) because a negative brightness will be multiplied by a negative f(x’,y’).
Approximating the Receptive Field:
The receptive field is approximated with 2 gaussian distributions. One narrow positive gaussian with a smaller standard deviation and one broad negative gaussian with a larger standard deviation.
This difference of gaussians filter works very well at detecting differences in brightness - i.e. a dark patch next to a bright patch. It cancels out regions in an image where there is constant intensity, thus it focuses mostly on edges.
This is mathematically indicated below:
w(x,y) is the difference of gaussians filter and you can see the broad gaussian being subtracted from the narrow gaussian.
Just as a refresher here is the gaussian formula so it is more clear:
6. Spatiotemporal Filtering:
Now that we understand spatial filtering and temporal filtering - we can combing them into spatial temporal filtering. Visual sensory neuronal responses depend on both spatial and temporal inputs. Thus, we need a filter that incorporates space and time. Such a filter would look like this:
As you can see that the frames of the receptive field defined in space by x and y are implemented over different points in time. It is also important to note how the spatial receptive field at varying time points is different.
The response function would be a function obviously also be a function of space (defined by x’ and y’) and time (defined by tau).
Shortcoming of the Linear Filters Talked about thus Far:
Linear filtering can result in response functions with negative firing rates or have firing rates that increase indefinitely as the input increases. These 2 situations are obviously impossible as a neuron cannot fire negatively and neurons also have a refractory period.
To fix this by adding another step: a non linear function applied to the filtered stimulus.
This is represented mathematically as such:
Now that we have our stimulus, our filtering feature/function, our nonlinear function, we have all the components of a basic encoding model!!
