Acquired Intelligence & Adaptive Behaviour Week 3 - 2
e) What are they good for?
- Powerful because the dynamics allows a form of memory and processing sequences of input.
- Language
e.g. I L O V E Y O U
i1 i2 i3 ...
f) How to train?
- Backpropagation through time
- Trian with BP through each time update
- Only works for small networks and has lots of problems
3) Attractor Neural Network
- Get rid of the complicated dynamics
- Symmetrical connected node gets rid of chaotic dynamics a leave just multiple fixed points
- Good for pattern completion
4) Echo state machines
- Weakly connected node exhibit only asingle fixed point attractor
- Use the transient dynamics to process
- Good time series analysis
*These models are more appropriate models of the brain which also is recurrent and show very rich dynamics. More biologically plausible than FFNN.