Interunit noise

The effects of Interunit noise occur primarily during training. As Interunit noise increases, it is more difficult for the network to converge when learning its original training set. In fact, if the Interunit noise is high enough the network will not converge during training unless the network's learning rate ($\eta$) is set very low.

Interunit noise also serves another very important function, and is the core of biases observed in the depressed network on the valence identification task. Recall that the network is effectively ``depressed'' by overtraining it on a particular stimulus, or small set of stimuli. The recurrent back-propagation algorithm used to train the network adjusts weights during training proportional to the discrepancy between the network's output and the true target value for the stimulus. Without noise, the network learns the ``depressogenic'' stimulus very quickly, and thus, does not adjust its weights a great deal on subsequent training. With noise there is more chance for discrepancy between the network's output and the target value at each time step, and thus weights may be adjusted more during training. After a long period of such adjustment the network effectively loses the weight configurations which allowed it to make determinations regarding the semantic and affective character of stimuli it had learned during its original training, and thus, is susceptible to biases, e.g., not being able to effectively identify information with a positive valence on the valence identification task.¹³

A possible cognitive correlate of this phenomenon is as follows. If depressed people, spend a great deal of time ruminating on some ever-slightly-changing depressogenic cognition, they may effectively lose their ability to process information which was somehow related to the deviations in their negative thought. Commonly, a depressed person thinking, e.g., of a friend's death will (potentially due to ``noise'' in their information processing) come to think negatively of the types of things they used to do with that friend (e.g., jogging). In this way, the entire usually positive act of jogging becomes negative for the depressed person. Were they to have ruminated only, and exactly, on their friend's death, their depression would not have spread to other areas of their life.