Information storage capacity of connectionist systems : the linear associator

Abstract: "The information-storage capacity of hetero-associative memory systems is addressed. The associator can be treated as an M-ary symmetric channel when M associations are stored. The maximum number of associations storable is bounded asymptotically by N/2 where N is the number of connection weights. Storage efficiency is bounded by M/N so that it never exceeds 1/2. Information capacity degrades as inter-vector correlations increase and also when classification tasks are performed. The correlation effect is most pronounced in high-dimensional systems storing a large number of associations."