Adrian Silvescu – Fourier Neural Networks (Article)

Adrian Silvescu – Fourier Neural Networks (Article)

size 587 KB

The first mathematical model of a neuron was proposed by McCulloch & Pitts [1943].

The underlying idea that this model tries to capture is that the response function of a neuron is a weighted sum of its inputs filtered through a nonlinear function:

y=h(Pwixi+í).

Much progress has been done in the field of neural networks since that time but this idea still remained a very fundamental one. Although the model of the computational unit(neuron) per se is simple, neural networks are powerful com-puters, higher levels of complexity being achieved by connecting many neurons together.

In this paper we try to propose more general and powerful models for the neuron as a computational unit. There are may motivations for this investiga-tion.

One of them is the fact that although the power of computers increased quite a lot since 1943 we are still not able to simulate and train but toy-size neural networks. So although from a theoretical point of view creating com-plexity out of very basic components is desirable, from a practical point of view more powerful models of the computational units(neurons) are more appealing because they can help reduce the size of the networks by some orders of magni-tude and are also more suitable to coarse grained paralelization. More complex and powerful computational imply also a more compact representation of the information stored in the network, making it an improvement from an Occam razor point of view.

Another motivation towards more general and elaborated models neurons comes from the discoveries in neurobiology that show more that more complex phenomena take place at the neuron level.

Although apparently different from the early model of McCulloch&Pitts our model is still based on the same kind of idea (although in a more general way) ”of computing the output of the neuron as weighted sum of the activations produced by the inputs”.

We will first introduce a general framework and discuss some of the issues that appear. Then a particular model, the Fourier Neural Networks is intro-duced and closely examinated. Next some specific theoretical results are pre-

sented followed by experimental results. Finally the conclusions and further de-velopment are discussed.

Note: The Fourier Neural Networks were introduced in Silvescu[1997].