Deep Learning in an Adaptive Function Neural Network

Artificial neural network learning is typically accomplished via adaptation between
neurons. This paper describes adaptation that is simultaneously between and within neurons. The
conventional neurocomputing wisdom is that by adapting the pattern of connections between neurons
the network can learn to respond differentially to classes of incoming patterns. The success of this
approach in an age of massively increasing computing power that has made high speed
neurocomputing feasible on the desktop and more recently in the palm of the hand, has resulted in
little attention being paid to the implications of adaptation within the individual neurons. The
computational assumption has tended to be that the internal neural mechanism is fixed. However,
there are good computational and biological reasons for examining the internal neural mechanisms of
learning. Recent neuroscience suggests that neuromodulators play a role in learning by modifying the
neuron’s activation function [Scheler] and with an adaptive function approach it is possible to learn
linearly inseparable problems fast, even without hidden nodes. The ADaptive FUction Neural
Network (ADFUNN) presented in this paper is based on a linear piecewise neuron activation function
that is modified by a novel gradient descent supervised learning algorithm [Palmer-Brown;Kang]. It
has been applied to the Iris dataset, and a natural language phrase recognition problem, exhibiting
impressive generalisation classification ability with no hidden neurons.

Citation:
Palmer-Brown, D. and Kang, M. (2006) ‘Deep Learning in an Adaptive Function Neural Network’ Proceedings of the AC&T, pp.78-83..