Introduction to Neural Networks

The theory of artificial neural networks includes a wide range of questions from different areas of science: biophysics, computer science, mathematics, circuit design, etc. We give the following definition:

Artificial neural networks are a collection of models of biological neural networks. Neural networks are a collection of elements interconnected by synaptic connections. The network processes the input information and in the process of changing its internal state of time forms output actions.

A wide interest in neural networks was initiated after the appearance of Hopfield's work (Hopfield JJ, 1982), which showed that the problem with Sizing neurons can be reduced to generalizations of a number of models developed by that time in the physics of disordered systems. The work of the Hopfield network (most discussed in the physical literature) consists in relaxing the initial “spin portrait” of a matrix of binary codes to one of the stationary states defined by the learning rule (Hebba rule). Thus, this network can be used for recognition tasks.

In 1986, the work of Rumelhart, Hinton and Williams (Rumelhart DE, Hinton GE, Williams RJ, 1986) appeared, containing an answer to a question that for a long time restrained the development of neuroinformatics - how hierarchical layered neural networks are taught, for which "classics" are still in 40 50 years has been proven universal for a wide class of problems. In subsequent years, the algorithm proposed by Hinton back-propagation of errors underwent countless variations and modifications.

The diversity of the proposed algorithms, characterized by varying degrees of detail, the possibilities of their parallel implementation, as well as the presence of hardware implementation, leads to special relevance of the study on the comparative characteristics of various techniques.

Let us highlight the main characteristics of artificial neural networks:

• Flexible model for approximation of multidimensional functions.
• A tool for predicting in time processes that depend on a large number of variables.
• Pattern Recognizer
• Association Search Tool
• Model for finding patterns in data arrays

Biological neuron.

The central nervous system has a cellular structure. A unit is a nerve cell, a neuron. The neuron has the following basic properties:

1. Participates in the metabolism and dissipates energy. Changes the internal state over time, reacts to input signals and generates output effects and therefore is an active dynamic system.

2. It has many synapses - contacts for information transfer.

3. A neuron interacts by exchanging electrochemical signals of two types: electrotonic (with attenuation) and nerve impulses (spikes) that propagate without attenuation.

Biological neuron contains the following structural units:

Cell body (t) - soma: contains the nucleus (s), mitochondria (provide the cell with energy), other organelles that support the vital activity of the cell.

Dendrites (d) - input fibers, collect information from other neurons. Activity in dendrites changes smoothly. Their length is usually no more than 1 mm.

Membrane - maintains a constant composition of the cytoplasm inside the cell, provides for the conduction of nerve impulses.

Cytoplasm is the internal environment of the cell. It differs in the concentration of K +, Na +, Ca ++ ions and other substances in comparison with the extracellular medium.

The axon (a), one or none of each cell, is a long, sometimes more than a meter, output cell nerve fiber. The impulse is generated in axonnomolikom (ah.). The axon provides impulse conduction and transmission to other neurons or muscle fibers (mV). Toward the end, the axon often branches.

Synapse (c) - the place of contact of nerve fibers - transmits excitation from cell to cell. Synapse transmission is almost always unidirectional. Presynaptic and postsynaptic cells are distinguished in the direction of impulse transmission.

Schwann cells (Swiss). Specific cells, almost entirely composed of myelin, an organic insulating substance. The nerve fiber is wrapped tightly with 250 layers of myelin. Uninsulated nerve fiber sites between Schwann cells are called Ranvier's interceptions (PR). Due to myelin isolation, the speed of propagation of nerve impulses increases 5 * 10 times and reduces the energy costs of conducting impulses. Myelinated fibers are found only in higher animals. In the human central nervous system there are from 100 to 1000 types of nerve cells, depending on the degree of detail chosen. They differ in the dendrite pattern, the presence and length of the axon and the distribution of synapses around the cell. Cells are strongly interconnected. A neuron may have more than 1000 synapses. Similar in function cells form clusters, spherical or parallel layered. There are hundreds of clusters in the brain. The cerebral cortex is also a cluster. The bark thickness is 2 mm, and the area is about a square foot.

Nerve impulse (spike) - the process of propagation of excitation along the axon from the cell body (axon knoll) until the end of the axon. This is the main unit of information transmitted through the fiber, so the model of generation and propagation of nerve impulses (NI) is one of the most important in the theory of NA.

Pulses are transmitted through the fiber in the form of surges in the potential of the intracellular environment with respect to the external environment surrounding the cell. The transmission speed is from 1 to 100 m / s. For myelinated fibers, the transfer rate is about 5 to 10 times higher than for unmyelinated ones. At distribution the form of spike does not change. Impulses do not fade. The spike shape is fixed, determined by the fiber properties and does not depend on how the pulse is created.

Biological neural network

In the human brain, neurons unite in a network, and three layers can be distinguished: sensory, decisive and motor. The sensory layer - receives information from the retina, tactile receptors, etc., in fact - receives information from sensors, then there is a primary processing and sending information to the brain.