The structure of the semantic neural network that implements the morphological and syntactic analysis of the text

The structure of the semantic neural network that implements the morphological and syntactic analysis of the text

The processing of text in a natural language can be divided into several levels: analysis, analysis and synthesis. We define the parse level as a function of transforming a natural language text from a non-formalized form into a formalized internal representation. The level of analysis is a function of converting data that exists in the internal representation and outputting new data on its basis, also in a formalized form. The level of synthesis is the function of forming the answer in a natural language that is adequate to the internal formalized presentation.

The level of analysis can be divided into several successive operations: morphological, syntactic and semantic analysis. Morphological analysis is carried out by selecting from the text of individual words and analysis of the selected words into morphemes. The operation of parsing a natural language text is a definition of all the syntactic features and syntactic links of these words necessary for semantic parsing. Semantic analysis requires the presence of a simulation model of the external world in the system’s memory. It consists in the final formation of the internal formalized representation of the text by comparing the facts that are directly in the text with the knowledge from the simulation model.

Consider the operation of morphological and syntactic analysis of the text of a natural language. The selection of individual words from the text in electronic form is not difficult, since in this case the words are separated by a special symbol "space". For the Russian language, the problem of morphological and syntactic analysis can be solved using the data contained in the grammatical dictionary of the Russian language. It provides information about word formation, inflection, and the definition of the syntactic features of individual words.

The implementation of the level of analysis of the text requires the presence of some formal model of this analysis. According to the Gödel theorem, the implementation of the formalism may be simpler than the description of this formalism. Referring to Gödel's theorem, von Neumann writes: "... In this case, it is faster to do something than to describe, to bring the scheme faster than to give a general description of all its functions and all conceivable circumstances. It is very important to understand that a network of formal neurons can do everything , which can be described in words, and this simplifies the matter unusually at low levels of complexity, but at high levels of complexity it will not necessarily be a simplification. It is quite possible that at high levels of complexity the value is the opposite statement of this Gödel theorem, i.e. it simplifies the matter because it guarantees the opposite: one can express the logic in the language of these constructions [formal neurons], and the direct statement may be incorrect. "

From this statement of Neumann, it follows that, perhaps, as a formal representation one should take not formal mathematical models, but the implementation of these models. One of the variants of such a formal representation can be a formal neural network. Therefore, you can try to use the formal neural network to solve problems of morphological and semantic analysis of the text, as well as the problem of inflection.

The paper describes the operation of the McCulloch-Pitts formal neural network. The individual neurons in the McCulloch-Pitts neural network are logical operations and, or and not. Depending on the result of the logical operation, the neuron has a state of excitement or rest, corresponding to the logical values ​​of true and false. "... The time lag associated with the operation of each neuron ensures the efficient and constructive nature of the logical system obtained here." Since the number of neurons in a network is finite and the number of states in a neuron is finite, such a neural network is an automaton with a finite number of states. A neuron in the McCulloch-Pitts network can only be in two logical states and provides only the functions of algebra-logic. Natural language operates with fuzzy and incomplete concepts. Therefore, the McCulloch-Pitts neural network without additional efforts does not provide processing of fuzzy concepts of a natural language text.

To solve problems of morphological and syntactic analysis of the text, as well as the problems of the analysis of inflection, we apply a semantic neural network that is similar in properties to the formal McCulloch-Pitts neural network. The difference between the semantic neural network and the McCulloch-Pitts network lies in the fact that Boolean algebra logic operations are performed on the McCulloch-Pitts network, and a fuzzy logic operation in the semantic neural network. In fuzzy logic, the confidence factor is used to determine the degree of truth of an assertion — a number that is in a certain interval, for example, from 0 to 1. The maximum value from this interval is usually interpreted as total confidence in the occurrence of an event, and the minimum value as total confidence in its absence.

In contrast to probability theory, the confidence factor expresses subjective confidence in the occurrence of an event and has no statistical meaning. Neurons in the semantic neural network correspond to the elementary concepts of the natural language and process discrete gradient values. Each neuron of this network has a finite number of states. Consequently, the semantic neural network can be considered as a finite state machine.

As the structure of the semantic neural network that performs morphological and syntactic analysis, we choose a synchronized linear tree. We will conduct a rough estimate of the volume of the neural network required for the implementation of the grammar dictionary of the Russian language. In the dictionary A.A. Zaliznyak contains about 100,000 entries. Suppose each dictionary entry contains 10 words of 10 characters each. Then the total amount of unpacked information will be 10 * 10 * 100000 = 107 characters. In the semantic neural network, each symbol corresponds to a separate neuron.

Let, on average, each neuron occupies 1000 bytes in storage. Then the total amount of storage for the semantic neural network that implements the grammatical dictionary A.A. Zaliznyak will be 1010 bytes or about 10 gigabytes. The accumulators of this volume are no longer exotic, therefore, it is possible to fully represent a grammatical dictionary in the form of a semantic network, in which each form of a word from a dictionary entry will correspond to a separate word in the form of a fragment of a neural network. In practice, the grammatical dictionary, implemented as a semantic neural network, will have a smaller volume. This is due to the fact that in a synchronized linear tree used to store word forms, the same character sequences of different words are stored as one fragment of a linear tree.

The layer of extraction of meaning in the form of a synchronized linear tree can be considered as a finite state machine, since the number of neurons in the network is limited, and they have a finite number of states and connections. The transition from one state to another occurs when the meaning of the next character of the input sequence is applied to the extraction layer. It is convenient to consider the extraction layer not as one automaton, but as several final sub-automata - according to the number of dictionary entries. It is also convenient to assume that one neuron has one gradient substate in the range from rest to excitation. Let each such substate is an elementary sense. It can be said that one or several excited neurons correspond to one active substate of a neuro-automaton. Then in one fragment of a synchronized linear tree (in one automaton) we will get several subautomats - according to the number of dictionary entries or several states of one automaton simultaneously. This solution will allow to cope with the ambiguity of natural language in the future. Consider the model of a neuron of a dictionary entry. The implementation of a semantic neural network on a sequential computing system imposes additional requirements on the speed of neurons. If possible, increase the speed of the individual neuron and reduce the number of neurons in the network, since in this case the neurons are processed sequentially, one after the other, and therefore the total calculation time for one system clock is equal to the number of processed neurons multiplied by the processing time of one neuron. To reduce the number of neurons and increase the speed of their work, combine the disjunctor and conjunctor in one neuron.

In this case, each neuron will have two dendritic trees: one - performing the function of disjunction of input gradient values, the other - performing the function of conjunction of input gradient values ​​and the result of the function of disjunction. The neuron will be designated by a circle, its disjunctor's dendrite will be designated to the left of this circle, the conjunctor's dendrite will be above or below the circle, the axon will be to the right of the circle. For convenience, inside the circle, we will write symbols corresponding to the conjuncture dendrite.

Consider the dictionary entry model. A separate dictionary entry is the main word that carries the main semantic load of the article, and a group of word forms, resulting from the main word by inflection (conjugation or declension). Let one dictionary entry be a group of neurons, or one neural subavtomat in a layer of extraction of meaning. Let the total number of substates of a dictionary entry be equal to the number of word forms of this article. Let each substate of such a sub-automaton be one excited neuron. At the same time, in the case of simultaneous excitation of two different neurons of one subautomat, we will say that the subautomatic machine simultaneously has two different substate states. Each entry has a main neuron corresponding to this entry. The main neuron of a dictionary entry is excited whenever the word belonging to its dictionary entry is recognized. Each word form corresponds to a separate neuron. It is excited if the word form is recognized.

In the sense extraction layer, there are neurons that do not belong to individual dictionary entries. These neurons correspond to the attributes of word forms common to many vocabulary entries, such as gender, case, number, time ... They are excited by the initiation of word forms with relevant characteristics. We assume that the states of these neurons, corresponding to the attributes of word forms, also belong to the subautomatic vocabulary entries with which these neurons form connections. Then, several dictionary entries can be in the same state at the same time. For example, all dictionary entries that recognize the word "spit" will simultaneously be in the "noun" state, or, equivalently, the common neuron corresponding to the sign "noun" will be in an excited state.

The set of excited neurons of the subautomat corresponds to the set of features belonging to a separate word form recognized by the subautomat. The task of classifying or defining a dictionary entry and word form according to a given symbol sequence is reduced to passing an excitation wave through the layer of extracting the meaning and exciting the corresponding subautomat for the corresponding dictionary entry. The task of inflection is reduced to a change in the state of such a subautomat from the initial state — the corresponding word form from which the inflection begins to the final state — the corresponding word form into which the original word form is to be converted.

Consider the structure of the neural network connections, providing a solution to the problems described. A synchronized linear tree provides a solution to the problem of classifying a word form into dictionary entries and determining the characteristics of this word form. In the case of the presence of ambiguity, in a synchronized linear tree all vocabulary entries and word forms, corresponding to all the individual meanings of the word form, are excited. For example, when entering the input of a synchronized linear tree of the word "spit", vocabulary entries corresponding to the values ​​"spit-hair", "spit-tool", "spit-landscape" will be excited. In this case, in each dictionary entry, two word forms will be excited corresponding to the substantive and the prepositional.

To solve the word change / word formation problems, you can also use a synchronized linear tree. In this case, it will act as a switching circuit, switching the excitation, transferring the subautomat from one state to another. Switching of subautomatic states will occur when special commands are sent to the input of a synchronized linear tree. These commands will be recognized by a synchronized linear tree and converted to a gradient value at the output of their corresponding effector neurons, which will cause excitation or inhibition of neurons corresponding to the states of the dictionary entry.