Artificial Intelligence and the Artificial Neural Network Approach

Artificial neural networks are an artificial-intelligence approach developed by modelling and imitating the human brain. The model is based on neurons, the biological nerve cells found in the brain. Mathematical models derived from medical knowledge about neural systems led to the neural-network techniques now used widely in engineering applications.

Solving problems often requires information represented in different structures to be identified and perceived rapidly. Artificial neural networks are highly effective in this respect. They can also improve through learning, including capabilities such as recognition and prediction.

Computer systems can outperform people in speed, accuracy, computational capacity, and performance. Nevertheless, computers do not yet possess a learning capability as comprehensive as that of the human brain. Artificial neural networks aim to give machines part of this power and capability.

Because of these properties, artificial neural networks have become a practical method for difficult problems, either independently or together with conventional solution techniques.

BIOLOGICAL NEURONS

When human intelligence is modelled for machine learning, neurons are taken as the primary example. Structurally, a human neuron is described through dendrites, an axon, a soma or nucleus, and its connections.

Dendrite

Dendrites carry signals received from other neurons towards the centre of the neuron.

Axon

The axon carries accumulated signals received from the soma to other neurons. These signals are transmitted after undergoing preliminary processing.

Connections

Connections are responsible for conveying newly produced signals to other neurons.

Communication between the soma and dendrites may vary. Some dendrites, for example, form dominant connections, while others communicate more passively.

Soma

The soma is the centre that collects signals delivered by the dendrites and transfers them to the axon. It is also described as the cell body or nucleus in this simplified model.

Synapse

A synapse performs preliminary processing on signals arriving from an axon and conveys them to the dendrites of other neurons.

APPLICATION AREAS

Artificial neural networks can store information, perform calculations, learn a problem from previously supplied examples, and produce solutions for new instances of that problem.

They are therefore used in many fields, including the following:

Digital forensics

Biomedicine

Durability analysis

Control systems

Language translation

Handwriting recognition

Electrical-signal recognition

Finance

Image processing

Image and data comparison

Weather forecasting

Quality control

Cost analysis

Automated vehicle control

Autopilot applications

Fingerprint recognition

Licence-plate recognition

Robotics

Speech recognition

Cybersecurity

Signal processing

System modelling

Word recognition

Flight simulation

Manufacturing-process control

Path following

Face matching

ADVANTAGES AND DISADVANTAGES

The strongest and most important property of an artificial neural network is its ability to learn. Its principal limitation is that analysing how the network reaches its result may be difficult and that the learning process may fail.

Advantages

A trained network can produce meaningful responses for data that was not included in the training set.

It can solve problems involving nonlinear subsystems.

If a problem occurs in an individual neural-network cell, the main system may continue operating safely.

It learns from problems and can make reasonable decisions in similar situations.

Because it works differently from conventional methods, it does not necessarily reproduce errors arising from those methods.

The loss of an individual cell does not necessarily cause meaningful information to disappear.

Data is stored in a distributed manner.

High accuracy can be achieved when the quantity and quality of example data used during training are increased and appropriately filtered.

The network may produce meaningful output even when some data is missing.

A separate explicit mathematical model is not always required.

The architecture can provide fault tolerance even when some cells fail.

It can be used for perception.

It can operate on the basis of pattern recognition.

It can provide highly accurate solutions to classification problems.

Self-learning and self-organisation are possible.

It can produce solutions more rapidly than conventional methods in suitable problems.

Disadvantages

With the exception of certain network families, stability analysis cannot generally be performed for artificial neural networks.

When a solution is produced, detailed information about how and why that solution was reached may not be available.

Because artificial neural networks operate on numerical data, source information must first be digitised by methods that preserve sufficient accuracy.

Parameters selected before a network is created may produce undesirable results in different application domains.

There is no universal rule that determines when training should end.

Some applications depend on specialised hardware because of their need for parallel computation.

HISTORY AND FUTURE

History

1969: Proposal of linear associators

1972: Development of correlation-matrix memory

1974: Design of the backpropagation model

1978: Design of the ART model

1982: Development of Kohonen learning

1982: Design of the SOM model

1982: Development of Hopfield networks

1982: Development of the multilayer perceptron

1984: Design of the Boltzmann machine

1985: Design of the delta learning rule

1988: Design of the RBF model

1988: Design of the PNN model

1991: Design of the GRNN model

Future

Research on artificial neural networks continues throughout the world. Networks are being redesigned to offer higher accuracy and greater speed, while new architectures are being developed for different classes of problems.

More efficient learning algorithms are being developed, together with networks capable of responding to models that change over time.

Artificial neural networks are also being implemented in hardware through silicon neural systems. Neural-network chips and circuits dedicated to this purpose are being developed, while optical-chip architectures are considered as an alternative.

Technologies such as FPGA and ASIC are being used to design more efficient hardware-based artificial neural networks.

BASIC COMPONENTS

As stated above, artificial neural-network cells operate in a manner inspired by biological nerve cells. Individual cells are connected to form larger networks.

Artificial neural networks consist of interconnected nodes and processing elements, with many subcomponents operating concurrently.

Like biological networks, artificial networks contain units that receive input signals, collect and process those signals, and transmit outputs.

Artificial neural networks possess capabilities such as generalisation, learning, and adaptation. In the human brain, learning occurs through the creation of new axons, stimulation of those axons, and changes in the strength of existing connections.

The counterpart of a biological nerve cell in an artificial neural network is the processing element. Each processing element contains five components: inputs, weights, a summation function, an activation function, and an output.

Inputs

Inputs are the information the network is expected to learn. The values obtained from these inputs are sent to the processing centre for aggregation, in a manner analogous to signals received by a biological neuron.

Weights

Weights are defined coefficient values that determine the effect of each input on the network.

Each input has an associated weight. A large weight indicates that the input is connected more strongly to the network or has greater importance.

A small weight indicates a weaker connection or lower importance.

Summation Function

The summation function multiplies each input by its corresponding weight, adds the products together with a threshold value, and passes the result to the activation function. In some systems this is a simple function; in others it may use more complex algorithms based on minimum, maximum, majority, or normalisation operations.

Activation Function

The activation function transforms the result of the summation operation, potentially as a function of time.

Scaling and Limiting

The result of the activation function may undergo scaling and/or limiting. In its simplest form, scaling is the multiplication of the activation value by a scale factor.

Limiting prevents the scaled result from exceeding defined minimum and maximum boundaries.

Output Function

The output function is the part through which the result of the activation function is sent to other neurons or to the external system. A neuron has a single output, which may be supplied as input to any number of subsequent neurons.

LEARNING

After an artificial neural network has learned, it is tested by supplying inputs that were not previously presented to it. If the network also produces good results for these unseen examples, the network is considered to have learned the problem.

To evaluate this, problem data should be divided into a training set and a test set. The training set contains input and output values supplied to enable the network to learn. The test set contains data not previously presented to the system; its expected outputs are known, but only the inputs are supplied to the network.

There is no universal rule for the proportion of data that should be assigned to training and testing. The important consideration is that the training set should adequately cover the problem space.

Learning Process

Examples representing the problem to be learned are collected.

The summation and activation functions are selected.

The network topology and output structure are determined.

Initial weight and threshold values are assigned.

Examples from the training set are presented.

The network output is calculated for the supplied training inputs.

The network output is compared with the expected output and error values are calculated.

A backward-calculation function is run and the weights are updated to reduce the error.

Supervised Learning

This method is described as not requiring a separate teaching agent, but both input values and their corresponding output values must be supplied to the artificial neural network. During learning, the weights are adjusted towards values that best represent the supplied examples. The difference between the network output and the true value is used to calculate the error, and the weights are determined accordingly.

Unsupervised Learning

Only input values are available in this method, so only those values are supplied to the network. Learning occurs as the network classifies or organises the input data.

Reinforcement Learning

This method also includes guidance, but the system is not given the level of detail used in supervised learning. At each iteration, the network receives information indicating whether the result is good or bad. It learns from this feedback and reorganises its internal representation.

TYPES OF ARTIFICIAL NEURAL NETWORK

Convolutional Neural Network

A convolutional neural network is a multilayer feed-forward architecture and is widely used in image processing, speech recognition, and natural-language processing.

Learning in a CNN is achieved by arranging and stacking convolutional layers with pooling operations. Multiple local kernels in the convolutional layers process raw input data to produce convolutional filters and locally invariant features. Subsequent layers extract increasingly important features.

Deep learning has provided advanced analytics for intelligent production. Pooling operations applied to raw input data include maximum pooling and average pooling. Maximum pooling is particularly suitable for extracting sparse prominent features during the pooling process.

After multilayer feature learning, fully connected layers transform a two-dimensional feature matrix into a one-dimensional vector.

Entropy values can be calculated to determine error rate and accuracy.

Recurrent Neural Network

Recurrent neural networks are used particularly for language sequences and for predicting the next point in sequential data.

Economic observations organised as a time series, for example, can be supplied as input to estimate the state of the values at a future time.

Another example is predicting the word that is likely to follow a sequence of words in a sentence.

Two commonly described forms are bidirectional RNNs and deep RNNs.

Long Short-Term Memory

Gaps and long intervals between relevant elements in a sequence make it difficult for a conventional RNN to predict later elements. Research intended to overcome this limitation led to long short-term memory, or LSTM, networks.

LSTM networks are structurally similar to RNNs but additionally contain memory cells. A memory cell retains information about the previous state and current input, and determines which information should be preserved or discarded.

The previous state is then combined with the current memory and input. This approach allows sequences to be continued while handling long-term dependencies more effectively.

Restricted Boltzmann Machine

A restricted Boltzmann machine consists of two neural layers, visible and hidden. It is used for classification, regression, dimensionality reduction, collaborative filtering, learning, and modelling, and it also forms a basis for deep belief networks.

Nodes in an RBM are connected across layers.

There is no connection or communication between two nodes in the same layer. Each node processes data and makes a stochastic decision about whether to transmit that data.

REFERENCES

Anderson, J. A.; Pellionisz, A.; and Rosenfeld, E. (eds.), 1990. Neurocomputing 2: Directions for Research. MIT Press.

Chakrapani, J., and Skorin-Kapov, J., 1992. “A connectionist approach to the quadratic assignment problem.” Computers and Operations Research, 19(3/4), 287–295.

Dagli, C. H.; Kumara, S. R. T.; and Shin, Y. C., 1991. Intelligent Engineering Systems Through Artificial Neural Networks. ASME Press, Fairfield, New Jersey.

Damodran, P. S.; Kolli, S. S.; and Alexander, S. M., 1993. “The investigation of new approaches to business data analysis.” Computers and Industrial Engineering, 25(1–4), 545–548.

De Carvalho, Luis A. V., and Barbosa, V. C., 1989. “Towards a stochastic neural model for combinatorial optimization.” Proceedings of the IEEE International Joint Conference on Neural Networks, Washington, D.C., vol. 2, p. 587.

DeSilets, L.; Golden, B.; Kumar, R.; and Wang, Q., 1992. “A neural network model for cell suppression of tabular data.” Working paper, College of Business and Management, University of Maryland, College Park, Maryland.

Glover, F., 1994. “Optimization by ghost image processes in neural networks.” Computers and Operations Research, 21(8), 801–822.

Jagota, A., 1996. “An adaptive, multiple restarts neural network algorithm for graph coloring.” European Journal of Operational Research, in press.

Kavcıoğlu, Ş., 2019. “A Comparison of Classical Methods and Artificial Neural Networks in Corporate Credit Scoring.” İstanbul Journal of Economics.

Keskenler, M., and Keskenler, E. “Artificial Neural Networks and Their History from Past to Present.” Takvim-i Vekayî.

Lee, B. W., and Sheu, B. J., 1990. “Combinatorial optimization using competitive Hopfield neural networks.” Proceedings of the IEEE International Joint Conference on Neural Networks, Washington, D.C., vol. 2, pp. 627–630.

Lee, J. K., and Jhee, W. C., 1994. “A two-stage neural network approach for ARMA model identification with ESCAF.” Decision Support Systems, 11(4), 461–479.

Lee, T. H.; White, H.; and Granger, C. W. J., 1993. “Testing for neglected nonlinearity in time series models.” Journal of Econometrics, 56, 269–290.

Looi, C., 1992. “Neural network methods in combinatorial optimization.” Computers and Operations Research, 19(3/4), 191–208.

Meriç, E., 2019. “Personnel Planning at Airports Using Artificial-Intelligence Methods.” Bursa Uludağ University.

Öztürk, K., and Şahin, M., 2018. “A General Overview of Artificial Neural Networks and Artificial Intelligence.” Takvim-i Vekayî.

Ramanujam, J., and Sadayappan, P., 1988. “Optimization by neural networks.” Proceedings of the IEEE International Conference on Neural Networks, San Diego, California, vol. 2, pp. 325–332.

Tank, D. W., and Hopfield, J. J., 1986. “Simple ‘neural’ optimization networks: An A/D converter, signal decision circuit, and linear programming circuit.” IEEE Transactions on Circuits and Systems, CAS-33(5), 533–541.

Yılmaz, Ş., and Özcan, B., 2019. “Estimation of Gross Domestic Product per Capita by Purchasing Power Parity Using an Artificial Neural Network and Regression.” International Journal of Social Sciences Academy.