Neural networks for encapsulating welding expertise

TWI Bulletin, March/April 1994

Bill Lucas is Technology Manager in the Arc and Laser Department of TWI. On graduating from Manchester, he received his doctorate from Queen's University of Belfast. In 1987, he became the first research engineer at TWI to be awarded Doctor of Science for his contribution to arc welding and computer technology.

After employment at Leyland Motors for four years, he joined the then Process, Application and Control Department at The Welding Institute. After some 16 years in Arc Welding, he became Head of the Arc Welding Department in 1986.

His research work has been largely devoted to process development and application studies in welding, cutting and surfacing. Over 60 papers have been published and he was awarded the Sir William Larke Medal of The Welding Institute in 1984.

He became a Fellow of The Welding Institute and of The Institute of Metals in 1983 and a European Welding Engineer in 1993.

Microcomputer software for welding engineers has been produced by TWI for about 10 years. There are now more than 15 packages ranging from databases for storing welding procedures to packages for carrying out repetitive calculations such as determining the preheat temperature to avoid hydrogen cracking. As Bill Lucas explains, possibly the most advanced packages have been the expert systems and neural networks which encapsulate the knowledge or expertise of expert welding engineers.

Expert systems have been found to be an ideal mechanism for storing a company's technical knowledge and best practice. Unfortunately, encapsulation of knowledge is a very time-consuming task as the knowledge must first be elicited from the expert, usually by means of a series of interviews. The knowledge is then expressed in the form of rules for embedding into the expert system. A typical expert system for welding engineering will require 6-12 months to be devoted to knowledge elicitation and can contain as many as a thousand rules. ^[1]

More recently, neural networks have been used as a mechanism for storing expertise. The principal characteristics of neural networks which distinguish them from conventional programs and expert systems are:

• The system can learn from examples or directly from experimental data, obviating the need to interview the expert;
• The source data may be incomplete or imprecise;
• As the relationship between the input data and the output is direct, there are no models or clearly stated algorithms to be derived.

Neural networks appear, therefore, to be ideally suited to knowledge-based systems in welding engineering where much of the knowledge is imprecise. For example, in recommending suitable welding parameters, a welding engineer will provide advice based on previous experience, otherwise a substantial number of welding trials must be carried out to establish even the most elementary relationships.

How neural networks work

A neural network operates in a similar manner to the thought processes in a human brain, which consists of a large number of processing elements. The act of learning is effected by the strengthening of the connections in the neurons linking the input (sensory areas) to the output (motor areas). In the neural network, there are a number of elementary mathematical processing units, likewise called neurons, which have several inputs and an output (Fig. 1).

Each input into the neuron has an associated weight which determines the strength of its influence on the network. The neuron acts as a trigger which will fire providing the total strength of the incoming signals is greater than a predetermined threshold level. Thus, the weights on the connection between neurons form the memory of the network.

In the neural computer, there are a large number of neurons (up to several thousand) which are interconnected to form complex networks. In a typical arrangement (Fig.2), there are at least three layers of neurons with each neuron connected to an adjacent neuron to form a forward-feed network. Data flow from the input layer, through the hidden layer (i.e. internal to the network) and out through the output layer. Each neuron progresses the input data through the network, layer by layer, to generate the output.

The neural network provides the same function as the human brain in its capacity to interpret data. For example, in a vision system, shown schematically in Fig.3, the neural network can interpret the output of a video camera in the same way as the human brain to provide information on objects of quite different size and shape, e.g. for recognition and identification purposes.

Training the network

The neural network can be taught in the same way as a human learns new skills, by being shown examples. In the case of a prediction type of network, two sets of examples are presented to the network, a training and a testing set. Each set contains both acceptable and unacceptable situations. The sets provide the input to the network and the corresponding results represent the derived output. Each neuron in the network processes the input data (as shown in Fig.2) and the resultant values cascade through the network layer by layer, until an output is generated.

The output is compared with the data from the examples in the training set and an error value is derived which is a measure of the discrepancy between the predicted and the actual outputs. The weights on the interconnections are then adjusted by propagating the error value backwards from the output layer, through the hidden layers, to the input.

Progressive modification of the interconnections as further examples are used, enables the network to generate a set of interconnection weights which will produce the derived output for all the training data. The network is now considered to be trained and when presented with data from an example from the testing set, an output will be produced on the basis of the trained interconnection weights.

How a neural network differs from conventional and expert system software

• Must be trained;
• Can handle multi-variables without any knowledge of their mathematical relationships;
• Can provide a solution when the data are incomplete ('fuzzy').

The interpretations or predictions can be made even from data not previously considered in the training examples, providing they fall within the boundaries which are set by the training data. Because only the neurons are stored in memory, the trained neural network uses very little memory. Consequently, the speed of operation is extremely high ensuring that neural networks are ideally suited to real-time control applications.

Neural networks are often considered to be an alternative to expert systems in that they encapsulate expertise or provide artificial intelligence. The two types of programs can be used for similar types of applications, e.g. solving problems, predictions and recommending best practice, but the major advantage of neural networks is that it is not necessary to devise rules by interviewing experts. However, this means that the underlying reason for the network reaching a particular conclusion cannot easily be defined. However, by comparing the input data with the output of the network, trends can be observed and explanations for the behaviour derived.

Whilst the time-consuming and costly activity in deriving rules is obviated, the difficult, and indeed costly training of the neural network should not be underestimated. Examples must be produced which clearly define the problem and the boundary conditions for training the network. When several inputs are involved, for example in specifying suitable welding parameters for producing defect-free welds, a large number of fully documented examples must be produced. As in human learning, the quality of the examples determines the performance of the system and its ability to reach a particular conclusion.

Application of neural networks

The most popular application for neural networks is in pattern recognition, e.g. in visual, speech and signal analysis. When there are wide variations in the quality of the patterns, it is almost impossible to write a conventional program which will be capable of accommodating all situations. A unique application has been the recognition of objects in remotely operated, deep-sea exploration (Fig.4). ^[3] Similar systems could be applied in welding inspection; for example, in assessing the quality of a weld, the operator must take into consideration not only the weld bead dimensions, but also the surface profile, presence of undercut, etc. However, a neural network could be trained to match the appearance of the weld with the closest fit from a bank of known patterns.

Other applications in welding engineering include:

• Modelling and control of TIG welding; ^[4]
• Control of submerged-arc welding using ultrasound; ^[5]
• Characterisation of weld defects by ultrasonic inspections ^[6]

It will also be possible to expand the network to predict the type of weld defect which will occur if other than the recommended set of parameters is used.

Programming the computer

Several software development packages are available similar to an expert system shell, for generating a neural network. ^[7] For Weld Predictor, the layered network, using error back-propagation learning algorithms, was built up easily, initially using Brainmaker and, more recently, using Neuralware Professional packages. The interface program was written in the 'C' language which can call the trained network as a subroutine. The interface program is necessary to input the welding parameters into the neural network, and to receive the computer results of the neural network.

A commercial graphics package was also used to display the output of the network, e.g. the predicted weld bead profile for selected input parameters. Weld Predictor consists of two parallel connected sub-networks, one for weld bead profile and the other for weld quality. The inputs and outputs and the parallel structure of the initial network are shown in Fig.5. It was found that by using two hidden layers, 160 in the first and 90 processing elements in the second, the network could be trained relatively quickly.

An example of the training data for weld bead profile sub-network in the most recent system is shown in Fig.6. The weld bead width, as predicted by the training network, for a range of values of welding current and welding speed, is shown in Fig.7. The trained neural network is now capable of predicting the weld bead profile and the quality level for a range of thicknesses of stainless steel and at various welding speeds.

The future

Much of the research on neural networks is being carried out in universities. TWI is supporting projects at The University of Liverpool (TIG welding parameter prediction and on-line control), and Brunel University (characterisation of weld defects by ultrasonic inspection). There is no doubt that neural networks are at an early stage of development. A successful conclusion to both projects will require innovation in design of neural computing especially in configuration of the neural network. Whilst the accuracy and performance of Weld Predictor have been generally satisfactory, further research is required on neural structures, number of hidden layers and techniques for deriving the weighting factors.

For the user, similar case studies should open up the use of computers to subject areas which were previously considered impossible, e.g. vision, pattern recognition and parameter prediction involving complex relationships. However, in using the packages, the user will be largely unaware of the computing techniques at the heart of the package, i.e. whether the solution has been achieved by conventional programming, expert systems or neural networks. But it is important to note that conventional programming remains the preferred technique for mathematical modelling types of programs. Neural networks will provide programmers with another tool for encapsulating expertise which should greatly enhance the range and quality of packages for the welding engineer.

Acknowledgements

References