Neural Computing in Pharmaceutical Formulation

Controlling the properties of pharmaceutical formulations demands an understanding of the links between end-use behaviour and the ingredients and processing conditions used to produce the formulation .  Such relationships are generally complex, with the rules governing cause and effect often known incompletely and anecdotally.  Recent advances in artificial intelligence technologies, particularly neural computing techniques, now permit useful information to be extracted directly from experimental data.   When neural networks are combined with other technologies, such as visualization, genetic algorithms and fuzzy logic, they can provide an easy to use and valuable tool for the formulator.  Because these techniques are ‘data driven’, they are exceptionally versatile and can be applied to any formulation type (e.g. tablet, injectable, suspension) where data are available.  This article introduces these technologies and reviews their use in pharmaceutical formulation.

Keywords:Artificial Intelligence, Product Formulation, Product Optimization, Neural Networks, Genetic Algorithms, Fuzzy Logic

Pharmaceutical formulation (irrespective of whether it is for oral products, for parenterals, or for other pharmaceutical products) is a complex process involving the interaction of many ingredient and process variables.  Consequently, it can be difficult to understand such systems, and even more of a challenge to develop useful models for them.  To date, statistics has been used as one approach to this problem.  This method has the advantage of generating clearly expressed models, with associated confidence factors.  However, for more than three or four inputs, statistical approaches rapidly become unwieldy, so that the formulator is tempted to oversimplify the problem (for example, restricting a study to three input variables) in order to model it.  Statistics also often require the assumption of a functional form (for example, linearity) in order to generate a model, and such assumptions can be inappropriate for complex tasks like formulation. 

In recent years, it has been shown that neural networks can provide an alternative approach (1).  Neural networks are mathematical constructs that are capable of “learning” relationships within data, with no prior knowledge required from the user.  The neural network makes no assumptions about the functional form of the relationships; it simply generates and assesses a range of models to determine one that will best fit to the experimental data provided to it.  As such, increasingly, artificial neural networks (often referred to as ANNs) are used to model complex behaviour in problems like pharmaceuticals formulation and processing. 

The models generated by neural networks allow 'what if' possibilities to be investigated easily.  However, their capabilities are enhanced substantially by combining them with other technologies.  For example, using genetic algorithms for optimization, together with neural networks models, has proved exceptionally powerful when the formulator must develop a formulation to meet stringent, often conflicting, objectives.  The objectives for the optimization can easily and intuitively be defined using another artificial intelligence technology, fuzzy logic.  Fuzzy logic has proved especially valuable when conflicting properties (for example, hard tablets that disintegrate quickly) are desired.  More recently, efforts have been made to integrate the technologies even more tightly, creating new methodologies like neurofuzzy logic, which combines the ability of neural networks to 'learn' from data, with fuzzy logic's capacity to express complex concepts in a simple fashion.  These techniques are capable of ‘mining’ information directly from data, presenting it in the form of easy to understand, actionable, rules that can guide the formulator’s future work. 

Here, the key neural computing technologies that have been used for formulation and formulation optimization are reviewed.

Neural networks can trace their roots back to the 1940’s, when initial attempts were made to see if machines could mimic mammalian intelligence.  Their name reflects this legacy.  The fundamental unit of a neural network is the ‘neuron’, a simple mathematical processing unit.  Like the biological neuron on which it is based, the mathematical neuron takes a range of inputs, weights them, adds them together, and produces an output signal which, if it is sufficiently strong, is passed on to the next neuron in the network.   Again analogous to the biological system, mathematical neurons are only effective when they are interconnected in a network.  One of the simplest types of network – and one that has proved exceptionally powerful in pharmaceutical formulation – is the so-called Multi Layer Perceptron (MLP) network illustrated schematically in Figure 1. 

Figure 1. Schematic drawing of Multi-Layer Perceptron neural network

The illustrated network has 3 ‘layers’.  One consists of the ‘input nodes’, and in the case of formulation there is one input node for each of the ingredient (including ingredient amounts) and processing conditions.  One layer consists of the ‘output nodes’; there is one output node for each of the measured properties (properties are, for example, tablet hardness, disintegration time, percentage release at a specific time point, and so on).  The input and output layers are connected via a ‘hidden layer’ with a user-specified number of nodes; here each node is one of the processing units described above.  In theory, neural networks can have more than one hidden layer.  In practice, for formulation problems, one hidden layer should suffice, and more may lead to over-complicated, less predictive, models.

The process of generating a model is straightforward, and requires experimental data that contain cause-and-effect information.  Typically these data would be obtained by performing a number of experiments where the inputs (ingredients and processing conditions) are varied, and the outputs (properties of the final product) are measured.  Within the network, the inputs are then given random weights, and passed to the hidden layer where the weighted inputs are summed together.  Then, these sums are ‘transformed’ with a smoothing function (usually a sigmoidal or S-shaped function), and summed again to produce an output.  The calculated output is then compared with the actual output known from the experimental results.  The difference, between the actual values and those predicted by the network, is used to calculate changes in the weights, in a process known as ‘back propagation of errors’.  The new weights are then used in the summations and transformations, to produce a revised set of predicted outputs.  This process continues iteratively until the calculated outputs match the experimental ones.  At that point, the weights that have been calculated provide the basis for a model that describes the relationship between the inputs (ingredients and process conditions) and the outputs. 

There are various risks associated with this process.  One is that the network will learn a model but that the model will not be predictive – in effect, the network will ‘memorize’ the data, including learning the noise in the system.  There are well-established techniques to avoid this, the most popular of which is a validation process in which some of the data are withheld from the ‘training’ and used exclusively to assess the performance of the network.  Training the network can be stopped when tests on this validation data show that the network risks losing predictivity if the training process were to continue. 

In recent years, neural networks have been used both in academe and industry for a range of pharmaceutical formulations.  For a complete review, see the papers cited in Reference 1. 

As with modelling, discussed above, pharmaceutical formulations make special demands of optimization techniques.  Because there are many variables whose interactions can interact in complex ways, it can be difficult to find the best solution (the global optimum) rather than a local optimum that lies nearest the starting point.  Historically, methods like steepest descent or conjugate gradients have been used, but these are directional techniques that almost inevitably will find a local minimum.  Genetic algorithms, on the other hand, explore the design space more effectively, and are able to find the global minimum.  However, until fast computers became readily available, they were too demanding of computer power to be used routinely.  Fortunately, now they are straightforward and quick to use. 

Genetic algorithms form one aspect of so-called ‘evolutionary computing’.  A trial population of solutions is generated (using the neural network models) and their ‘fitness’ is assessed.  The ‘fittest’ solutions are used as the parents of the next generation, with new solutions being created by mathematical operations that mimic reproduction and mutation.  This process continues until an optimum solution – one that most closely matches our design criteria – evolves.  By controlling the amount of the parent solution’s characteristics that are passed to the next generation, the optimization either can be tightly controlled or can be allowed to explore the design space by ‘jumping around’ previously unexplored areas. 

From this description, it is clear that genetic algorithms rely on an ability to describe a criterion of ‘fitness’ for any proposed solution.  This ‘fitness’ is generally a measure of how important each property is to the formulator, and what values or ranges the property should take.  To define the formulation objectives, fuzzy logic has proved exceptionally useful.  Based on the theory of fuzzy sets introduced in the 1960’s, fuzzy logic allows more flexibility than traditional ‘crisp’ logic, which demands a true or false answer.  To illustrate this, consider the case where the formulator wants a tablet that has a disintegration time less than 180 seconds.  Tablets that meet this criterion have a desirability of 100%.  Traditional logic would demand that a tablet that took 181 seconds would be unacceptable, but fuzzy logic would allow it to have a ‘desirability’ close to 100% (perhaps, say, 98%).  In this way, fuzzy logic allows a more flexible and intuitive description of when a formulation is ‘good enough’. 

These concepts have been used in two software packages, CAD/Chem (now no longer available) and INForm (from Intelligensys Ltd).  Published papers show how such software packages can be used in conjunction with genetic algorithms and fuzzy logic for optimization to show for example the trade-offs in trying to produce hard tablets that disintegrate quickly (2,3). 

Recently, a new technique, neurofuzzy computing, has been introduced.  This combines the strengths of both parent technologies – the adaptive ‘learning’ capabilities of neural networks, and the ability of fuzzy logic to express complex concepts in a form readily intelligible to humans.  The combination allows rules, in the form IF (ingredient 1) AND (ingredient 2) THEN (property), to be derived directly from data.  No assumptions need to be made by the human operator; the technology simply discovers for itself which models best describe the relationships in the data, then presents this in the form of rules with associated confidence levels.  It is often said that neurofuzzy computing adds a degree of transparency to the ‘black box’ models of neural networks, so that these techniques are referred to as ‘grey box’ modelling. 

As yet, the use of this technology in pharmaceutical formulation is in its early stages, having been pioneered by Ray Rowe in his work at the PROFITS Group at the UK’s University of Bradford (4, 5).  Their work has shown that although both neural networks and neurofuzzy methods develop good models from data, neurofuzzy techniques had the added advantage that they generated understandable rules that were consistent with known pharmaceutical practice.  This is encouraging since it means that this technology can be expected to work successfully in new formulation domains where little knowledge is already available.

Neural computing techniques applied to pharmaceutical formulation are now reaching maturity, with a proven track record for modelling complex formulation domains.  They must be applied with some care to ensure that the results are valid and meaningful, and require a sufficient amount of good data, since like all computer methods it is a case of ‘garbage in, garbage out’.   However, with these caveats, they are now poised to take their place as full partners to well-established statistical techniques. 

The author would like to acknowledge a long-standing collaboration with Professor Ray Rowe and his students at the PROFITS Group at the University of Bradford. 

1.R C Rowe and E A Colbourn, Pharmaceutical Visions 2002 Spring Edition, 4-7; E A Colbourn and R C Rowe, The Chemical Educator, 2003 8, 211-218; E A Colbourn and R C Rowe, Pharmaceutical Technology, IT Innovations 2003, 22-24

2.R C Rowe and E A Colbourn, Pharm. Tech. Eur. 1996 8(9), 46-55

3.E A Colbourn, Chapter 6 in Intelligent Software for Product Formulation, R C Rowe and R J Roberts (Taylor & Francis, London, UK, 1998)

4.R C Rowe and C G Woolgar,  Pharmaceutical Science and Technology Today  1999 2,495-497

5.Q Shao, R C Rowe and P York, A comparison of two data mining technologies in the modelling of tablet formulation.  Poster presentation at AAPS, Toronto, Canada 2002

See www.intelligensys.co.uk for more information about the technologies and for a wider range of applications to pharmaceutical formulation.  The new edition of the Enclyclopedia of Pharmaceutical Technology will also have a review on this subject.

Elizabeth Colbourn has been involved in materials modelling for 25 years. In ICI plc she was one of the first women appointed to the prestigious Scientific Ladder before she left to help to found Intelligensys Ltd. She has published over 50 articles in materials modelling, and for the past few years has concentrated especially on the application of artificial intelligence in formulation.

Tel: +44 1642 343411,Fax: +44 1642 714305
e-mail: colbourn@intelligensys.co.uk