An Intensive Insulinotherapy Mobile Phone Application Built on Artificial Intelligence Techniques

Abstract

Background

Software to help control diabetes is currently an embryonic market with the main activity to date focused mainly on the development of noncomputerized solutions, such as cardboard calculators or computerized solutions that use “flat” computer models, which are applied to each person without taking into account their individual lifestyles. The development of true, mobile device-driven health applications has been hindered by the lack of tools available in the past and the sheer lack of mobile devices on the market. This has now changed, however, with the availability of pocket personal computer handsets.

Method

This article describes a solution in the form of an intelligent neural network running on mobile devices, allowing people with diabetes access to it regardless of their location. Utilizing an easy to learn and use multipanel user interface, people with diabetes can run the software in real time via an easy to use graphical user interface. The neural network consists of four neurons. The first is glucose. If the user's current glucose level is within the target range, the glucose weight is then multiplied by zero. If the glucose level is high, then there will be a positive value multiplied to the weight, resulting in a positive amount of insulin to be injected. If the user's glucose level is low, then the weights will be multiplied by a negative value, resulting in a decrease in the overall insulin dose.

Results

A minifeasibility trial was carried out at a local hospital under a consultant endocrinologist in Belfast. The short study ran for 2 weeks with six patients. The main objectives were to investigate the user interface, test the remote sending of data over a 3G network to a centralized server at the university, and record patient data for further proofing of the neural network. We also received useful feedback regarding the user interface and the feasibility of handing real-world patients a new mobile phone. Results of this short trial confirmed to a large degree that our approach (which also can be known as intensive insulinotherapy) has value and perhaps that our neural network approach has implications for future intelligent insulin pumps.

Conclusions

Currently, there is no software available to tell people with diabetes how much insulin to inject in accordance with their lifestyle and individual inputs, which leads to adjustments in software predictions on the amount of insulin to inject. We have taken initial steps to supplement the knowledge and skills of health care professionals in controlling insulin levels on a daily basis using a mobile device for people who are less able to manage their disease, especially children and young adults.

Introduction

The International Diabetes Federation¹ estimated that there were 30 million people with diabetes in 1985. It now affects approximately 194 million people worldwide.² In just 20 years, the number of people with diabetes increased over six times. The number of people who have diabetes is expected to more than double from its current number within the next 25 years. Beaglehole and Lefèbvre³ from the World Health Organization and International Diabetes Foundation have announced that “the world is facing a growing diabetes epidemic of potentially devastating proportions.”

Diabetes is a metabolic disorder. The body digests carbohydrates in food that has been eaten and moves the by-product (glucose) into the blood. From circulation of the blood, glucose can be used or stored by any part of the body. Glucose is the body's main energy source and is moved from the blood into cells by insulin. The amount of insulin in the body of an individual with diabetes is either in inefficient supply or nonexistent. Many people with diabetes are faced with having to inject insulin in order to live. If too much insulin is injected, there will not be enough glucose in the blood. If too little insulin is injected, there will be an oversupply. Both circumstances can have fatal consequences. It is very important that insulin-dependent people with diabetes inject a proper amount of insulin. As the amount of glucose in the blood is dynamic, the ideal amount of insulin to inject should not be static—it should match the rise and fall of glucose. If a dynamic amount of insulin is injected, how is a person with diabetes to know the amount to inject? As this is a metabolic problem and people have different metabolic rates, the answer cannot be a “one size fits all” solution. This current health problem requires an “intelligent” solution—a program that is able to learn about how each unit of insulin affects its current user.

We propose a solution in the form of an artificial neural network. We have knowledge of the concentration of glucose there should be in the blood. With a desired result and an ability to read the current amount of glucose, an error rate can be found. The error rate can be used to adjust the weights of inputs that affect the glucose level. This feedback and changing of weights can cause a neural network to learn and adapt itself to every user. The current method of guessing a dynamic amount of insulin to inject can then be replaced by this intelligent solution. The emergence of personal computers into our daily lives in the past 20 years means that this neural network program can be implemented for the first time in history. Even now, most people are not in front of a computer at all times of every day; however, within the past few years, mobile phones have become our minicomputers with ever-increasing memory and processing power. Because most people have mobile phones with them throughout the day, this is an ideal platform for something people will need at any time of the day.

A study done in 2006 by the American Diabetes Association said one in three Americans has diabetes or a precursor to the disease. Untreated, diabetes can be fatal or cause blindness and circulation problems. This therefore makes it difficult to “quantify” the value to a customer. However, a study by Feltbower and colleagues⁴found that if 1% of current patients under the age of 15 years with diabetes were to move onto insulin pump therapy, this would impose an additional cost of £400–1300 per year for each primary care trust. Therefore, the additional resources required to pay for a solution such as proposed here for a small proportion of the diabetes population would be minimal given the potential benefits to these patients of improved control and anticipated reduction in long-term morbidity.

Health care is the most information-intensive of industries and is evolving rapidly as new information supplants old information. The mobile phone may be the perfect platform for applications such as remote medical monitoring, home screening tests, “lite” telemedicine visits, medication management, patient education, and, specifically here, diabetes management. Already, a number of medical device manufacturers and mobile phone companies have joined forces to develop and pilot various kinds of applications using specially configured mobile phones, services, and Bluetooth-enabled physiological sensors such as blood glucose meters, cardiac monitors, sleep apnea detectors, exercise and dietary intake monitors, peak flow meters for asthma management, cuffless blood pressure monitors for hypertension screening, and medication management dispensers. All of this holds great promise for the system described here.

Diabetes

How does an insulin-dependent diabetic individual know when he/she injects insulin how much glucose will be in his/her blood later in the day? This is the major disadvantage insulin-dependent people with diabetes have—predicting the future. On paper, this would be best handled by eating the same amount of carbohydrates at the same time each day and exercising, which increases the absorption of glucose in the muscle cells, the same amount every day. However, in reality, this 7-day-a-week static routine can range from being very awkward to nearly impossible.⁵ The insulin-dependent diabetic individual has the following choices:

1. Live with this restrictive plan, eating the same number of carbohydrates and getting the same amount of exercise at the same time every day.

2. Follow a more normal, dynamic lifestyle while guessing the amount of insulin to have.

While the second choice is more ideal, it comes at a heavy price. The guess might be wrong. If too much insulin is injected, there will not be enough glucose in the body. This affects the brain first and foremost. Unlike muscles, the brain does not store glycogen. The brain is almost entirely dependent on glucose (in the blood) for normal functioning.⁶ If the amount of glucose in the blood is too low, the brain does not function as well. If the amount of glucose continues to drop beyond a certain level, a coma will result. If too little insulin is injected, there will be too much glucose in the blood, which results in every organ in the body being harmed over time. Let us suppose that a hypothetical, insulin-dependent diabetic individual follows the first choice. If he/she tested his/her glucose level in the morning and it was high, the best thing he/she could do is inject more short-term insulin to bring the high amount of glucose down. Let us suppose he/she usually sits at a desk at work every day, but has this day off work. If he/she wanted to hike in hills in the afternoon, he/she would either have to have less insulin before the activity or have to eat more carbohydrates in the afternoon to compensate for the exercise. These two situations—being in need of more and less insulin—can best be resolved by following a dynamic insulin regime.

Factors Affecting Insulin Need

Three main factors are involved in deciding how much insulin a person with diabetes would need to inject⁷: (1) current glucose level, (2) future amount of carbohydrates to be eaten, and (3) future amount of exercise. As the first two items rise, the amount of insulin should also rise. As the third item rises, the amount of insulin should decrease. The reverse is also true for all these factors. A few other minor factors, such as a disruption to the sleep pattern and a high stress level, can affect glucose levels. However, as the American Diabetes Association⁵ reports, because these factors can affect glucose levels in an unpredictable way, insulin cannot always be predicatively adjusted for these other factors.

Current Glucose Level

Ideally, any time the level of glucose in the blood is raised, there should be extra fast-acting insulin injected to counter the raised glucose—more glucose resulting in more insulin. This is, after all, how the body of a person without diabetes maintains the balance of glucose in the bloodstream. If, contrary to this, the amount of glucose is low, then less fast-acting insulin should be injected. The lower the level of glucose, the less insulin should be injected—as long as a “basal” amount of insulin is always present in the blood.

Future Amount of Carbohydrates To Be Eaten

The more carbohydrates a person eats, the more insulin should be injected to counteract the higher amount of glucose in the blood. Eating a meal with many carbohydrates results in proportionally more glucose in the blood. Eating a light meal, such as a salad, with little or no carbohydrates would put little or no glucose in the blood and would thus require little or no insulin to counteract such a meal. There is an additional factor at work in the digestion of carbohydrates—the glycemic index. This index accounts for how quickly carbohydrates are digested into glucose.⁸

For purposes of adjusting insulin levels, the glycemic index only needs to be taken into consideration if the person with diabetes intends to eat food with a very high or very low glycemic index shortly before the 4-hour period after injecting insulin. If food has a very high glycemic index and will be consumed 3.5 hours after injecting insulin, it might fall into the short-term insulin range; otherwise it might not be digested until after a longer term insulin range begins. Although some carbohydrates are digested more quickly than others, for our purposes of controlling blood glucose, a carbohydrate is a carbohydrate is a carbohydrate.^9,10

Future Amount of Exercise

Exercise reduces the amount of glucose in the bloodstream. Exercise helps transport glucose into working muscle cells, and muscle contraction has some of the same effects as insulin.¹¹ The more exercise that is taken, the more glucose is transported into the muscle cells. This causes the body to need less insulin to transport the same amount of glucose out of the bloodstream—a state of insulin sensitivity. Contrary to this, reduced exercise moves less glucose into muscle cells—similar to insulin resistance in people with type 2 diabetes. Other factors affect the amount of glucose in the blood. One of these is the changing metabolic rate of the body throughout the day. There is a whole field of study devoted to this called chronobiology.¹² Chronobiology has found “overwhelming” evidence that daily variations of human physiology (including heartbeat frequency, blood pressure, body temperature, renal activity, liver metabolism, and the secretion of many hormones), as well as the incidence and severity of many diseases, have an underlying circadian basis.¹³ This “circadian basis” is our body's 24-hour cycle. A meal eaten in the morning may need a different amount of immediate insulin than if it were eaten at night. This is something that can be observed in a neural network.

Related Work

The unique technological aspect of this project is to incorporate neural network-driven software, which is fronted by an intuitive user interface that strives to make the application simple and accessible for both the user and the clinician. Existing offerings do not offer the end-to-end package of Intelligent Neural Network for Suggesting Unambiguous Levels of Insulin via Needle (INNSULIN), which offers not only an intuitive user interface on a mobile phone, but also the communication of data to a central server where data are analyzed and presented to the user and clinician in a usable and user-friendly format. There are competitors but they are fragmented in terms of the end user profile their products address.

InsuCalc is a small cardboard calculator. Patients turn a disk representing the amount of carbohydrates about to be eaten in order to get a result of the amount of insulin to inject. InsuCalc uses the selling point that it is “nonelectronic, fits in a pocket”; however, the output is only for short-term insulin. Therefore, any person who injects long-term insulin would not get any benefit from this product for periods over 4 hours.

Another related product is the Diabetes Mentor from Mimosa Wireless Ltd. This application, however, is more of an intelligent tutoring system that mentors people with diabetes in an attempt to give them the ability to develop the necessary expertise to manage their intake. It contains a minimal knowledge of managing diabetes and has not yet moved beyond the prototype stage.

Online Tools

The vast majority of tools are online calculators of some sort. Most do not have applications targeted at the mobile phone market. For instance, the DiabetesNet Wizard tool assesses current insulin doses and gives suggestions on how they might be changed to improve control. Users enter the total units of insulin used each day along with control patterns. My Sugar Level is an online tracking and charting service for all people with diabetes. It is a visual management tool but no mobile phone application exists.

Personal Device Assistance Software

UTS Diabetes is Palm personal device assistance software that tracks the user's blood glucose level, insulin injections, carbohydrates consumed, medication intakes, and other diabetes statistics. Its purpose, however, is to serve as a traditional paper diabetic logbook. Diabetes Pilot is Palm software that helps users see trends in their blood sugars with various reports and graphs. It also categorizes records by time of day or any other system that the user requests. This again is more a diary and is not “active” in the sense of INNSULIN.

Insulin Pumps

An insulin pump is a small device about the same size as a pack of cards, which looks similar to a pager. Inside is a reservoir containing insulin, which is attached via a long piece of thin tubing. At the end of this tubing is a needle or cannula inserted under the skin, enabling insulin to flow into the body. The pump is not automatic, but is programmed to deliver insulin constantly at varying rates that the user determines; however, there are problems.

Because insulin pumps use the very fast-acting lispro insulin, which only lasts 3 to 4 hours after injection, diabetic ketoacidosis (DKA), a life-threatening condition caused by a lack of insulin, becomes a real possibility if the pump stops delivering the required amount of insulin. DKA could happen if (1) the pump tubing becomes blocked or comes out from under the skin or (2) the cartridge is allowed to run out of insulin.

We believe that no other company is utilizing neural network-based software for mobile phones in the control of diabetes insulin intake.

Neural Network

What is needed for a more accurate projection of insulin requirements is for exercise, current glucose level, intended carbohydrate intake, and period of the day to be used as inputs. There should be feedback based on any glucose testing within the next several hours so that the product can “learn” and adjust itself to the user. A patient with increased insulin sensitivity will require less insulin, even given the same inputs. A patient with increased insulin resistance will need more insulin. The insulin sensitivity/resistance will differ from person to person and with one person over time. Also, given the same inputs and insulin sensitivity/resistance, two insulin-dependent people with diabetes will also require different amounts of insulin based on their mass. For example, if one diabetic individual weighs 7 stone, another weighs 14 stone, and they both have the same percentage of glucose in their blood, the person who weighs twice as much will generally have twice as much glucose in the blood, which will result in generally needing twice as much insulin.

The neural network consists of four neurons. The first is glucose. If the user's current glucose level is within the target range, the glucose weight is then multiplied by zero. If the glucose level is high, then there will be a positive value multiplied to the weight, resulting in a positive amount of insulin to be injected. If the user's glucose level is low, then the weights will be multiplied by a negative value, resulting in a decrease in the overall insulin dose. The long-term “basal” dose will be removed completely from the network so that there will always be a guaranteed minimal amount of insulin present in the body, regardless of how low the glucose level falls. The weight will answer the question “how much insulin is required to lower the glucose level by 1 unit?” This way, for every unit the current glucose is above the target range, the adapting weight will output how much insulin is needed to move the glucose to the target range.

The second neuron is the carbohydrate neuron. The weight of this neuron is the amount of insulin required to keep the amount of glucose in the bloodstream at the same level for every carbohydrate eaten. If, for example, the user of the program was going to eat 10 carbohydrates, then this number would be multiplied by the carbohydrate weight to output the required amount of insulin needed to nullify the 10 carbohydrates from the bloodstream. The amount of insulin for the glucose output could be:

1. positive for a high glucose reading

2. zero for an “in the target zone” glucose level

3. negative if the current glucose level is low

The carbohydrate neuron can only be:

1. positive if the user is going to eat carbohydrates

2. zero if the user will not be eating any carbohydrates

After the glucose and carbohydrate outputs are computed, results are summed. A negative result is possible if the user has a low glucose level and will not eat many carbohydrates. If this is the case, then the final result of insulin to inject will automatically be zero units. A diabetic individual cannot, after all, inject a negative amount of insulin. It might be a good idea for the program in this case to recommend the user eat a calculated amount of carbohydrates in order to raise the glucose level to the target range.

The exercise neuron is the most challenging. After the glucose and carbohydrate outputs are summed, the result is divided by the number of minutes in the acting range of the short-term insulin. This gives the amount of insulin that will be working—on average—for each minute over the course of the range of the dose. Exercise intensities have been divided into 13 categories. The lowest number, which is 1, would be for “sitting around” type of activities, and the highest number would be for the most physically demanding activities. Exercise intensity is multiplied by the weight of the exercise. This solution would work if it was linear; however, the difference in the effect in glucose may not be—a straight line on a graph—linear.

To explain this using an oversimplified, hypothetical situation, let us suppose the weight for exercise is 1.0. A person is to be at activity level 1 for an hour. Then the person is going to participate in an activity at level 2 for an hour. Training at exercise level 2 would output exactly half of the insulin required than the time spent at level 1. If the solution is a linear one, then exercising at activity level 4 would require half the insulin again. Training at level 8 would require half the insulin than that of level 4. If the solution to exercise is a linear one, then this would work perfectly. Through thorough searching, no evidence was found to show how exercise should be computed.

Our solution, which caters to both linear and nonlinear weights, is to have 13 different weights. Whichever level of activity a person participates in will be multiplied by the weight of that level. During the update of weights, only the weights of those levels will be updated. A cleaner, single weight, quadratic equation can then be implemented after rigorous testing is complete on this setup. After the level is multiplied by the weight, the result is then multiplied by the duration. The output from this calculation is a percentage of total insulin to inject for the activity duration. The amount of insulin per minute is multiplied by the duration and then multiplied by the percentage. After then dividing by 100, the exercise portion of the neuron is complete.

The weight of the time neuron is an array of size 24—one for each hour. The array value for that time will be multiplied by the result after the exercise neuron. The value will be a double data type very near 1.0, which is the value all of the weights will start off with. This will cater to the metabolism of the body for different times of the day and will affect the overall insulin dose very slightly. The National Health Service¹⁴ recommends that the level of glucose in the blood of an individual with diabetes be “between 4 and 7 mmol/liter before meals and less than 10 mmol/liter 2 hours after meals.” This is the target for the neural network. We adopt an error-correction rule that, as Kecman¹⁵ states, “changes the weights proportional to the error e = d – o between the actual output o and the desired output d.” Every time a blood glucose test is taken, the result is entered into the neural network and is subtracted from the target (desired) output and the error will then change the weights (unless desired output equals target output).

INNSULIN

INNSULIN is a java application for mobile phones, as we believe that there would be very few people with diabetes who would sit in front of their personal computer every waking hour. However, programs can be written, stored, and run on mobile phones, and these phones are in the hands of a large majority of the population. The introduction screen for mobile phones is shown in Figure 1A.

Figure 1.

INNSULIN initial screenshots.

After pressing the “Start” button, the user arrives at the glucose screen shown in Figure 1B. To maneuver from the regular user interface (UI) to the help UI, the user only has to push the joystick to the right. To move back involves pushing the joystick to the left, as illustrated in Figure 1C.

The user will select their food through a list sourced from the U.S. Department of Agriculture¹⁶ of over 7000 foods with their nutritional values. Two thousand foods were removed from this list, such as pop-corn, salted, as there is no carbohydrate content in salt. We wanted the user to be able to look up this list to select their food because most users might not know the carbohydrate count of many foods, but it would be embedded for them to use—on the list (see Figure 1D). After selecting the food that will be eaten, the user then has to select how many servings they will eat, as shown in Figure 2A and Figure 2B. The “Options” screen is shown in Figure 2C. If the user cannot find the correct food they intend to eat, but knows the carbohydrate count of the food, they can select the “Input Carbs” option as illustrated in Figure 2D. The exercise component is broken down into 13 user-selectable categories. A list of many activities (from carpet sweeping to flying an airplane) and the amount of calories each of the activities expend is used. Activity level 1 (basically sitting around) is shown in Figure 3A. The next level of activity is shown in Figure 3B. Figure 3C and 3D show higher levels of exercise activity.

Figure 2.

Selecting servings.

Figure 3.

Exercise activity level screens.

After the activity is selected, the user must enter what time the activity will begin—starting at the current time, as shown in Figure 4a. After this, the user must select what time the activity will last, as illustrated in Figure 4B.

Figure 4.

Activity start and end times.

The network then calculates the correct dose and displays results to the user, as shown in Figure 4C. Figure 4D confirms in this case that all data are up to date. If, however, the program is in the “training” stage, the next screen will involve asking the user how much she/he intends to inject.

Evaluation

A minifeasibility trial was carried out at a local hospital under a consultant endocrinologist in Belfast. The short study ran for 2 weeks with six patients. The main objectives were to investigate the user interface, test the remote sending of data over a 3G network to a centralized server at the University of Ulster, and record patient data for further proofing of the neural network. We also received useful feedback regarding the user interface and the feasibility of handing real-world patients a new mobile phone.

Results of this short trial confirmed to a large degree that our approach (which also can be known as intensive insulinotherapy) has value and that perhaps our neural network approach has implications for future intelligent insulin pumps. Indeed, lately, the United Kingdom Prospective Diabetes Study demonstrated that intensive insulinotherapy has a protective value in ameliorating frequency and severity of macro- and microvascular complications of diabetes mellitus. The method of intensive insulinotherapy is based on multiple injections of rapid-acting insulin before each meal and a once-daily dose of long-acting insulin as a baseline dose. Close monitoring of blood glucose with several times a day performed tests is indispensable. Our system provides all the inputs necessary to assist those who are following this approach.

A key feature of our solution is the ability to store large amounts of past glucose results. The field of diabetes care is arguably the most chronically data-intensive discipline in medicine. Our current system has worked well in a preliminary trial with Ulster Hospital and allows results to be sent in real time from a phone to a secure Web site where the consultant endocrinologist can view the insulin levels of each patient. This resulted in the most favorable feedback to date from those involved, as previously they relied on feedback from the patients, which obviously had to be taken on trust.

We have had initial discussions with companies working in the field of software development for the health care market, with one in particular declaring that they have development plans for a similar product, but without our unique feature of using a neural network. The office of innovation at the university also commissioned a preliminary report from external consultants and revealed some positive findings for the INNSULIN concept.

Future Work

There is a drive to use online technology to expand patient knowledge and support patient–physician communication to improve diabetes management and treatment outcomes. Stanley and Malakar¹⁷ have shown that monitoring glucose levels from patients' homes resulted in participants reporting high levels of satisfaction and were more likely to monitor their blood glucose levels regularly. They were also more knowledgeable about their condition and more confident in their ability to control their blood glucose.

This project will integrate home-based glucose monitoring results securely and easily within the clinical workflow of diabetes care. Glucometers and blood pressure devices linked to smartphones and iPhones will transmit personal health data into our secure clinical server side diabetes management application. Currently, our system can deal with at least two kinds of insulin inputs (see Figure 5). This is because patients use two kinds of insulin together: one for background and one for the meal. To calculate the dosage of the insulin, we use a neural network.

Figure 5.

Current system model.

A number of key areas need to be focused on to allow us to reach the commercialization stage:

1. Accumulate longer term insulin data from clinical data sets to train the neural network.

2. Add fuzzy logic to the system to ensure that predictions are accurate.

3. Add a “startup” mathematical formula to product a general result and then use the neural network to fix the mistake, which will raise the accuracy of the neural network.

4. Conduct more visits to hospitals and clinics. Speak to doctors and patients to ascertain what system is most needed and how a system such as this can be developed to be useful and convenient as a real-time monitoring tool for both patient and clinician.

5. Continue testing of current interface.

6. Build a kind of program called “diabetes diary”; this kind of software is only used to collect information for training the neural network.

7. Use MATLAB to find a proper mathematic model and the structure of the neural network from data and then implement it.

8. Build another demo (see Figure 6) and add the calculation function.

9. Loop: test, update program, and support different kinds of mobile phones, including the iPhone.

Figure 6.

Planned system model.

Figure 7.

Outline of neural network. CH, carbohydrate.

Our unique application of a modified Levenberg– Marquardt method to train the neural network and the integration of wireless glucometers in the system are especially worth investigating further. A more detailed outline of the neural network is outlined in Appendix A.

Conclusion

The prediction is that application development for mobile devices will explode, with high growth in social, recreational, and educational applications. The mobile market offers considerable potential for educational applications with an emergent trend in mobile learning for supporting health and leisure activities. There is growing governmental awareness of the need for patients with chronic or long-term health conditions to become skilled at managing their own condition instead of relying on the intervention of health care professionals. Diabetes is a critical illness that is being diagnosed throughout society at a very large rate. It is a result of the body not being able to transport digested carbohydrates to where they can be used or stored. For many people with diabetes, the only way to continue living with this illness is to inject insulin. Currently, there is no software that tells people with diabetes how much insulin to inject in accordance with their lifestyle and individual inputs, which lead to adjustments in software preductions on an amount of insulin to inject. INNSULIN attempts to supplement the knowledge and skills of health care professionals in controlling insulin levels on a daily basis using a mobile device for people who are less able to manage their disease, especially children and young adults.

Abbreviations

DKA

diabetic ketoacidosis

INNSULIN

An Intelligent Neural Network for Suggesting Unambiguous Levels of Insulin via Needle

UI

user interface

Appendix A

(1) Hard limit or step function:

$$y=f\left(net\right)=\{\begin{array}{r}\hfill +1\text{\hspace{0.17em}if\hspace{0.17em}}net\ge 0\\ \hfill 0\text{\hspace{0.17em}if\hspace{0.17em}}net<0\end{array}$$

(2) Symmetric hard limit function:

$$y=f\left(net\right)=\{\begin{array}{r}\hfill +1\text{\hspace{0.17em}if\hspace{0.17em}}net\ge 0\\ \hfill -1\text{\hspace{0.17em}if\hspace{0.17em}}net<0\end{array}$$

(3) Linear function:

$$y=f\left(net\right)=k·net$$

where k is a constant.

(4) Symmetric saturating linear or ramp function:

$$y=f\left(net\right)=\{\begin{array}{ll}\text{max}\hfill & \text{if\hspace{0.17em}}net>\text{\hspace{0.17em}upper\hspace{0.17em}limit}\hfill \\ k·net\hfill & \text{if\hspace{0.17em}upper\hspace{0.17em}limit\hspace{0.17em}}>net>\text{\hspace{0.17em}lower\hspace{0.17em}limit}\hfill \\ \text{min}\hfill & \text{if\hspace{0.17em}}net<\text{\hspace{0.17em}lower\hspace{0.17em}limit}\hfill \end{array}$$

(5) Sigmoid function

$$y=f\left(net\right)\frac{1}{1+e^{-net}}$$

(6) Hyperbolic tangent or tan–sigmoid function

$$y=f\left(net\right)\frac{2}{1+e^{-net}}-1=\frac{1-e^{-net}}{1+e^{-net}}$$