Machine learning with human in the loop

One of the biggest issues with machine learning is that it is often very easy to get an algorithm to 80 percent accuracy but nearly impossible to get an algorithm to 99 percent accuracy. The “human in the loop” computing solves this issue by using the human judgment, to be fed back into the algorithm to make it smarter. In this section, we will describe how the “human in the loop” affects the performance of a machine learning algorithm, as well as mention some applications where this aforementioned computing method has been used. We will also relate this computing method to our Pref-CBR framework.

6.2.2.1 Human in the loop

There are several ways of including a human in the loop in machine learning algorithms. One way is to use a human as a means for information extraction, where users are allowed to specify the nature of the information structures they desire. As explained by [130], the strengths of humans and machines can be combined in the following way:

6.2 Machine learning

the human is proficient at judging an information structure as desirable or undesirable, while the machine is proficient at quickly and efficiently locating similar examples from large quantities of data. Another way of using a human in the loop is to apply programming by feedback, which involves a sequence of interactions between the active computer and the user, in which the user provides preference judgments on pairs of solutions supplied by the active computer [131]. Another approach includes having a human user going one step further and not only provide feedback about past actions, but also provide future directed rewards to guide subsequent actions [132]. Users can be included in an interactive machine learning model for classification, which allows users to train, classify/view and correct the classifications [133, 134].

Another view of including a human in the loop is suggested by [135]. They propose to change the limitations of present day technology, by engaging machines implicitly and indirectly in a world of humans; computers would be put in the human interaction loop, rather than the other way around. Multiple audio-video sensors can be attached in what is called Computers in the Human Interaction Loop rooms, to “observe” humans and can then be analyzed. As explained by [135], the analysis of all audio-video signals in the environment (speech, faces, signs, bodies, gestures, objects, attitudes, events and situations) provide answers, which allow computers to engage and interact with humans in a human-like manner.

6.2.2.2 Applications of human in the loop in machine learning

Many safety-critical systems are interactive, they interact with a human being, and the human operator’s role is central to the correct working of the system [136]. Examples of such interactive systems (human-in-the-loop control systems) include fly-by-wire aircraft control systems (interacting with a pilot), automobiles with driver assistance systems (interacting with a driver), and medical devices (interacting with a doctor, nurse, or patient). Another type of interactive systems is presented by [132], which integrates machine learning and human-robot interaction. In the latter system, the reinforcement learning algorithm benefits from the human-robot interaction and learns how humans teach the robots. Accordingly the learning algorithm modifies the action selection mechanism, and there is a significant improvement in the learning performance of the agent when the robot is tested later in a second study. Learning from humans is also applied in assistance systems (e.g. email categorizing, conference planning),

6. RELATED METHODOLOGIES

where humans are in the loop for both the learning and evaluation steps [137]. These assistance systems consist of multiple machine learning components, natural language processing and optimization techniques.

A human in the loop approach has been also used for image characterization for medical images. In this image characterization approach, an expert radiologist in each anatomic region, selects images for the database and provides differential diagnosis and includes treatment information. This information can be useful to a less experienced practitioner, enabling him/her to use the stored expertise, and provide the role of an expert consultant if confronted with a similar image [138]. [134] apply the human in the loop in visual recognition of images, by asking users some questions and accordingly classify correctly the object in the image. In the work of the latter authors, they show that this interactive, hybrid human-computer method for object classification, drives up recognition accuracy to levels that are not only acceptable, but can be considered good enough for practical applications. Machine learning in interactive settings is also proposed by [133], where machine learning and computer vision techniques are applied for image classification. The goal is to have a human user upon receiving an image, to do some manual classification, thus training a classifier. The user can then later refine the classifier by adding more manual classification, if the classifier is still not satisfactory. The proposed method by [133] replaces the analysis of many feature combinations by the machine learning algorithm, especially if there are many features, by an interactive machine-learning model that allows users to train, classify/view and correct the classifications.

It is clear that having a human in the loop can aid the learning in machine learning algorithms. The benefits of having a human in the loop has been stated; whether the feedback is given by the user in an interactive manner, continuously updating the learning process, or whether the feedback of the user is during testing of the algorithm for improving the learning. In our Pref-CBR framework, our oracle, is the human in the loop component of our algorithm. Although this oracle does not necessarily represent a human, it represents an expert of some form. It can alternatively be a human or some expert program, which in turn also represents expertise knowledge. We can conclude that the human in the loop can be a special case in our Pref-CBR framework, where our framework is more generic and can operate also without necessarily having a human in the loop. Our image correction application, described in detail in the proceeding

6.2 Machine learning

chapter shows a nice example of a human in the loop integrated in our Pref-CBR search process.