Lecture
simultaneous training on a group of interrelated tasks, each of which has its own «situation, required solution» pairs
Multi-Task Learning is a machine learning method that is the opposite of single-task learning. In machine learning, the standard theory of algorithms consists in studying one task at a time, that is, in the case where the output of the system is a real number. A complex learning problem is first decomposed into theoretically independent subtasks, then each subtask is studied separately, and finally, the mathematical model of the complex task is established by combining the results of learning the subtasks. Multi-task learning is a type of joint learning in which several tasks are studied in parallel, and the results influence each other.
Let's make a simple comparison with the school data that everyone often uses. The school data is a dataset used to predict the regression problem of student performance. A total of 15,362 students study in 139 secondary schools, and each secondary school can be viewed as a prediction task. Single-task learning consists in ignoring the possible relationships between tasks and learning 139 regression functions to predict grades, or directly combining all the data of the 139 schools to learn a single regression function for prediction. Multi-task learning emphasizes the relationship between tasks. Through joint learning, different regression functions are learned simultaneously for the 139 tasks, which takes into account both the differences between the tasks and the relationship between the tasks. This is also one of the most important ideas of multi-task learning.
In machine learning, we usually care about optimizing a specific indicator, regardless of whether that indicator is a standard metric or a corporate KPI. To achieve this goal, we train a single model or a set of several models to perform the specified tasks. Then we improve the model by tuning the parameters until performance stops improving. Although this may provide acceptable performance for the task, we may miss some information that would help improve the metrics that matter to us. Specifically, this information consists of observation data for related tasks. By sharing representation information among related tasks, our model achieves better generalization performance on the original task. This method is called multi-task learning, and it is the main topic of this blog post.
There are many forms of multi-task learning, such as joint learning, learning to learn, learning with auxiliary tasks, and so on. These are just some of the aliases. So, as soon as you find that you are optimizing more than one objective function, you can effectively solve it with multi-task learning (in general, as soon as you find that you are optimizing more than one loss function, you are effectively doing multi-task learning (as opposed to single-task learning)). In such a scenario, it helps to understand what we are actually going to do, and to draw inspiration from it.
Even if your optimization goal is only one in a very special situation, an auxiliary task can help you improve the learning efficiency of the main task. Rich Caruana concluded: «Multi-task learning improves generalization performance by leveraging the domain knowledge contained in the training signals of related tasks». In studying this blog post, we will try to give a brief overview of the current state of research on multi-task learning, especially multi-task learning for deep neural networks. First, we examined the sources of inspiration for multi-task learning. Then we present the two most common methods of multi-task learning. Then we describe the mechanism that makes multi-task learning effective in practice. Before summarizing more advanced neural-network-based multi-task learning methods, we reviewed some of the basic knowledge from earlier literature on multi-task learning. Then, this article presents some powerful multi-task learning methods based on deep neural networks that have been proposed in recent years. Finally, we discussed the types of commonly used auxiliary tasks and the characteristics of auxiliary tasks that are suitable for multi-task learning.
Early research on multi-task learning arose from studying an important machine learning problem, namely the problem of «inductive bias». The machine learning process can be viewed as a process of analyzing empirical data related to a problem and generalizing from it a model that reflects the nature of the problem. The function of inductive bias is to guide the learning algorithm in how to search the model space. Inductive bias directly affects the performance of the model being searched for, and any learning system that lacks inductive bias cannot be effective. Learning. Different learning algorithms (for example, decision trees, neural networks, support vector machines, etc.) have different inductive biases. People need to manually decide which learning algorithm to use when solving practical problems. In fact, they choose different algorithms subjectively. Inductive bias strategy. It is very intuitive to ask whether the process of determining the inductive bias can also be completed automatically during learning, that is, adopting the idea of «learning to learn». Multi-task learning provides a real way of implementing the above idea, that is, using the useful information contained in related tasks to provide a stronger inductive bias for learning the corresponding task.
Current multi-task learning methods can be roughly divided into two categories: one uses the same shared parameters for different tasks , while the other searches for hidden functions that are hidden between different tasks .
We propose to diversify the starting points of multi-task learning:
(1) From a biological point of view, we regard multi-task learning as a simulation of human learning. To learn a new task, we usually use knowledge gained from tasks related to the training. For example, infants first learn to recognize faces, and then use this knowledge to recognize other objects.
(2) From a pedagogical point of view, the first task we learn is those skills that can help us master more complex techniques. This is a very correct method for teaching martial arts and programming. As an example not usually appreciated by the public, in the movie «The Karate Kid», Mr. Miyagi teaches the karate kid to polish the floor and wax the car, which suggests that this doesn't matter. However, it turned out that it was precisely these trivial chores that gave him the necessary skills to learn karate.
(3) From a machine learning point of view, we regard multi-task learning as inductive transfer. Inductive transfer improves the model by introducing an inductive bias, making the model more inclined toward certain assumptions. For example, a common inductive bias is L1 regularization, which makes the model more inclined toward sparse solutions. In multi-task learning scenarios, the inductive bias is provided by the auxiliary tasks, which makes the model more inclined toward solutions that can explain several tasks at once. Next we will see that this improves the generalization characteristics of the model.
Earlier we discussed the theoretical source of multi-task learning. To make the idea of multi-task learning more concrete, we will show two methods commonly used in multi-task learning based on deep neural networks: hard sharing and soft sharing of hidden layer parameters.
(1) Hard-sharing mechanism of parameters: The hard parameter-sharing mechanism is the most common method of multi-task learning for neural networks, which can be found in the literature. Generally speaking, it can be applied to all the hidden layers of all tasks, while keeping the task-specific output layer. The hard-sharing mechanism reduces the risk of overfitting. In fact, the literature proves that the order of the overfitting risk of these shared parameters is N, where N is the number of tasks, which is smaller than the overfitting risk of the task-specific parameters. Intuitively, this is very important. The more tasks we learn at the same time, the more tasks our model can cover with the same representation, which reduces the risk of overfitting our original tasks.
(2) Soft parameter-sharing mechanism: each task has its own model and its own parameters. We regularize the distance between the model parameters to ensure that the parameters are similar. Some works use L2 distance regularization, while others use the trace norm. The constraints of the soft-sharing mechanism used in deep neural networks are strongly influenced by regularization methods from traditional multi-task learning. Let's discuss this further.
Although the reduced-bias explanation for multi-task learning is very popular, to better understand multi-task learning, we need to study its underlying mechanism. Most of these mechanisms were proposed by Caruana back in 1998. To simplify the description of the distance, let's assume there are two related tasks A and B, which share a hidden-layer representation F.
(1) Implicit data augmentation mechanism. Multi-task learning effectively increases the number of training examples. Since all tasks have more or less noise, for example, when we train a model on task A, our goal is to obtain a good representation of task A while ignoring the noise associated with the data and the generalization performance. Since different tasks have different noise patterns, a model that learns two tasks at the same time can learn a more general representation.). Whereas learning task A alone carries the risk of overfitting to task A, learning task A and task B at the same time averages out the noise pattern, so the model can obtain a better representation F.
(2) Attention mechanism. If a task is noisy, the amount of data is small, and the dimensionality of the data is high, it becomes difficult for the model to distinguish relevant from irrelevant features. Multi-tasking helps focus the model's attention on the features that are really important, since the other tasks can provide additional evidence of the correlation and non-correlation of features.
(3) Eavesdropping mechanism. Some feature G is easy to learn for task B, but this feature is hard to learn for task A. This may be because the interaction between task A and feature G is more complex, or because other features hinder the learning of feature G. Through multi-task learning, we can let the model eavesdrop, that is, use task B to learn feature G. The simplest way to achieve this is to use hints, that is, to train the model to directly predict which features are the most important.
(4) Representation bias mechanism. Multi-task learning is more inclined to learn the type of model that emphasizes the part of the representation that is also emphasized by other tasks. Since a hypothesis space that works well for a sufficient number of training tasks will also work well for new tasks from the same environment, this helps the model demonstrate its ability to generalize to new tasks.
(5) Regularization mechanism. Multi-task learning plays the same role as regularization by introducing an inductive bias. In this way, it reduces the risk of overfitting the model, while at the same time reducing its Rademacher complexity, that is, its ability to fit random noise.
Although many recent works on deep learning explicitly or implicitly use multi-task learning as part of their model, the method used still doesn't go beyond the two methods we mentioned earlier: hard parameter sharing and soft parameter sharing. In contrast, only a small part of the work is devoted to proposing good multi-task learning mechanisms in deep neural networks.
In multi-task scenarios for computer vision, these existing methods usually use shared convolutional layers and treat the fully connected layer as task-specific. Reference [34] proposes a deep relationship network. In addition to the shared-layer and task-specific-layer structure, they add a matrix before the fully connected layer. This allows the model to learn the relationships between tasks. This is similar to the Bayesian method we saw earlier. However, the problem is that this method still relies on a predefined shared structure. This is enough for computer vision problems, but for new tasks it may turn out to be wrong.
On the other hand, reference [35] proposes a bottom-up approach. Starting from a thin network, using indicators that automatically group similar tasks, it greedily and dynamically expands the network. This expansion process dynamically creates branches, as shown in Figure 4. However, this greedy approach doesn't yield the overall best result. Assigning a specific task to each branch prevents the model from learning more complex interactions between tasks.
Reference [36] connects two independent networks with soft parameter sharing. They then described how to use a so-called cross-stitch module to determine how to use knowledge gained from other tasks in the networks associated with those tasks, and linearly combine the outputs of the previous layer. This structure is shown in Figure 5, and the cross-stitch module is added only after the pooling layer and the fully connected layer.
In contrast, in recent years the focus of multi-task learning in the field of natural language processing has been on finding a good hierarchical structure: reference [37] shows that some basic tasks in NLP, such as part-of-speech tagging, named entity recognition, and so on, should be used as an auxiliary task, with supervised learning performed at a lower level.
Based on this discovery, [38] predefines a hierarchical structure containing several NLP tasks, as shown in Figure 6, and uses it as a joint model for multi-task learning.
In addition to splitting the learning structure, [39] uses an orthogonal method to consider the uncertainty of each task. They adjust the relative weight of each task in the cost function and achieve the goal of multi-task learning based on the principle of maximizing the probability function of task-related uncertainty. Figure 7 shows the structure of three tasks: depth regression, semantic segmentation, and instance segmentation for each pixel u.
5.7 Tensor Decomposition in Multi-Task Learning
Many recent works have attempted to extend the existing multi-task learning model to deep learning: [40] extends some of the existing tensor decomposition methods to model parameter splitting, in order to decompose the correlation coefficients between the shared parameters of each layer and the task. .
Finally, let us talk about the sluice network mentioned in [41], which is a generalization of many multi-task learning methods based on deep neural networks. As shown in Figure 8, this model can learn which subspaces in each layer should be shared, and which are used to learn a good representation of the input sequence.
Having reviewed this related work, let us summarize what information should be shared in a deep multi-task learning model. In most multi-task learning cases, the tasks come from the same distribution. Although this scenario is favorable for sharing, it is not always true. To develop a more robust multi-task model, we must deal with unrelated tasks.
Early multi-task models used for deep learning required a predefined shared structure between tasks. This strategy does not scale well and depends heavily on the multi-task structure. The hard parameter-sharing technique, which was proposed as far back as 1997, still remains mainstream today, 20 years later. Although the hard parameter-sharing mechanism is useful in many scenarios, if the relationship between tasks is not close enough or multi-level reasoning is required, the hard-sharing technique will soon break down. Recently, some research has also been carried out to learn what can be shared, and the performance of this work is generally better than the hard-sharing mechanism in a general sense. In addition, if the model is known, it is also useful to learn the capacity of the task hierarchy, especially in scenarios with multiple levels of granularity.
As mentioned at the beginning, as soon as we need to solve a multi-objective optimization problem, we turn to multi-task learning. Multi-tasking should not only be limited to expressing knowledge about all tasks in the form of the same parameter space, but should also pay more attention to how to make our model learn the interactive mode between tasks (thus, it is useful to apply the advances in MTL that we discussed, and let our model learn how tasks should interact with each other).
For scenarios in which the prediction results of several tasks are obtained simultaneously, multi-task learning is naturally suitable. This scenario is common in financial or economic forecasting. For example, we may need to know both the relevant influencing factors and the forecast results. In bioinformatics, we may want to learn the symptoms of several diseases at once. But in most cases we focus on only one task. In this section we discuss how to find an auxiliary task that will benefit multi-task learning.
Using related tasks as auxiliary tasks is a typical choice for multi-task learning. If you want to know what a «relevant task» is, here we show a few intuitive examples. Caruana used the prediction of characteristics of various roads in 1997 to learn to control autonomous driving. [42] uses head pose estimation and facial attribute inference to facilitate the task of facial landmark detection. [43] studies both query classification and web search. [44] simultaneously predicts the category and location of objects in an image. [45] also predicts the duration and frequency of phonemes in the text-to-speech process.
Typically, annotated data for a related task is unavailable. However, in some cases the tasks we can use are the opposite of what we want to achieve . Such data can be used to work against the loss. This loss is used not for minimization, but to apply a gradient-reversal layer to maximize the training error . [46] shows a successful example of this scenario in domain adaptation. The adversarial task in this scenario is used to predict the input field. By reversing the gradient of the adversarial task, the loss of the adversarial task is increased to a maximum. This is useful for the main task and can encourage the model to learn a representation of the two domains without distinguishing between them.
As mentioned earlier, multi-task learning makes it possible to learn features that cannot be learned when performing a single task. Using hints is one such effective mechanism: predicting features in auxiliary tasks. A recent example is natural language processing. In [47], the question of whether the input sentence contains words with positive and negative sentiment is used as an auxiliary task in sentiment analysis. [48] judges whether a sentence contains a name, as an auxiliary task in misspelled-name recognition.
Auxiliary tasks can be used to focus attention on specific parts of an image that the network might otherwise ignore. For example, for steering-direction-supervised tasks, a single-task model usually ignores fine details in these images, such as road signs. Predicting road signs can then be used as an auxiliary task. By making the model learn to depict them, such knowledge can be used for the main task. Similarly, for face recognition, since faces vary, we can use the prediction of facial landmark positions as an auxiliary task.
For multi-tasking, the optimization objective is quantized. The continuous type is usually preferred, but the available labels form discrete sets. In most cases, manual labeling is required for data collection, for example, for disease-risk prediction or sentiment analysis (positive, negative, neutral). Since the objective function is smooth, using a less quantized auxiliary task will make the learning easier.
In some cases, using certain features as input does not help predict the target output. However, they can guide the supervised training process. In these situations, the feature is part of the output rather than the input. [49] shows scenarios for solving these problems in practical applications.
In many scenarios, some features are only available after the fact, i.e., after the prediction has been made. For example, in autonomous driving, when a car passes obstacles or road signs, they can be measured precisely afterward. Caruana gave the example of pneumonia in 1997. Additional diagnostic cases only become available after the incident. For these examples, this additional data is unavailable at input time, so it cannot be used as a feature. However, it can be used as an auxiliary task to give the model additional knowledge to help with training.
The goal of auxiliary tasks in multi-task learning is to make the model learn a shared representation to help learn the main task. The auxiliary tasks we have discussed so far do this implicitly. Because they are closely related to the main task, they can allow these models to learn representations that benefit the main task during training. A more explicit approach is to use an auxiliary task specifically to learn a transferable representation. This role was played by the work of Cheng et al. 2015 and by the language-model objective used in [50]. Similarly, an autoencoder can also be used for auxiliary tasks.
Although in practice we may only care about one auxiliary task, we have discussed various auxiliary tasks that can be used in multi-task learning. Although we do not know what will work in practice, the basic assumption for finding auxiliary tasks is: auxiliary tasks should be closely related to the main task, or be useful for the training process of the main task.
However, we do not know which two tasks are related or similar. In 1997, Caruana gave the following definition: if two tasks use the same features to make decisions, then the two tasks are similar. In 2000, Baxter added: theoretically related tasks have the same optimal hypothesis class, i.e., the same inductive bias. [50] proposes that if the data in two tasks is generated by the same type of transformation F to obtain a fixed probability distribution, then the two tasks are F-related. Although this can be used for the same classification task, it cannot be used for tasks involving different problems. Xue et al. proposed in 2007 that if the classification boundary (parameter vector) of two tasks is close, then the two tasks are similar.
Although some progress has been made at an early stage in understanding the theoretical definition of task relevance, recent results have not. Task similarity is not binary but graded. Two more similar tasks derive more benefit from multi-task learning, and vice versa. This allows our model to learn which parameters to use, which can only temporarily overcome the theoretical shortcomings and make better use of tasks that are unrelated to each other . However, we also need a theoretical understanding of task similarity, which will help us understand how to choose auxiliary tasks.
[52] found that auxiliary tasks with a full and uniform label distribution should be more useful for the main sequence-labeling task , which was verified experimentally. In addition, [53] found that auxiliary tasks without plateaus also improve main tasks that plateau.
However, the scale of these experiments is limited. These recent research results provide only some clues for our further understanding of multi-task learning in neural networks.
This article is devoted to the history of multi-task learning and the latest developments in multi-task learning in deep neural networks. Although multi-task learning is widely used, the hard parameter-sharing mechanism of the last 20 years still remains the main paradigm of multi-task learning in neural networks. Work on learning what information to share looks more promising. At the same time, our understanding of task similarity, the relationships between tasks, task hierarchy, and the benefits of multi-task learning remains limited. We need to learn more in order to understand the generalization capabilities of multi-task learning in deep neural networks.
Comments