A Survey on the State-of-the-art Multi-Task Learning Models: Techniques, Open Problems and Future Directions

Abstract

Multi-Task Learning (MTL) is a type of machine learning technique where a model is trained to perform multiple 
tasks simultaneously. In deep learning, MTL refers to training a neural network to perform multiple tasks by sharing 
some of the network’s layers and parameters across tasks. In MTL, the goal is to improve the generalization 
performance of the model by leveraging the information shared across tasks. By sharing some of the network’s 
parameters, the model can learn a more efficient and compact representation of the data, which can be beneficial 
when the tasks are related or have some commonalities. MTL can be useful in many applications such as natural 
language processing, computer vision, and healthcare, where multiple tasks are related or have some commonalities. 
Despite the recent progress in deep learning, most approaches still go for a silo-like solution, focusing on learning 
each task in isolation: training a separate neural network for each individual task. Many real-world problems, 
however, call for a multi-modal approach and, therefore, for multi-tasking models. Multi-task learning (MTL) aims 
to leverage useful information across tasks to improve the generalization capability of a model. Hence, a detailed 
survey on the summarization on MTL model will be carried out in this abstract. In this survey, we provide a well
rounded view on state-of-the-art MTL techniques within the context of deep learning. The contributions of the MTL 
model concern the following. First the MTL from a network architecture point-of-view will be considered. The 
comprehensive overview as well as examine the pros and cons of the recent familiar MTL techniques will be included. 
A variety of optimization schemes to undertake the joint learning of many tasks will be examined. The qualitative 
constituents of the works, their commonalities and the dissimilarities will be explored. Finally, a huge investigational 
assessment over various databases will be provided to examine the advantages and disadvantages of different models, 
including both architectural as well as optimization strategies.