Different types of learning have been popping up as we approach a more artificial general intelligence. Artificial General intelligene (AGI) aims to achieve computers that are capable of performing tasks humans can.

Transfer Learning

Transfer learning refers to using the weights learned from a given problem to solve a different related problem. For example, a neural network can be trained using the ImageNet dataset as a starting point. It is then retrained to work on the CUBS dataset. Transfer Learning helps speed up the learning process and obtain higher accuracy for the new dataset by using the pre-trained model instead of training from scratch.

Few-Shot Learning

One shot learning refers to learning from a limited number of examples per class. More particularly learning using only one example per class is called one-shot learning.

Meta-learning

Meta learning is learning how to learn the learning process. Most neural networks learn the parameters to perform specific task, Meta-learning refers to learning the ability to learn tasks such that when a given unknown task is shown it can perform it properly with little training. Few-shot learning is a good application of the meta-learning concept.

Incremental Learning

Incremental Learning refers to starting with pre-trained model and adding new tasks while retaining its ability to perform previous task. This is in contrst with transfer learning in which the network loses its ability to perform previous task because of catastrophic forgetting.

Multitask Learning

Multitask Learning refers to a type of learning in which learning from a given task can help improve learning on another task. This is similar to transfer learning in this sense, however the tasks are learned in parallel but not consecutively. This can also be confused with multilabel classification in which multiple labels are assigned per image. In the case of multitask learning a single image can have multiple labels, i.e. a picture with a car, stop sign and a pedestrian.

Continual learning

Continual learning (CL) is a type of sequential learning that aims to extend its abilities to new tasks as they come while retaining its ability to perform previous tasks. This ability is important for continuously changing environments and is an import aspect of AGI. One of the core issues of continual learning is to address the issue of catastrophic forgetting. This has been used interchangeably with lifelong learning.

CL is a type of online learning in which learning occurs at the presence pf a task. There is no set boundaries between task and the tasks can continuously grow as time progresses. The system must be able to use its previous experience to generalize and improve its capabilities with the presence of new information but must be resilient with catastrophic forgetting in which new information can be detrimental to the system’s performance with previous tasks. CL is also challenged by limited resources such as memory, and the system hence must use its resources, learning what information to retain and what to forget in the process.