Modality refers to how a particular subject is experienced or represented. Our experience of the world is multimodal—we see, feel, hear, smell and taste things. Multimodal deep learning tries to link and extract information from data of different modalities. Just as the human brain processes signals from all senses at once, a multimodal deep learning model extracts relevant information from different types of data in one go.
(more…)
