Home > Research > Publications & Outputs > Multimodal analysis and prediction of latent us...

Links

Text available via DOI:

View graph of relations

Multimodal analysis and prediction of latent user dimensions

Research output: Contribution in Book/Report/Proceedings - With ISBN/ISSNConference contribution/Paperpeer-review

Published
Close
Publication date15/09/2017
Host publicationSocial Informatics - 9th International Conference, SocInfo 2017, Proceedings
EditorsGiovanni Luca Ciampaglia, Taha Yasseri, Afra Mashhadi
PublisherSpringer-Verlag
Pages323-340
Number of pages18
ISBN (Print)9783319672168
<mark>Original language</mark>English
Event9th International Conference on Social Informatics, SocInfo 2017 - Oxford, United Kingdom
Duration: 13/09/201715/09/2017

Conference

Conference9th International Conference on Social Informatics, SocInfo 2017
Country/TerritoryUnited Kingdom
CityOxford
Period13/09/1715/09/17

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10539 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference9th International Conference on Social Informatics, SocInfo 2017
Country/TerritoryUnited Kingdom
CityOxford
Period13/09/1715/09/17

Abstract

Humans upload over 1.8 billion digital images to the internet each day, yet the relationship between the images that a person shares with others and his/her psychological characteristics remains poorly understood. In the current research, we analyze the relationship between images, captions, and the latent demographic/psychological dimensions of personality and gender. We consider a wide range of automatically extracted visual and textual features of images/captions that are shared by a large sample of individuals Using correlational methods, we identify several visual and textual properties that show strong relationships with individual differences between participants. Additionally, we explore the task of predicting user attributes using a multimodal approach that simultaneously leverages images and their captions. Results from these experiments suggest that images alone have significant predictive power and, additionally, multimodal methods outperform both visual features and textual features in isolation when attempting to predict individual differences.