Our research establishes the value of infusing expertise (domain knowledge, perceptual/visual behaviors) into the design of computing systems for complex tasks. It is motivated by recognition that design of effective computing systems requires understanding of the relationships between people and technologies. Such interdisciplinary research builds upon and feedbacks the fields of image understanding, multimodal data fusion, multimedia information retrieval, and human-centered computing. The primary objective is to strike the appropriate balance between human and machine capabilities and to integrate them in such a way that the system naturally supports the human end user.

By modeling and fusing data derived from domain experts, the human cognitive processing can be brought into the computational processes, which integrates human intelligence and machine intelligence that is essential for complex visual problem solving. The research will also inform a human-centered image retrieval (HCIR) system to allow natural semantic-level exploration of images that are organized according to many different dimensions of expertise-based processing, as opposed to highly structured, rigidly annotated relationship databases. The computational models and the fusion framework will provide novel ways to objectively discover latent conceptual/perceptual behaviors and their relationship that are related to different levels of expertise and domain knowledge.

We also focus on research towards intelligent human-computer interaction---using computers to enhance human performance involving complex cognitive activities, for example during visual reasoning and decision-making. Intelligent human-computer interaction combines human cognitive, physical, and affective factors to model human behavior for the design of adaptive systems.

There are several research threads in the HCM3 lab at RIT. These share a common experimental framework based on the idea that human behavior is both observable (e.g. motor, verbal, visual) and hidden (mental states) and that we can capture and model representative data via a variety of physical and physiological sensors. Our interdisciplinary group integrates methods and approaches of human-centered design, visual perception, imaging science, and computational linguistics to study the influence of perceptual expertise and domain knowledge on complex visual tasks. The HCM3 closely collaborates with the Multidisciplinary Vision Research Lab (MVRL) in RIT’s Center for Imaging Science.

More stories on our research focus and research tracks can be found here. A complete list of our publications can be found here.