To advance image information systems the fundamental problem of image understanding needs to be tackled. Computing solutions that bridge the "semantic gap" must capture high-level knowledge of end users. The goal of our research is to extract and utilize the tacit knowledge of domain experts towards building a pipeline of image processing algorithms (IPA) that could closely parallel the underlying cognitive processes. We believe experts' eye movements can serve as an objective measure to help us understand the perceptual and conceptual processes. The eye movements provide us with key visual features and diagnostic regions of the images. Comparison of union of all subjects' fixations and intersection of 75% of all subjects' fixations with local features such as SIFT (scale invariant feature transform) shows similarity indicating the usefulness of SIFT features in the identification and narrowing of regions of interest [1].

Experts' eye movements can also be used to evaluate image processing algorithms (IPA).  We use medical experts' fixations as a metric to evaluate the correlation of perceptually-relevant regions with individual clusters identified through k-means clustering algorithm. Our results demonstrate the possibility of using this technique to determine if a particular IPA will be useful in identifying image regions with high visual interest [2]

[1] Li, R.Vaidyanathan, P.Mulpuru, S; Pelz, J.Shi, PCalvelli, C.Haake, A.(2010, November). Human-centric approaches to image understanding and retrieval. In Image Processing Workshop (WNYIPW), 2010 Western New York(pp. 62-65). IEEE. 

[2] Vaidyanathan, P.Pelz, J.LiRMulpuru, S.WangD.Shi, P.Calvelli, C.Haake, A., "Using human experts' gaze data to evaluate image processing algorithms," IVMSP Workshop, 2011 IEEE 10th , vol., no., pp.129-134. 

Vaidyanathan P., Pelz J.Ovesdotter Alm C., Calvelli C., Shi P., Haake A. (2013) Integration of Eye Movements and Spoken Description for Medical Image Understanding. Book of Abstracts of the 17th European Conference on Eye Movements, p.40. 

Vaidyanathan, P.; et al. Visualinguistic Approach to Medical Image Understanding. Proceedings of the AMIA 2012 Annual Symposium, Chicago, November, 2012. Ed. William Hersh. Chicago, IL: AMIA, Web.