Exploring scientific literature by textual and image content using DRIFT

Digital libraries represent the most valuable resource for storing, querying, and retrieving scientific literature. Traditionally, the reader/analyst aims to compose a set of articles based on keywords, according to his/her preferences, and manually inspect the resulting list of documents. Except for the articles which share citations or common keywords, the results retrieved will be limited to those which fulfill a syntactic match. Besides, if instead of having an article as a reference, the user has an image, the process of finding and exploring articles with similar content becomes infeasible. This paper proposes a visual analytic methodology for exploring and analyzing scientific document collections that consider both textual and image content. The proposed technique relies on combining multiple Content-Based Image Retrieval (CBIR) components and multidimensional projection to map the documents to a visual space based on their similarity, thus enabling an interactive exploration. Moreover, we extend its analytical capabilities with visual resources to display complementary information on selected documents that uncover hidden patterns and semantic relations. We evidence the effectiveness of our methodology through three case studies and a user evaluation, which attest to its usefulness during the process of scientific collections exploration.