Data Entry: Please note that the research database will be replaced by UNIverse by the end of October 2023. Please enter your data into the system https://universe-intern.unibas.ch. Thanks

Login for users with Unibas email account...

Login for registered users without Unibas email account...

 
MM-DocTable: Multimedia Document Engineering Workflows on Tabletop Devices.
Third-party funded project
Project title MM-DocTable: Multimedia Document Engineering Workflows on Tabletop Devices.
Principal Investigator(s) Schuldt, Heiko
Project Members El-Kabary, Ihab
Organisation / Research unit Departement Mathematik und Informatik / Databases and Information Systems (Schuldt)
Project Website http://dbis.cs.unibas.ch/projects/ongoing-projects/mm-doctable/dbis_project_view
Project start 01.10.2011
Probable end 30.09.2014
Status Completed
Abstract

Multi-touch devices have recently enjoyed a surge of popularity by supported by an ever-increasing wealth of applications which is a testimony that digital natural user interfaces (NUI) have gained attraction in the public. While paper is still far from extinct, we observe that habits and interaction patterns are more and more shifting towards those types of digital devices, as they become user-friendlier and more effective at performing regular document tasks. Digital interactive tabletops form an important subcategory, which receives considerable attention in the different HCI-research communities, yet surprisingly has not seen a great number of advanced document engineering applications that take advantage of this type of platform. We postulate that bimanual pen-and-touch-operated tabletops, as enhanced virtual office desks, are very suitable platforms to perform a number of document-engineering tasks and are a natural extension of the paper-digital interface paradigm, the obvious advantage of tabletops being that they provide immediate, interactive feedback to users upon their actions. Currently, though, systems only demonstrate the novel UI concepts in rather isolated, single-document environments and are not yet integrated in any particular document engineering workflow. Very often, documents are engaged in a process involving several interactions with interfaces of sometimes heterogeneous systems and so the need to integrate tabletop UIs with a variety of legacy workflows arises.

The advent of NUIs has also strongly influenced the multimedia retrieval community. In particular, novel approaches that make use of the interaction capabilities of NUIs for posing queries, such as query by sketch applied to digital image collections, have become increasingly popular. While the general idea to search for images on the basis of a (rough) user-provided sketch is very appealing, it is currently limited to a particular search task, namely known-item search. With the availability of large digital video collections, query by sketch will also become highly relevant for content-based video retrieval. However, in order to go beyond the comparison of a sketch with a single video frame, support for dedicated gestures to express the motion of objects over several frames are needed.

The MM-DocTable project aims at providing a sound architecture and tools to support document engineering tasks performed on digital tabletops in a wider range of contexts, in particular by enabling rapid and seamless access to reference material for multimedia document creation or editing.  Most importantly, the proposed architecture will support several querying methods (keyword search, query by example, and query by sketch) with a view to apply them interchangeably in a number of retrieval scenarios. The pen-and-touch-operated UI of the tabletop will be designed so that the different retrieval operations are all seamlessly executable within the main document task, that is, without requiring tedious context switches.

To achieve these goals, the project will consist of four main parts. First, we will investigate what scenarios can benefit from complex document engineering workflows on tabletop devices that jointly use different query paradigms. Second, we will develop new algorithms for the detection of prominent objects in sketch-based image retrieval in order to apply query by sketch to other interaction intentions beyond known item search. Third, we will extend sketch-based retrieval to digital video collections, by adding gestures for specifying the motion of objects. Fourth, we will seamlessly integrate all these services into the tabletop UI so that users are able to directly access elements of the multimedia database and use this integrated system in user studies.

Financed by Swiss National Science Foundation (SNSF)
Follow-up project of 66439 PADIR: Paper-Digital System for Information Capture and Retrieval.

Published results ()

  ID Autor(en) Titel ISSN / ISBN Erschienen in Art der Publikation
1444474  Springmann, Michael  Building Blocks for Adaptable Image Search in Digital Libaries      Publication: Thesis (Dissertationen, Habilitationen) 
1444539  Al Kabary, Ihab; Schuldt, Heiko  Sketch-based image similarity search with a pen and paper interface  978-1-4503-1472-5    Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
1444542  Al Kabary, Ihab; Schuldt, Heiko  SKETCHify - an adaptive prominent edge detection algorithm for optimized query-by-sketch image retrieval  978-3-319-12092-8    Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
1444511  Kreuzer, Roman; Springmann, Michael; Al Kabary, Ihab; Schuldt, Heiko  An interactive paper and digital pen interface for query-by-sketch image retrieval  978-3-642-28997-2 (E-Book) ; 978-3-642-28996-5 (Print)    Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
1444512  Giangreco, Ivan; Springmann, Michael; Al Kabary, Ihab; Schuldt, Heiko  A user interface for query-by-sketch based image retrieval with color sketches  0302-9743 ; 978-3-642-28996-5 ; 978-3-642-28997-2    Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
2185203  Matulic, Fabrice; Norrie, Moira C.; Al Kabary, Ihab; Schuldt, Heiko  Gesture-supported document creation on pen and touch tabletops  978-1-4503-1899-0    Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
2185244  Al Kabary, Ihab; Buechler, Marcel; Schuldt, Heiko  TOUCHify: bringing pen-based touch screen functionality to flat panel display screens      Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
2185294  Al Kabary, Ihab; Schuldt, Heiko  SportSense: using motion queries to find scenes in sports videos      Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
2185301  Al Kabary, Ihab; Giangreco, Ivan; Schuldt, Heiko; Matulic, Fabrice; Norrie, Moira  QUEST: towards a multi-modal CBIR framework combining query-by-example, query-by-sketch, and text search  978-1-4799-2171-3    Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
2185309  Al Kabary, Ihab; Schuldt, Heiko  Towards sketch-based motion queries in sports videos      Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 
2301776  Al Kabary, Ihab; Schuldt, Heiko  Using hand gestures for specifying motion path queries in sketch-based video retrieval  Springer     Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) 

Cooperations ()

  ID Kreditinhaber Kooperationspartner Institution Laufzeit - von Laufzeit - bis
2301755  Schuldt, Heiko  Norrie, Moira, Professor  ETH Zürich  01.10.2011  30.09.2014 
   

MCSS v5.8 PRO. 0.441 sec, queries - 0.000 sec ©Universität Basel  |  Impressum   |    
28/03/2024