|
MM-DocTable: Multimedia Document Engineering Workflows on Tabletop Devices.
Third-party funded project |
Project title |
MM-DocTable: Multimedia Document Engineering Workflows on Tabletop Devices. |
Principal Investigator(s) |
Schuldt, Heiko
|
Project Members |
El-Kabary, Ihab
|
Organisation / Research unit |
Departement Mathematik und Informatik / Databases and Information Systems (Schuldt) |
Project Website |
http://dbis.cs.unibas.ch/projects/ongoing-projects/mm-doctable/dbis_project_view |
Project start |
01.10.2011 |
Probable end |
30.09.2014 |
Status |
Completed |
Abstract |
Multi-touch devices have recently enjoyed a surge of popularity by supported by an ever-increasing wealth of applications which is a testimony that digital natural user interfaces (NUI) have gained attraction in the public. While paper is still far from extinct, we observe that habits and interaction patterns are more and more shifting towards those types of digital devices, as they become user-friendlier and more effective at performing regular document tasks. Digital interactive tabletops form an important subcategory, which receives considerable attention in the different HCI-research communities, yet surprisingly has not seen a great number of advanced document engineering applications that take advantage of this type of platform. We postulate that bimanual pen-and-touch-operated tabletops, as enhanced virtual office desks, are very suitable platforms to perform a number of document-engineering tasks and are a natural extension of the paper-digital interface paradigm, the obvious advantage of tabletops being that they provide immediate, interactive feedback to users upon their actions. Currently, though, systems only demonstrate the novel UI concepts in rather isolated, single-document environments and are not yet integrated in any particular document engineering workflow. Very often, documents are engaged in a process involving several interactions with interfaces of sometimes heterogeneous systems and so the need to integrate tabletop UIs with a variety of legacy workflows arises.
The advent of NUIs has also strongly influenced the multimedia retrieval community. In particular, novel approaches that make use of the interaction capabilities of NUIs for posing queries, such as query by sketch applied to digital image collections, have become increasingly popular. While the general idea to search for images on the basis of a (rough) user-provided sketch is very appealing, it is currently limited to a particular search task, namely known-item search. With the availability of large digital video collections, query by sketch will also become highly relevant for content-based video retrieval. However, in order to go beyond the comparison of a sketch with a single video frame, support for dedicated gestures to express the motion of objects over several frames are needed.
The MM-DocTable project aims at providing a sound architecture and tools to support document engineering tasks performed on digital tabletops in a wider range of contexts, in particular by enabling rapid and seamless access to reference material for multimedia document creation or editing. Most importantly, the proposed architecture will support several querying methods (keyword search, query by example, and query by sketch) with a view to apply them interchangeably in a number of retrieval scenarios. The pen-and-touch-operated UI of the tabletop will be designed so that the different retrieval operations are all seamlessly executable within the main document task, that is, without requiring tedious context switches.
To achieve these goals, the project will consist of four main parts. First, we will investigate what scenarios can benefit from complex document engineering workflows on tabletop devices that jointly use different query paradigms. Second, we will develop new algorithms for the detection of prominent objects in sketch-based image retrieval in order to apply query by sketch to other interaction intentions beyond known item search. Third, we will extend sketch-based retrieval to digital video collections, by adding gestures for specifying the motion of objects. Fourth, we will seamlessly integrate all these services into the tabletop UI so that users are able to directly access elements of the multimedia database and use this integrated system in user studies. |
Financed by |
Swiss National Science Foundation (SNSF)
|
Follow-up project of |
66439 PADIR: Paper-Digital System for Information Capture and Retrieval.
|
Published results () |
|
ID |
Autor(en) |
Titel |
ISSN / ISBN |
Erschienen in |
Art der Publikation |
|
1444474 |
Springmann, Michael |
Building Blocks for Adaptable Image Search in Digital Libaries |
|
|
Publication: Thesis (Dissertationen, Habilitationen) |
|
1444539 |
Al Kabary, Ihab; Schuldt, Heiko |
Sketch-based image similarity search with a pen and paper interface |
978-1-4503-1472-5 |
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
1444542 |
Al Kabary, Ihab; Schuldt, Heiko |
SKETCHify - an adaptive prominent edge detection algorithm for optimized query-by-sketch image retrieval |
978-3-319-12092-8 |
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
1444511 |
Kreuzer, Roman; Springmann, Michael; Al Kabary, Ihab; Schuldt, Heiko |
An interactive paper and digital pen interface for query-by-sketch image retrieval |
978-3-642-28997-2 (E-Book) ; 978-3-642-28996-5 (Print) |
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
1444512 |
Giangreco, Ivan; Springmann, Michael; Al Kabary, Ihab; Schuldt, Heiko |
A user interface for query-by-sketch based image retrieval with color sketches |
0302-9743 ; 978-3-642-28996-5 ; 978-3-642-28997-2 |
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
2185203 |
Matulic, Fabrice; Norrie, Moira C.; Al Kabary, Ihab; Schuldt, Heiko |
Gesture-supported document creation on pen and touch tabletops |
978-1-4503-1899-0 |
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
2185244 |
Al Kabary, Ihab; Buechler, Marcel; Schuldt, Heiko |
TOUCHify: bringing pen-based touch screen functionality to flat panel display screens |
|
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
2185294 |
Al Kabary, Ihab; Schuldt, Heiko |
SportSense: using motion queries to find scenes in sports videos |
|
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
2185301 |
Al Kabary, Ihab; Giangreco, Ivan; Schuldt, Heiko; Matulic, Fabrice; Norrie, Moira |
QUEST: towards a multi-modal CBIR framework combining query-by-example, query-by-sketch, and text search |
978-1-4799-2171-3 |
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
2185309 |
Al Kabary, Ihab; Schuldt, Heiko |
Towards sketch-based motion queries in sports videos |
|
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
2301776 |
Al Kabary, Ihab; Schuldt, Heiko |
Using hand gestures for specifying motion path queries in sketch-based video retrieval |
Springer |
|
Publication: ConferencePaper (Artikel, die in Tagungsbänden erschienen sind) |
|
Cooperations () |
|
ID |
Kreditinhaber |
Kooperationspartner |
Institution |
Laufzeit - von |
Laufzeit - bis |
|
2301755 |
Schuldt, Heiko |
Norrie, Moira, Professor |
ETH Zürich |
01.10.2011 |
30.09.2014 |
|
|
|
|