Publications
Below, you will find a collection of publications concerning the MASP project.
Download the complete bibliography as BibTex: 
Meta-Modeling Runtime Models
Grzegorz Lehmann, Marco Blumendorf, Frank Trollmann, Sahin Albayrak
5th Workshop on Models@run.time, Oslo, Norway, 2010.
Abstract:
Runtime models enable the implementation of highly adaptive applications but also require a rethinking in the way we approach models. Metamodels of runtime models must be supplemented with additional runtime concepts that have an impact on the way how runtime models are built and reflected in the underlying runtime architectures. The goal of this work is the generalization of common concepts found in different approaches utilizing runtime models and the provision of a basis for their meta-modeling. After analyzing recent works dealing with runtime models, we present a meta-modeling process for runtime models. Based on a meta-metamodel it guides the creation of metamodels combining design time and runtime concepts.
Bridging Models and Systems at Runtime to Build Adaptive User Interfaces
Marco Blumendorf, Grzegorz Lehmann, Sahin Albayrak
EICS '10: Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems, 2010, Berlin, Germany.
Abstract:
Adapting applications and user interfaces at runtime requires a deeper understanding of the underlying design. Models formalize this design, express the underlying concepts and make them accessible to machines. In our work we utilize runtime models to reflect the state of the interactive system (its UI respectively) and to change its underlying configuration. So called executable models combine design information, runtime state, and execution logic. From the perspective of adaptive UIs this allows the dynamic reconfiguration of UIs according to design information and the current state of the application at runtime. Dedicated elements of the model create a causal interconnection between model and user interface and facilitate a continuous information exchange between the two. This creates a feedback loop between model and UI where external stimulations influence the model execution and where projections to the outside allow the dynamic alteration of user interfaces.
Development of context-adaptive applications on the basis of runtime user interface models
Grzegorz Lehmann, Marco Blumendorf, Sahin Albayrak
EICS '10: Proceedings of the 2nd ACM SIGCHI symposium on Engineering interactive computing systems, 2010, Berlin, Germany.
Abstract:
One of the challenges faced by developers of applications for smart environments is the diversity of contexts of use. Applications in smart environments must cope with continuously changing context of use, so the developers need to prepare them for a possibly broad range of situations. Since the developer has no access to all environments, in which her application will be executed, it must be possible to simulate different environments and evaluate the behavior of the application at design time. In our demonstration the designer has the possibility to simulate and modify a runtime context model and observe as her application adapts on the fly. In the underlying runtime architecture applications, defined as sets of models, are adapted automatically on the basis of the information held in the runtime context model. A visual tool enables the user interface developer to access and modify the models at any time and immediately observe the behavior of the application.
Runtime Models for Ubiquitous User Interfaces
Grzegorz Lehmann
W3C Workshop on Future Standards for Model-Based User Interfaces, May 13-14th, 2010, Rome, Italy.
Statement of Interest
Presentation
A Multimodal User Interface Model For Runtime Distribution
Dirk Roscher, Marco Blumendorf, Sahin Albayrak
CHI'10 Workshop on Model Driven Development of Advanced User Interfaces, April 10th, 2010, Atlanta, Georgia, USA.
Abstract:
Smart environments provide numerous networked interaction resources (IRs) allowing users to interact with services in many different situations via many different (combinations of) IRs. In such environments it is necessary to adapt the user interface dynamically at runtime to each new situation to allow an ongoing interaction in changing contexts. Our approach allows to dynamically select the combination of IRs that are suitable for the interaction in the current context at any time. The decision is based on information from a user interface model executed at runtime and a context model gathering information about the environment. The user interface model supports the CARE properties to specify flexible multimodal interaction.
Adaptive user interface assistance in smart environments
Maximilian Kern, Frank Trollmann, Marco Blumendorf, Sahin Albayrak
International Symposium on Matching and Meaning at the AISB 2010 convention, 29 March – 1 April 2010, De Montfort University, Leicester, UK.
Abstract:
More than half of the products, which appear to be malfunctioning are often brought back to stores by the customers although they are in full working order. The reason for this is either the customers’ lack of understanding of how to use such products or the unmanageable complexity of the products. This fact claims for intelligent user interfaces which adapt to the user at runtime. Such systems also require new kinds of help systems, which also need to be adaptive. In this paper, the work in progress of a user interface assistance approach in smart environments is presented. This approach aims at the utilization of model-based development in order to build self-explanatory systems which are able to guide and help the user dynamically. We take benefit of the self-describing nature of models and generate help for navigation within user interfaces. In the future, we will extend this approach in order to use the same semantics of the model which we consider for help generation in order to offer the user an alternative goal-driven interaction pattern.
A 3-Layer Architecture for Smart Environment Models
Grzegorz Lehmann, Andreas Rieger, Marco Blumendorf, Sahin Albayrak
IEEE PerCom Workshop on Smart Environments, March 29, 2010, Mannheim,Germany.
Abstract:
Enriched with more and more intelligent devices modern homes rapidly transform into smart environments. Their growing capabilities enable the implementation of a new generation of ubiquitous applications, but also raise the complexity of the development. Developers of applications for smart environments must cope with a multitude of sensors, devices, users and thus contexts. We present a model-based approach for modeling of, reasoning about and controlling smart environments. A context model provides adaptive applications with a unified access to the smart home environment and, through a unique approach of utilizing executable models, also reflects its state at runtime. The presented approach supports runtime user interface adaption and reconfiguration for seamless interaction and has been successfully utilized to build several context-adaptive applications running in our smart home testbed.
Using User Interface Models at Runtime
Marco Blumendorf, Grzegorz Lehmann
IFIP working group 2.7 / 13.4 on User Interface Engineering, December 10-12th, 2009, London, United Kingdom.
Abstract:
The utilization of models and modeling technologies is a promising approach to handle the increasing complexity of user interface development. In our work we develop a model-based approach that combines executable user interface models with a runtime architecture to handle interaction in smart environments. We identify the common building blocks of dynamic, self-contained and executable models that integrate the design-time and runtime aspects of user interface modeling. On the one side bridging the gap betwenn design- and runtime models enables the reasoning about and modifications of the design decisions at runtime. On the other side it allows describing dynamic information about the interaction and the environment on a high abstraction level of models. Using the notion of executable models we aim at implementing fully model-based, adaptive and adaptable applications for smart environments with multimodal and distributed user interfaces.
A Novel Approach for Device Independent Gesture Recognition
Mathias Wilhelm, Marco Blumendorf, Dirk Roscher, Sahin Albayrak
4th International Workshop on Haptic and Audio Interaction Design, September 10-11th, 2009, Dresden, Germany.
Abstract:
In this poster we present a novel approach for a device independent gesture recognition system. Our system is trained by an arbitrary gesture device and recognises dynamic hand gestures. Input is recorded by a vision system, accelerometers or a pen-based device. Training the system with one device then allows applying the recognition mechanism to the other devices as well..
Using Meta User Interfaces to Control Multimodal Interaction in Smart Environments
Dirk Roscher, Marco Blumendorf, Sahin Albayrak
IUI'09 Workshop on Model Driven Development of Advanced User Interfaces, February 8, 2009, Sanibel Island, Florida, USA.
Abstract:
Smart environments bring together multiple users, (interaction) resources and services. This creates complex and unpredictable interactive computing environments that are hard to understand. Users thus have difficulties to build up their mental model of such interactive systems. To address this issue users need possibilities to evaluate the state of these systems and to adapt them according to their needs. In this work we describe the requirements and functionalities for evaluating and controlling interactive spaces in smart environments from the system and the user perspective. Furthermore we present a model-based implementation of these capabilities which is accessible for the user in form of a meta user interface.
A meta user interface to control multimodal interaction in smart environments
Dirk Roscher, Marco Blumendorf, Sahin Albayrak
International Conference on Intelligent User Interfaces, February 8-11, 2009, Sanibel Island, Florida, USA.
Abstract:
Smart environments bring together multiple users, (interaction) resources and services. This creates complex and unpredictable interactive computing environments that are hard to understand. Users thus have difficulties to build up their mental model of such interactive systems. To address this issue users need possibilities to evaluate the state of these systems and to adapt them according to their needs. In this work we present our implementation of the functionalities to evaluate and control multimodal interaction in smart environments, which is accessible for users through a meta user interface.
Behavior-sensitive User Interfaces for Smart Environments
Veit Schwartze, Sebastian Feuerstack, Sahin Albayrak
Accepted for:
HCI International 2009
In 'Digital Human Modeling' track in 'Modeling Human Behavior for Smart Environments' session.
19-24 July 09, Town and Country Resort & Convention Center, San Diego, CA, USA
Abstract:
In smart environments interactive assistants can support the user's daily life by being ubiquitously available through any interaction device that is connected to the network. Focusing on graphical interaction, user interfaces are required to be flexible enough to be adapted to the actual context of the user. In this paper we describe an approach, which enables flexible user interface layout adaptations based on the current context of use (e.g. by changing the size of elements to visually highlight the important elements used in a specific situation). In a case study of the '4-star Cooking Assistant' application we prove the capability of our system to dynamically adapt a graphical user interface to the current context of use.
Towards a Framework for the Development of Adaptive Multimodal User Interfaces for Ambient Assisted Living Environments
Marco Blumendorf, Sahin Albayrak
HCI International 2009
In 'Design of Ambient Assisted Living Environments' track in 'Universal Access in Human-Computer Interaction' session.
19-24 July 09, Town and Country Resort & Convention Center, San Diego, CA, USA
Abstract:
In this paper we analyse the requirements and challenges ambient assisted living and smart environments pose on interactive systems. We present a framework for the provisioning of user interfaces for such environments. The framework incorporates model-based user interface development technologies to create a runtime system that manages interaction resources and context information to adapt interaction. This approach allows the creation of adaptive and multimodal interactive ambient assisted living applications.
Ubiquitous User Interfaces: Multimodal Adaptive Interaction for Smart Environments
Marco Blumendorf, Grzegorz Lehmann, Dirk Roscher, Sahin Albayrak
To appear in:
Multimodality in Mobile Computing and Mobile Devices: Methods for Adaptable Usability
A book edited by Dr. Stan Kurkovsky
Central Connecticut State University, USA
Abstract:
The widespread use of computing technology raises the need for interactive systems that adapt to user, device and environment. Multimodal user interfaces provide the means to support the user in various situations and to adapt the interaction to the user’s needs. In this chapter we present a system utilizing design-time user interface models at runtime to provide flexible multimodal user interfaces. The server-based system allows the combination and integration of multiple devices to support multimodal interaction and the adaptation of the user interface to the used devices, the user and the environment. The utilization of the user interface models at runtime allows exploiting the design information for advanced adaptation possibilities. An implementation of the system has been successfully deployed in a smart home environment throughout the Service Centric Home project (www.sercho.de).
Automated Usability Evaluation during Model-based Interactive System Development
Sebastian Feuerstack, Marco Blumendorf, Maximilian Kern, Michael Kruppa, Michael Quade, Mathias Runge and Sahin Albayrak
Tamodia 2008, September 25-26, 2008, Pisa, Italy
Abstract:
In this paper we describe an approach to efficiently evaluate the usability of an interactive application that has been realized to support various platforms and modalities. Therefore we combine our Multi-Access Service Platform (MASP), a model-based runtime environment to offer multimodal user interfaces with the MeMo workbench, a tool supporting an automated usability analysis. Instead of deriving a system model by reverse-engineering or annotating screenshots for the automated usability analysis, we use the semantics of the runtime models of the MASP. This allows us to reduce the evaluation effort by automating parts of the testing process for various combinations of platforms, modalities and user groups that should be addressed by the application. Furthermore, by testing the application at runtime, the usability evaluation can also consider system dynamics and information that are unavailable at design time.
Executable Models for Human-Computer Interaction
Marco Blumendorf and Grzegorz Lehmann and Sebastian Feuerstack and Sahin AlbayrakDSV-IS 2008, Kingston, Ontario, Canada, 16.-18.7.2008
Abstract:
Interacting with smart devices and smart homes becomes increasingly complex but also more and more important. Computing systems can help the aging society to live autonomously in their own home. However, interacting with such systems is a challenge not only for older people. Multimodal user interfaces and new interaction paradigms can help addressing these problems, but are not yet mature enough to be of direct use. In this paper we describe our work in the area of smart home environments and multimodal user interaction. We present the Ambient Assisted Living Testbed set up at the Technical University of Berlin and the Multi-Access Service Platform, allowing multimodal interaction in this smart environment with adaptivity, session management, migration, distribution and multimodality as key features for future services.
Utilizing Dynamic Executable Models for User Interface Development
Grzegorz Lehmann and Marco Blumendorf and Sebastian Feuerstack and Sahin AlbayrakDSV-IS 2008, Kingston, Ontario, Canada, 16.-18.7.2008
Abstract:
In this demonstration we present the Multi Access Service Platform (MASP), a model-based runtime architecture for user interface development based on the idea of dynamic executable models. Such models are self-contained and complete as they contain the static structure, the dynamic state information as well as the execution logic. Utilizing dynamic executable models allows us to implement a rapid prototyping approach and provide mechanisms for the extension of the UI modeling language of the MASP.
Multimodal User Interfaces for Smart Environments: The Multi-Access Service Platform
Marco Blumendorf, Sebastian Feuerstack, Sahin Albayrak
ACM Advanced Visual Interfaces Conference 2008; Napoli, Italy
Abstract:
User interface models are a well accepted approach to handle increasing user interface complexity. The approach presented in this paper utilizes user interface models at runtime to provide a basis for user interface distribution and synchronization. Task and domain model synchronize workflow and dynamic content across devices and modalities. A cooking assistant serves as example application to demonstrate multimodality and distribution. Additionally a debugger allows the inspection of the underlying user interface models at runtime.
Model-based Layout Generation
Sebastian Feuerstack, Marco Blumendorf, Veit Schwartze, Sahin Albayrak
ACM Advanced Visual Interfaces Conference 2008; Napoli, Italy
Abstract:
Offering user interfaces for interactive applications that are flexible enough to be adapted to various context-of-use scenarios such as supporting different display sizes or addressing various input styles requires an adaptive layouting. This is still a problem since user interface design is often done independently from the software application design. We describe an approach for layout derivation that is embedded in a model-based user interface generation process. By an interactive and tool-supported process we can efficiently create a layout model that is composed of interpretations of the other design models and is consistent to the application design. By shifting the decision about which interpretations are relevant to support a specific context-of-use scenario from design-time to run-time we can flexible adapt the layout to support adaptations to new device capabilities, user demands like interface distributions. We present our runtime environment that is able to evaluate the relevant model layout information to constraints as they are required and to reassemble the user interface parts regarding the updated containment, order, orientation and sizes information of the layout-model. Finally we present results of an evaluation we did to test the design and runtime efficiency of our model-based layouting approach.
Multimodal Smart Home User Interfaces
Marco Blumendorf, Sebastian Feuerstack, Sahin Albayrak
International Workshop on Intelligent User Interfaces for Ambient Assisted Living (IUI4AAL), at International Conference on Intelligent User Interfaces (IUI), Canary Islands, Spain, 13.-16.1.2008
Abstract:
Interacting with smart devices and smart homes becomes increasingly complex but also more and more important. Computing systems can help the aging society to live autonomously in their own home. However, interacting with such systems is a challenge not only for older people. Multimodal user interfaces and new interaction paradigms can help addressing these problems, but are not yet mature enough to be of direct use. In this paper we describe our work in the area of smart home environments and multimodal user interaction. We present the Ambient Assisted Living Testbed set up at the Technical University of Berlin and the Multi-Access Service Platform, allowing multimodal interaction in this smart environment with adaptivity, session management, migration, distribution and multimodality as key features for future services.
Multimodal User Interaction in Smart Environments: Delivering Distributed User Interfaces
Marco Blumendorf, Sebastian Feuerstack and Sahin Albayrak
European Conference on Ambient Intelligence: Workshop on Model Driven Software Engineering for Ambient Intelligence Applications, Darmstadt, Germany, 7.-10.11.2007
Abstract:
The ongoing utilization of computer technologies in all areas of life leads to the development of smart environments comprising numerous networked devices and resources. Interacting in and with such environments requires new interaction paradigms, abstracting from single interaction devices to utilize the environment as interaction space. Using a networked set of interaction resources allows supporting multiple modalities and new interaction techniques, but also requires the consideration of the set of devices and the adaptation to this set at runtime. While the generation of user interfaces based on UI models, although still challenging, has been widely researched, the runtime processing and delivery of the derivable user interfaces has gained less attention. Delivering distributed user interfaces while maintaining their interdependencies and keeping them synchronized is not a trivial problem. In this paper we present an approach to realize a runtime environment, capable of distributing user interfaces to a varying set of devices to support multimodal interaction based on a user interface model and the management of interaction resources.
Prototyping of Multimodal Interactions for Smart Environments based on Task Models
Sebastian Feuerstack, Marco Blumendorf, Sahin Albayrak
European Conference on Ambient Intelligence: Workshop on Model Driven Software Engineering for Ambient Intelligence Applications, MDSE-AmI 2007, Darmstadt, Germany
Abstract:
Smart environments offer interconnected sensors, devices, and appliances that can be considered for interaction to substantially extend the potentially available modality mix. This promises a more natural and situation aware human computer interaction. Technical challenges and differences in interaction principles for distinct modalities restrict multimodal systems to specialized systems that support specific situations only. To overcome these limitations enabling an easier integration of new modalities to enhance interaction in smart environments, we propose a task-based notation that can be interpreted at runtime. The notation supports evolutionary prototyping of new interaction styles for already existing interactive systems. We eliminate the gap between design- and runtime, since support for additional modalities can be prototyped at runtime to an already existing interactive system.
A Notation supporting Runtime Interpretation of Task-Models for Interactive Applications in Smart Environments
Sebastian Feuerstack, Marco Blumendorf, Sahin Albayrak
NewMet - Interact 2007
Abstract:
In this paper we present our approach to serve user interfaces for smart environments based on an enhanced ConcurTaskTree notation that can be interpreted at runtime. Different to other approaches our task model includes an explicit description of the domain concepts as well as a separation of the user interface definition from the functional backend. This enables us to interpret the task tree at runtime and keep the interactive system always synchronized to the actual state of the interaction in case of context changes or modifications of the task tree and domain concepts for prototyping applications at runtime.
Event-based Synchronization of Model-Based Multimodal User Interfaces
Marco Blumendorf, Sebastian Feuerstack, Sahin Albayrak
Second International Workshop on Model Driven Development of Advanced User Interfaces, MoDELS 2006, Genova, Italy, 1.-5.10.2006
Abstract:
Smart environments utilize computers as tools supporting the user in his daily life, moving interaction with computers from a single system to a complex, distributed environment. User interfaces available in this environment need to adapt to the specifics of the various available devices and are distributed across several devices at the same time. A problem arising with distributed user interfaces is the required synchronization of the different parts. In this paper we present an approach allowing the event-based synchronization of distributed user interfaces based on a multi-level user interface model. We also describe a runtime system we created, allowing the execution of model-based user interface descriptions and the distribution of user interfaces across various devices and modalities using channels established between the system and the end devices.
Bridging the Gap between Model and Design of User Interfaces
Sebastian Feuerstack, Marco Blumendorf, Sahin Albayrak
Workshop Modellbasierte Entwicklung von Interaktionsanwendungen, Informatik 2006, Dresden, 1.-6.10.2006
Abstract:
The creation of user interfaces usually involves various people in different roles and several tools that are designed to support each specific role. In this paper we propose a tool for rapid prototyping that allows all parties involved to directly interact with the system under development. The tool is based on task tree development and integrates the system designer, the user interface designer, the usability expert, and the user interface developer in a common process. The final system is derived from two sources, the task model specified by the system architect and the final user interface specified by the user interface developer and designer. Aggregating the runtime system and the design tools into one complete integrated system is our approach to bridge the gap between the user interface designer working on system mock-ups and the actual developers implementing the system.
Seamless Home Services
Sebastian Feuerstack, Marco Blumendorf, Grzegorz Lehmann, Sahin Albayrak
AmI.d 2006, Sophia Antipolis, France, 20.-22.09.2006
Abstract:
The growing number of smart devices providing services in peoples' homes and the increasing number of services available via the Internet creates two separate worlds that have not been successfully integrated yet. To bridge these two worlds, we propose a Home Service Platform supporting the seamless integration of home-based and net-centric services in a smart home environment. A common Service Engine supporting inter-service communication and aggregation provides human accessible user interfaces via a Service Portal and allows representing devices through a Home Device Controller. The integration of the two worlds allows us to create a new type of service: Seamless Home Services.
