• Small text Medium text Large text
  • Default colours Black text on white background Yellow text on black background

An Introduction to the Key Issues Relating to Accessible User Interfaces


Iosif Klironomos
Foundation for Research and Technology
Hellas, Institute of Computer Science

Julio Abascal
Laboratory of HCI for Special Needs
University of the Basque Country/Euskal Herriko Unibertsitatea

Table of Contents
1        Introduction
1.1     CARDIAC project
1.2     Scope of the SDDP in Inclusive Human-Machine Interaction
1.3     Accessible Human-Machine interaction and Assistive Technology
1.4     The human factor
2        HCI and Accessibility
2.1     Human-Computer Inclusive design
2.1.1       Advantages of Inclusive Design
2.1.2       Current barriers for Inclusive Design
2.1.3       Conditions for Inclusive Design
2.2     Web accessibility
2.2.1       Mobile Web
2.2.2       The adaptive web
2.2.3       Web personalization
2.2.4       Data mining for web personalization
2.3     Human interfaces for accessible Ambient Intelligence system
2.3.1       Accessible AmI supporting technology
2.3.2       Accessible adaptive interfaces for AmI
2.4     Interfaces for robot control
2.4.1       Requirements of the Human-Robot Interface
3        A summary of Current Research Trends in HCI Design
3.1     Some Design Frameworks and tools
3.2     Current research trends
3.3     HCI and AmI
3.4     Emerging Issues
4        References
4.1     Further readings


1 Introduction

1.1 CARDIAC project

One of the main objectives of the CARDIAC project is to generate a roadmap identifying issues in the area of Inclusive uman Computer Interaction (HCI) research and development priorities.  This roadmap will be a document outlining essential areas and subsequent types of research missing that could facilitate the development of inclusive HCI technologies.

The purpose of this document is to provide a background and context to the second Structured Dialogue Design Process (SDDP) co-laboratory of the CARDIAC Project which is scheduled for the 28th – 29th of June in Donostia-San Sebastian, Spain. The purpose of this event is to generate a roadmap in response to the Triggering Question "What type of research is missing that could facilitate development of inclusive HCI".

When considering the various methodologies for generating roadmaps, the Structured Dialogic Design Process methodology was selected due to its robustness and efficiency in gathering the collective wisdom of a wide range of different takeholders. The SDDP methodology supports democratic and structured dialogue among a group of stakeholders and is especially effective in resolving multiple conflicts of purpose and values, and in generating consensus on organizational and inter-organizational strategy. A full description of the methodology and how exactly the methodology will guide the process of generating a roadmap is presented in a separate document.

1.2 Scope of the SDDP in Inclusive Human-Machine Interaction

Several research activities in the field of Ambient Assisted Living (AAL) focus on more user involvement in the design process. The ISO standard 13407 Human-centred design process for interactive systems provides guidance on human-centred design activities throughout the life cycle of interactive computer-based systems. However also other research methods are available, for instance participatory and co-design. These approaches have in common that they all express the belief that all people have something to offer to the design process.

Moreover, adaptivity/intelligence on the one hand, and the analysis of the implications, from an eAccessibility perspective, of the emerging Ambient Intelligence (AmI) paradigm (with a clear orientation to creating "natural" interfaces) on the other, are becoming increasingly important aspects. The main difficulty lies in understanding and utilising the whole range of possibilities for inclusive Human-Computer Interaction.

Therefore, it seems necessary to propose a road-map towards achieving inclusive HCI based on the accumulated experience by diverse European actors. This could be addressed through a network of multidisciplinary experts, who can bring in their expertise in the different aspects of the issues involved, as well as propose solutions, in order to elaborate a balanced model incorporating different approaches.

1.3 Accessible Human-Machine interaction and Assistive Technology

According to the Wikipedia (http://en.wikipedia.org/wiki/Assistive_technology), Assistive technology or adaptive technology (AT) is an umbrella term that includes assistive, adaptive, and rehabilitative devices for people with disabilities and also includes the process used in selecting, locating, and using them. AT promotes greater independence by enabling people to perform tasks that they were formerly unable to accomplish, or had great difficulty accomplishing, by providing enhancements to or changed methods of interacting with the technology needed to accomplish such tasks. Likewise, disability advocates point out that technology is often created without regard to people with disabilities, creating unnecessary barriers to hundreds of millions of people. Even the makers of AT technologies will often still argue that universal design is preferable to the need for AT and that universal design projects and concepts should be continuously expanded.

People with disabilities usually need assistive devices and programs that have been specifically designed to cover their needs taking into account their capabilities. These devices (e.g. Braille lines) are frequently used to access services or other devices (e.g. computers) that have not being specifically designed for them. The later also have to be designed in a way that does not impose extra barriers to people with disabilities.

Therefore, it is crucial to determine what are the technologies, methodologies and tools that allow the design of accessible and inclusive human interaction systems. These accessibility procedures must be applied to the design of both assistive technologies and main stream technologies in order to avoid any type of barrier or exclusion.

1.4 The human factor

The most distinctive characteristic of the accessible human-machine interaction is the critical importance of the human. Systems designed without taking into account the characteristics, needs, interests, likes, behaviours, etc, of the users are bound to the failure. Unfortunately, generally interaction technologies are designed for the mythical “normal user” ignoring the huge human diversity. One of the reasons is the lack of suitable methodologies and tools to include the users in the whole process of design and development. For this reason, CARDIAC aims to propose a road map on inclusive and accessible human-machine technologies, methodologies and tools that are rooted and centred in the users.

2 HCI and Accessibility

In recent years, many research activities have focused on design that aims to produce universally accessible systems, taking into account special needs of various user groups. These special needs are associated with many user factors, such as impairments of speech, hearing or vision, cognitive limitations, aging, as well as with various environmental factors. Fields that address this problem, such as Usability, Universal Accessibility, Universal Design, or Inclusive Design have been developed as relatively independent domains, but they share many aspects with other HCI disciplines. However, researchers and practitioners are often not aware of interconnections among concepts of universal accessibility and “ordinary” HCI. In view of this situation, [Obrenovic, 2007] show that there is a fundamental connection between multimodal interface design and universal accessibility, and that awareness of these links can help both disciplines. Researchers from these areas may use different terminology, but the concepts they use often have essentially the same meaning.

2.1 Human-Computer Inclusive design

Experience shows that a large amount of HCI systems are designed for someone conceived as the "standard man" leaving out the scope all the people with different physical, sensory or cognitive features. Having in mind that the most common human characteristic is just variety, most designs do not completely fit individual user’s needs.

The problem of matching product features with users' characteristics is most frequently addressed by the own user adapting himself or herself as much as possible to the interface. As a consequence, people that are no able to adapt themselves are simply left out of the possibility of using these products or services.

There exist design techniques and methodologies able to address users’ diversity, by means of modelling and adaptation (Kules, B. User Modeling for Adaptive and Adaptable Software Systems (2000), http://www.otal.umd.edu/UUGuide) . Nevertheless, they are not enough known and used. In fact, the marginalization of large sectors of users was –and frequently is– justified by limitations of technology. Nowadays we know that technology can be designed in a most inclusive way avoiding the inclusion of unnecessary barriers. Inclusive Design aims to consider the needs of all the users in main stream applications and not only in the systems especially designed for people with physical, sensorial or cognitive restrictions.

2.1.1 Advantages of Inclusive Design

Inclusive Design is based on the convincement that humans are naturally very diverse. The partition into "normal users" and "other users" is artificial and the frontiers between both populations are arbitrary. In fact, there are abundant examples where technology eliminated or alleviated these frontiers. Something as simple as glasses –nowadays of common use– allow several people with eyesight restrictions to enhance their vision. More complex technologies, such as computers, give people with motor and speech impairments a way to personal and remote communication, and to control their environment.
Evidently, Inclusive Design has ethic and social fundaments. Universal Accessibility is supported by the convincement that all the human beings have the same rights. In practical terms this means that they should be able to access to the same services and to enjoy the same opportunities. Technological designs that unnecessarily establish barriers to universal use effectively exclude users with physical, sensory or cognitive restrictions.

In addition, to its ethics roots, inclusive technology is highly practical and useful. It frequently has a higher impact over the market because accessible products are directed to a broader population of potential consumers. In fact, people without disabilities usually find inclusive technology easier and more usable. On the other hand, the new ways to interact with mobile and ubiquitous technology frequently require hand and sight free interaction, and as a result they can very much benefit from Inclusive Design. For instance, people wanting to read their email while they drive to work do need auditory interfaces, similarly to several vision impaired people. In addition they will need voice input to enter commands to the system, similarly to many people with severe motor restrictions.

It is frequently argued that Inclusive Design (for instance accessible web design) is not more expensive that standard design. We cannot deny that accessible design requires higher effort from the designer. That means the need of knowledge and experience on this kind of design, longer development periods, etc. Nevertheless, it is proved that accessible products are of higher quality. In fact, usability and accessibility are included as quality measurement figures of merit in a number of software methodologies (for instance, [ISO 2008a] and [ISO 2008b]).

2.1.2 Current barriers for Inclusive Design

Even if Inclusive Design advantages have been frequently acknowledged, there is little advancement in Inclusive Design of commercial ICT products. This may be due to a combination of factors, such as:

  • The lack of awareness of universal design. Numerous HCI designers frequently ignore that they can design for a broader population simply avoiding the inclusion of certain features that put difficulties to the accessibility by a number of users.
  • The lack of knowledge about user needs. Most HCI professionals usually design without having a previous analysis of the users needs. Their designs are frequently based in their own mental model of the task, their own capabilities and likes, etc.

2.1.3 Conditions for Inclusive Design

There are numerous ethical and social risks that must be taken into account when Inclusive Design paradigm is adopted [Abascal 2005]. For instance:

  • User autonomy: Avoid taking automatic decisions about the user without her or his consent.
  • User privacy: Avoid to store, transmit or process data about the user or his/her activities that are not strictly necessary for the current application.
  • User consent: Always ask for the informed consent from the user.
  • Human contact: Compensate the social impact of services that produce isolation.
  • Finally, Inclusive Design is hardly possible without the full participation of the users in the whole design and development process. That means that the users must be present: In all the phases of the process; as full participants; being paid; under a code of conduct for experiments.

2.2 Web Accessibility

It is well known that the web has spread over the last few years in an unexpected way. The Web has become a major part of many people's everyday life since it facilitates the fulfilment of daily tasks related to communication, entertainment, work, study, etc. In addition, the number of eServices that, in many cases, substitute or complement traditionally delivered services.
One of the human groups that may easily suffer exclusion is that of people with disabilities, as in their case many commercial interfaces fail to prove accessible. Different initiatives have been taken in order to avoid this situation, such as inclusive laws promulgated in several countries. However, these efforts are insufficient if technological advancements do not support universal design. For this reason, many national authorities are supporting projects to achieve web accessibility.

  • Even if designers are convinced (or compelled) to create accessible products, they usually have to face a lack of knowledge and experience on accessible design. Therefore, methods, tools and guidelines, are needed to help designers in this difficulty. Guidelines have frequently been used to collect design knowledge and experience. Even if they may present problems, such as incoherence and unreliability, and be difficult to handle (when the set of guidelines is too large), guidelines nevertheless prove to be the best method in order to transmit satisfactory design experiences within large design groups or for to the external world [Nicolle 2001].

For this reason, a crucial advance in web accessibility is the provision of sets of guidelines and tools to apply them. Since web technology is rapidly changing, and web accessibility guidelines have to be frequently updated, appropriate tools must be able to easily modify the existing, or include new, sets of guidelines. The most relevant sets of guidelines are those developed by the Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C). W3C-WAI has established three sets of W3C recommendations to improve the accessibility of the Web. These are:

  • WCAG (Web Content Accessibility Guidelines), which concern how to make Web sites sufficiently accessible so that people with disabilities are able to use them alongside with today’s technologies. The version 2.0 was released on December 2008.
  • ATAG (Authoring Tool Accessibility Guidelines), which provide guidance for software developers in designing authoring tools that produce accessible web content and in creating accessible authoring interfaces. A working draft of ATAG 2.0 was released on July 2010.
  • UAAG (User Agent Accessibility Guidelines), released in December 2002, which concern how to make browsers and multimedia players more accessible, as well as compatible with some of the assistive technology that people with disabilities use. A working Draft of UAAG was released on June 2010.

Nevertheless, guidelines compliance does not guarantee web accessibility. Users and experts evaluations are required to find accessibility barriers that are hardly specified by guidelines.

The strengths of WAI guidelines are both their universal acceptance and the way they are produced (cooperatively, in a limited period of time and in a clear way). However, these broadly accepted and used guidelines are far from being definitively established. As web technology is rapidly evolving, the production of guidelines is a continuous process that periodically offers new versions. However the paramount importance of WAI guidelines cannot be ignored.  In addition to their contribution to web accessibility, WAI guidelines are an incredible pioneering experience of advancement towards Universal Accessibility that has to be taken into account for all the other actions in favour of the accessibility.

2.2.1 Mobile Web

The mobile web has become more widespread as the computing performance of mobile devices and their availability has steadily increased over the last few years. Mobile devices such as PDAs, mobile phones, videogame consoles and a number of home appliances (TV, video, etc.) currently have XHTML (and similar) browsers, thus enabling the advent of the ubiquitous web. Even if these devices have very dissimilar input and output modalities, the fact that several of them have reduced keyboards and small screens causes poor interactive experience. Low input rate, lack of a pointing device and low bandwidth are the key factors that cause a decline in the quality of interaction.

Since mobile devices are used in everyday situations they may cause the so-called situationally-induced impairment and disabilities. For example, users interacting with a touch screen while experiencing turbulence during a flight or texting while hands are busy can be considered temporary impaired. Therefore, there is a strong relationship between web accessibility and the mobile web because the problems encountered while interacting with the Mobile Web can be referred to as accessibility barriers for the able-bodied. Similarly, the existence of an overlap between mobile web usability recommendations and guidelines for physically impaired users is evident. Consequently by applying accessibility-related good practices, navigation in the WWW can be enhanced for a wider audience, reinforcing the experience showing that content accessibility benefits all users.

Therefore, the problems that able-bodied individuals encounter in the Mobile Web and the barriers found by users with physical, sensory or cognitive disabilities while browsing the Desktop Web are related to: Small display size; No support for mark-up, scripting or data formats.

Mobile web guidelines aim at providing developers with guidance to develop web content suitable for mobile devices. Yet, a number of mobile web guidelines refer to specific device features such as screen size, support for particular picture formats or support for pointing device.

Even if there are some concerned developers who follow Design for All principles and methods to ensure that all resources are accessible by everyone, it is not always possible to accomplish the Design for All paradigm, given the diversity of users and access devices that exist. An alternative approach is the personalization of user interfaces, thus interfaces should be adapted considering users’ specific needs. Systems which adapt websites to handheld device constraints (screen size, etc.) have been developed. Others adapt a determined type of web pages to users with visual impairments. Additionally, users’ preferences, as well as access device’s features are taken into account for adaptation purposes. Nevertheless, none of them considers a wide range of disabilities, and they are focused on adapting user interfaces of a determined type of web pages or standalone applications.

2.2.2 The adaptive web

The possibility to access to Information and Communication technologies is frequently challenged by diverse factors such as (a) characteristics of the user (including physical, sensory or cognitive restrictions and literacy problems); (b) limitations of the equipment (obsolete devices or applications, narrow band access, small displays or keyboards); (c) barriers imposed by context of use (noisy or dark environments; performing parallel activities, such as working or driving). Frequently these issues have a deep root in social causes such as poverty and limited access to education.

The design of effective adaptive user interfaces may help in closing the digital divide. These interfaces usually allow automatic adaptation to the user considering user’s own characteristics, the context where the interaction is carried out, the task to be performed, the technology being used, etc. These methods may give people suffering restrictions higher opportunities to access to ITCs.

Currently, classic user modelling techniques are being extended to model the environment and the technology involved in the process. The supporting paradigms are evolving to the use of advanced technologies, such as ontologies, which are extremely useful to store and process the information for modelling. In addition, machine learning techniques allow the compilation of user models with information obtained by data mining of interaction logs. But, while current technology allows for more effective personalization systems, some issues, such as privacy impact, condition the applicability of these techniques.

2.2.3 Web personalization

The main objective of Web personalization is to adapt the browsing or navigation, the presentation and the contained information of the web pages to the needs of the user without him or her doing an explicit demand. The goal is to enhance the speed and the achievement using the web and to decrease the physical and cognitive effort to perform it. Therefore, adaptation becomes especially helpful when the users have special needs.

Web personalization seeks for the adaptation to the user in three areas [Brusilowsky 2007] that in the case of users with disabilities are crucial in order to easy and speed up the interaction:

  • Navigation. Browsing pages is the most common task when using the web. This task can be slowed, for instance, if the page contains a high number of links that are not interesting for the user. Knowing the objectives and interests of the user the browser can make easier the navigation task giving more emphasis to the most interesting or probable links for a specific user. The inclusion of a specific navigation menu with selected links is another navigation facilitating possibility.
  • Presentation: The presentation of a web page can be adapted to the specific needs of each user applying cascading stile sheets (CSS ). The most convenient stile sheet(s) may be stored in the (static) model of the user.
  • Content. Even if it is not convenient to automatically change the content of a web page, some inclusions may enhance its readability. For instance, the automatic inclusion of text captions in simple language used to explain longer and more difficult texts are useful for people with reading difficulties.

Web personalization is based on modelling user features such as interests, navigational behaviour, preferences, physical sensory or cognitive restrictions, etc. The information stored in the model is used to make assumptions about the current user-system interaction that allows adapting the system to the actual user needs or preferences. User adaptation methods have been frequently adopted by intelligent interface designers to adjust the interface to the user (contrarily to the usual situation where the user adapts him or herself to the interface). These models could be directly designed by experts in the area (rule based approach), they can be built based on previous information from that user such as the logs of previous navigations (content based approach) or they can be induced from information about groups of users with similar characteristics (collaborative approach). The first approach is somehow static and requires previous knowledge of the users and redesigning when new behaviours appear in the users. However, the last two approaches focus on automatic techniques for user characterization. Currently the user component is usually built by means of ontologies that allow storing, manipulating, and extracting assumptions from data about the user, its context, tasks, etc. [Miñón 2010].

2.2.4 Data mining for web personalization

In order to be able to model the user, the modelling component must collect information about a number of observable parameters such as interest, characteristics, etc. This information can be requested to the user in a previous session, but this is annoying, disruptive and can produce false assumptions. Another option is to collect this information while the user is accessing the web. In this way the system can learn its interests, likes, etc. Learning from the own interaction allows maintaining a dynamic profile of the user, avoiding the application of all assumptions when the interest, characteristics or circumstances of the user change.

Data mining for web personalization has many advantages. It is not disruptive, is based in statistical data obtained by real navigation exercises (decreasing the possibility of false assumptions) and is itself adaptive (when the characteristics of the user change, collected data allows the automatic change of the interaction schema). When the user is a person with physical, sensory or cognitive restrictions, data mining is the easiest (and frequently almost the only) way to obtain information about the uses of the person.

Data mining in this context has also some drawbacks. The most important one is its impact over privacy, due to the need of storing large quantities of data about the users. Diverse laws in different countries protect user rights for privacy. Even if it is difficult to reach a balance among privacy and personalization, some appealing proposals have been recently published.

2.3 Human interfaces for accessible Ambient Intelligence system

The advances in networking, computing and sensing technologies allow the design of intelligent environments able to give support to people located inside them. The Ambient Intelligence paradigm benefits from ubiquitous and wearable computers, communicated by wireless networks with static computers –which can be also connected to wired networks–, that are able to process enormous quantities of contextual information coming from networks of sensors. This technological infrastructure will allow the deployment of intelligent applications that proactively give support to the users [Streitz 2006].
Ambient Intelligence obviously provides an extraordinary opportunity to develop assistive environments for people with sensory, physical or cognitive restrictions due to aging, disability, illness, etc. All these intelligent environments perform several supportive activities that are usually ignored by the user, such as adjusting the temperature, humidity, lights, etc., or verifying the safety of gas, electricity or water installations. In addition, intelligent environments have to communicate with the user to provide information or to request commands. The user interfaces are supposed to be as natural as possible, allowing a communication similar to the interaction between humans. That means that the system should be able to produce voice messages –and maybe to display some of them via wall screens or data glasses– and to recognize natural language and gestures. Nevertheless some of these communication methods may not be appropriate for people with specific sensory or cognitive restrictions.

2.3.1 Accessible AmI supporting technology

Elderly and disabled people belong to a segment of the population that would profit very much from Ambient Intelligence if it is accessible. This is only possible if accessibility barriers are early detected in the evolution of the Ambient Intelligence concept and opportune standardization measures are provided.

Autonomy and quality of life of elderly and disabled people living in smart private or public homes designed under the Ambient Intelligence paradigm can experience significant enhancements due to the increased support received from the environment. This support includes facilities for environmental control, information access, communication, monitoring, etc., built over diverse technologies and using different operation ways. Nevertheless, users can find accessibility barriers frequently related to the diverse user interfaces with heterogeneous devices and procedures. These problems include both, physical difficulties to handle the devices, and cognitive barriers to understand use procedures and navigation. As a result, accessible unified interfaces to control all the appliances and services are needed. This is only possible if the network technology used for smart homes is able to support interoperability and systems integration. Therefore, the needs of senior and disabled users can only be provided by means of interoperable systems in an integrated intelligent environment. Consequently, only a convergence policy based on inclusive design guidelines and standards can guaranty the accessibility of the future intelligent ambient [Sevillano 2004] [Emiliani 2008].

2.3.2 Accessible adaptive interfaces for AmI

As previously mentioned, the growing ubiquitous computing paradigm allows the provision of context-aware services to mobile users. In addition to the usual computing requirements, these environments entail wireless network infrastructures and special management software, usually called middleware. When a mobile computing device (smart phone, PDA, etc.) enters into an Ambient Intelligence environment the middleware establishes the communication with the local network in a way that is transparent to the user. After the “discovering” and “presentation” phases, the available local services are offered to the user. In order to be operated some of them may require a specific user interface that is downloaded to the user’s mobile device.

This type of environment can be extremely helpful for people with disabilities who have mobile devices adapted to their characteristics. In this way, using their own device they can access several local services that can otherwise be inaccessible to them, such as ATMs, vending machines, information kiosks, smart home appliances, etc. This is only possible if the downloaded user interface is rendered to the mobile device in an accessible mode. The large variety of user characteristics and restrictions (due to the broad range of disabilities) and the peculiarities of the devices used by them makes it necessary to adapt the “basic” user interface supplied by the service provider to the specific needs of the user and his or her device. Therefore the system has to automatically generate user interfaces adapted to the features and preferences of users with disabilities. To automatically adapt the interface to the user characteristics, it is necessary to take into account what the most suitable communication modalities are for each user, mapping them to the appropriate media.

Ubiquitous systems handle a huge quantity of information that can be used to infer knowledge about the user, the environment and the tasks. Modelling this knowledge would contribute to enhancing the generation of adapted user interfaces. Ubiquitous Computing itself frequently includes user modelling and personalization as a goal, in order to take into account the human context. This goal requires a component that can manage the adaptation of the information resources and make the interaction comfortable for each user of the ubiquitous environment.

2.4 Interfaces for robot control

Mobile Robotics has experienced a notable development in recent years. For instance, sensors are more and more reliable and accurate at lower prices. In addition, processors are also more powerful and memory availability is larger and cheaper. For these reasons, it is possible nowadays to speak about “consumer robotics”. Similarly to the evolution of personal computers, robots are finding new applications in the home, outside of the factories. One of the most promising fields among the non-industrial applications of robots is Assistive Robotics. Assistive Robotics is proposing new ways of supporting people with motor restrictions to develop tasks that were previously impossible for them. Among the diverse applications that are being developed, assisted mobility and manipulation stand out [Abascal 2008a].

Augmentative and Augmentative Mobility, AAM, (similarly to Augmentative and Alternative Communication ) attempts to provide people with methods and devices to enhance or restore their mobility. The application of the advancements in Mobile Robotics to AMM allowed the design of very advanced assisted mobility systems. Among them the most sophisticated are smart wheelchairs [Abascal 2008b]. Smart wheelchairs are intended for people with severe motor restrictions that experience great difficulty in driving standard electric wheelchairs. They are usually provided with diverse types of sensors and embedded computers that receive information from the sensory system, handle the interaction with the user and control the motors through the power stage. The number and quality of the sensors determine the accuracy of the control. For this reason many experimental Smart Wheelchairs are provided with extremely advanced and expensive sensors that convert them into impressive mobile robots but are too expensive and sophisticated to be marketed.

The interaction between the user and the wheelchair is again a key factor. As previously mentioned, users of smart wheelchairs are people with severe motor restrictions that impede the use of standard input devices. Therefore, the design of interfaces for AAM has also to take into account specific guidelines to satisfy the needs of the users.

Another important human need is to manipulate objects in the surroundings. There are specific technologies applicable to people with severe motor disabilities or to people with amputations. Light articulated arms come from the adaptation of articulated industrial manipulators to allow people with severe hand movement restrictions to grasp and move objects. It is evident that is not possible just to use industrial robots at home. There are problems of size, height, security (the user and the robot share the work space), etc... Nevertheless, the most important barrier is human-robot interaction. Robots are designed to handle objects based on their position and orientation, using diverse types of coordinates. Users describe objects in terms of names, properties (colour, shape, size...), function, etc. In the user’s mind positions are usually related to other objects or to the room. It is not expected that a user should have to give numeric parameters with the position, orientation and size of the object to be manipulated. Therefore, intelligent mediator applications are necessary to understand object description in natural language and to translate this into coordinates.

2.4.1 Requirements of the Human-Robot Interface

Some basic principles have to be taken into account in designing the interface to control both robotic assistant devices and intelligent environments. The first one is the rehabilitation goal. Numerous people with disabilities are able to enhance their cognitive abilities, personal attitudes and social integration when they are provided with adequate user interfaces. To this end the interface must encourage the use of all the capabilities of the user, and avoid taking decisions on behalf of the user when it is not absolutely necessary.

For instance, in the case of autonomous smart wheelchairs, they are able to automatically navigate requiring little or no interaction from the user. After the destination is somehow specified, the wheelchair is able to take all the necessary decisions to arrive at the selected place. Although this procedure is very convenient for people with extreme motor restrictions, many users have some remaining abilities that could be lost if they are not used. These abilities may even be enhanced when they are trained. Therefore the interface must facilitate, as much as possible, user participation in order to enhance their cognitive abilities, personal attitudes and social integration. In addition, the user must feel that he or she is the one who controls the device in order to avoid frustration and passivity That includes ease of switching between automatic/assisted/manual functioning.

Safety and reliability are also important requirements. Several of these systems interact with the environment in various ways that could be dangerous in the case of failure or malfunction. The designer must ensure that the system is safe, reliable and fault tolerant.

Another key issue is the final price. Inexpensive solutions are needed to prevent unaffordable systems. In the case of AAM, that means using cheap sensors (for example, infrared and ultrasonic sensors instead of laser, to measure distances). Currently processors are cheap and the inferior quality of the sensors can be balanced by a much greater processing capacity. Moreover, since most intelligent wheelchairs are built on commercial electric wheelchairs, carrying out any major changes in order to facilitate its future potential marketing by the industry without making large investments should be avoided.

3 A summary of Current Research Trends in HCI Design

3.1 Some Design Frameworks and tools

Modern user interface builders provide graphical environments for user interface prototyping, usually following a WYSIWYG (“What You See Is What You Get”) design paradigm, and offering graphical editing facilities that allow designers to perform rapid prototyping visually. Such editors may be standalone or embedded in integrated environments (IDEs), i.e., programming environments which allow the direct development of the application functionality for the created prototypes. Commonly used IDEs are Microsoft Visual Studio, NetBeans, and Eclipse. IDEs are very popular in application development because they greatly simplify the transition from design to implementation, thus speeding up considerably the entire process, while also supporting look-and-feel consistency through the availability of common sets of UI widgets.
Research efforts related to frameworks and tools for user interface design include a reference framework proposed by Calvary, Coutaz and Thevenin [Calvary 2001], which follows a model-based approach and that structures the development process of plastic user interfaces. Based on this proposed model, a design tool, named ARTStudio, has been implemented to support the development process. TERESA is another tool, which supports the design and development of nomadic applications, providing general solutions that can be tailored to specific cases [Mori 2003]. Through TERESA designers can either specify the appearance of common UI elements for the supported platforms or even modify some general design assumptions.

Several research efforts have suggested a variety of components and tools to facilitate the development of user interfaces capable of adaptation, including accessibility features:

  • MENTOR, is a tool providing (a) practical integrated support for all phases of adaptation design, through appropriate editing facilities; (b) practical support for a 'smooth transition' from design to development through the availability of automated verification mechanisms for the designed adaptation logic, as well as the automated generation of 'ready-to-implement' interface specifications; and (c) support for the progressive accumulation of design cases and of the related design experience and knowledge, in particular regarding adaptation [Antona 2006].
  • EAGER is a toolkit built to support WUI (Web User Interface) adaptation and facilitate the design of web interfaces that can adapt to the diversity of the target user population [Doulgeraki 2009]. By means of EAGER, a developer can produce Web portals that have the ability to adapt to the interaction modalities, metaphors and UI elements most appropriate to each individual user, according to profile information containing user and context specific parameters.
  • On the other hand, MAID is a multi-platform accessible interface design framework, facilitating the development and customization (through skins) of structured User Interfaces (templates) through the use a set of predefined UI components (widgets), while promoting reusability and supporting dynamic content population [Korozi 2009].


3.2 Current research trends

Research efforts in recent years have elaborated comprehensive and systematic approaches to user interface adaptations in the context of Universal Access and Design for All [Stephanidis  2001]. The Unified User Interfaces methodology was conceived and applied [Savidis 2004] as a means to efficiently and effectively ensure, through an adaptation-based approach, the accessibility and usability of User Interfaces to users with diverse characteristics, supporting also technological platform independence, metaphor independence and user-profile independence. In such a context, automatic UI adaptation seeks to minimize the need for a posteriori adaptations and deliver products that can be adapted for use by the widest possible end user population (adaptable user interfaces).

This implies the provision of alternative interface manifestations depending on the abilities, requirements and preferences of the target user groups, as well as the characteristics of the context of use (e.g., technological platform, physical environment). The main objective is to ensure that each end-user is provided with the most appropriate interactive experience at run-time.
In more detail, a unified user interface comprises a single (unified) interface specification that exhibits the following properties:

  • It embeds representation schemes for user- and usage-context- parameters and accesses user- and usage-context- information resources (e.g., repositories, servers), to extract or update such information.
  • It is equipped with alternative implemented dialogue artefacts appropriately associated to different combinations of values for user- and usage-context- related parameters.
  • It embeds design logic and decision making capabilities that support activating, at run-time, the most appropriate dialogue patterns according to particular instances of user- and usage-context- parameters, and is capable of interaction monitoring to detect changes in parameters.

As a consequence, a unified user interface realises:

  • User-adapted behaviour (user awareness), i.e., the interface is capable of automatically selecting interaction patterns appropriate to the particular user.
  • Usage-context adapted behaviour (usage context awareness), i.e., the interface is capable of automatically selecting interaction patterns appropriate to the particular physical and technological environment.

At run-time, the adaptations may be of two types:

  • adaptations driven from initial user- and context- information known prior to the initiation of interaction, and
  • adaptations driven by information acquired through interaction monitoring analysis.

The former behaviour is referred to as adaptability (i.e., initial automatic adaptation) reflecting the interface’s capability to automatically tailor itself initially to each individual end-user in a particular context. The latter behaviour is referred to as adaptivity (i.e., continuous automatic adaptation), and characterizes the interface’s capability to cope with the dynamically changing or evolving user and context characteristics.

3.3 HCI and AmI

While many problems connected with interaction with the present Information Society are actually linked to a suitable structuring of information and an accessible human system interface, integration within the ambient intelligence environment is much more complex, due to the interplay of different levels, e.g. the physical level with a multiplicity and heterogeneity of intelligent objects in the environment and their need for a continuous and high-speed connection, the level of identification and consideration of the variety of contexts of use, and the level of elicitation of the diversity of user goals and help in their fulfilment [Emiliani 2008].

Thus, in the context of the emerging paradigm of AmI, current research trends have to address:

  • The diversity in the User Population
  • Diversity in the enabling technologies for AmI
  • The need for research on the Lifecycle of User Interfaces (requirements in AmI Environments, Design for All, Development requirements, user experience evaluation)
  • User Interface Development (architectures, Components, Tools) itself
  • Interaction Techniques and Devices
  • The variety of Application Domains
  • A number of non-technological issues (social, ethical, legal issues, privacy, security)

Ambient Intelligence is defined according to its properties as technology which is:

  • Embedded in the physical and social environment of people;
  • Context Aware - employing machine perception a model of activities of people and their social and physical context can be obtained;
  • Personalized - addressing each user as an individual person;
  • Adaptive to context and activities of the person;
  • Anticipatory - predicting user's needs and taking action to support them.

In that respect, the different approaches to Ambient Intelligence are themselves challenging areas for research in HCI, such as: 

  • Ubiquitous Computing and communication
  • The disappearing computer and Calm Technology
  • Pervasive and Embedded Everywhere Computing
  • Internet of Devices and Web of Things
  • Ubiquitous Networking
  • Ambient Computing and Ambient Displays
  • Tangible Interfaces

With regards to User Interfaces in particular, an interesting approach could be based on the introduction of the Ambient User Interfaces (AmUIs) paradigm. In contrast to typical graphical user interfaces (GUIs), which are always instantiated on a computer screen, AmUIs can take advantage of the available Ambient Intelligence Infrastructure, in order to support interaction that is tailored to the current needs and characteristics of a particular user and context of use. Thus, they could be multimodal and distributed in the environment (e.g., employ the TV screen and stereo speakers to provide output, and get input through both speech and gestures). These interfaces could allow the interaction between humans and the ambient technological environment in an efficient, effective and intuitive way which also guarantees their well-being, privacy and safety, while on the other hand they could creatively combine the available, dispersed computing devices in order to provide useful, added-value, services. This could support seamless, high-quality, unobtrusive, and fault-tolerant user interaction, by creating software frameworks for developing and orchestrating ambient interactions, and by designing and developing useful ambient  interactive systems that cater in the best possible way for the real needs of their users.

3.4 Emerging Issues

Lastly, a number of emerging key issues that arise from the evolution of the Information Society towards Ambient Intelligence environments have to be considered [Stephanidis 2009]. These include:

  • The investigation of human characteristics, abilities, and requirements in the context of AmI
  • Suitable approaches to non-functional characteristics such as accessibility, privacy, security, safety
  • Suitable models of the context of use
  • Appropriate interaction devices and techniques for diverse users and contexts of use,
  • Interaction Design for continuous and implicit interaction
  • Elaboration of design methods suitable for very complex interactive environments,
  • Mechanisms for interaction adaptation,
  • A balance of policy, standardisation and legislation intervention

4 References

 [Abascal 2005]        Abascal J., Nicolle C. Moving towards inclusive design guidelines for socially and ethically aware HCI. Interacting with Computers. 17(5), 2005. Pp. 484-505. 

[Abascal 2008a]      Abascal J. Users with Disabilities: Maximum Control with Minimum Effort. In: Perales F. J., Fisher R.B. (Eds.) AMDO 2008, LNCS 5098, Springer-Verlag, Berlin, 2008. Pp. 449–456.

[Abascal 2008b]      Abascal J., Bonail B., Cagigas D., Garay N., Gardeazabal L. Trends in Adaptive Interface Design for Smart Wheelchairs. In: Lumsden, J. (Ed.) Handbook of Research on User Interface Design and Evaluation for Mobile Technology. Idea Group Reference, Pennsylvania, 2008.

[Antona 2006]         Antona M., Savidis A., Stephanidis C. A Process–Oriented Interactive Design Environment for Automatic User Interface Adaptation. International Journal of Human Computer Interaction, 20 (2), 2006. Pp. 79-116.

[Bergman 1995]      Bergman E., Johnson E. Towards Accessible Human-Computer Interaction. In Nielsen J. (ed.) Advances in Human-Computer Interaction, V. 5, 1995. https://labs.oracle.com/features/tenyears/volcd/papers/Johnson.pdf

[Brusilowsky 2007]          Brusilowsky P. Adaptive Navigation Support. In: Brusilovsky P., Kobsa A., Nejdl W. (Eds.) The Adaptive Web. Methods and Strategies of Web Personalization.  Springer-Verlag, Berlin-Heildeberg, 2007. Pp. 263-290.

[Calvary 2001]         Calvary G., Coutaz J., Thevenin D. A Unifying Reference Framework for the Development of Plastic User Interfaces. In Proc. Eng. Human-Computer Interaction Conf, 2001. Pp. 173-192.

[Doulgeraki 2009]  Doulgeraki C., Partarakis N., Mourouzis A., Stephanidis C. Adaptable Web-based user interfaces: methodology and practice. eMinds, Vol 1, No 5 (2009).

[EC ISTP]                    European Commission – Information Society Thematic Portal: Overview of ongoing Projects FP6 – FP7 – CIP, available at: http://ec.europa.eu/information_society/activities/einclusion/research/projects/index_en.htm

[Emiliani 1999]         Emiliani, P. L., Hine N., Stephanidis, C. (1999), Definition of Models of Interactions, IPSNI Project, Deliverable 2.2.

[Emiliani 2008a]      Emiliani P. L., Billi M., Burzagli L., Gabbanini F.  Design for All in the Ambient Intelligence Environment. Computers Helping People with Special Needs. LNCS Vol. 5105, Springer-Verlag, Berlin-Heildeberg, 2008.

[Emiliani 2008b]      Emiliani P. L., Burzagli L., Billi M., Gabbanini F., Palcheti E. Report on the impact of technological developments on eAccessibility. DfA@eInclusion Coordination Action, 2008. http://www.dfaei.org/deliverables/D2.1.pdf

[Emiliani 2009a]      Emiliani, P., L., Burzagli, L., Billi, M., Gabbanini, F., Palchetti, E. (2009). Report on the impact of technological developments on eAccessibility. DfA@eInclusion Deliverable D2.1. Retrieved available at: http://www.dfaei.org/deliverables/D2.1.pdf

[Emiliani 2009b]      Emiliani, P., L., Aalykke, S., Antona, M., Burzagli, L., Gabbanini, F., Klironomos, I. (2009). Document on necessary research activities related to DfA. DfA@eInclusion Deliverable D2.6. available at: http://www.dfaei.org/deliverables/D2.6.pdf

[Emiliani 2011]         Emiliani P.L. (2001), “Anyone, Anywhere Access to Community-Oriented Services”, 1st International Conference on “Universal Access in Human-Computer Interaction”, New Orleans, pp. 803 – 807.

[Fink 1997]                Fink, J., Kobsa, A., & Nill, A. (1997), “Adaptable and Adaptive Information Access for All Users, Including the Disabled and Elderly”, In A. Jameson, C. Paris, & C. Tasso (Eds.), Proceedings of the 6th International Conference on User Modelling (UM '97), Sardinia, Italy (pp. 171-173), New York: Springer-Verlag.

[ISO 2008a]               ISO 9241-20. 2008. Accessibility guidelines for information/communication technology (ICT) equipment and services.

[ISO 2008b]              ISO 9241-171. 2008. Ergonomics of human-system interaction -- Part 171: Guidance on software accessibility.

[Kobsa 1995]            Kobsa, A., & Pohl, W. (1995), “The user modelling shell system BGP-MS”, User Modelling and User-adapted interaction, 4 (2), 59-106.

[Korozi 2009]           Korozi M., Leonidis A., Margetis G., Stephanidis C. MAID: a Multi-platform Accessible Interface Design Framework. In: C. Stephanidis (Ed.) Universal Access in Human-Computer Interaction -Applications and Services– Vol. 7 Procs. of the 13th Int. Conf. on Human-Computer Interaction (HCI International 2009), San Diego, CA, USA, 2009. Pp. 725-734.

[Miñón 2010]           Miñón R., Aizpurua A., Cearreta I., Garay N., Abascal J. Ontology-Driven Adaptive Accessible Interfaces in the INREDIS project. In: Procs. of the Int. Workshop on Architectures and Building Blocks of Web-Based User-Adaptive Systems, Hawaii. CEUR Workshop Proceedings Vol. 609, 2010. Pp. 37-39.

[Mori 2003]              Mori G., Paterno F., Santoro C. Tool Support for Designing Nomadic Applications. In: Proc. of 7th ACM Int. Conf. on Intelligent User Interfaces IUI’2003. ACM Press, New York, 2003. Pp. 141–148.

[Nicolle  2001]         Nicolle C., Abascal J. (Eds.) Inclusive design guidelines for HCI. Taylor & Francis, London, 2001.
[Obrenovic 2007]   Obrenovic Z., Abascal J., Starcevic D. Universal Accessibility as a Multimodal Design Issue.  Comms. of the ACM. 50(5), 2007. Pp. 27-29.

[Savidis 2004]          Savidis A., Stephanidis C. Unified User Interface Design: Designing Universally Accessible Interactions. Interacting with Computers, 16(2), 2004. Pp. 243-270.

[Savidis 2005]          Savidis A., Stephanidis C. Distributed Interface Bits: Dynamic Dialogue Composition from Ambient Computing Resources. Personal and Ubiquitous Computing, 9(3), 2005. Pp. 142-168.

[Sevillano 2004]      Sevillano J. L., Falcó J., Abascal J., Civit-Balcells A., Jiménez G., Vicente S., Casas R. On the Design of Ambient Intelligent Systems in the Context of Assistive Technologies. In: Miesenberger, K., Klaus, J., Zagler, W., Burger, D. (eds.) ICCHP 2004. LNCS vol. 3118, Springer, Heidelberg, 2004. Pp. 914–921.

[Stephanidis 1995]           Stephanidis, C., Mitsopoulos, Y. (1995), “INTERACT: An interface builder facilitating access to users with disabilities”, HCI International '95, Tokyo, Japan,, pp. 923-928.

[Stephanidis 1997]           Stephanidis, C., Savidis, A., Akoumianakis, D. (1997), “Unified Interface Development: Tools for Constructing Accessible and Usable User Interfaces”, Tutorial Notes 13, 7th International Conference on “Human-Computer Interaction” (HCI International ’97), San Francisco, California, USA.

[Stephanidis 1998]           Stephanidis, C., Salvendy, G., Akoumianakis, D., Bevan, N., Brewer, J., Emiliani, P. L., Galetsas, A., Haataja, S., Iakovidis, I., Jacko, J., Jenkins, P., Karshmer, A., Korn, P., Marcus, A., Murphy, H., Stary, C., Vanderheiden, G., Weber, G., Ziegler, J. (1998a), “Towards an Information Society for All: An International R&D Agenda”, International Journal of Human-Computer Interaction, vol. 10(2), pp. 107-134.

[Stephanidis 2001a]        Stephanidis C. The concept of Unified User Interfaces. In: Stephanidis C. (Ed.) User Interfaces for All - Concepts, Methods, and Tools. Mahwah, NJ: Lawrence Erlbaum Associates, 2001. Pp. 371-388.

[Stephanidis 2001b]        Stephanidis, C. (ed.) (2001), “User Interfaces for All – Concepts, Methods and Tools”, Mahwah, NJ: Lawrence Erlbaum Associates.

[Stephanidis 2009]           Stephanidis C. Emerging Challenges. In: Stephanidis C. (Ed.) The Universal Access Handbook. Taylor & Francis 2009. Pp. 61-1 - 61-9.

[Streitz 2006]           Streitz, N. A. From Human–Computer Interaction to Human–Environment Interaction: Ambient Intelligence and the Disappearing Computer. In: C. Stephanidis and M. Pieper (Eds.) Universal Access in Ambient Intelligence Environments. Procs. of the 9th ERCIM Workshop on User Interfaces for All. Königswinter, Germany, 2006. Pp. 3-13.

[Vogel 2004]            Vogel D., Balakrishnan R. Interactive Public Ambient Displays: Transitioning From Implicit to Explicit, Public to Personal, Interaction with Multiple Users. In: Procs. of the ACM Symp. on User Interface Software and Technology (UIST’04), 2004. Pp. 137–146.

4.1 Further readings

 [Ahonen 2008]       Ahonen P., Alahuhta P., Daskala B., Delaitre S., De Hert P., Lindner R, Maghiros I., Moscibroda A., Schreurs W., Verlinden M. Dark scenarios. In: Safeguards in a World of Ambient Intelligence. The International Library of Ethics, Law and Technology, Vol. 1, 2010. Pp. 33-142.

[Ferscha 2006]       Ferscha, A., Resmerita, S., Holzmann, C. Human Computer Confluence. In C. Stephanidis and M. Pieper (Eds.) Universal Access in Ambient Intelligence Environments. Procs. of the 9th ERCIM Workshop on User Interfaces for All. Königswinter, Germany, 2006. Pp. 14-27.

[ISTAG 2001]            IST Advisory Group. “Scenarios for ambient intelligence in 2010” Final Report. Compiled by K. Ducatel, et al. Feb-2001. EC. Brussels, 2001.

[Klironomos 2006] Klironomos I., Antona M., Basdekis I., Stephanidis C. Promoting Design for All and e-Accessibility in Europe. Universal Access in the Information Society, 5(1), 2006. Pp. 105-119.

[Murugesan 2006] Murugesan, S. Understanding Web 2.0. IT Professional 9(4), 2006. Pp. 34-41.

[Roe 2006]                Roe, P. (Ed.) Towards an Inclusive Future: Impact and Wider Potential of Information and Communication Technologies. COST219ter, Brussels, 2006.

[Savidis 2010]          Savidis, A., Stephanidis, C. A unified software architecture for user interface adaptation. In:  C. Stephanidis (Ed.) The Universal Access Handbook. CRC Press, 2010. Pp. 21-1/21-17.

[Stephanidis 1998]           Stephanidis C., Akoumianakis D.,  Sfyrakis M.,  Paramythis A. Universal accessibility in HCI: Process-oriented design guidelines and tool requirements. In: Stephanidis C., Waern A. (Eds.) Procs. of the 4th ERCIM Workshop on User Interfaces for All, 1998. [http://ui4all.ics.forth.gr/UI4ALL-98/proceedings.html]

[Stephanidis 1999]           Stephanidis, C., Salvendy, G., Akoumianakis, Arnold, A, D., Bevan, N., Dardailler, D., Emiliani, P. L., Iakovidis, I., Jenkins, P., Karshmer, A., Korn, P., Marcus, A., Murphy, H., Oppermann, C., Stary, Tamura, H., Tscheligi, M., Ueda, H.,G., Weber, G., Ziegler, J. Toward an Information Society for All: HCI challenges and R&D recommendations. International Journal of Human-Computer Interaction, 11(1), 1999. Pp. 1-28