• Small text Medium text Large text
  • Default colours Black text on white background Yellow text on black background

D3.1 Report with background material needed to support the SDDP-2 Meeting

An Introduction to the Key Issues Relating toAccessible User Interfaces

Iosif Klironomos
Foundation for Research and Technology
Hellas, Institute of Computer Science

Julio Abascal
Laboratory of HCI for Special Needs
University of the Basque Country/Euskal Herriko Unibertsitatea

Ilse Bierhoff


The CARDIAC Project is a Coordination Action funded by the EU’s 7th Framework Programme which aims to improve the overall success of Challenge 7, ICT 2009 7.2 ‘Accessible and Assistive ICT’ by preparing research agenda roadmaps that highlight research priorities that will favour eAccessibility.

It aims to do this by looking into the wide range of issues that play a role in the availability of accessible and assistive ICT. The issues range from future research priorities, development and design aspects, right through to making the business case and the adoption or non-adoption of a particular technology or service.

In recent years, a large number of international projects had to address the need for guaranteeing accessibility and usability in Human Computer Interaction. To this end, a number of diverse approaches, methodologies and technologies have been proposed. Many research and development activities have been carried out on different aspects of accessibility of ICT equipment and services with an Assistive Technology (AT) approach, and more recently, the Design for All approach has been explored.

Positive results have been achieved combining both approaches. In particular, accessibility problems of specific groups of users have been addressed through AT based adaptations, and systematic Design for All approaches have been elaborated and applied in various domains at a research level.

Still, the field is currently in need of a breakthrough towards the adoption in practice of design approaches, based on the accumulated knowledge, leading to accessible and usable inclusive interfaces.

One of the main objectives of the CARDIAC project is to generate a roadmap identifying issues in the area of Inclusive Human Computer Interaction (HCI) research and development priorities.  This roadmap will be a document outlining essential areas and subsequent types of research missing that could facilitate the development of inclusive HCI technologies.

The purpose of this document is to provide a background and context to the second Structured Dialogue Design Process (SDDP) co-laboratory of the CARDIAC Project which is scheduled for the 28th – 29th of June in Donostia-San Sebastian, Spain. The purpose of this event is to generate a roadmap in response to the Triggering Question "What type of research is missing that could facilitate development of inclusive HCI".

When considering the various methodologies for generating roadmaps, the Structured Dialogic Design Process methodology was selected due to its robustness and efficiency in gathering the collective wisdom of a wide range of different stakeholders. The SDDP methodology supports democratic and structured dialogue among a group of stakeholders and is especially effective in resolving multiple conflicts of purpose and values, and in generating consensus on organizational and inter-organizational strategy. A full description of the methodology and how exactly the methodology will guide the process of generating a roadmap is presented in a separate document.

Scope of the SDDP in Inclusive Human-Machine Interaction

Several research activities in the field of Ambient Assisted Living (AAL) focus on more user involvement in the design process. The ISO standard 13407 Human-centred design process for interactive systems provides guidance on human-centred design activities throughout the life cycle of interactive computer-based systems. However also other research methods are available, for instance participatory and co-design. These approaches have in common that they all express the belief that all people have something to offer to the design process.

Moreover, adaptivity/intelligence on the one hand, and the analysis of the implications, from an eAccessibility perspective, of the emerging Ambient Intelligence (AmI) paradigm (with a clear orientation to creating "natural" interfaces) on the other, are becoming increasingly important aspects. The main difficulty lies in understanding and utilising the whole range of possibilities for inclusive Human-Computer Interaction.

Therefore, it seems necessary to propose a road-map towards achieving inclusive HCI based on the accumulated experience by diverse European actors. This could be addressed through a network of multidisciplinary experts, who can bring in their expertise in the different aspects of the issues involved, as well as propose solutions, in order to elaborate a balanced model incorporating different approaches.

Accessible Human-Machine interaction and Assistive Technology

According to the Wikipedia , Assistive technology or adaptive technology (AT) is an umbrella term that includes assistive, adaptive, and rehabilitative devices for people with disabilities and also includes the process used in selecting, locating, and using them. AT promotes greater independence by enabling people to perform tasks that they were formerly unable to accomplish, or had great difficulty accomplishing, by providing enhancements to or changed methods of interacting with the technology needed to accomplish such tasks. Likewise, disability advocates point out that technology is often created without regard to people with disabilities, creating unnecessary barriers to hundreds of millions of people. Even the makers of AT technologies will often still argue that universal design is preferable to the need for AT and that universal design projects and concepts should be continuously expanded.

People with disabilities usually need assistive devices and programs that have been specifically designed to cover their needs taking into account their capabilities. These devices (e.g. Braille lines) are frequently used to access services or other devices (e.g. computers) that have not being specifically designed for them. The later also have to be designed in a way that does not impose extra barriers to people with disabilities.

Therefore, it is crucial to determine what are the technologies, methodologies and tools that allow the design of accessible and inclusive human interaction systems. These accessibility procedures must be applied to the design of both assistive technologies and main stream technologies in order to avoid any type of barrier or exclusion.

The human factor

The most distinctive characteristic of the accessible human-machine interaction is the critical importance of the human. Systems designed without taking into account the characteristics, needs, interests, likes, behaviours, etc, of the users are bound to the failure. Unfortunately, generally interaction technologies are designed for the mythical “normal user” ignoring the huge human diversity. One of the reasons is the lack of suitable methodologies and tools to include the users in the whole process of design and development. For this reason, CARDIAC aims to propose a road map on inclusive and accessible human-machine technologies, methodologies and tools that are rooted and centred in the users.

A review of the advancements in the field of inclusive HCI from past and current European projects

Universal Access implies the accessibility and usability of information and telecommunications technologies by anyone at any place and at any time and their inclusion in any living context. This can be considered as the right for all citizens to be granted availability of all information and communication facilities in the Information Society. This can be partially addressed by making them accessible to all citizens. Therefore access and accessibility are used as an approach toward eInclusion. Traditionally, accessibility problems have been solved with adaptations and the use of Assistive Technology products has been a technical approach to obtain adaptations. Presently, there is a shift toward the “Design for All” approach.

Numerous projects funded by European Commission Programmes, for almost 20 years, have pursued an evolutionary path, initially adopting reactive, and subsequently advocating proactive strategies to accessibility. It is important to notice that these projects have progressively shifted towards more generic solutions to accessibility.

The purpose of this document is to provide a summary of the advancements in the field of inclusive Human Computer Interaction, carried out by past and current European EC funded projects. The overview aims to offer a basis for the elaboration of a catalogue of HCI methodologies and tools developed and/or used by European projects in the e-Accessibility area. A number of pioneering projects in the field is first presented, followed by the presentation of recently completed and currently running projects.

Pioneering projects in the field of inclusive HCI

As an exploratory activity, the IPSNI project (Integration of People with Special Needs in the Broadband Communication Network) investigated the possibilities offered by the multimedia communication network environment, and in particular B-ISDN (Broadband Integrated Services Digital Network), for the benefit of people with activity limitations.

Different types of solutions were proposed, addressing the specific user abilities and requirements, at three different levels:

  • Adaptations within the user-to-terminal and the user-to-service interface, through the integration of additional input/output devices and the provision of appropriate interaction techniques, taking into account the abilities and requirements of the specific user group;
  • Service adaptations through the augmentation of the services with additional components capable of providing redundant or transduced information;
  • Introduction of special services, only in those cases where the application of the two previously mentioned types of adaptation are not possible or effective.

The IPSNI-II project built on the results of the IPSNI project, and demonstrated the technical feasibility of providing access to people with activity limitations to multimedia services running over a broadband network. Adaptations of terminals and services were implemented and evaluated. Special emphasis was placed on the adaptation of the user interfaces, and for this purpose, a user interface design and construction tool was designed, named INTERACT (Stephanidis and Mitsopoulos, 1995). INTERACT takes into account the interaction requirements of impaired users and builds on the notion of separating an interactive system in two functional components, namely the application functional core and the user interface component, thus allowing the provision of multiple user interfaces to the same application functionality.

The IPSNI-II project allowed an in-depth analysis of services and applications for the broadband telecommunications environment from the point of view of usability by people with activity limitations, leading to the identification and testing of necessary adaptations and/or special solutions. This work led to the conclusion that if emerging services, applications and terminals were designed considering usability requirements of users with activity limitations, many of their access problems would be automatically reduced with a negligible expense.

The TIDE-GUIB and TIDE-GUIB-II projects aimed to identify and provide the technological means to ensure continued access by blind users to the same computer-based interactive applications used by sighted users. The short-term goal of the GUIB project was to improve adaptation methodologies of existing GUIs. The GUIB approach to interface adaptation for blind users was based on a transformation of the desk-top metaphor to a non-visual version combining Braille, speech and non-speech audio. Access to basic graphical interaction objects (e.g., windows, menus, buttons), utilisation of the most important interaction methods, and extraction of internal information from the graphical environment were investigated.

The GUIB project also investigated a variety of issues related to user interaction in a graphical environment, particularly for users who cannot see. For example, the project investigated different input methods that can be used instead of the mouse. It also studied the problem of how blind users can efficiently locate the cursor on the screen, and examined issues related to combining spatially localised sounds (both speech and non-speech) and tactile information in order to present available information. Finally, the project addressed the design and implementation of real-world metaphors in a non-visual form and the development of an optimal method to present graphical information from within applications.

The ACCESS project (Development Platform for Unified ACCESS to Enabling Platforms) aimed to develop new technological solutions for supporting the concept of User Interfaces for all, i.e., universal accessibility of computer based applications, by facilitating the development of user interfaces automatically adaptable to individual user abilities, skills, requirements, and preferences. The project approached the problem at two levels:

  • the development of appropriate methodologies and tools for the design and implementation of accessible and usable User Interfaces;
  • the validation of the approach through the design and implementation of demonstrator applications in two application domains, namely interpersonal communication aids for speech-motor and language-cognitive impaired users, and hypermedia systems for blind users.

The ACCESS approach enables designers to deal with problems of rehabilitation and access to technology in a consistent, systematic and unified manner. The ACCESS project has proposed the concept of Unified User Interface development (U2ID), with the objective of supporting platform independence and target user-profile independence, i.e., possibility of implementation in different platforms and adaptability to the requirements of individual users (Stephanidis, Savidis and Akoumianakis, 1997).

The EC ACTS AVANTI project (Adaptive and Adaptable Interactions for Multimedia Telecommunications Applications) developed a new approach to the implementation of Web-based information systems, by putting forward a conceptual framework for the construction of systems that support adaptability and adaptivity at both the content and the user interface levels (Emiliani, 2001). The AVANTI framework comprises five main components:

  • A collection of multimedia databases, which contain the actual information and are accessed through a common communication interface (Multimedia Database Interface - MDI);
  • The User Modelling Server (UMS) (Kobsa and Pohl, 1995), which maintains and updates individual user profiles, as well as user stereotypes;
  • The Content Model (CM), which retains a meta-description of the information available in the system;
  • The Hyper-Structure Adaptor (HSA) (Fink at al., 1997), which adapts the information content, according to user characteristics, preferences and interests;
  • The User Interface (UI) component (Stephanidis et al, 1998; Stephanidis et al., 2001), which is also capable of adapting itself to the users' abilities, skills and preferences, as well as to the current context of use.

PALIO (Personalised Access to Local Information and Services for Tourists) was a project funded by the EC’s IST Programme. The main challenge of the PALIO project was the creation of an open system for accessing and retrieving information without constraints and limitations (imposed by space, time, access technology, etc.). The PALIO system envisaged the adaptation of both the information content and the way in which it is presented to the user, as a function of user characteristics (e.g. abilities, needs, requirements, interests); user location with the use of different modalities and granularities of the information contents; context of use; the current status of interaction (and previous history); and, lastly, the technology (e.g., communications technology, terminal characteristics, special peripherals) used.

2WEAR (A Run Time for Adaptive and Extensible Wireless Wearables) was a project aiming to explore the vision of a distributed personal computing system that is built on-the-fly by combining several different devices. The project developed and experimented with a wearable system, focusing on extensibility and adaptation issues.

State of the art of recently completed or running projects in the area of e-Inclusion

It is interesting to observe that in most EC funded projects in the area of eInclusion and e-Accessiblity attention is not only focused on Human-Computer interaction per se, but most of them are based on the concept of services and applications set up in order to support people (including people with activity limitations). Many of the projects are not aiming to offer specific solutions for single groups of people with activity limitations, but to the production of platforms for implementing systems and applications that are accessible and supportive, and to the development of methodologies for developing and evaluating accessibility technologies.

The list of recently completed and currently running projects that follows is by no means exhaustive, however the aim is to provide a concise picture of the recent developments in the area of Inclusive HCI, as a result of these activities.

Projects addressing Personalisation according to the user characteristics and/or the context of use

Many projects have recognised and investigated further the concept that systems, services and applications must be able to behave differently with different users and in different contexts is recognised in.

ASK-IT (Ambient intelligence system of agents for knowledge-based and integrated services for mobility impaired users) aimed to enable the provision of personalised, self-configurable, intuitive and context-related applications and services, with a self-configurable User Interface.

DIADEM (Delivering Inclusive Access for Disabled or Elderly Members of the community) plans to produce an adaptable web browser interface, to enable people who suffer a reduction in cognitive skills to remain active and independent members of society both at work and at home.

I2HOME (Intuitive interaction for everyone with home appliances based on industry standards) aims to provide intelligent and adaptable interfaces that are particularly targeted to persons with cognitive disabilities and older persons, using multi-modal communication and activity management.

SHARE-IT (Supported Human Autonomy for Recovery and Enhancement of cognitive and motor abilities using information technologies) aimed at the development of adaptive systems as transparent and easy to use to the person as possible, making significant contributions to fundamental, long-term research in: (i) verifying system adaptation to persons with special needs: both at design and run-time - as operating conditions and governing norms change - to establish e.g. safety, regulatory and security requirements; (ii) incorporating shared autonomy: ensuring that individual components can be designed to operate in a given intelligent ambience and adapt to possible changes both in the needs of the user or in the environment.

SOPRANO (Service oriented programmable smart environments for older Europeans) is focusing on the design and development of highly innovative, context-aware, smart services with natural and comfortable interfaces for older people, meeting requirements of users, family and care.

AEGIS (Open accessibility everywhere: groundwork, infrastructure, standards) identifies user needs and interaction models for several user groups, (users with visual, hearing, motion, speech and cognitive impairments as well as application developers) and develops open source-based generalised accessibility support into mainstream ICT devices/applications: (i) desktop, (ii) W3C/WAI standards-abiding accessible rich web applications and (iii) embedded generalized accessibility in terms of user interfaces and applications running into standard as well as rich features cell phones and PDAs.

The GUIDE (Gentle User Interfaces for Disabled and Elderly Citizens) project develops a toolbox of adaptive, multi-modal user interfaces (UIs) that target the accessibility requirements of elderly users in their home environment, making use of TV set-top boxes as processing and connectivity platform beside the common PC platform. With its software, hardware and documented knowledge, this toolbox aims to put developers of ICT applications in the position to easier implement truly accessible applications using the most recent user interface technologies with reduced development effort.

Projects developing platforms for integrated solutions

A second very important aspect is that many projects do not aim to produce single solutions addressing the requirements of a group of users, but the trend is toward the use of platforms on which solutions are implemented.

In the context of ASK-IT content and tools developed are integrated within an Ambient Intelligent Framework, by a Multi Agent System of Intelligent Agents that offer service personalisation according to user profile, habits, preferences and context of use.

EU4ALL (European Unified Approach for Accessible Lifelong Learning), is an integrated project that seeks to make a widespread impact on the way universities and educational institutions deliver lifelong learning services to the whole population. Support services and a technical open service infrastructure will enable teaching, technical and administrative staff of educational institutions to offer their teaching and services in a way that is accessible to disabled learners. The goals of EU4ALL are to:

  • Design an open service-oriented architecture for ALL
  • Develop the software infrastructure for ALL services (content, support and access services)
  • Provide technical standards/specifications for ALL applications integrated with current and emerging eLearning standards
    Validate the results in large-scale higher education settings

I2HOME (Intuitive interaction for everyone with home appliances based on industry standards) uses an architecture with a Universal Control Hub (UCH) as core component that communicates to networked (off-the-shelf) home appliances and consumer electronics devices.

MONAMI (Mainstreaming on Ambient Intelligence) is focusing on services, platforms and usability: and the technology platform is being derived from mainstream technology.

eMPOWER (Middleware platform for eMPOWERing cognitively disabled and elderly) defines and implements an open platform for the integration of the smart house and sensor technology and the interoperability between profession and institution specific systems (e.g. Hospital Information System), in order to simplify and speed up the task of developing and deploying services for persons with cognitive disabilities and elderly.

PERSONA (Perceptive spaces promoting independent aging) is aiming to develop a technological platform that allows the seamless and natural access to services. It will develop a scalable open standard technological platform to build a broad range ofAAL Services, to demonstrate and test the concept in real life implementations, assessing their social impact and establishing the initial business strategy for future deployment of the proposed technologies and services

OASIS (Open architecture for accessible services integration and standardization) aims to introduce an innovative, Ontology-driven, Open Reference Architecture and Platform, which will enable and facilitate interoperability, seamless connectivity and sharing of content between different services and ontologies in all application domains relevant to applications for the elderly and beyond. The OASIS platform is supposed to be open, modular, holistic, easy to use and standards abiding. It includes a set of novel tools for content/services connection and management, for user interfaces creation and adaptation and for service personalization and integration.

Projects setting up Platforms for the development of accessible software

Another stream of activities aims to the set up of platforms for the development of accessible software.

The ÆGIS project seeks to determine whether 3rd generation access techniques will provide a more accessible, more exploitable and deeply embeddable approach in mainstream ICT (desktop, rich Internet and mobile applications). It develops and explores this approach with the Open Accessibility Framework (OAF) through which aspects of the design, development and deployment of accessible mainstream ICT are addressed. The OAF is supposed to provide embedded and built-in accessibility solutions, as well as toolkits for developers, for "engraving" accessibility in existing and emerging mass-market ICT-based products, thus making accessibility open, plug & play, personalised & configurable, realistic & applicable in various contexts.

The goal of ACCESSIBLE (Accessibility assessment simulation environment for new applications design and development)  is to improve the accessibility of software development products, by introducing a harmonised accessibility methodology into the software design and development processes, using significantly better measurement strategies and methodologies. It develops a process for collating and merging different methodological tools, checking the coherence with the W3C/WAI ARIA and other standards in order to produce an Open Source Assessment Simulation Environment. This is supposed to enable large organisations, SMEs or individuals (developers, designers, etc.) to produce software products of superior accessibility and quality, accompanied with appropriate measures and proposals for best practice.

HAPTIMAP (Haptic, audio and visual interfaces for maps and location-based services) is supposed to embed accessibility into digital mainstream maps and mobile location-based services, firstly developing tools that make it easier for developers to add adaptable multimodal components (designed to improve accessibility) to their applications; and secondly, raising the awareness of these issues via new guidelines and suggesting extensions to existing design practices so that accessibility issues are considered throughout the design process. The concrete outcomes of the project will be an open, interoperable and standardized adaptable toolkit together with a set of design guidelines that help developers of mainstream applications make maps in general more accessible and easier to use (not only for disabled users but for everyone.

The goal of eACCESS+, (The eAccessibility Network) is to create a platform for collecting and providing guidance on how to use in practice the body of knowledge on eAccessibility. eAccess+ is a best-practice network to facilitate co-operation between the community of practitioners (found in research institutions and consultancies) and all the other stakeholders (policy makers, administrators in the public sector, technical staff in the private sector…). The purpose is to accelerate the take-up of e-accessibility specifications and technical solutions, and to contribute to a common approach at European level. The network will support the development of common guidelines and standards, and, where needed, will provide rationale for harmonised political and legal measures. The main focus is on web accessibility, however digital TV and self-service terminals are also addressed.

Projects involved in the Development of evaluation - assessment Methodologies

A cluster of European projects (The WAB Cluster – Web Accessibility Benchmarking Cluster) has recently completed activities to develop a harmonised European methodology for evaluation and benchmarking of websites. The primary goals of the Cluster are:

  • to develop a EU-harmonised assessment methodology for Web accessibility, based on W3C/WAI and to be synchronised with the foreseen migration from WCAG1.0 to WCAG2.0.
  • to ensure that evaluation tools and methods developed for global monitoring or for local evaluation, are compatible and coherent among themselves (and with WAI)
  • to provide a strong European feedback and contribution to WAI and others for future guidelines or versions of guidelines.

The WAB Cluster involved the following activities/projects:

The European Internet Accessibility Observatory (EIAO) concerns the preparation of a platform for a possible observatory (measurement machine with modular tests, site inventory for jurisdictions, results management and aggregation). The platform prototype, once sufficiently advanced, will provide a facility for testing aspects of the WAB Cluster methodology

BenToWeb (www.bentoweb.org)
BentoWeb refers to the production of test suites for evaluation tools, and evaluation modules for checkpoints difficult to automatise. Research into integration of testing modules in CMS and issues related to dynamic multiversion webpages

SupportEAM (www.support-eam.org)
Support EAM (Supporting the creation of a e-Accessibility Quality Mark) has proposed a certification mechanism and authority, training material and tools supporting a unified European approach to Web Accessibility evaluation. This includes third party and self-certification

UWEM (www.wabcluster.org/uwem/)
Accessibility checks can be carried out in different ways even if the checks are based on the same guidelines. The Unified Web Evaluation Methodology (UWEM1.1) is the result of a joint harmonisation effort by 23 European organisations of the three European projects mentioned above. They have developed UWEM to ensure that large scale monitoring and local evaluation are compatible and coherent among themselves and with the Web Content Accessibility Guidelines from W3C/WAI. The UWEM methodology has already incorporated support for the migration from WCAG 1.0 to WCAG 2.0. Thus, UWEM is the ideal instrument to support evaluation, (self)certification, and benchmarking of web content in Europe and beyond.The WAB Cluster is building an observatory for large scale European evaluation and benchmarking of website accessibility. This supports large scale and local evaluation. The UWEM methodology is conformant with the W3C Web Content Accessibility Guidelines and based on an interpretation of WCAG agreed among stakeholders. In this way, it can offer unprecedented guidance for evaluation and benchmarking.

The Web Cluster has delivered a European instrument for evaluation and benchmarking of websites to the EU.
The UWEM methodology contains a complete methodology including detailed tests for the evaluation of websites for WCAG1.0 conformance. The document is separated into a Core document and a Tests document. The UWEM has been developed in order to improve the Tool and Browser Independence, the Unique Interpretability, Repeatability and Translatability of the WCAG1.0 guidelines. The Methodology can be downloaded from: http://www.wabcluster.org.

Projects investigating future technology developments

Finally, a number of projects are addressing technologies important for the emergence of the Ambient Intelligence environment. In that respect, the aim is at highly innovative ICT-based solutions that are cost effective, reliable and user friendly for assisted living taking into account design-for-all principles where applicable, with the goal to lead to integrated environments bringing together progress in various ICT building blocks and responding to key user requirements.

EASY LINE+ (Low cost advanced white goods for a longer independent life of elderly people) foresees using the integrated RFID, Neuronal Networks and HMI technologies to build a system that can capture data of the home environment, and can control via wireless communication (Zigbee) or the mains electricity (EMS PLC), any white good in the home. The users, elderly persons, may actuate by themselves any white good in the home, or may leave the "e-servant" to do the actuation. The e-servant is a white good control system, based on the sensor information and the habits of the user that can program any application without/or with user cooperation.

VAALID (Accessibility and usability validation framework for AAL interaction design process) aims at creating new tools and methods that facilitate and streamline the process of creation, design, construction and deployment of technological solutions in the context of AAL (Ambient Assisted Living) assuring that they are accessible and usable for senior citizens. The main objective of the project is to develop a 3D-Immersive Simulation Platform for computer aided design and validation of User-Interaction subsystems that improve and optimise the accessibility features of Ambient Assisted Living services for social inclusion and independent living. The simulation environment is composed by software and hardware components that constitute a physical ensemble that in conjunction allow the ICT designer to implement actual Virtual Reality and Augmented Reality scenarios of AAL. It can be used to verify interaction designs and validate the accessibility of the AAL products by means of immersing the users in 3D virtual spaces.

COMPANIONABLE (Integrated cognitive assistive and domotic companion robotic systems for ability and security) explores the synergetic combination of the strengths of a mobile robotic companion with the advantages of a stationary smart home, since neither of those approaches alone can accomplish the demanding tasks to be solved. Positive effects of both individual solutions are supposed to be combined to demonstrate how the synergies between a stationary smart home solution and an embodied mobile robot companion can make the care and the care person's interaction with her assistive system significantly better.

The UNIVERSAAL (The UNIVERsal open platform and reference Specification for Ambient Assisted Living) project aims to produce an open platform providing a standardised approach to make it technically feasible and economically viable to develop AAL solutions. The platform will be produced by a mixture of new development and consolidation of state-of-the-art results from existing initiatives. Work on establishing and running a sustainable community will achieve attention, with promotion of existing results gradually evolving into promotion of the universAAL platform, as it develops into one consolidated, validated and standardised European open AAL platform. The platform will provide runtime support for the execution of AAL applications in accordance with a reference architecture, development support through core AAL services and an online developer depot of various development resources. universal results will be standardised in European (CEN) and international (OMG, Continua) standardisation bodies.

The VAALID (Accessibility and Usability Validation Framework for AAL Interaction Design Process) research project aims at facilitating and streamlining the process of creation, design, construction and deployment of technological solutions in the context of AAL. A 3D-Immersive Simulation Platform for computer aided design and validation of User-Interaction subsystems will support the design of the Human Interaction aspects in all the stages of user centred design, putting in practice the guidelines for verification and validation of the accessibility and usability facets. Virtual Reality and Augmented Reality scenarios will be used to verify interaction designs and validate the accessibility of the AAL products. The overall goal is to help European industry, ICT companies specialized in Human Factors and User Interaction design, Research and Academia in streamlining their respective business for the Independent Living and Inclusion.

VERITAS , (Virtual and Augmented Environments and Realistic User Interactions To achieve Embedded Accessibility DesignS) aims to develop, validate and assess an open framework for built-in accessibility support at all stages of ICT and non-ICT product development, including specification, design, development and testing. The goal is to introduce simulation-based and VR testing at all stages of product design and development into the automotive, smart living spaces, workplace, infotainment and personal healthcare applications areas. The goal is to ensure that future products and services are being systematically designed for all people including those with disabilities and functional limitations.
Although this does not claim to be an exhaustive account of every project in the field, it nevertheless gives a concise picture of the recent developments in the area of Inclusive HCI as a result of these activities. It has to be stressed that the purpose of this document has not been to evaluate the actual impact of these tools and methodologies, but rather to list relevant and important advancements in the field of inclusive HCI, and to provide a basis for the elaboration of a catalogue of HCI methodologies and tools developed and/or used by European projects in the e-Accessibility area.

HCI and Accessibility

In recent years, many research activities have focused on design that aims to produce universally accessible systems, taking into account special needs of various user groups. These special needs are associated with many user factors, such as impairments of speech, hearing or vision, cognitive limitations, aging, as well as with various environmental factors. Fields that address this problem, such as Usability, Universal Accessibility, Universal Design, or Inclusive Design have been developed as relatively independent domains, but they share many aspects with other HCI disciplines. However, researchers and practitioners are often not aware of interconnections among concepts of universal accessibility and “ordinary” HCI. In view of this situation, [Obrenovic, 2007] show that there is a fundamental connection between multimodal interface design and universal accessibility, and that awareness of these links can help both disciplines. Researchers from these areas may use different terminology, but the concepts they use often have essentially the same meaning.

Human-Computer Inclusive design

Experience shows that a large amount of HCI systems are designed for someone conceived as the "standard man" leaving out the scope all the people with different physical, sensory or cognitive features. Having in mind that the most common human characteristic is just variety, most designs do not completely fit individual user’s needs.

The problem of matching product features with users' characteristics is most frequently addressed by the own user adapting himself or herself as much as possible to the interface. As a consequence, people that are no able to adapt themselves are simply left out of the possibility of using these products or services.

There exist design techniques and methodologies able to address users’ diversity, by means of modelling and adaptation . Nevertheless, they are not enough known and used. In fact, the marginalization of large sectors of users was –and frequently is– justified by limitations of technology. Nowadays we know that technology can be designed in a most inclusive way avoiding the inclusion of unnecessary barriers. Inclusive Design aims to consider the needs of all the users in main stream applications and not only in the systems especially designed for people with physical, sensorial or cognitive restrictions.

Advantages of Inclusive Design

Inclusive Design is based on the convincement that humans are naturally very diverse. The partition into "normal users" and "other users" is artificial and the frontiers between both populations are arbitrary. In fact, there are abundant examples where technology eliminated or alleviated these frontiers. Something as simple as glasses –nowadays of common use– allow several people with eyesight restrictions to enhance their vision. More complex technologies, such as computers, give people with motor and speech impairments a way to personal and remote communication, and to control their environment.
Evidently, Inclusive Design has ethic and social fundaments. Universal Accessibility is supported by the convincement that all the human beings have the same rights. In practical terms this means that they should be able to access to the same services and to enjoy the same opportunities. Technological designs that unnecessarily establish barriers to universal use effectively exclude users with physical, sensory or cognitive restrictions.

In addition, to its ethics roots, inclusive technology is highly practical and useful. It frequently has a higher impact over the market because accessible products are directed to a broader population of potential consumers. In fact, people without disabilities usually find inclusive technology easier and more usable. On the other hand, the new ways to interact with mobile and ubiquitous technology frequently require hand and sight free interaction, and as a result they can very much benefit from Inclusive Design. For instance, people wanting to read their email while they drive to work do need auditory interfaces, similarly to several vision impaired people. In addition they will need voice input to enter commands to the system, similarly to many people with severe motor restrictions.

It is frequently argued that Inclusive Design (for instance accessible web design) is not more expensive that standard design. We cannot deny that accessible design requires higher effort from the designer. That means the need of knowledge and experience on this kind of design, longer development periods, etc. Nevertheless, it is proved that accessible products are of higher quality. In fact, usability and accessibility are included as quality measurement figures of merit in a number of software methodologies (for instance, [ISO 2008a] and [ISO 2008b]).

Current barriers for Inclusive Design

Even if Inclusive Design advantages have been frequently acknowledged, there is little advancement in Inclusive Design of commercial ICT products. This may be due to a combination of factors, such as:

  • The lack of awareness of universal design. Numerous HCI designers frequently ignore that they can design for a broader population simply avoiding the inclusion of certain features that put difficulties to the accessibility by a number of users.
  • The lack of knowledge about user needs. Most HCI professionals usually design without having a previous analysis of the users needs. Their designs are frequently based in their own mental model of the task, their own capabilities and likes, etc.

Conditions for Inclusive Design

There are numerous ethical and social risks that must be taken into account when Inclusive Design paradigm is adopted [Abascal 2005]. For instance:

  • User autonomy: Avoid taking automatic decisions about the user without her or his consent.
  • User privacy: Avoid to store, transmit or process data about the user or his/her activities that are not strictly necessary for the current application.
  • User consent: Always ask for the informed consent from the user.
  • Human contact: Compensate the social impact of services that produce isolation.

Finally, Inclusive Design is hardly possible without the full participation of the users in the whole design and development process. That means that the users must be present: In all the phases of the process; as full participants; being paid; under a code of conduct for experiments.

Web accessibility

It is well known that the web has spread over the last few years in an unexpected way. The Web has become a major part of many people's everyday life since it facilitates the fulfilment of daily tasks related to communication, entertainment, work, study, etc. In addition, the number of eServices that, in many cases, substitute or complement traditionally delivered services.
One of the human groups that may easily suffer exclusion is that of people with disabilities, as in their case many commercial interfaces fail to prove accessible. Different initiatives have been taken in order to avoid this situation, such as inclusive laws promulgated in several countries. However, these efforts are insufficient if technological advancements do not support universal design. For this reason, many national authorities are supporting projects to achieve web accessibility.

  • Even if designers are convinced (or compelled) to create accessible products, they usually have to face a lack of knowledge and experience on accessible design. Therefore, methods, tools and guidelines, are needed to help designers in this difficulty. Guidelines have frequently been used to collect design knowledge and experience. Even if they may present problems, such as incoherence and unreliability, and be difficult to handle (when the set of guidelines is too large), guidelines nevertheless prove to be the best method in order to transmit satisfactory design experiences within large design groups or for to the external world [Nicolle 2001].

For this reason, a crucial advance in web accessibility is the provision of sets of guidelines and tools to apply them. Since web technology is rapidly changing, and web accessibility guidelines have to be frequently updated, appropriate tools must be able to easily modify the existing, or include new, sets of guidelines. The most relevant sets of guidelines are those developed by the Web Accessibility Initiative (WAI) of the World Wide Web Consortium (W3C). W3C-WAI has established three sets of W3C recommendations to improve the accessibility of the Web. These are:

  • WCAG (Web Content Accessibility Guidelines), which concern how to make Web sites sufficiently accessible so that people with disabilities are able to use them alongside with today’s technologies. The version 2.0 was released on December 2008.
  • ATAG (Authoring Tool Accessibility Guidelines), which provide guidance for software developers in designing authoring tools that produce accessible web content and in creating accessible authoring interfaces. A working draft of ATAG 2.0 was released on July 2010.
  • UAAG (User Agent Accessibility Guidelines), released in December 2002, which concern how to make browsers and multimedia players more accessible, as well as compatible with some of the assistive technology that people with disabilities use. A working Draft of UAAG was released on June 2010.

Nevertheless, guidelines compliance does not guarantee web accessibility. Users and experts evaluations are required to find accessibility barriers that are hardly specified by guidelines.

The strengths of WAI guidelines are both their universal acceptance and the way they are produced (cooperatively, in a limited period of time and in a clear way). However, these broadly accepted and used guidelines are far from being definitively established. As web technology is rapidly evolving, the production of guidelines is a continuous process that periodically offers new versions. However the paramount importance of WAI guidelines cannot be ignored.  In addition to their contribution to web accessibility, WAI guidelines are an incredible pioneering experience of advancement towards Universal Accessibility that has to be taken into account for all the other actions in favour of the accessibility.

Mobile Web

The mobile web has become more widespread as the computing performance of mobile devices and their availability has steadily increased over the last few years. Mobile devices such as PDAs, mobile phones, videogame consoles and a number of home appliances (TV, video, etc.) currently have XHTML (and similar) browsers, thus enabling the advent of the ubiquitous web. Even if these devices have very dissimilar input and output modalities, the fact that several of them have reduced keyboards and small screens causes poor interactive experience. Low input rate, lack of a pointing device and low bandwidth are the key factors that cause a decline in the quality of interaction.

Since mobile devices are used in everyday situations they may cause the so-called situationally-induced impairment and disabilities. For example, users interacting with a touch screen while experiencing turbulence during a flight or texting while hands are busy can be considered temporary impaired. Therefore, there is a strong relationship between web accessibility and the mobile web because the problems encountered while interacting with the Mobile Web can be referred to as accessibility barriers for the able-bodied. Similarly, the existence of an overlap between mobile web usability recommendations and guidelines for physically impaired users is evident. Consequently by applying accessibility-related good practices, navigation in the WWW can be enhanced for a wider audience, reinforcing the experience showing that content accessibility benefits all users.

Therefore, the problems that able-bodied individuals encounter in the Mobile Web and the barriers found by users with physical, sensory or cognitive disabilities while browsing the Desktop Web are related to: Small display size; No support for mark-up, scripting or data formats.

Mobile web guidelines aim at providing developers with guidance to develop web content suitable for mobile devices. Yet, a number of mobile web guidelines refer to specific device features such as screen size, support for particular picture formats or support for pointing device.

Even if there are some concerned developers who follow Design for All principles and methods to ensure that all resources are accessible by everyone, it is not always possible to accomplish the Design for All paradigm, given the diversity of users and access devices that exist. An alternative approach is the personalization of user interfaces, thus interfaces should be adapted considering users’ specific needs. Systems which adapt websites to handheld device constraints (screen size, etc.) have been developed. Others adapt a determined type of web pages to users with visual impairments. Additionally, users’ preferences, as well as access device’s features are taken into account for adaptation purposes. Nevertheless, none of them considers a wide range of disabilities, and they are focused on adapting user interfaces of a determined type of web pages or standalone applications.

The adaptive web

The possibility to access to Information and Communication technologies is frequently challenged by diverse factors such as (a) characteristics of the user (including physical, sensory or cognitive restrictions and literacy problems); (b) limitations of the equipment (obsolete devices or applications, narrow band access, small displays or keyboards); (c) barriers imposed by context of use (noisy or dark environments; performing parallel activities, such as working or driving). Frequently these issues have a deep root in social causes such as poverty and limited access to education.

The design of effective adaptive user interfaces may help in closing the digital divide. These interfaces usually allow automatic adaptation to the user considering user’s own characteristics, the context where the interaction is carried out, the task to be performed, the technology being used, etc. These methods may give people suffering restrictions higher opportunities to access to ITCs.

Currently, classic user modelling techniques are being extended to model the environment and the technology involved in the process. The supporting paradigms are evolving to the use of advanced technologies, such as ontologies, which are extremely useful to store and process the information for modelling. In addition, machine learning techniques allow the compilation of user models with information obtained by data mining of interaction logs. But, while current technology allows for more effective personalization systems, some issues, such as privacy impact, condition the applicability of these techniques.

Web personalization

The main objective of Web personalization is to adapt the browsing or navigation, the presentation and the contained information of the web pages to the needs of the user without him or her doing an explicit demand. The goal is to enhance the speed and the achievement using the web and to decrease the physical and cognitive effort to perform it. Therefore, adaptation becomes especially helpful when the users have special needs.

Web personalization seeks for the adaptation to the user in three areas [Brusilowsky 2007] that in the case of users with disabilities are crucial in order to easy and speed up the interaction:

  • Navigation. Browsing pages is the most common task when using the web. This task can be slowed, for instance, if the page contains a high number of links that are not interesting for the user. Knowing the objectives and interests of the user the browser can make easier the navigation task giving more emphasis to the most interesting or probable links for a specific user. The inclusion of a specific navigation menu with selected links is another navigation facilitating possibility.
  • Presentation: The presentation of a web page can be adapted to the specific needs of each user applying cascading stile sheets (CSS ). The most convenient stile sheet(s) may be stored in the (static) model of the user.
  • Content. Even if it is not convenient to automatically change the content of a web page, some inclusions may enhance its readability. For instance, the automatic inclusion of text captions in simple language used to explain longer and more difficult texts are useful for people with reading difficulties.

Web personalization is based on modelling user features such as interests, navigational behaviour, preferences, physical sensory or cognitive restrictions, etc. The information stored in the model is used to make assumptions about the current user-system interaction that allows adapting the system to the actual user needs or preferences. User adaptation methods have been frequently adopted by intelligent interface designers to adjust the interface to the user (contrarily to the usual situation where the user adapts him or herself to the interface). These models could be directly designed by experts in the area (rule based approach), they can be built based on previous information from that user such as the logs of previous navigations (content based approach) or they can be induced from information about groups of users with similar characteristics (collaborative approach). The first approach is somehow static and requires previous knowledge of the users and redesigning when new behaviours appear in the users. However, the last two approaches focus on automatic techniques for user characterization. Currently the user component is usually built by means of ontologies that allow storing, manipulating, and extracting assumptions from data about the user, its context, tasks, etc. [Miñón 2010].

Data mining for web personalization

In order to be able to model the user, the modelling component must collect information about a number of observable parameters such as interest, characteristics, etc. This information can be requested to the user in a previous session, but this is annoying, disruptive and can produce false assumptions. Another option is to collect this information while the user is accessing the web. In this way the system can learn its interests, likes, etc. Learning from the own interaction allows maintaining a dynamic profile of the user, avoiding the application of all assumptions when the interest, characteristics or circumstances of the user change.

Data mining for web personalization has many advantages. It is not disruptive, is based in statistical data obtained by real navigation exercises (decreasing the possibility of false assumptions) and is itself adaptive (when the characteristics of the user change, collected data allows the automatic change of the interaction schema). When the user is a person with physical, sensory or cognitive restrictions, data mining is the easiest (and frequently almost the only) way to obtain information about the uses of the person.

Data mining in this context has also some drawbacks. The most important one is its impact over privacy, due to the need of storing large quantities of data about the users. Diverse laws in different countries protect user rights for privacy. Even if it is difficult to reach a balance among privacy and personalization, some appealing proposals have been recently published.

Human interfaces for accessible Ambient Intelligence system

The advances in networking, computing and sensing technologies allow the design of intelligent environments able to give support to people located inside them. The Ambient Intelligence paradigm benefits from ubiquitous and wearable computers, communicated by wireless networks with static computers –which can be also connected to wired networks–, that are able to process enormous quantities of contextual information coming from networks of sensors. This technological infrastructure will allow the deployment of intelligent applications that proactively give support to the users [Streitz 2006].
Ambient Intelligence obviously provides an extraordinary opportunity to develop assistive environments for people with sensory, physical or cognitive restrictions due to aging, disability, illness, etc. All these intelligent environments perform several supportive activities that are usually ignored by the user, such as adjusting the temperature, humidity, lights, etc., or verifying the safety of gas, electricity or water installations. In addition, intelligent environments have to communicate with the user to provide information or to request commands. The user interfaces are supposed to be as natural as possible, allowing a communication similar to the interaction between humans. That means that the system should be able to produce voice messages –and maybe to display some of them via wall screens or data glasses– and to recognize natural language and gestures. Nevertheless some of these communication methods may not be appropriate for people with specific sensory or cognitive restrictions.

Accessible AmI supporting technology

Elderly and disabled people belong to a segment of the population that would profit very much from Ambient Intelligence if it is accessible. This is only possible if accessibility barriers are early detected in the evolution of the Ambient Intelligence concept and opportune standardization measures are provided.

Autonomy and quality of life of elderly and disabled people living in smart private or public homes designed under the Ambient Intelligence paradigm can experience significant enhancements due to the increased support received from the environment. This support includes facilities for environmental control, information access, communication, monitoring, etc., built over diverse technologies and using different operation ways. Nevertheless, users can find accessibility barriers frequently related to the diverse user interfaces with heterogeneous devices and procedures. These problems include both, physical difficulties to handle the devices, and cognitive barriers to understand use procedures and navigation. As a result, accessible unified interfaces to control all the appliances and services are needed. This is only possible if the network technology used for smart homes is able to support interoperability and systems integration. Therefore, the needs of senior and disabled users can only be provided by means of interoperable systems in an integrated intelligent environment. Consequently, only a convergence policy based on inclusive design guidelines and standards can guaranty the accessibility of the future intelligent ambient [Sevillano 2004] [Emiliani 2008].

Accessible adaptive interfaces for AmI

As previously mentioned, the growing ubiquitous computing paradigm allows the provision of context-aware services to mobile users. In addition to the usual computing requirements, these environments entail wireless network infrastructures and special management software, usually called middleware. When a mobile computing device (smart phone, PDA, etc.) enters into an Ambient Intelligence environment the middleware establishes the communication with the local network in a way that is transparent to the user. After the “discovering” and “presentation” phases, the available local services are offered to the user. In order to be operated some of them may require a specific user interface that is downloaded to the user’s mobile device.

This type of environment can be extremely helpful for people with disabilities who have mobile devices adapted to their characteristics. In this way, using their own device they can access several local services that can otherwise be inaccessible to them, such as ATMs, vending machines, information kiosks, smart home appliances, etc. This is only possible if the downloaded user interface is rendered to the mobile device in an accessible mode. The large variety of user characteristics and restrictions (due to the broad range of disabilities) and the peculiarities of the devices used by them makes it necessary to adapt the “basic” user interface supplied by the service provider to the specific needs of the user and his or her device. Therefore the system has to automatically generate user interfaces adapted to the features and preferences of users with disabilities. To automatically adapt the interface to the user characteristics, it is necessary to take into account what the most suitable communication modalities are for each user, mapping them to the appropriate media.

Ubiquitous systems handle a huge quantity of information that can be used to infer knowledge about the user, the environment and the tasks. Modelling this knowledge would contribute to enhancing the generation of adapted user interfaces. Ubiquitous Computing itself frequently includes user modelling and personalization as a goal, in order to take into account the human context. This goal requires a component that can manage the adaptation of the information resources and make the interaction comfortable for each user of the ubiquitous environment.

Interfaces for robot control

Mobile Robotics has experienced a notable development in recent years. For instance, sensors are more and more reliable and accurate at lower prices. In addition, processors are also more powerful and memory availability is larger and cheaper. For these reasons, it is possible nowadays to speak about “consumer robotics”. Similarly to the evolution of personal computers, robots are finding new applications in the home, outside of the factories. One of the most promising fields among the non-industrial applications of robots is Assistive Robotics. Assistive Robotics is proposing new ways of supporting people with motor restrictions to develop tasks that were previously impossible for them. Among the diverse applications that are being developed, assisted mobility and manipulation stand out [Abascal 2008a].

Augmentative and Augmentative Mobility, AAM, (similarly to Augmentative and Alternative Communication ) attempts to provide people with methods and devices to enhance or restore their mobility. The application of the advancements in Mobile Robotics to AMM allowed the design of very advanced assisted mobility systems. Among them the most sophisticated are smart wheelchairs [Abascal 2008b]. Smart wheelchairs are intended for people with severe motor restrictions that experience great difficulty in driving standard electric wheelchairs. They are usually provided with diverse types of sensors and embedded computers that receive information from the sensory system, handle the interaction with the user and control the motors through the power stage. The number and quality of the sensors determine the accuracy of the control. For this reason many experimental Smart Wheelchairs are provided with extremely advanced and expensive sensors that convert them into impressive mobile robots but are too expensive and sophisticated to be marketed.

The interaction between the user and the wheelchair is again a key factor. As previously mentioned, users of smart wheelchairs are people with severe motor restrictions that impede the use of standard input devices. Therefore, the design of interfaces for AAM has also to take into account specific guidelines to satisfy the needs of the users.

Another important human need is to manipulate objects in the surroundings. There are specific technologies applicable to people with severe motor disabilities or to people with amputations. Light articulated arms come from the adaptation of articulated industrial manipulators to allow people with severe hand movement restrictions to grasp and move objects. It is evident that is not possible just to use industrial robots at home. There are problems of size, height, security (the user and the robot share the work space), etc... Nevertheless, the most important barrier is human-robot interaction. Robots are designed to handle objects based on their position and orientation, using diverse types of coordinates. Users describe objects in terms of names, properties (colour, shape, size...), function, etc. In the user’s mind positions are usually related to other objects or to the room. It is not expected that a user should have to give numeric parameters with the position, orientation and size of the object to be manipulated. Therefore, intelligent mediator applications are necessary to understand object description in natural language and to translate this into coordinates.

Requirements of the Human-Robot Interface

Some basic principles have to be taken into account in designing the interface to control both robotic assistant devices and intelligent environments. The first one is the rehabilitation goal. Numerous people with disabilities are able to enhance their cognitive abilities, personal attitudes and social integration when they are provided with adequate user interfaces. To this end the interface must encourage the use of all the capabilities of the user, and avoid taking decisions on behalf of the user when it is not absolutely necessary.

For instance, in the case of autonomous smart wheelchairs, they are able to automatically navigate requiring little or no interaction from the user. After the destination is somehow specified, the wheelchair is able to take all the necessary decisions to arrive at the selected place. Although this procedure is very convenient for people with extreme motor restrictions, many users have some remaining abilities that could be lost if they are not used. These abilities may even be enhanced when they are trained. Therefore the interface must facilitate, as much as possible, user participation in order to enhance their cognitive abilities, personal attitudes and social integration. In addition, the user must feel that he or she is the one who controls the device in order to avoid frustration and passivity That includes ease of switching between automatic/assisted/manual functioning.

Safety and reliability are also important requirements. Several of these systems interact with the environment in various ways that could be dangerous in the case of failure or malfunction. The designer must ensure that the system is safe, reliable and fault tolerant.

Another key issue is the final price. Inexpensive solutions are needed to prevent unaffordable systems. In the case of AAM, that means using cheap sensors (for example, infrared and ultrasonic sensors instead of laser, to measure distances). Currently processors are cheap and the inferior quality of the sensors can be balanced by a much greater processing capacity. Moreover, since most intelligent wheelchairs are built on commercial electric wheelchairs, carrying out any major changes in order to facilitate its future potential marketing by the industry without making large investments should be avoided.


A summary of Current Research Trends in HCI Design

Some Design Frameworks and tools

Modern user interface builders provide graphical environments for user interface prototyping, usually following a WYSIWYG (“What You See Is What You Get”) design paradigm, and offering graphical editing facilities that allow designers to perform rapid prototyping visually. Such editors may be standalone or embedded in integrated environments (IDEs), i.e., programming environments which allow the direct development of the application functionality for the created prototypes. Commonly used IDEs are Microsoft Visual Studio, NetBeans, and Eclipse. IDEs are very popular in application development because they greatly simplify the transition from design to implementation, thus speeding up considerably the entire process, while also supporting look-and-feel consistency through the availability of common sets of UI widgets.

Research efforts related to frameworks and tools for user interface design include a reference framework proposed by Calvary, Coutaz and Thevenin [Calvary 2001], which follows a model-based approach and that structures the development process of plastic user interfaces. Based on this proposed model, a design tool, named ARTStudio, has been implemented to support the development process. TERESA is another tool, which supports the design and development of nomadic applications, providing general solutions that can be tailored to specific cases [Mori 2003]. Through TERESA designers can either specify the appearance of common UI elements for the supported platforms or even modify some general design assumptions.

Several research efforts have suggested a variety of components and tools to facilitate the development of user interfaces capable of adaptation, including accessibility features:

  • MENTOR, is a tool providing (a) practical integrated support for all phases of adaptation design, through appropriate editing facilities; (b) practical support for a 'smooth transition' from design to development through the availability of automated verification mechanisms for the designed adaptation logic, as well as the automated generation of 'ready-to-implement' interface specifications; and (c) support for the progressive accumulation of design cases and of the related design experience and knowledge, in particular regarding adaptation [Antona 2006].
  • EAGER is a toolkit built to support WUI (Web User Interface) adaptation and facilitate the design of web interfaces that can adapt to the diversity of the target user population [Doulgeraki 2009]. By means of EAGER, a developer can produce Web portals that have the ability to adapt to the interaction modalities, metaphors and UI elements most appropriate to each individual user, according to profile information containing user and context specific parameters.
  • On the other hand, MAID is a multi-platform accessible interface design framework, facilitating the development and customization (through skins) of structured User Interfaces (templates) through the use a set of predefined UI components (widgets), while promoting reusability and supporting dynamic content population [Korozi 2009].

Current research trends

Research efforts in recent years have elaborated comprehensive and systematic approaches to user interface adaptations in the context of Universal Access and Design for All [Stephanidis  2001]. The Unified User Interfaces methodology was conceived and applied [Savidis 2004] as a means to efficiently and effectively ensure, through an adaptation-based approach, the accessibility and usability of User Interfaces to users with diverse characteristics, supporting also technological platform independence, metaphor independence and user-profile independence. In such a context, automatic UI adaptation seeks to minimize the need for a posteriori adaptations and deliver products that can be adapted for use by the widest possible end user population (adaptable user interfaces).

This implies the provision of alternative interface manifestations depending on the abilities, requirements and preferences of the target user groups, as well as the characteristics of the context of use (e.g., technological platform, physical environment). The main objective is to ensure that each end-user is provided with the most appropriate interactive experience at run-time.
In more detail, a unified user interface comprises a single (unified) interface specification that exhibits the following properties:

  • It embeds representation schemes for user- and usage-context- parameters and accesses user- and usage-context- information resources (e.g., repositories, servers), to extract or update such information.
  • It is equipped with alternative implemented dialogue artefacts appropriately associated to different combinations of values for user- and usage-context- related parameters.
  • It embeds design logic and decision making capabilities that support activating, at run-time, the most appropriate dialogue patterns according to particular instances of user- and usage-context- parameters, and is capable of interaction monitoring to detect changes in parameters.

As a consequence, a unified user interface realises:

  • User-adapted behaviour (user awareness), i.e., the interface is capable of automatically selecting interaction patterns appropriate to the particular user.
  • Usage-context adapted behaviour (usage context awareness), i.e., the interface is capable of automatically selecting interaction patterns appropriate to the particular physical and technological environment.

At run-time, the adaptations may be of two types:

  • adaptations driven from initial user- and context- information known prior to the initiation of interaction, and
  • adaptations driven by information acquired through interaction monitoring analysis.

The former behaviour is referred to as adaptability (i.e., initial automatic adaptation) reflecting the interface’s capability to automatically tailor itself initially to each individual end-user in a particular context. The latter behaviour is referred to as adaptivity (i.e., continuous automatic adaptation), and characterizes the interface’s capability to cope with the dynamically changing or evolving user and context characteristics.

HCI and AmI

While many problems connected with interaction with the present Information Society are actually linked to a suitable structuring of information and an accessible human system interface, integration within the ambient intelligence environment is much more complex, due to the interplay of different levels, e.g. the physical level with a multiplicity and heterogeneity of intelligent objects in the environment and their need for a continuous and high-speed connection, the level of identification and consideration of the variety of contexts of use, and the level of elicitation of the diversity of user goals and help in their fulfilment [Emiliani 2008].

Thus, in the context of the emerging paradigm of AmI, current research trends have to address:

  • The diversity in the User Population
  • Diversity in the enabling technologies for AmI
  • The need for research on the Lifecycle of User Interfaces (requirements in AmI Environments, Design for All, Development requirements, user experience evaluation)
  • User Interface Development (architectures, Components, Tools) itself
  • Interaction Techniques and Devices
  • The variety of Application Domains
  • A number of non-technological issues (social, ethical, legal issues, privacy, security)

Ambient Intelligence is defined according to its properties as technology which is:

  • Embedded in the physical and social environment of people;
  • Context Aware - employing machine perception a model of activities of people and their social and physical context can be obtained;
  • Personalized - addressing each user as an individual person;
  • Adaptive to context and activities of the person;
  • Anticipatory - predicting user's needs and taking action to support them.

In that respect, the different approaches to Ambient Intelligence are themselves challenging areas for research in HCI, such as:

  • Ubiquitous Computing and communication
  • The disappearing computer and Calm Technology
  • Pervasive and Embedded Everywhere Computing
  • Internet of Devices and Web of Things
  • Ubiquitous Networking
  • Ambient Computing and Ambient Displays
  • Tangible Interfaces

With regards to User Interfaces in particular, an interesting approach could be based on the introduction of the Ambient User Interfaces (AmUIs) paradigm. In contrast to typical graphical user interfaces (GUIs), which are always instantiated on a computer screen, AmUIs can take advantage of the available Ambient Intelligence Infrastructure, in order to support interaction that is tailored to the current needs and characteristics of a particular user and context of use. Thus, they could be multimodal and distributed in the environment (e.g., employ the TV screen and stereo speakers to provide output, and get input through both speech and gestures). These interfaces could allow the interaction between humans and the ambient technological environment in an efficient, effective and intuitive way which also guarantees their well-being, privacy and safety, while on the other hand they could creatively combine the available, dispersed computing devices in order to provide useful, added-value, services. This could support seamless, high-quality, unobtrusive, and fault-tolerant user interaction, by creating software frameworks for developing and orchestrating ambient interactions, and by designing and developing useful ambient  interactive systems that cater in the best possible way for the real needs of their users.

Emerging Issues

Lastly, a number of emerging key issues that arise from the evolution of the Information Society towards Ambient Intelligence environments have to be considered [Stephanidis 2009]. These include:

  • The investigation of human characteristics, abilities, and requirements in the context of AmI
  • Suitable approaches to non-functional characteristics such as accessibility, privacy, security, safety
  • Suitable models of the context of use
  • Appropriate interaction devices and techniques for diverse users and contexts of use,
  • Interaction Design for continuous and implicit interaction
  • Elaboration of design methods suitable for very complex interactive environments,
  • Mechanisms for interaction adaptation,
  • A balance of policy, standardisation and legislation intervention


Roadmapping the development of inclusive accessible technology

CARDIAC project aims to propose a roadmap to guide the European Commission in the development of R&D policies that promote accessibility and inclusion. There exist several other initiatives that, using diverse methods, have conveyed to prospective studies, roadmaps or guidelines with similar purposes. Far to be ignored, all the results from these efforts must be also used as an input to CARDIAC discussions. They can provide inspiration and orientation to the CARDIAC work.
Here we have listed a number of reports from diverse institutions that can serve as examples of technological prospective and roadmapping for the development of accessible and inclusive human-machine interaction.

[Camarinha-Matos 2011] Camarinha-Matos L. M. , Rosas J. D6.1 Interim Roadmap for ICT and Ageing. BRAID European Project. http://braidproject.eu/sites/default/files/BRAID-D6.1final.pdf

[Rodriguez-Ascaso 2010] Rodriguez-Ascaso A., Zetterström E., BöckerM., Hüttenrauch H:, Pluke M., Schneider M. Inclusive E-Services for All: Identifying Accessibility Requirements for Upcoming Interaction Technologies. In Miesenberger K. et al. (Eds.): ICCHP 2010, Part I, LNCS 6179, pp. 135–138. Springer, Berlin, 2010.

[van den Broek 2010]     van den Broek G., Cavallo F., Odetti L., Wehrmann C. (eds.) AALIANCE Ambient Assisted Living Roadmap. IOS Ppress, 2010.

[ETSI EG 202 848-2011]   Human Factors; Inclusive eServices for all: Optimizing the accessibility and the use of upcoming user-interaction technologies. ETSI EG 202 848 V1.1.1, 2011. http://www.etsi.org/deliver/etsi_eg/202800_202899/202848/ 01.01.01_60/eg_202848v010101p.pdf

[European Commission 2007] European Commission. Accelerating the development of the ehealth market in Europe. Luxembourg: Office for Official Publications of the EC. 2007. KK-70-07-022-EN-C. http://ec.europa.eu/information_society/ activities/health/docs/publications/lmi-report-final-2007dec.pdf


 [Abascal 2005]        Abascal J., Nicolle C. Moving towards inclusive design guidelines for socially and ethically aware HCI. Interacting with Computers. 17(5), 2005. Pp. 484-505. 

[Abascal 2008a]      Abascal J. Users with Disabilities: Maximum Control with Minimum Effort. In: Perales F. J., Fisher R.B. (Eds.) AMDO 2008, LNCS 5098, Springer-Verlag, Berlin, 2008. Pp. 449–456.

[Abascal 2008b]      Abascal J., Bonail B., Cagigas D., Garay N., Gardeazabal L. Trends in Adaptive Interface Design for Smart Wheelchairs. In: Lumsden, J. (Ed.) Handbook of Research on User Interface Design and Evaluation for Mobile Technology. Idea Group Reference, Pennsylvania, 2008.

[Antona 2006]         Antona M., Savidis A., Stephanidis C. A Process–Oriented Interactive Design Environment for Automatic User Interface Adaptation. International Journal of Human Computer Interaction, 20 (2), 2006. Pp. 79-116.

[Bergman 1995]      Bergman E., Johnson E. Towards Accessible Human-Computer Interaction. In Nielsen J. (ed.) Advances in Human-Computer Interaction, V. 5, 1995. https://labs.oracle.com/features/tenyears/volcd/papers/Johnson.pdf

[Brusilowsky 2007]          Brusilowsky P. Adaptive Navigation Support. In: Brusilovsky P., Kobsa A., Nejdl W. (Eds.) The Adaptive Web. Methods and Strategies of Web Personalization.  Springer-Verlag, Berlin-Heildeberg, 2007. Pp. 263-290.

[Calvary 2001]         Calvary G., Coutaz J., Thevenin D. A Unifying Reference Framework for the Development of Plastic User Interfaces. In Proc. Eng. Human-Computer Interaction Conf, 2001. Pp. 173-192.

[Doulgeraki 2009]  Doulgeraki C., Partarakis N., Mourouzis A., Stephanidis C. Adaptable Web-based user interfaces: methodology and practice. eMinds, Vol 1, No 5 (2009).

[EC ISTP]                    European Commission – Information Society Thematic Portal: Overview of ongoing Projects FP6 – FP7 – CIP, available at: http://ec.europa.eu/information_society/activities/einclusion/research/projects/index_en.htm

[Emiliani 1999]         Emiliani, P. L., Hine N., Stephanidis, C. (1999), Definition of Models of Interactions, IPSNI Project, Deliverable 2.2.

[Emiliani 2008a]      Emiliani P. L., Billi M., Burzagli L., Gabbanini F.  Design for All in the Ambient Intelligence Environment. Computers Helping People with Special Needs. LNCS Vol. 5105, Springer-Verlag, Berlin-Heildeberg, 2008.

[Emiliani 2008b]      Emiliani P. L., Burzagli L., Billi M., Gabbanini F., Palcheti E. Report on the impact of technological developments on eAccessibility. DfA@eInclusion Coordination Action, 2008. http://www.dfaei.org/deliverables/D2.1.pdf

[Emiliani 2009a]      Emiliani, P., L., Burzagli, L., Billi, M., Gabbanini, F., Palchetti, E. (2009). Report on the impact of technological developments on eAccessibility. DfA@eInclusion Deliverable D2.1. Retrieved available at: http://www.dfaei.org/deliverables/D2.1.pdf

[Emiliani 2009b]      Emiliani, P., L., Aalykke, S., Antona, M., Burzagli, L., Gabbanini, F., Klironomos, I. (2009). Document on necessary research activities related to DfA. DfA@eInclusion Deliverable D2.6. available at: http://www.dfaei.org/deliverables/D2.6.pdf

[Emiliani 2011]         Emiliani P.L. (2001), “Anyone, Anywhere Access to Community-Oriented Services”, 1st International Conference on “Universal Access in Human-Computer Interaction”, New Orleans, pp. 803 – 807.

[Fink 1997]                Fink, J., Kobsa, A., & Nill, A. (1997), “Adaptable and Adaptive Information Access for All Users, Including the Disabled and Elderly”, In A. Jameson, C. Paris, & C. Tasso (Eds.), Proceedings of the 6th International Conference on User Modelling (UM '97), Sardinia, Italy (pp. 171-173), New York: Springer-Verlag.

[ISO 2008a]               ISO 9241-20. 2008. Accessibility guidelines for information/communication technology (ICT) equipment and services.

[ISO 2008b]              ISO 9241-171. 2008. Ergonomics of human-system interaction -- Part 171: Guidance on software accessibility.

[Kobsa 1995]            Kobsa, A., & Pohl, W. (1995), “The user modelling shell system BGP-MS”, User Modelling and User-adapted interaction, 4 (2), 59-106.

[Korozi 2009]           Korozi M., Leonidis A., Margetis G., Stephanidis C. MAID: a Multi-platform Accessible Interface Design Framework. In: C. Stephanidis (Ed.) Universal Access in Human-Computer Interaction -Applications and Services– Vol. 7 Procs. of the 13th Int. Conf. on Human-Computer Interaction (HCI International 2009), San Diego, CA, USA, 2009. Pp. 725-734.

[Miñón 2010]           Miñón R., Aizpurua A., Cearreta I., Garay N., Abascal J. Ontology-Driven Adaptive Accessible Interfaces in the INREDIS project. In: Procs. of the Int. Workshop on Architectures and Building Blocks of Web-Based User-Adaptive Systems, Hawaii. CEUR Workshop Proceedings Vol. 609, 2010. Pp. 37-39.

[Mori 2003]              Mori G., Paterno F., Santoro C. Tool Support for Designing Nomadic Applications. In: Proc. of 7th ACM Int. Conf. on Intelligent User Interfaces IUI’2003. ACM Press, New York, 2003. Pp. 141–148.

[Nicolle  2001]         Nicolle C., Abascal J. (Eds.) Inclusive design guidelines for HCI. Taylor & Francis, London, 2001.
[Obrenovic 2007]   Obrenovic Z., Abascal J., Starcevic D. Universal Accessibility as a Multimodal Design Issue.  Comms. of the ACM. 50(5), 2007. Pp. 27-29.

[Savidis 2004]          Savidis A., Stephanidis C. Unified User Interface Design: Designing Universally Accessible Interactions. Interacting with Computers, 16(2), 2004. Pp. 243-270.

[Savidis 2005]          Savidis A., Stephanidis C. Distributed Interface Bits: Dynamic Dialogue Composition from Ambient Computing Resources. Personal and Ubiquitous Computing, 9(3), 2005. Pp. 142-168.

[Sevillano 2004]      Sevillano J. L., Falcó J., Abascal J., Civit-Balcells A., Jiménez G., Vicente S., Casas R. On the Design of Ambient Intelligent Systems in the Context of Assistive Technologies. In: Miesenberger, K., Klaus, J., Zagler, W., Burger, D. (eds.) ICCHP 2004. LNCS vol. 3118, Springer, Heidelberg, 2004. Pp. 914–921.

[Stephanidis 1995]           Stephanidis, C., Mitsopoulos, Y. (1995), “INTERACT: An interface builder facilitating access to users with disabilities”, HCI International '95, Tokyo, Japan,, pp. 923-928.

[Stephanidis 1997]           Stephanidis, C., Savidis, A., Akoumianakis, D. (1997), “Unified Interface Development: Tools for Constructing Accessible and Usable User Interfaces”, Tutorial Notes 13, 7th International Conference on “Human-Computer Interaction” (HCI International ’97), San Francisco, California, USA.

[Stephanidis 1998]           Stephanidis, C., Salvendy, G., Akoumianakis, D., Bevan, N., Brewer, J., Emiliani, P. L., Galetsas, A., Haataja, S., Iakovidis, I., Jacko, J., Jenkins, P., Karshmer, A., Korn, P., Marcus, A., Murphy, H., Stary, C., Vanderheiden, G., Weber, G., Ziegler, J. (1998a), “Towards an Information Society for All: An International R&D Agenda”, International Journal of Human-Computer Interaction, vol. 10(2), pp. 107-134.

[Stephanidis 2001a]        Stephanidis C. The concept of Unified User Interfaces. In: Stephanidis C. (Ed.) User Interfaces for All - Concepts, Methods, and Tools. Mahwah, NJ: Lawrence Erlbaum Associates, 2001. Pp. 371-388.

[Stephanidis 2001b]        Stephanidis, C. (ed.) (2001), “User Interfaces for All – Concepts, Methods and Tools”, Mahwah, NJ: Lawrence Erlbaum Associates.

[Stephanidis 2009]           Stephanidis C. Emerging Challenges. In: Stephanidis C. (Ed.) The Universal Access Handbook. Taylor & Francis 2009. Pp. 61-1 - 61-9.

[Streitz 2006]           Streitz, N. A. From Human–Computer Interaction to Human–Environment Interaction: Ambient Intelligence and the Disappearing Computer. In: C. Stephanidis and M. Pieper (Eds.) Universal Access in Ambient Intelligence Environments. Procs. of the 9th ERCIM Workshop on User Interfaces for All. Königswinter, Germany, 2006. Pp. 3-13.

[Vogel 2004]            Vogel D., Balakrishnan R. Interactive Public Ambient Displays: Transitioning From Implicit to Explicit, Public to Personal, Interaction with Multiple Users. In: Procs. of the ACM Symp. on User Interface Software and Technology (UIST’04), 2004. Pp. 137–146.

Further readings

 [Ahonen 2008]       Ahonen P., Alahuhta P., Daskala B., Delaitre S., De Hert P., Lindner R, Maghiros I., Moscibroda A., Schreurs W., Verlinden M. Dark scenarios. In: Safeguards in a World of Ambient Intelligence. The International Library of Ethics, Law and Technology, Vol. 1, 2010. Pp. 33-142.

[Ferscha 2006]       Ferscha, A., Resmerita, S., Holzmann, C. Human Computer Confluence. In C. Stephanidis and M. Pieper (Eds.) Universal Access in Ambient Intelligence Environments. Procs. of the 9th ERCIM Workshop on User Interfaces for All. Königswinter, Germany, 2006. Pp. 14-27.

[ISTAG 2001]            IST Advisory Group. “Scenarios for ambient intelligence in 2010” Final Report. Compiled by K. Ducatel, et al. Feb-2001. EC. Brussels, 2001.

[Klironomos 2006] Klironomos I., Antona M., Basdekis I., Stephanidis C. Promoting Design for All and e-Accessibility in Europe. Universal Access in the Information Society, 5(1), 2006. Pp. 105-119.

[Murugesan 2006] Murugesan, S. Understanding Web 2.0. IT Professional 9(4), 2006. Pp. 34-41.

[Roe 2006]                Roe, P. (Ed.) Towards an Inclusive Future: Impact and Wider Potential of Information and Communication Technologies. COST219ter, Brussels, 2006.

[Savidis 2010]          Savidis, A., Stephanidis, C. A unified software architecture for user interface adaptation. In:  C. Stephanidis (Ed.) The Universal Access Handbook. CRC Press, 2010. Pp. 21-1/21-17.

[Stephanidis 1998]           Stephanidis C., Akoumianakis D.,  Sfyrakis M.,  Paramythis A. Universal accessibility in HCI: Process-oriented design guidelines and tool requirements. In: Stephanidis C., Waern A. (Eds.) Procs. of the 4th ERCIM Workshop on User Interfaces for All, 1998. [http://ui4all.ics.forth.gr/UI4ALL-98/proceedings.html]

[Stephanidis 1999]           Stephanidis, C., Salvendy, G., Akoumianakis, Arnold, A, D., Bevan, N., Dardailler, D., Emiliani, P. L., Iakovidis, I., Jenkins, P., Karshmer, A., Korn, P., Marcus, A., Murphy, H., Oppermann, C., Stary, Tamura, H., Tscheligi, M., Ueda, H.,G., Weber, G., Ziegler, J. Toward an Information Society for All: HCI challenges and R&D recommendations. International Journal of Human-Computer Interaction, 11(1), 1999. Pp. 1-28