ICT and HCI between multimodality and
cognitive personalization
Prof. Licia Sbattella
ACCESSIBILITY
Politecnico di Milano - Como
Contents and time schedule
Special Users & Apps
(11/3, 15/3, 18/3, 5/4, 12/4,
20/4, 22/4)
Devices & HC
I (9/3, 16/3, 30/3, 1/4, 8/4, 15/4)
Web Accessibility
(6/4, 13/4, 20/4)
Meeting Users
(18/3, 30/3, 9/4, 13/4, 22/4)
Models, applications, research
presentation, meeting users
l Contents:
1) Universal accessibility: national and international contexts
l The UN Convention on the Rights of Persons with Disabilities
(2007) and its influence on ICT
l The international network: (the role of governmental, non
governmental, professional, industrial and academic subjects)
2) A global reference model to profile users
l Human mind and behavior (embodiment and relational skills)
l Perception, attention, and temporal skills / Emotional and affective
skills
l Interaction and communication / Learning and cooperation
l Personality and subjective differences
l Impairments, disabilities and handicaps
Models, applications, research
presentation, meeting users
3) Multimodal devices and contexts
l Multimodality for innovation and accessibility
l Innovative devices and best accessible solutions for
different abilities (sensorial, physical, cognitive, and relational)
l Best and worst practices and programs in the field of
learning, professional education, autonomy,
communication, rehabilitation and self expression
Models, applications, research
presentation, meeting users
4) Web accessibility
l Accessing the interface and the contents
l Disabilities relied to personal, social or cultural diversities
l How to use the input and the output redundancies: additional
and complementary multimodality
l The WCAG guidelines and other relied documents
l The personal style and the access paradigm: graphics, texts,
gestures, and voices
l The wireless and the mobile context.
l Accessibility of the active web: best and worst practices and
programs.
l How to evaluate the web accessibility taking into account
different points of view
Models, applications, research
presentation, meeting users
5) Focus On
l Different disabilities (cognitive, sensorial, physical, or
relational) but also aged persons or persons with dyslexia (DSA)
l presentation of innovative research programs and
projects
l presentation of a meaningful multimodal and
HCI, Innovative ICT , Accessible Computing,
Assistive Technology
l
Strength and weakness
l Slow / Quick
l Very small / very big
l
A relationship of mutual help
l To be legitimated l To be identified l To be integrated
New attention
l
The OMS paradigm for Persons with Disability (and
not only for them!)
l OMS: ICF (impairment, disability, handicap) – Personal
attributes? Context relied properties?)
l
Motor, Sensorial, Cognitive, Relational Impairments
l Congenital or acquired disabilities
l
The central role of persons
l Wish, responsibility … adult life
l
Different ages
Complexity and Integration
l
Complexity
l Of the mind
l Of the existential dimension
l
Integration
l Neuroscience l Pedagogy
l Psychology and Cognitive Science l Clinic
ICT - Multichannel - Multimodal
l
Accessibility and Navigation
lMultichannel:
l Information systems / adaptive systems
l accessibility and heterogeneity
l
Multimodal:
l Different modality are available simultaneously for accessibility
(visual, auditory, and tactile … ) ... Preferred or constricted?
l
Context awareness:
l technological, relational / cooperative or communicative,
ICT - Wireless & Peer-to-peer
l
Distributed Architecture –
“
user-centered
”
l Cloud Solutions
l peer-to-peer interaction
l Distributed databases and context adaptability
l Flexibility of connections in wireless contexts
l
VSDBs:
l Personal profiles: health, learning, daily life, ….
l
Position detection and navigation
l RFID (passive or active)
l WI-FI contexts
ICT – Adaptability and
Personalization
l
User profile
l behavior analysis: cooperative, communicative, action
oriented …
l preferences, motivation, wish, … l Personal habits and characters, ...
l
Adaptability
l At different levels (interface, design, functional
specification, requirements …)
l Through time
ICT – Devices and Interfaces
l
New integrated modalities are studied to enhance
the interaction
l Auditory / visual / tactile / neurological / relied to specific
actions
l Brain-computer interaction
l Tablet – PC and touch screens
l Gestural interface
l
Main points to be stressed:
l The synaesthesic properties of perception
l Te integration between different steps of the project (design,
ICT – Virtual reality
l
From 2D to 3D spaces
l
Space and time: explorative styles in space
and time
l
The simulated reality: different contexts to
communicate, cooperate, learn, leave,
…
l
Virtual /Real networking
ICT - Robotics
l
Devices
l Actuators / manipulators
l To help or to substitute human action
l
Different level of:
l programmability l personalization
l
The role of the object
ICT – Signal Processing
l
Less invasive devices and processing
l Signal detection is possible also in complex situations,
taking into account daily life conditions.
l
The detection of meaningful signals
l Allows a better detection of the brain activity l Conditions are still complex
l
New applications make innovative studies
more and more significant
ICT – Computational Linguistics
l The analysis and the production of written or spoken texts
l At different level (morphological, lexical, syntactical, semantic and
pragmatic levels, …)
l Making navigation more efficient and accessible
l the analysis of complexity, summarization, the detection of the text
structure, data mining,…
l Translation between different formats and languages.
l Iconic, oral, or written languages ...
l Guidelines, standards and protocols,…
l Linguistic Prediction
AT – Learning and playing
l
Playing: new opportunities
l real / virtual l space / time
l cooperation/ personal exploration
l
Learning: new opportunities
l Self and world exploration
l Kids as actors – authors or protagonists
l A constructive paradigm and a cooperative context l Imbalance between costs, results in time
AT – Multimodal Communication
lMultimodal integration
l Linguistics / Textual
l Auditory / vocal / musical l Tactile / gestural / animated
l Iconic / dealing with representation
l
Rewarding and prestigious
l
Solutions for
AT - User Empowerment
l
Daily life supports
l Functional supports
l Communicative Supports l Cognitive Assistants
l Decision Making Supports
l
The user centered paradigm
AT – Sensorial Rehabilitation
lSensorial Disability:
l visual l auditive l … lVirtual Contexts
l
More or less invasive techniques
AT- Rehabilitation and Robotics
l
Main topics:
l functionality
l programmability
l
Support to functional rehabilitation
l Definition to the rehabilitative strategy l Its evaluation and validation
AT - BCI
l
Neuroscience, Neuropsychology , Cognitive
Psychology, Clinics
l
Signal Detection
l Monitoring rehabilitative process
l Modeling mental process at different level
l
Applications
l Rehabilitation
R&D – The Relational Mind
l The Mind is ‘multimodal’ and ‘complex’
l Perception l Synaesthesia l Spatial Dimensions l Temporal Dimensions l Memory
l Attention / Shared Attention l Emotional and Affective Aspects l Language / Dialogue / Linguistic acts
l Identification (adult and prestigious artifacts)
l Responsibility / Management
R&D – Imitation and Simulation
l
Mirror Neurons:
l A neurophysiological base for imitation and simulation
l
The relationship is stressed
l To learn
l To cooperate
l To communicate
l To self identify
l
“
I know what You do
”
l Empathy and non-verbal resonant channel
l Imitation / embodied simulation
POLIMI – The ATGroup
Assistive Technology Group, the mission:
Identify, forecast, implement, promote and apply
innovative method and technology from ICT
for the development of sustainable solutions
for disabled and elderly
which guarantees the recovery of functionality,
social integration, equal opportunities, health,
POLIMI - MultiChancePoliTeam
l
The wireless Campus
l 400 students with motor, sensorial disability and specific
learning impairments
l Accessible ICT and innovative projects
l Personalized access to distributed services
l On-line studying (but only when really needed)
l Special projects
l UbiCampus l 4Wheels
l Vocalization, prediction and correction for Dyslexia l PoliNotes, PoliSpell, PoliLips,
l SPARTA2 l KASPER
POLIMI - ESAGRAMMA
Multimodal Communication
l
For Kids , Young and Adults with autism and
mental retard
l
Cooperative activities of:
l storytelling
l multimodal interaction (verbal, textual, auditory- musical,
tactile, gestural, iconic, AAC, …)
Universal Accessibility: UN policies
lG3ict The Global Initiative for Inclusive ICTs. A Flagship
Advocacy Initiative of the United Nations Global Alliance
for ICT and Development
–
www.g3ict.com
l
International Projects:
– Digital Accessibility Index for Persons with DIsabilities
(G3ict /ITU / POLIMI: DEI, Mcpt )
– Toolkit for Policy Makers on e-Accessibility and Service
Needs for Persons with Disabilities (G3ict / ITU / POLIMI: DEI, Mcpt )
– ICT4IAL – Guidelines for accessiiblity of contents and
R&D - EU policy
l
Strategic research items:
l Standardization and protocols
l Cognitive Disability (and elderly)
l Multimodal Interaction
l Brain Computer Interaction
l
User profiling
l and validation of strategies
l
Low cost availability of devices and best practices
l Different stackholders
l Not a specific strategy but a “user-centered” DESIGN POLICY
AT – International Contexts
l Rehabilitation Engineering and AT Society of North
America - RESNA– www.resna.org
l North Caroline State Univ. Center for Universal Design - www.design.ncsu.edu/cud/
l International Conferences and Workshops
l AAATE (Europe)
l ICCHP (IFIP)
l ASSETS (ACM)
l HCI, CHI
l EDULEARN, ICERI, …
l Magazine and Scientific Journals
l Computer Science (HCI, e-learning, Web Services, MM e MC,..)
l Psychology – pedagogy – neuro-psychology
l …
l Specific references: ASPHI, SIVA, ESTEEN (portal), ATIS4all (portal), HANDYMATICA (Bologna)
Bibliography
l 1) INTERACTION DESIGN. BEYOND HUMAN-COMPUTER
INTERACTION, 3rd Edition, Y. Rogers, H. Sharp, J. Preece, Wiley,
2011
l 2) ASSISTIVE TECHNOLOGY, Principles and Practice, 3rd Edition,
A. M. Cook, J. Miller Polgar, S. M. Hussey (author emerita), Elsevier, 2008.
l 3) The SLIDES prepared by the teachers
Further readings:
l 1) INCLUSIVE DESIGN GUIDELINES FOR HCI, Colette Nicolle, Julio Abascal, Taylor & Francis, 2001
l 2) JUST ASK: INTEGRATING ACCESSIBILITY THROUGHOUT DESIGN, Henry, Shawn Lawton, 2007
Bibliography
l 4) UNDERSTANDING ACCESSIBILITY , Robert B. Yonaitis, HiSoftware, 2002 - pdf on-line
l 5) ASSISTIVE TECHNOLOGY. Matching device and consumer for successful rehabilitation, M. J, Scherer (editor), American Psychological Association, 2001.
l 6) CONNECTING TO LEARN. Educational and Assistive Technology for People with disabilities, M. Scherer, American Psychological Association, 2003.
l 7) ASSISTIVE TECHNOLOGY IN THE WORKPLACE, D. de Jonge, M. J. Scherer, S. Rodger, Mosby Elsevier, 2007
l 8) EXTRA-ORDINARY HUMAN-COMPUTER INTERACTION. Interfaces for users with disabilities, A. D. N. Edwards (editor), Cambridge University Press, 1995.
l 9) AUGMENTATIVE AND ALTERNATIVE COMMUNICATION: European Perspectives, S. Von Tetzchner, M. H. Jensen (editors), Whurr Publishers Ltd, London, 1997
l 10) The G3ict publications (see www.g3ict.org)
l 11) Selected papers from the Proceeding of International Conferences and Workshops HCI, AAATE, ICCHP, ASSETS
