ICT and HCI between multimodality and cognitive personalization

33 

Loading....

Loading....

Loading....

Loading....

Loading....

Full text

(1)

ICT and HCI between multimodality and

cognitive personalization

Prof. Licia Sbattella

ACCESSIBILITY

Politecnico di Milano - Como

(2)

Contents and time schedule

Special Users & Apps

(11/3, 15/3, 18/3, 5/4, 12/4,

20/4, 22/4)

Devices & HC

I (9/3, 16/3, 30/3, 1/4, 8/4, 15/4)

Web Accessibility

(6/4, 13/4, 20/4)

Meeting Users

(18/3, 30/3, 9/4, 13/4, 22/4)

(3)

Models, applications, research

presentation, meeting users

l  Contents:

1) Universal accessibility: national and international contexts

l  The UN Convention on the Rights of Persons with Disabilities

(2007) and its influence on ICT

l  The international network: (the role of governmental, non

governmental, professional, industrial and academic subjects)

2) A global reference model to profile users

l  Human mind and behavior (embodiment and relational skills)

l  Perception, attention, and temporal skills / Emotional and affective

skills

l  Interaction and communication / Learning and cooperation

l  Personality and subjective differences

l  Impairments, disabilities and handicaps

(4)

Models, applications, research

presentation, meeting users

3) Multimodal devices and contexts

l  Multimodality for innovation and accessibility

l  Innovative devices and best accessible solutions for

different abilities (sensorial, physical, cognitive, and relational)

l  Best and worst practices and programs in the field of

learning, professional education, autonomy,

communication, rehabilitation and self expression

(5)

Models, applications, research

presentation, meeting users

4) Web accessibility

l  Accessing the interface and the contents

l  Disabilities relied to personal, social or cultural diversities

l  How to use the input and the output redundancies: additional

and complementary multimodality

l  The WCAG guidelines and other relied documents

l  The personal style and the access paradigm: graphics, texts,

gestures, and voices

l  The wireless and the mobile context.

l  Accessibility of the active web: best and worst practices and

programs.

l  How to evaluate the web accessibility taking into account

different points of view

(6)

Models, applications, research

presentation, meeting users

5) Focus On

l  Different disabilities (cognitive, sensorial, physical, or

relational) but also aged persons or persons with dyslexia (DSA)

l  presentation of innovative research programs and

projects

l  presentation of a meaningful multimodal and

(7)

HCI, Innovative ICT , Accessible Computing,

Assistive Technology

l 

Strength and weakness

l  Slow / Quick

l  Very small / very big

l 

A relationship of mutual help

l  To be legitimated l  To be identified l  To be integrated

(8)

New attention

l 

The OMS paradigm for Persons with Disability (and

not only for them!)

l  OMS: ICF (impairment, disability, handicap) – Personal

attributes? Context relied properties?)

l 

Motor, Sensorial, Cognitive, Relational Impairments

l  Congenital or acquired disabilities

l 

The central role of persons

l  Wish, responsibility … adult life

l 

Different ages

(9)

Complexity and Integration

l 

Complexity

l  Of the mind

l  Of the existential dimension

l 

Integration

l  Neuroscience l  Pedagogy

l  Psychology and Cognitive Science l  Clinic

(10)

ICT - Multichannel - Multimodal

l 

Accessibility and Navigation

l 

Multichannel:

l  Information systems / adaptive systems

l  accessibility and heterogeneity

l 

Multimodal:

l  Different modality are available simultaneously for accessibility

(visual, auditory, and tactile … ) ... Preferred or constricted?

l 

Context awareness:

l  technological, relational / cooperative or communicative,

(11)

ICT - Wireless & Peer-to-peer

l 

Distributed Architecture –

user-centered

l  Cloud Solutions

l  peer-to-peer interaction

l  Distributed databases and context adaptability

l  Flexibility of connections in wireless contexts

l 

VSDBs:

l  Personal profiles: health, learning, daily life, ….

l 

Position detection and navigation

l  RFID (passive or active)

l  WI-FI contexts

(12)

ICT – Adaptability and

Personalization

l 

User profile

l  behavior analysis: cooperative, communicative, action

oriented …

l  preferences, motivation, wish, … l  Personal habits and characters, ...

l 

Adaptability

l  At different levels (interface, design, functional

specification, requirements …)

l  Through time

(13)

ICT – Devices and Interfaces

l 

New integrated modalities are studied to enhance

the interaction

l  Auditory / visual / tactile / neurological / relied to specific

actions

l  Brain-computer interaction

l  Tablet – PC and touch screens

l  Gestural interface

l 

Main points to be stressed:

l  The synaesthesic properties of perception

l  Te integration between different steps of the project (design,

(14)

ICT – Virtual reality

l 

From 2D to 3D spaces

l 

Space and time: explorative styles in space

and time

l 

The simulated reality: different contexts to

communicate, cooperate, learn, leave,

l 

Virtual /Real networking

(15)

ICT - Robotics

l 

Devices

l  Actuators / manipulators

l  To help or to substitute human action

l 

Different level of:

l  programmability l  personalization

l 

The role of the object

(16)

ICT – Signal Processing

l 

Less invasive devices and processing

l  Signal detection is possible also in complex situations,

taking into account daily life conditions.

l 

The detection of meaningful signals

l  Allows a better detection of the brain activity l  Conditions are still complex

l 

New applications make innovative studies

more and more significant

(17)

ICT – Computational Linguistics

l  The analysis and the production of written or spoken texts

l  At different level (morphological, lexical, syntactical, semantic and

pragmatic levels, …)

l  Making navigation more efficient and accessible

l  the analysis of complexity, summarization, the detection of the text

structure, data mining,…

l  Translation between different formats and languages.

l  Iconic, oral, or written languages ...

l  Guidelines, standards and protocols,…

l  Linguistic Prediction

(18)

AT – Learning and playing

l 

Playing: new opportunities

l  real / virtual l  space / time

l  cooperation/ personal exploration

l 

Learning: new opportunities

l  Self and world exploration

l  Kids as actors – authors or protagonists

l  A constructive paradigm and a cooperative context l  Imbalance between costs, results in time

(19)

AT – Multimodal Communication

l 

Multimodal integration

l  Linguistics / Textual

l  Auditory / vocal / musical l  Tactile / gestural / animated

l  Iconic / dealing with representation

l 

Rewarding and prestigious

l 

Solutions for

(20)

AT - User Empowerment

l 

Daily life supports

l  Functional supports

l  Communicative Supports l  Cognitive Assistants

l  Decision Making Supports

l 

The user centered paradigm

(21)

AT – Sensorial Rehabilitation

l 

Sensorial Disability:

l  visual l  auditive l  … l 

Virtual Contexts

l 

More or less invasive techniques

(22)

AT- Rehabilitation and Robotics

l 

Main topics:

l  functionality

l  programmability

l 

Support to functional rehabilitation

l  Definition to the rehabilitative strategy l  Its evaluation and validation

(23)

AT - BCI

l 

Neuroscience, Neuropsychology , Cognitive

Psychology, Clinics

l 

Signal Detection

l  Monitoring rehabilitative process

l  Modeling mental process at different level

l 

Applications

l  Rehabilitation

(24)

R&D – The Relational Mind

l  The Mind is ‘multimodal’ and ‘complex’

l  Perception l  Synaesthesia l  Spatial Dimensions l  Temporal Dimensions l  Memory

l  Attention / Shared Attention l  Emotional and Affective Aspects l  Language / Dialogue / Linguistic acts

l  Identification (adult and prestigious artifacts)

l  Responsibility / Management

(25)

R&D – Imitation and Simulation

l 

Mirror Neurons:

l  A neurophysiological base for imitation and simulation

l 

The relationship is stressed

l  To learn

l  To cooperate

l  To communicate

l  To self identify

l 

I know what You do

l  Empathy and non-verbal resonant channel

l  Imitation / embodied simulation

(26)

POLIMI – The ATGroup

Assistive Technology Group, the mission:

Identify, forecast, implement, promote and apply

innovative method and technology from ICT

for the development of sustainable solutions

for disabled and elderly

which guarantees the recovery of functionality,

social integration, equal opportunities, health,

(27)

POLIMI - MultiChancePoliTeam

l 

The wireless Campus

l  400 students with motor, sensorial disability and specific

learning impairments

l  Accessible ICT and innovative projects

l  Personalized access to distributed services

l  On-line studying (but only when really needed)

l  Special projects

l  UbiCampus l  4Wheels

l  Vocalization, prediction and correction for Dyslexia l  PoliNotes, PoliSpell, PoliLips,

l  SPARTA2 l  KASPER

(28)

POLIMI - ESAGRAMMA

Multimodal Communication

l 

For Kids , Young and Adults with autism and

mental retard

l 

Cooperative activities of:

l  storytelling

l  multimodal interaction (verbal, textual, auditory- musical,

tactile, gestural, iconic, AAC, …)

(29)

Universal Accessibility: UN policies

l 

G3ict The Global Initiative for Inclusive ICTs. A Flagship

Advocacy Initiative of the United Nations Global Alliance

for ICT and Development

www.g3ict.com

l 

International Projects:

–  Digital Accessibility Index for Persons with DIsabilities

(G3ict /ITU / POLIMI: DEI, Mcpt )

–  Toolkit for Policy Makers on e-Accessibility and Service

Needs for Persons with Disabilities (G3ict / ITU / POLIMI: DEI, Mcpt )

–  ICT4IAL – Guidelines for accessiiblity of contents and

(30)

R&D - EU policy

l 

Strategic research items:

l  Standardization and protocols

l  Cognitive Disability (and elderly)

l  Multimodal Interaction

l  Brain Computer Interaction

l 

User profiling

l  and validation of strategies

l 

Low cost availability of devices and best practices

l  Different stackholders

l  Not a specific strategy but a “user-centered” DESIGN POLICY

(31)

AT – International Contexts

l  Rehabilitation Engineering and AT Society of North

America - RESNA– www.resna.org

l  North Caroline State Univ. Center for Universal Design - www.design.ncsu.edu/cud/

l  International Conferences and Workshops

l  AAATE (Europe)

l  ICCHP (IFIP)

l  ASSETS (ACM)

l  HCI, CHI

l  EDULEARN, ICERI, …

l  Magazine and Scientific Journals

l  Computer Science (HCI, e-learning, Web Services, MM e MC,..)

l  Psychology – pedagogy – neuro-psychology

l  …

l  Specific references: ASPHI, SIVA, ESTEEN (portal), ATIS4all (portal), HANDYMATICA (Bologna)

(32)

Bibliography

l  1) INTERACTION DESIGN. BEYOND HUMAN-COMPUTER

INTERACTION, 3rd Edition, Y. Rogers, H. Sharp, J. Preece, Wiley,

2011

l  2) ASSISTIVE TECHNOLOGY, Principles and Practice, 3rd Edition,

A. M. Cook, J. Miller Polgar, S. M. Hussey (author emerita), Elsevier, 2008.

l  3) The SLIDES prepared by the teachers

Further readings:

l  1) INCLUSIVE DESIGN GUIDELINES FOR HCI, Colette Nicolle, Julio Abascal, Taylor & Francis, 2001

l  2) JUST ASK: INTEGRATING ACCESSIBILITY THROUGHOUT DESIGN, Henry, Shawn Lawton, 2007

(33)

Bibliography

l  4) UNDERSTANDING ACCESSIBILITY , Robert B. Yonaitis, HiSoftware, 2002 - pdf on-line

l  5) ASSISTIVE TECHNOLOGY. Matching device and consumer for successful rehabilitation, M. J, Scherer (editor), American Psychological Association, 2001.

l  6) CONNECTING TO LEARN. Educational and Assistive Technology for People with disabilities, M. Scherer, American Psychological Association, 2003.

l  7) ASSISTIVE TECHNOLOGY IN THE WORKPLACE, D. de Jonge, M. J. Scherer, S. Rodger, Mosby Elsevier, 2007

l  8) EXTRA-ORDINARY HUMAN-COMPUTER INTERACTION. Interfaces for users with disabilities, A. D. N. Edwards (editor), Cambridge University Press, 1995.

l  9) AUGMENTATIVE AND ALTERNATIVE COMMUNICATION: European Perspectives, S. Von Tetzchner, M. H. Jensen (editors), Whurr Publishers Ltd, London, 1997

l  10) The G3ict publications (see www.g3ict.org)

l  11) Selected papers from the Proceeding of International Conferences and Workshops HCI, AAATE, ICCHP, ASSETS

Figure

Updating...

References

Updating...

Related subjects :