Development of a computer vision-based technique for analysis of hand therapy exercise

Universiti

Malaysia

PAHANG

SUPERVISOR'S DECLARATION

We hereby declare that we have checked this thesis and in our opinion, this thesis is adequate in terms of scope and quality for the award of the degree of Master of Engineering.

(Supervisor's Signature)

Full Name : DR. ROSDIY ANA BINTI SAMAD Position

Date

: SENIOR LECTURER ~~

I If

I

UJI'l

(Co-supervisor's Signature) Full Name : DR. DWI PEBRIANTI Position

Date

: SENIOR LECTURER

:

~

6(£+(

~l'b

Universiti

Malaysia

PAHANG

STUDENT'S DECLARATION

I hereby declare that the work in this thesis is based on my original work except for quotations and citations which have been duly acknowledged. I also declare that it has not been previously or concurrently submitted for any other degree at Universiti Malaysia Pahang or any other institutions.

(Student's Signature)

Full Name : MUHAMMAD ZABRI BIN ABU BAKAR ID Number : MEE14001

DEVELOPMENT OF A COMPUTER VISION-BASED TECHNIQUE FOR ANALYSIS OF HAND THERAPY EXERCISE

MUHAl\1MAD ZABRI BIN ABU BAKAR

Thesis submitted in fulfillment of the requirements for the award of the degree of

Master of Engineering

Faculty ofElectrical & Electronics Engineering UNIVERSITI MALAYSIA PAHANG

ACKNOWLEDGEMENTS

In the name of Allah, the most Beneficent, the most Merciful, lord of the universe, Praise is to Allah who gave me the power, the strength, the motivation, help and patience to complete this study while going through so many hurdles and obstacles. Blessings and peace be upon our final prophet and messenger Muhammad (peace be upon him) who was sent by Allah to be a great teacher of human kind. First and above all, I would like to thank God for giving me great health and all the internal resources for the fulfilment of this master endeavour.

I would like to express my heartiest thanks to my supervisor, Dr. Rosdiyana Binti Samad, for receiving me with open arms and providing endless support, advice as well

as devotional time throughout my research. Thank you for teaching me on how to

arrange my research and innovate my ideas to achieve my goal. You have been my inspiration in both personal and professional aspects and do not forget to my co-supervisor Dr. Dwi Pebrianti for the support and advice. Thank you.

I would also like to extend my acknowledgement to my family for their affectionate understanding and love throughout my master study, my father Abu Bakar bin Mamat, my mother Zanariah binti Isa, my wife Noraziah binti Razali, my sister and my brother. No words could express my deepest gratitude for them. Their support and endless efforts have relieved my worries and enabled me to concentrate on my research.

Lastly, my deepest gratitude goes to all the people who have directly or indirectly

involved in this research. Without them, this research could never have been completed.

ABSTRAK

Senaman terapi tangan berdasarkan sistem penglihatan komputer mempunyai banyak manfaat dan telah menarik minat para penyelidik ke arah membangunkan aplikasi penglihatan komputer. Inovasi alat pengukuran untuk senaman telah mengalami penambahbaikan yang berterusan hingga ke hari ini, di mana sebahagiannya menggunakan "wearable sensor", bantuan robot dan penglihatan komputer. Sistem yang telah dibina menggunakan kaedah penglihatan komputer melibatkan struktur badan manusia yang fleksibel dan mennyebabkan "self-occlusion". Masalah lain ialah teknik pengesanan menggunakan kamera biasa yang tidak mudah dan memerlukan masa untuk dibangunkan. Tujuan penyelidikan ini adalah untuk memban!:,runkan teknik berasaskan penglihatan komputer untuk menganalisis senaman terapi tangan dan untuk menilai pengukuran sudut yang telah dicadangkan dengan alat pengukuran konvensional, iaitu goniometer. Dalam tesis ini, senaman Latihan Bergerak (Range of Motion (ROlvf)) untuk pergelangan tangan dan siku dengan menggunakan sensor Kinect dibentangkan dan senaman ini memberikan maklum balas dalam masa nyata untuk setiap pergerakan. Kedua-dua senaman, iaitu pergelangan tangan "radial/ulnar" dan lenturan siku ".flexion/extension" adalah terdiri daripada pengukuran sudut pada sendi. Algoritma yang digunakan adalah pengesanan dan penjejakkan tangan, pengesanan hujung jari, pengesanan sendi rangka dan juga algoritma pengukuran sudut pada sendi. Kemudian, nilai-nilai sudut sendi boleh disimpan secara automatik. Hasil keputusan bagi pengukuran sudut sisihan pergelangan tangan "radial/ulnar" menunjukkan perbezaan daripada sudut rujukan sebanyak ±6°. Manakala, hasil bagi pengukuran sudut sendi siku "jlexion1_{extension" adalah sebanyak ±8° dari sudut rujukan. Nilai-nilai ukuran hampir} sama dengan nilai-nilai rujukan piawai ROM iaitu 20° untuk "radial", -30° untuk "ulnar", 140° untuk ·:flexion" dan 0° untuk "extension". Algoritma telah berjalan dengan baik dan boleh mengikuti pergerakan pada masa nyata. Sistem senaman menggunakan sensor Kinect mampu menjadi alat latihan yang mudah alih, mudah untuk dipasang dan diletakkan di mana-mana. Data pengukuran sudut boleh disimpan dan boleh digunakan untuk rujukan pada masa hadapan. Pengesahan dengan alat konvensional dalam pengukuran sudut menunjukkan bahawa ukuran sudut adalah lebih tepat berbanding goniometer kerana ia boleh mengukur skala yang lebih kecil.

ABSTRACT

Hand therapy exercises based on computer vision system have many benefits and this has attracted the interest of researchers towards building a computer vision application. The innovation of measurement tool in computer vision application for the exercise also have undergone continuously improvement which is using wearable sensor, robot aid and computer vision. The hand therapy exercise system that was built using computer vision method involved highly flexible structure of the human body and causes a self-occlusion. Another problem is tracking techniques using an ordinary camera are not easy and require extensive time to develop. The aim of this research is to develop a computer vision-based technique for analysis of hand therapy exercise and to evaluate the propose angle measurement with the conventional measurement tool, which is goniometer. In this thesis a Range of Motion (ROM) exercises for wrist and elbow by using the Kinect sensor is presented and this exercise provides a real-time feedback for every movement. Both exercises, which are wrist radial/ulnar deviation and elbow flexion/extension also consists of joint angle measurement. For this purpose, the ROM exercise was developed by using the hand detection and tracking, fingertips detection, skeleton joints detection algorithms and joint angle measurement algorithm. Then, the

joint angle values can be saved automatically. The result for the measurement of wrist

radial/ulnar deviation angle shows its value is different from the reference angle of ±6°.

Meanwhile, the result for the measurement of elbow flexion/extension joint angle was different at ±8° from the reference angle. These measurement values are almost similar to the standard ROM reference values which are 20° for radial deviation, -30° for ulnar deviation, 140° for flexion and 0° for extension. The algorithm worked well and could

follow the movement of the upper arm and forearm in real-time. This exercise system

that uses Kinect sensor is able to be a portable exercise tool, which is easy to install and to be placed anywhere. The joint angle measurement data can be saved and can be used for reference in the future. The validation with the conventional tool for angle measurement (goniometer) shows that the joint angle measurement with the proposed technique is more precise compared to the goniometer because it can measure the small scale of angle in degree.

TABLE OF CONTENT DECLARATION TITLE PAGE ACKNOWLEDGEMENTS ABSTRAK ABSTRACT TABLE OF CONTENT LIST OF TABLES LIST OF FIGURES LIST OF SYMBOLS LIST OF ABBREVIATIONS CHAPTER 1 INTRODUCTION 1.1 Introduction 1.2 Problem Statement

1.2.1 Conventional monitoring physical therapy program

1.2.2 The joint angle measurement approach in computer vision based physical therapy 1.3 Objectives of Research 1.4 Scope of Thesis 1.5 Contribution of Research 1.6 Thesis Outline v ii iii iv v

viii

ix xii ii 1 1 2 2 3 5 5 5 6

CHAPTER 2 LITERATURE REVIEW 7

2.1

Introduction

7

2.2

Hand Injuries

8

2.

3

Hand Therapy Exercise

12

2.4

Technologies in Hand Physical Exercise 13

2.4.1

Kinect Sensor Approaches in Physical Exercise

14

2.4.2

Other Approaches in Physical Exercise

18

2

.5

Swnmary

20

CHAPTER 3 METHODOLOGY 23

3.1

Introduction

23

3.2

Hardware Setup

24

3.3

Software Requirement

27

3.4

Experimental Setup

28

3.5

hnageNideo Acquisition by Kinect Sensor

30

3.6

Data Collection and Exercise Protocol

31

3.7

Wrist Radial/Ulnar Deviation Exercise System

33

3.7.1

Hand Detection and Tracking using NiTE

34

3

.

7.2

Hand Segmentation

37

3.7.3

Centre of Palm Detection and Tracking

41

3.7.4

Fingertips Detection

44

3.7.5

Deviation Angle Calculation

45

3.7.6

User Interface

48

3.7

.7

Experiments

49

3.8 Elbow Flexion/Extension Exercise System

3.8.1 Skeleton Detection and Tracking using NiTE 3.8.2 Joint Detection and Angle Calculation 3.8.3 User Interface

3.8.4 Experiments

3.9 Summary

CHAPTER 4 RESULTS AND DISCUSSION

4.1 Introduction

4.2 Result of Wrist Radial/Ulnar Deviation Exercise

4.2.1 Hand Detection and Tracking 4.2.2 Fingertips Detection

4.2.3 Wrist Radial and Ulnar Deviation Exercise System

4.2.4 Skeleton Detection

4.2.5 Elbow Flexion and Extension Exercise System

CHAPTER 5 CONCLUSION AND FUTURE WORK 5.1 5.2 5.3 Introduction Conclusion Future Works REFERENCES LIST OF PUBLICATIONS Vll 52 54 56 58 60 62 63 63 64 64 68 72 82 85 93 93 93 95 96 104

LIST OF TABLES

Table 3.1 Steps for the center of palm detection using convex hull. Table 4.1 Result of the correct fingertips detection.

Table 4.2 Processing time performance

viii

43 71 72

LIST OF FIGURES

Figure 1.1 Goniometer and the scale of the goniometer. 4

Figure 1.2 Two type of ROM measuring angle (a) wrist radial/ulnar (b) elbow

flexion/extension. 4

Figure 2.1 A diagram illustrating of the hand rahabilitation from hand injuries to

physical therapy 7

Figure 2.2 illustration of the carpal Immel Syndrome. 9

Figure 2.3 Rheumatoid arthritis hand sympthoms 10

Figure 2.4 Types of carpal tunnel syndrome exercise 11

Figure 2.5 Goniometer 13

Figure 2.6 Example of measurement using goniometer 13

Figure 2.7 A game for hand exercise (a) illustration of the system's setup (b)

Screenshot of the game 15

Figure 2.8 The Tai Chi movement process 17

Figure 2.9 The scoring of the Tai Chi perfonning 18

Figure 2.10 The motion assistive exoskeleton robot (a) ETS-MARSE (b) A few

types of ROM 19

Figure 2.11 The hand movement with 3D cuboid (a) Setup for rehabilitation (b)

Virtual 3D space with different colours and shapes 19

Figure 3.1 Experimental Analysis for Validation of the Proposed Method for

Hand Therapy Exercise 25

Figure 3.2 Working principle! ofK.inect depth image. 26

Figure 3.3 Kinect sensor structure 26

Figure 3.4 Software and hardware diagram for this hand exercise technique 28

Figure 3.5 The limited range of Kinect view 29

Figure 3.6 illustration of the experimental setup. 29

Figure 3.7 Sample of data (a) RGB data stream (b) Depth data stream collected in

OpenCV matrix. 30

Figure 3.8 The location where that the expetiment was conducte. 32

Figure 3.9 Flow chart of the wrist radial/ulnar exercise system 35

Figure 3.10 Hand initialisation (a) Hand wave (b) Hand Raise 36

Figure 3.11 Depth image of hand tracking (a) Depth image before the detection (b)

Detected hand in depth image 36

Figure 3.12 Automated resizing ofROI (a) Size ofROI increase (b) Decrease for

auto resize of ROI. 38

Figure 3.13 Illustration of the parameter original ROI. 38

Figure 3.14 The illustration of the depth threshold 40

Figure 3.15 Hand image (a) Binary image of hand detection (b) Contour of hand

detection 41

Figure 3.16 K-Curvature parameter and additional parameter. 45

Figure 3.17 Candidate for fingertips (a) before (b) after grouping. 45

Figure 3.18 Wrist ROM movement (a) Ulnar deviation movement (b) Radial

deviation movement. 46

Figure 3.19 Flow map of the deviation angle calculation 47

Figure 3.20 Wrist deviation angle calculation 48

Figure 3.21 Template of wrist radial/ulnar deviation exercise. 49

Figure 3.22 Illustration of the reference coordinate rule 51

Figure 3.23 Example of evaluation table that automatically generated for wrist

exercise. 52

Figure 3.24 Flow chart of the elbow f1exion/extension exercise system 53

Figure 3.25 NiTE Skeleton 56

Figure 3.26 Upper limb skeleton detection. 56

Figure 3.27 Angle calculation between two vectors (a) Vector at skeleton joint (b)

lllustration 3D vector 57

Figure 3.28 Template of elbow flexion/extension exercise (right hand)

Figure 3.29 Template of elbow flexion/extension exercise (left hand)

Figure 3.30 Example of the hand in flexion and extension position and also the

reference points.

Figure 3 .31. Exan1ple of evaluation table that automatically generated for elbow exercise

Figure 4.1 The effect ofROI (a) The output using automated resize ROI (b)

Shows the output using fixed ROI

Figure 4.2 Threshold of hand at a distance (z-axis) of (a) origin+40mm (b)

origin-60mm (c) origin+40mm and origin-origin-60mm

Figure 4.3 The process to remove unwanted object near to the hand (a-d)

Figure 4.4 Invariant rotation of hand movement (a) Sequence of rotation hand (b)

Invariant rotation of hand tracking and center of palm detection.

Figure 4.5 Finger counting base on fingertips detection (a)One finger, (b)Two

finger, (c) Three finger, (d)Four finger and (e) Five finger.

Figure 4.6 Average time taken during the wrist exercise.

Figure 4.7 Progress result for subject 1 for one times exercise

Figure 4.8 Pogress result for five selected subject for one times exercise

Figure 4.9 Result of the wrist radial/ulnar deviation exercise for right hand

Figure 4.10 Result of the wrist radial/ulnar deviation exercise for left hand

Figure 4.11 Comparison of the angle measurement for the wrist radial deviation of

the right hand

Figme 4.12 Comparison of the angle measurement for the wrist ulnar deviation of

the right hand

Figure 4.13 Comparison of the angle measurement for the wrist radial deviation of

the left hand

Figure 4.14 Comparison of the angle measurement for the wrist ulnar deviation of

the right hand

Xl 58 59 61 62 65 66 67 68 70 73 74 74 76 77 78 78 79 79

Figure 4.15 Image for wrist radial deviation exercise. 80 Figure 4.16 linage for wrist ulnar deviation exercise. 81 Figure 4.17 Image for f1tll template wrist radial and ulnar deviation exercise. 81 Figure 4.18 Skeleton detection (a) RGB image (b) depth image during skeleton

initialization 82

Figure 4.19 Result for cross product angle calculation 83 Figure 4.20 Position of elbow joint point at x-axis during the measurement of

angle 130° 84

Figure 4.21 Position of elbow joint point at y-axis during the measurement of

angle 130° 85

Figure 4.22 Position of elbow joint point at z-axis during the measurement of

angle 130° 85

Figure 4.23 Elbow movement for flexion and extension. 86 Figure 4.24 Result of the elbow flexion/extension exercise for right hand 87 Figure 4.25 Result of the elbow flexion/extension exercise for left hand 87 Figure 4.26 Comparison of the angle measurement for the elbow flexion of the

tight hand 89

Figure 4.27 Comparison of the angle measurement for the elbow extension of the

right hand 89

Figure 4.28 Comparison of the angle measurement for the elbow flexion of the left

hand 90

Figure 4.29 Comparison of the angle measurement for the elbow extension of the

left hand 90

Figure 4.30 Image of elbow extension exercise. 91

Figure 4.31 Image of elbow flexion exercise. 91

Figure 4.32 Image of overall user interface of elbow flexion/extension exercises. 92

m mm em

s

ms 0 dz TZI TZ2 DpiJ

P;

pi

c

H R (J P J (x,y) H(x,y) V1 V2 LIST OF SYMBOLS . Metre Millimetres Centimetre Seconds Milliseconds Degree Scaling Factor

Frame Pixel, x and y coordinate Frame Pixel, x and y coordinate Frame Pixel, x and y coordinate Frame Pixel, x and y coordinate

Distance from Kinect sensor to hand detection Threshold value of hand depth

Threshold value of hand depth

Distance between two point of hand contour Hand contour point

Hand contour point Hand contour Convex hull

Radius of the maximum inscribed circle Theta

Reference palm point Detection palm point Vector of hand Vector of hand

LIST OF ABBREVIATIONS

3D Three Dimensions

API Application Programming Interfaces

csv

comma separated values

CTS Carpal Tunnel Syndrome

FES Functional Electrical Stimulation fps Frame per seconds

GUI Graphical User Interface

lR Infrared

MIRA Medical Interactive Recovery Assistant

OS Operating System ROI Region of Interest ROM Range ofMotion

REFERENCES

Abu Bakar, M. Z., Samad, R., Pebrianti, D., Mustafa, M., & Abdullah, N. R. H. (2017). Experiment and analysis of computer vision-based wrist radial and ulnar deviation exercises. In Proceedings - 2016 IEEE International Conference on Automatic

Control and Intelligent Systems, I2CACIS 2016, pp. 190-195.

Abu Bakar, M. Z., Samad, R., Pebrianti, D., & Aan, N. L. Y. (2014). Real-time rotation invariant hand tracking using 3D data. In 2014 IEEE International Conference on

Control System, Computing and Engineering (ICCSCE 2014), pp. 490-495.

Abu Bakar, M. Z., Samad, R., Pebrianti, D., Mustafa, M., & Abdullah, N. R. H. (2015). Finger Application Using K-Curvature Method and Kinect Sensor in Real-Time, 213-217.

Abu Bakar, M. Z., Samad, R., Pebrianti, D., Mustafa, M., & Abdullah, N. R. H. (2016). Computer vision-based hand deviation exercise for rehabilitation. In

Proceedings-5th IEEE International Conference on Control System, Computing and

Engineering, ICCSCE 2015, pp. 389-394.

Blair, T. B., & Davis, C. E. (2013). Innovate engineering outreach: A special application of the Xbox 360 Kinect sensor. In Proceedings - Frontiers in Education

Conference, FIE, pp. 1279-1283.

Bradski, G., & Kaehler, A. (2008). Learning openCV : Computer vision with the openCV library (1st ed). Sebastopol, CA: O'Reilly.

Bunger, M. (2013). Evaluation of Skeleton Trackers and Gesture Recognition for Interaction, 90. (Master Thesis).

Calin, A, Cantea, A, Dascalu, A., Mihaiu, C., & Suciu, D. (2011). Mira-Upper Limb Rehabilitation System Using Microsoft Kinect. Sudia University Babes-Bolyai,

Informatica, LVI(4), 63-74.

Chang, C. Y., Lange, B., Zhang, M., Koenig, S., Requejo, P., Somboon, N., & Rizzo,

A

A (2012). Towards pervasive physical rehabilitation using microsoft kinect. In 2012 6th International Conference on Pervasive Computing Technologies for Healthcare and Workshops, PervasiveHealth 2012, pp. 159-162.

Chang, Y. J., Chen, S. F., & Huang, J. Da. (2011). A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in

Developmental Disabilities, 32, 2566-2570.

Chang, Y. J., Han, W. Y., & Tsai, Y. C. (2013). A Kinect-based upper limb rehabilitation system to assist people with cerebral palsy. Research in Developmental Disabilities, 34(11), 3654-3659.

Christian, N. (2014). What is carpal tunnel syndrome? What causes carpal tunnel

syndrome? Retrieved June 28, 2015, from

http://www.medicalnewstoday.com/articles/184337.php

Corte, J. De. (2011). Gesture Recognition from 3D-image Stream. Science And

Technology, Qune ).

D. Lelis Baggio, S. Emami, D. Millan Escriva, K. Ievgen, N. Mahmood, J. Saragih, R. S. (2014). Mastering OpenCV with Practical Computer Vision Projects. Sexual health

(Vol. 11).

Daniels JM 2nd, Zook EG, Lynch JM. (2014) Hand and wrist mJunes: Part I.

Nonemergent evaluation. Am Fam Physician 2004;69:1941-8.

Dan Matthew, M. (2008). INJURIES TO THE HAND AND WRIST, (June). Retrieved from www.sportsmed.org

Derebery, J. (2006). Work-related carpal tunnel syndrome: the facts and the myths.

Clinics in Occupational and Environmental Medicine, 5(2), 353-67, viii.

http://doi.org/1 0.10 16/j .coem.2005 .11.014

Devices, G., & Sense, P. (2010). Prime Sensor 1M NITE 1 . 3 Algorithms notes. Control.

Erap, S. (2012). Gesture based Pc interface with kinect sensor. Thesis Master.

Exell, T., Freeman, C., Meadmore, K., Kutlu, M., Rogers, E., Hughes, A M., &

Burridge, J. (20 13 ). Goal orientated stroke rehabilitation utilising electrical stimulation, iterative learning and Microsoft Kinect. In IEEE International Conference on Rehabilitation Robotics.

Exell, T., Hallewell, E., Meadmore, K., Freeman, C., Kutlu, M., Hughes, A-M.,

Burridge, J. (2013) Goal-orientated functional rehabilitation using electrical

stimulation and iterative learning control for motor recovery in the upper extremity post-stroke. In: 18th Annual Conference of the International Functional Electrical Stimulation Society (IFESS); 2013; Belgrade, Serbia. Enschede: University of

Twente; 2013

Falahati, S. (2013). OpenNI Cookbook. Birmingham. Packtpub

Flexion, P. I. P. (2015). Goniometric Assessment Form, 213. Retrieved from http://www .nasm .org/ docs/ default -source/

conference-sessions-2015/2015 _advanced_ 2 _day _sessions.pdf?sfvrsn=2

Galna, B., Jackson, D., Schofield, G., Mcnaney, R., Webster, M., Barry, G., ... Rochester, L. (2014). Retraining function in people with Parkinson's disease using the Microsoft kinect: game design and pilot testing. Journal of NeuroEngineering and Rehabilitation, 11, 1-12.

Gama, D. A, Chaves, T., Figueiredo, L., & Teichrieb, V. (2012). Poster: Improving motor rehabilitation process through a natural interaction based system using

Kinect sensor. In IEEE Symposium on 3D User Inteifaces 20I2, 3DUI 20I2

-Proceedings, pp. 145-146.

Ganesan, S., & Anthony, L. (2012). Using the kinect to encourage older adults to exercise: a prototype. Proceedings of CHI 2012 Extended Abstracts, 2297-2302.

Gopura, R. A R. C., Kiguchi, K., & Horikawa, E. (2010). A Study on Human

Upper-Limb Muscles Activities during Daily Upper-Upper-Limb Motions. International Journal

of Bioelectromagnetism, I2(2), 54-61.

Healthwise Staff. (2012). Finger, Hand, and Wrist Injuries. Retrieved November 14, 2014, from http://www.healthlinkbc.ca/healthtopics/content.asp?hwid=handi

Jana, A (2012). Kinectfor Windows SDK Programming Guide. PACT Publishing.

Jayaram, M. A, & Fleyeh, H. (2016). Convex Hulls in Image Processing: A Scoping Review. American Journal of Intelligent Systems, 6(62), 48-58.

Jolly, K., Taylor, R., Lip, G. Y., Greenfield, S., Raftery, J., Mant, J., ... Stevens, A (2007). The Birmingham Rehabilitation Uptake Maximisation Study (BRUM). Home-based compared with hospital-based cardiac rehabilitation in a multi-etlmic

population: cost-effectiveness and patient adherence. Health Technology

Assessment (Winchester, England).

Katrasnik, J., Veber, M., & Peer, P. (2005). Using computer vision in a rehabilitation method of a human hand, 2-4.

Katz, J. N., & Simmons, B. P. (2002). Carpal Tunnel Syndrome. New England Journal

qfMedicine, 346(23), 1807-1812.

Khoshelham, K. (2011). ACCURACY ANALYSIS OF KINECT DEPTH DATA, XXXVIII(August), 29-31.

Laganiere, R. (2011). OpenCV 2 Computer Vision Application Programming Cookbook.

Pattern Recognition.

Lee, J.-D., Hsieh, C.-H., & Lin, T.- Y. (2014). A Kinect-based Tai Chi exercises evaluation system for physical rehabilitation. 2014 IEEE International Conference

on Consumer Electronics (ICCE), 177-178.

Li, Z. M., Kuxhaus, L., Fisk, J. A., & Christophel, T. H. (2005). Coupling between wrist flexion-extension and radial-ulnar deviation. Clinical Biomechanics, 20(2), 177-183.

Lin, T., Hsieh, C., & Lee, J. (2013). A Kinect-based System for Physical Rehabilitation: Utilizing Tai Chi Exercises to Improve Movement Disorders in Patients with Balance Ability.

Lister, G. D., Kleinert, H. E., Kutz, J. E., & Atasoy, E. (1977). Primary flexor tendon

repair followed by immediate controlled mobilization. The Journal of Hand

Surgery, 2(6), 441-51.

Llorens, R., Alcafiiz, M., Colomer, C., & Navarro, M. D. (2012). Balance recovery through virtual stepping exercises using kinect skeleton tracking: A follow-up study with chronic stroke patients. Studies in Health Technology and Informatics, 181, 108-112.

Logan, A. J., Makwana, N., Mason, G., & Dias, J. (2004). Acute hand and wrist injuries in experienced rock climbers, 3(M), 545-549.

Logan, A. J., Makwana, N., Mason, G., Dias, J., & Logan, M. (2004). Acute hand and wrist injuries in experienced rock climbers. Br J Sports Med, 38(M), 545-548.

Lovett, W. L., & McCalla, M. A. (1983). Management and rehabilitation of extensor tendon injuries. The Orthopedic Clinics of North America, 14( 4 ), 811-26.

Luttgens, K., Hamilton, N., & Weimar, W. (2011). Kinesiology: Scientific Basis of Hmnan Motion, 640.

Mark, S. (2014). Carpal tunnel cases need prompt treatment. Retrieved June 28, 2015,

from

http://www.sfgate.com/health/article/Carpal-tunnel-cases-need-prompt-treatment-4458056.php

Meislin, M. A, Wagner, E. R., & Shin, A Y. (2016). A Comparison of Elbow Range of

Motion Measurements: Smmtphone-Based Digital Photography Versus

Goniometric Measurements. Journal of Hand Surgery, 41(4), 510-515.

OpenCV User Guide. (2013). Retrieved April 23, 2014, from

http://docs. opencv.org/2. 4/opencv _user.pdf

OpenNI I NiTE 2 Migration Guide. (2013), Retrieved April 23, 2014, from http:// a penni. rulopenni-migration-guide/index. html.

Pagliari, D., Menna, F., Roncella, R., Remondino, F., Pinto, L., Map, D., & Libraries, F. (2014). KINECT FUSION IMPROVEMENT USING DEPTH CAMERA

CALIBRATION, XL(June), 23-25.

Pagliari, D., & Pinto, L. (2015). Calibration of Kinect for Xbox One and Comparison

between the Two Generations of Microsoft Sensors, 27569-27589.

Pan, Z., Li, Y., Zhang, M., Sun, C., Guo, K., Tang, X., & Zhou, S. Z. (2010). A real-time multi-cue hand tracking algorithm based on computer vision. 2010 IEEE Virtual Reality Conference (VR), 219-222.

Pastor, I., Hayes, H. A, & Bamberg, S. J. M. (2012). A feasibility study of an upper

limb rehabilitation system using Kinect and computer games. Conference

Proceedings: ... Annual International Conference of the IEEE Engineering in Medicine and Biology Society. IEEE Engineering in Medicine and Biology Society. Conjerence,2012, 1286-1289.

Peer, P. (2013). A computer vision based system for a rehabilitation of a human hand, 115(4), 535-544.

Pirozzolo, jason. (2008). CIS Exercise. Retrieved Febmary 25, 2015, from

http://www.jasonpirozzolo.com/patientinfo/sma _ xcarphm _ art.htm

Polygerinos, P., Galloway, K. C., Savage, E., Herman, M., Donnell, K. 0., & Walsh, C. J. (2015). Soft Robotic Glove for Hand Rehabilitation and Task Specific Training, 2913-2919.

PrimeSense. (2010). Prime Sensor TM. NITE 1. 3 Controls Programmer Guide. Retrived

May 22, 2014, from http// pr.cs.cornell.edu!humanactivitiesldata/NITE.pdf

PrimeSense. (2011). OpenNI User Guide. OpenNI User Guide, l(June), 44.

http://doi .org/1 0.1007 /SpringerReference _27988

Raheja, J. L., Chaudhary, A., & Singal, K. (2011). Tracking of fingertips and centers of palm using KINECT. In Proceedings - CIMSim 2011: 3rd International Conference on Computational Intelligence, Modelling and Simulation, pp. 248-252.

Rahman, M. H., & Archambault, P. S. (2013). Robot Aided Passive Rehabilitation using Nonlinear Control Techniques, 0-5.

Range of Joint Motion Evaluation Chart. (2003), 2002-2003.

Range of Motion Exercises. (2004). Retrieved January 10, 2016, from http://www.alsa-or.org/treatment/ROMExercises

Reinold, M. M., Wilk, K. E., Macrina, L. C., Sheheane, C., Dun, S., Fleisig, G. S., & James R. Andrews. (2008). Changes in Shoulder and Elbow Passive Range of Motion After Pitching in Professional Baseball Players, (April2008).

Resnik, C. S. (2000). Wrist and hand injuries. Seminars in Musculoskeletal Radiology.

Rettig, a C. (2001). Wrist and hand overuse syndromes. Clinics in Sports Medicine,

20(3), 591-611.

Rimkus, K.

C

.

,

Bukis, A., & Sinkevi, S. (20 13). 3D Human Hand Motion Recognition System, 180-183.

Ryan, D. J. (2012). Finger and gesture recognition with Microsoft Kinect. University of Stavanger.

Samad, R., Abu Bakar, M. Z., Pebrianti, D., Mustafa, M., & Abdullal1, N. R. H. (2017). Elbow flexion and extension rehabilitation exercise system using marker-less kinect-based method. International Journal of Electrical and Computer Engineering, 7(3), 1602-1610.

Sardelli, M., Tashjian, R. Z., & MacWilliams, B. A. (2011). Functional Elbow Range of Motion for Contemporary Tasks. The Journal of Bone & Joint Surgery, 93(5), 471 -477.

Shotton, J., Fitzgibbon, A., Cook, M., Sharp, T., Finocchio, M., Moore, R., ... Blake,

A

(2013). Real-time hmnan pose recognition in parts from single depth images.

Studies in Computational Intelligence, 411, 119-135.

Silas, F., Alves, R., Uribe-quevedo, A J., Nunes, I., & Filho, H. F. (2014). Pomodoro , a

Mobile Robot Platfonn for Hand Motion Exercising, 970-974.

Simmons, B. P. (2015). 5 exercises to improve hand mobility. Retrieved December 12,

2015, from http:/ /www.health.harvard.edu/pain/5

-exercises-to-improve-hand-mobility-and -reduce-pain

Sosa, G. D., Sanchez, J., & Francoy, H. (2015). Improved front-view tracking ofhmnan

skeleton from Kinect data for rehabilitation support in Multiple Sclerosis. 2015 20th Symposium on Signal Processing, Images and Computer Vision, STSIVA 2015

-Conference Proceedings.

Stephanie, W. (2016). 7 Hand Exercises to Ease Arthritis Pain. Retrieved July 3, 2016,

from

http://www.healthline.com/health/osteoarthritis/arthritis-hand-exercises#Overview 1

Stone, E. E., & Skubic, M. (2012). Capturing habitual, in-home gait parameter trends

using an inexpensive depth camera. In Proceedings of the Annual International Conference of the IEEE Engineering in Medicine and Biology Society, EMBS, pp. 5106-5109.

Su, C.-J. (2013). Personal Rehabilitation Exercise Assistant with Kinect and Dynamic

Time Warping. International Journal of Infonnation and Education Technology, 3,

448.

Tim, K. (2013). Tendinopathy and Tenosynovitis. Retrieved April 21, 2016, from

http://patient.info/health/tendinopathy-and-tenosynovitis

Torpy, J. M. (2010). Rheumatoid Arthritis. The Journal of the American Medical Association (Vol. 305).

Tscherter, V., Ravasio, P., & Guttormsen-schar, S. (1986). The Qualitative Experiment

in HCI: Defnition, Occurrences, Value and Use. ACM Transactions on

Computer-Human Interaction, V, 1-24.

Uhl, T. L., Blazar, P., Pitts, G., Otr, L., & Ramsdell, K. (2001). Wrist and Hand Injuries in the Athlete.

Vinh, T. Q., Ho, C., Minh, V., & Tri, N. T. (2015). Hand Gesture Recognition Based on

Depth Image Using Kinect Sensor, 34-39.

Wen, Y., Hu, C., Yu, G., & Wang, C. (2012). A Robust Method of Detecting Hand

Gestures Using Depth Sensors, 0-5.

Willing, R. T., Nishiwaki, M., Johnson, J. A, King, G. J. W., & Athwal, G. S. (2014).

Evaluation of a computational model to predict elbow range of motion. Computer

Aided Surgery, 9088(April2016), 1-7.

Witter, J., & Dionne, R. A (2004). What can chronic arthritis pain teach about

developing new analgesic drugs? Arthritis Research & Therapy, 6(6), 279.

Y amaura, H., Matsushita, K., Kato, R., & Yokoi, H. (2009). Development of hand rehabilitation system for paralysis patient - Universal design using wire-driven

mechanism. In Proceedings of the 31st Annual International Conference of the

IEEE Engineering in Medicine and Biology Society: Engineering the Future of

Biomedicine, EMBC 2009, pp. 7122-7125.

Yeo, H. S., Lee, B. G., & Lim, H. (2013). Hand tracking and gesture recognition system

for human-computer interaction using low-cost hardware. Multimedia Tools and

Applications, 1-29.

Zhou, H., & Hu, H. (2008). Human motion tracking for rehabilitation- A survey, 3,

1-18.