Human-machine interaction for unmanned surface systems

HUMAN-MACHINE INTERACTION FOR UNMANNED SURFACE SYSTEMS

BY SAKI HANDA

THESIS

Submitted in partial fulfillment of the requirements for the degree of Master of Science in Civil Engineering

in the Graduate College of the

University of Illinois at Urbana-Champaign, 2015

Urbana, Illinois

Adviser:

A

This research investigated the machine interaction (HMI) technologies for human-robot teams operating as unmanned surface systems (USS). An pilot role was found to be the most prevalent in the USS-related literature but additional human roles were determined to likely be necessary (e.g., Mission Specialist) though were not documented; interface needs have not yet been determined for any role. The human interfaces used by 67 Micro and Small X, Intermediate, Harbor, Fleet, and E,F,G-Class platforms were examined and it was determined that: i) the research literature does not well characterize the human roles present in unmanned surface systems, ii) domain complexity may necessitate increased automation of the robot platform for the human team, and iii) that unmanned surface vehicles likely lay on the human-machine interaction spectrum between unmanned ground vehicles and unmanned aerial vehicles. This work is expected to serve as a reference for future design and refinement of human interfaces for USSs and as a foundation for better understanding HMI in USSs.

A

I would first like to thank my graduate advisor, Prof. Joshua Peschel for his valuable and creative insights. He understood my non-engineering backgrounds and turned my uniqueness to my strength.

I sincerely appreciate Prof. Cassandra Rutherford for her help and support throughout my master’s program at the University of Illinois. She lead me through difficult time and showed me the importance of being persistent and always strives to grow. Without her help, I would have never accomplished my degree. She is truly my role model.

I also feel grateful to have a wonderful family who supported me, and believed in me all the way from Japan. Without them, I would never have came this far. Thank you so much. I would also like to mention my colleagues and friends: Elizabeth Depew, Chris Chini, Meng Han, Fernanda Maciel, Tianyu He, Adam Burns, Jeff Wallace, Adrian Naranjo and Lopa Bhaumik at Human Infrastructure Interaction Lab and friends: Andrew Rehn and Sarah Hoyle-Katz. Thank you very much and best wishes to you all.

Lastly, I love to say thank you to my best friend, Beth Cherry and her family for the continuous support and encouragement. They are my family in the United States and I am so glad that I met you all.

T

C

L

T

CHAPTER 1 INTRODUCTION . . . 1

1.1 Research Question . . . 1

1.2 Why focus on USV . . . 2

1.3 Understanding Unmanned Surface Vehicles . . . 2

1.4 Importance to Civil Engineering . . . 3

1.5 Contributions . . . 3

1.6 Organization of this Thesis . . . 4

CHAPTER 2 RELATED WORK . . . 5

2.1 Categories of Unmanned Surface Vehicles . . . 5

2.2 Human-Machine Interaction Literature on USV . . . 10

2.3 Human Roles . . . 11

CHAPTER 3 HUMAN-MACHINE INTERACTION ANALYSIS . . . 18

3.1 Human-Machine Interaction for X-Class . . . 19

3.2 Human-Machine Interaction for Intermediate-Class . . . 23

3.3 Human-Machine Interaction for Harbor-Class . . . 24

3.4 Human-Machine Interaction for Fleet-Class . . . 25

3.5 Human-Machine Interaction for EFG-Class . . . 26

CHAPTER 4 HUMAN-MACHINE INTERACTION FINDINGS . . . 27

4.1 Finding 1 . . . 27

4.2 Finding 2 . . . 28

4.3 Finding 3 . . . 29

CHAPTER 5 CONCLUSIONS . . . 30

APPENDIX A TABLE FOR ALL EXISTING USV . . . 32

L

T

2.1 Classifications of Selected Unmanned Surface Vehicles Currently in Operation 15 2.2 Hardware Based HMI of Selected Unmanned Surface Vehicles Currently

in Operation . . . 16

2.3 Software Based HMI of Selected Unmanned Surface Vehicles Currently in Operation . . . 17

A.1 Summary of X-Class Unmanned Surface Vehicles Currently in Operation . . 33

A.1 Table A.1 (Cont.) . . . 34

A.1 Table A.1 (Cont.) . . . 35

A.1 Table A.1 (Cont.) . . . 36

A.1 Table A.1 (Cont.) . . . 37

A.1 Table A.1 (Cont.) . . . 38

A.2 Summary of Intermediate-Class Unmanned Surface Vehicles Currently in Operation . . . 39

A.2 Table A.2 (Cont.) . . . 40

A.2 Table A.2 (Cont.) . . . 41

A.2 Table A.2 (Cont.) . . . 42

A.2 Table A.2 (Cont.) . . . 43

A.2 Table A.2 (Cont.) . . . 44

A.3 Summary of Harbor-Class Unmanned Surface Vehicles Currently in Operation 45 A.3 Table A.3 (Cont.) . . . 46

A.4 Summary of Fleet-Class Unmanned Surface Vehicles Currently in Operation 47 A.5 Summary of E, F, G-Class Unmanned Surface Vehicles Currently in Operation 48 A.6 Summary of X-Class Unmanned Surface Vehicles Hadrware Based HMI . . . 49

A.6 Table A.6 (Cont.) . . . 50

A.7 Summary of Intermediate-Class Unmanned Surface Vehicles Hadrware Based HMI . . . 51

A.7 Table A.7 (Cont.) . . . 52

A.8 Summary of Harbor-Class Unmanned Surface Vehicles hardware Based HMI 53 A.9 Summary of Fleet-Class Unmanned Surface Vehicles Hardware Based HMI . 54 A.10 Summary of E,F,G-Class Unmanned Surface Vehicles Hardware Based HMI 55 A.11 Summary of X-Class Unmanned Surface Vehicles Software HMI . . . 56

A.12 Summary of Intermediate-Class Unmanned Surface Vehicles Software HMI . 58 A.12 Table A.12 (Cont.) . . . 59 A.13 Summary of Harbor-Class Unmanned Surface Vehicles Software based HMI . 60 A.14 Summary of Fleet-Class Unmanned Surface Vehicles Software Based HMI . . 61 A.15 Summary of E,F,G-Class Unmanned Surface Vehicles Software Based HMI . 62

CHAPTER 1

INTRODUCTION

This research surveys the current state of understanding and technological accessibility of human-machine interaction (HMI) of the human roles for Unmanned Surface Systems (USSs).

1.1

Research Question

Unmanned Surface Vehicles (USVs) have become a part of a core branch of Unmanned Systems in the globe due to their stability, robust communication, and potential for future development. But in order for USVs to be successful, the lack of detailed information of the Unmanned Surface Systems (USVs) and its solutions are needed to be identified.

The primary research question of this thesis work is

What is a current state of Human-Machine Interaction (HMI) for small unmanned surface systems?

HMI in general but specifically for small USVs has not been discussed or mentioned in many studies. The identification of HMI can potentially improve the control mode and especially the level of automation so that the variety of mission applications can be expanded. The USVs seems to have a potential to be a leading devices in many different field of applications due to advantages discussed below.

The majority of the platforms are in the smaller size group that are not developed by military. Reasons behind the void of information is caused by the simplicity of the platforms and single operator system. Smaller systems have restricted limitations in payloads, range, and endurance and these limitations influence each other. Especially primary missions for smaller USVs are single missions in environmental monitoring so that risk of human oper-ators are low, and in disaster recovery & rescue that require urgent human decisions. The

main interest of the operation tends to be simplicity and efficiency instead of depending on automation with less human in loop operation. The level of HMI are not considered in terms of automation or the level of dependence in computer, but the level of ease for the human operators.

1.2

Why focus on USV

Unmanned systems are dramatically becoming popular due to following points: the cost of manned systems are the significantly more expensive than unmanned system even with hu-man operators; the coverage and awareness of environment situation for Unhu-manned systems are improved due to advanced technology of sensors and, localization systems; productivity of missions as a whole will be more efficient due to operator specific mission (human operator can concentrate on manned missions); Unmanned systems keep human presence away from dangerous environment [1]. Addition to USVs, Unmanned systems consist of Unmanned Ground Vehicles (UGVs), Unmanned Ariel Vehicles (UAVs), Unmanned Underwater Vehi-cles (UUVs), and Unmanned Space VehiVehi-cles. Especially USVs are given a recent popularity due to effective utilization for not only wartime missions, but also peacetime missions [1].

The reasons behind this popularity are tied to accessibility and communication. For environmental monitoring mission, air regulation and land ownership can cause limitation in accessibility. Surface water such as watercourses, lake and wetland are public water. For deep water mission, UUVs have difficulty communicating with the ground control station due to a limitation of signal transmission under water. USVs belongs to the border of air and under water, so that for heterogeneous mission, USVs can act as the link between ground station or other unmanned and manned systems.

1.3

Understanding Unmanned Surface Vehicles

USVs have been developed and utilized for many of decades, Unmanned implies removing human presence from the targeted area [2]. They are either remotely operated or pre-programmed to be auto-piloted through radio, WiFi, cellular network, satellite, etc. Due to drastic improvement of global positioning systems in terms of compactness, effectiveness and affordability, USVs started to show their strength in Unmanned System field [2]. USVs especially take advantage of their air-sea interface to serve as a bridge in the networks of

heterogeneous manned/unmanned air, ground, and marine platforms to introduce new and advanced understandings in environmental monitoring, disaster rescue, surveillance, warfare, and defence applications [3].

1.4

Importance to Civil Engineering

UAVs, UGVs and UUVs have been popularly used for a variety of civil applications. For example: UAVs have been utilized for land surveying and monitoring, construction manage-ment, and disaster response; UUVs have been used for underwater monitoring, underwater inspection and maintenance and repair for marine infrastructure or vehicles; UGVs are pop-ularly used for military, and explosives & bomb disabling missions.

Despite high popularity and prestigious technology of these systems, the existing prob-lems and application difficulties prohibit researchers from finding complete solutions. In particular, UAVs and UUVs share the same restriction of signal communication range. Es-pecially for the team (multi-vehicle) missions, their environment becomes obstacle between these platforms. USVs are considered to fill this void and with additional factors such as long endurance, high payload capacity, user friendly interfaces and mostly reliable signal com-munication. USVs have a potential to be powerful additional tool or team-player tool for civil applications such as disaster rescue, offshore & onshore infrastructure inspection, and environmental monitoring. For instance, bridge inspection after Hurricane WILMA and IKE were conducted by the Center for Robot-Assisted Search and Rescue (CRASAR) with USVs and Underwater ROV. This case study led post disaster recovery inspection economically feasible by reduction of time, cost, and risk of human presence [4].

1.5

Contributions

There are two distinct contributions that are presented in this thesis regarding to HMI for USVs. In the existing literature, specific identifications of human roles for USVs are not always stated for both research and commercial systems. Especially for smaller platforms, number of team members, human roles, or interfaces are barely found in contrast of growing numbers of smaller platforms.

1.5.1

Contribution 1

Filling the void of HMI information provides better accessibility to match high demand and supply of multi-mission smaller USVs by large range of users. Especially this void can provide solution to key components when level of autonomy increases.

1.5.2

Contribution 2

Identifying specific human roles provides efficacy in number of crew member needed and on the other hand, shared crew can be analyzed and reevaluated depending on risk and safety of the mission.

1.6

Organization of this Thesis

This thesis is organized as follows:

• Chapter 2 begins with a review of the research and industrial literature associated with categories and characteristics of USVs, Human Robot Team roles and HMI for existing USVs platforms. Actively operating USVs are identified and surveyed in Table 1. • Chapter 3 analyzes identified HMI and its trend according to the NAVY classification

scale of USV of Micro and Small X-Class, Intermediate-Class, Harbor-Class, Fleet-Class and E,F,G-Fleet-Class. Hardware and Software based Human-Machine interfaces are analyzed by the surveyed data in Table 2 and Table 3.

• Chapter 4 introduces 3 findings of Human-Machine Interaction for Operator role and its limitation.

• Chapter 5 finally presents conclusions regarding to each findings and limitations, and then future work and recommendation.

CHAPTER 2

RELATED WORK

This section provides a comprehensive review of the research and industrial literature, identifying 67 existing Unmanned Surface Vehicles (USVs). The USVs are categorized into four different groups based on size, weight, range, endurance, and types of the mission. The present human roles and human machine interaction (HMI) relationships are also presented including hardware and software interfaces for each USV.

2.1

Categories of Unmanned Surface Vehicles

Currently, there is not an official classification system for USVs [5]; however, the most common descriptions for USVs follow the 2007 U.S. Navy Master Plan standards that include four classes of vehicles organized by length (the distance between the forward-most and after-most parts of the vehicle): X-Class, Harbor, Snorkeler, and Fleet [1]. The National Defense Research Institute (NDRI) provides an additional class, EFG, for USVs that are greater in length than the Fleet-Class [6]. This survey combines the U.S. Navy, NDRI category structures and, additional adjustment classes which are created for this survey.

Additional category modifications are included in the survey: (1) The Snorkeler-Class (submersible) is not described in this survey due to the similarity with Harbor-Class as well as this class of SUV is outside the scope of the thesis. (2) The Intermediate-Class has been added to cover USVs with lengths between 3 m and 7 m. Currently, USVs with lengths between 3 m 0 7 m not classified in the literature. The vehicles that fall into this size range have been found to be show unique characteristics and a large number of USVs are available in this class. (3) The X-Class is divided into two sub-groups: Micro X-Class defined as a length under 1.5 m and Small X-Class defined with a length of 3 m. These sub-classifications are used to distinguished the X-Class USVs by concept and operational system.

A total of 6 classifications are use to described the state-of-the-art for USVs: Micro X-Class, Small X-Class, Intermediate-Class, Harbor-Class, Fleet-Class, and E,F,G-Class.

Three vehicles are described for each classification type in the section below.

2.1.1

X-Class

The X-Class category is defined as vehicles that are under 3 m in length and can operate on the open water in choppy sea-surface conditions. However, a majority of the USVs in this class are more fitted for shallow water use.

These vehicles are considered to provide ”low-end” Intelligence, Surveillance and Recon-naissance (ISR) and also have ability to support manned or other unmanned missions [1]. Several different types of missions are supported with the most common use for observation and data collection for non-military purposes [6]. The missions range from environmental studies to search & rescue to surveillance. This class tends to be more cost effective and user friendly due to the simpler platforms than larger systems. These advantages encourage use in academic and civilian operations.

The X-Class is divided into 2 sub-categories by sizes to describe the differences in geome-tries, mission types, and advantages.

2.1.2

Micro X-Class

Micro X-Class is a sub-category of the X-Class with the length of the vehicles under 1.5 m. While smaller platforms have limited payloads, their small size and simplicity allow ease in accessibility and deployment. This class tends to be low-cost and operate on very simple control and sensing systems. The restrictions of the payload limit the variety of sensors that can be equipped. The small platform also has a higher risks of damage by environmental conditions. Missions are dominated by environmental monitoring, infrastructure inspection, flood study and search & rescue. One advantage is the ability to operate in narrow channels and shallow water depths. Due to the small range of communication, the distance to targets and operational endurance are very limited.

The Ziphius [7] is the smallest app-controlled platform among existing USVs in the indus-try. It is made for recreational usages but can potentially be utilized for different applications such as environmental monitoring or infrastructure inspection due to customizable body and open source programmable systems. It’s small hull draft allows operation in shallow to deep water as long as it is in within the limited communication range.

Platypus LLC., which was industrialised after many experiments conducted by Carnegie Mellon University. It is also small and used for environmental monitoring and flood studies. The Lutra has a very durable control system for its size and the ability to be operated as a multi-vehicle team by a single operator. An Android smart-phone interface and piloting interface allows multi-vehicle operations.

Finally, the Pioneer [9] is a twin pontoon type platform that are especially suitable for shallow water and it is usually controlled remotely with RC-controller by an operator though his or her line of sight (LOS). The Pioneer can also be semi-autonomous if additional control systems are equipped. A live stream video feedback is recorded by its gimbal camera as well as piloted by LOS. It is used for filming, environmental clean up, and infrastructure inspections.

2.1.3

Small X-Class

The Small X-Class is another sub-category wit the length of the vehicles are between 1.5 m and 3 m. It is the larger end of the X-Class and it has a diverse range of characteristics. One of the main differences from Micro X-Class is wider range of the endurance due to com-munication and payload improvements. While the mission types stays within environmental monitoring and search & rescue, the increase in their endurance and communication range allows these platform to obtain larger scale of data such as climate and hurricane monitoring instead of short term point monitoring such as water sampling or inspection.

The Hurricane Tracker [10, 11, 12] is a buoy system similar to the Emergency Integrated Lifesaving Lanyard (EMILY) platform implemented for hurricane tracking. The Hurria-cane Tracker is dropped to the target area by other vessels and controlled remotely. Pre-programmed operations can be activated via text message from a smart-phone. This system allows close and detailed study of the eye of hurricane.

The CNR-ISSA Charlie [13] is also controlled with a smart-phone but more directly and hands-on. A smart-phone is utilized as a controller instead of just an activator. Its cata-maran platform is especially well-fitted for shallow water environmental monitoring. The control system was experimented to improve user’s ”easy-to-use” feeling. The operator is not required to have special training to use this system.

The Wave Glider [14] is the first hybrid water and solar propelled USV. It is used for long term environmental and climate monitoring, and also for the surveillance patrol. Their stored solar energy and wave-powered energy allow this platform to conduct a long endurance

missions without increasing the cost of operation. It is monitored from a base station on-shore with open source software developed by Liquid Robotics.

2.1.4

Intermediate-Class

The Intermediate-Class was added specifically for this survey and includes the vehicles that are between 3 m and 7 m in length. Some can be on the shallow water but majority of others are well-fitted for choppy open water due to the size, weight, and payloads of the vehicle. Many inflatable type of vehicles are commonly utilized as the platform for this class. The mission for this class includes environmental monitoring, long term ocean monitoring, rescue, and port surveillance.

the Intermediate-Class seems to be still have a site specific domain even though com-plex sensing and control systems can be equipped. The payload is chosen based on the environmental condition or types of mission the VSU will conduct.

The smaller end of the Intermediate-Class is utilized for surveillance missions and Jet-ski or inflatable boats with high speed engines are favorable. On the other hand, the larger end of this class is utilized for long endurance environmental missions. The fast mono-hull vehicles are used to reached to target position quickly and turn off to float while collecting data. They also be solar powered depending on the needs of the operating conditions. The accuracy of the positioning and sensing is improved for this class due to allowance of the higher payload to equip complex sensing systems.

The VaCAS [15] group used a laser line scanner to real-time map and identify the river pathway. It also maps the bottom of the river with sonar to determine underwater obstacles. The data collected allows for accurate navigation and path planning for autonomous control of the USV.

The WAM-V [16, 17] is the another inflatable vehicle in this class. They are ultra light-weight and can be operated in choppy water conditions allowing for real time sensing. Its unique light weight platform also sustains the maximum payload capacity allowing extra sensing devices to be equipped. Inflatable platforms become very beneficial for maximization of payload and portability. The ability to be disassembled and inflatable the platform are one some of the advantages of these USVs.

The Ocean Atmosphere Sensor Integration System (OASIS) [18] is a floating type plat-form rather than a moving boat used for climate monitoring, water quality monitoring, and military operations. It is piloted to target location and floats to collect data while saving

power. It intended to be low-cost, long duration, and has a reusable battery platform so that it utilizes the solar power and recharges as it run out the power. For these reasons, this platform requires open water locations with minimal overhead obstacles.

2.1.5

Harbor-Class

The Harbor-Class is defined for vehicles that are 7 m in length and fitted for open sea-surface. For this size and larger, the platforms can be both manned or unmanned. This class can conduct major missions in maritime security with robust ISR and it is expected to have mature autonomy, launch and recovery, and weapons & payloads technologies [1]. The X-Class had wide variety of level in autonomy but for vehicles that are the Harbor-Class and larger, adaptive planning/group behavior, obstacle avoidance, and communication become necessities to achieve the robust autonomy level that are required [1]. The Harbor-Class includes both warfare and environmental monitoring missions [19].

2.1.6

Fleet-Class

The Fleet Class categorizes vehicles that are 11 m in length. The fleet-class also includes both warfare and environmental monitoring missions. This class support ”high-end” surface mission for military operations [1]. This class can be operated to be very fast or moderate speed when it is supporting other missions such as towing other manned vehicles or USVs. Occasionally, they are used as support vehicles for other missions [19].

One of advantages of this class and larger USVs are that their vehicles can also hold manned missions. From this size class, the numbers of the vehicles drastically decrease due to the high cost of vessels and operations. To overcome this limit,in 2014, US NAVY created retro fitting projects for fleet class vehicle for full automation update. The system is described in detail in the next section. This retrofitting will allow the old manned vehicles to be retrofitted to unmanned vehicles with low cost.

2.1.7

EFG-Class

The E and F-Class is 26 m in length but they have a variety of widths. The G-Class is 41 m in length. Since there are no platforms that are larger than 26 m or 41 m [6], in this survey, E,F and G-Classes are all combined in the same group. Commonly, these size vehicles are

utilized for the manned mission due to the cost of vehicles and the range of complexity of the mission. Their missions are dominated by warfare and surveillance. They tend to have large of cost and are not an economical solution due to their size and required accessibility. Due to large payloads and the cost of system and operation, this class vehicle completes multiple missions with manned and unmanned situations.

2.2

Human-Machine Interaction Literature on USV

The government states in USV master plan 2007 [1] that USVs need major improvements in the level of autonomy to reduce data inflow and outflow to and from the operator[20]. Most of the time, USVs are semi-autonomy and this level of autonomy can be adjusted during the mission. There are some tele-operated systems(non-autonomy), and there is no full autonomy system running even though it has a technological potential to be fully autonomy.

Huang [21] developed a framework to characterize and articulate the autonomy of un-manned systems. He introduced a visual framework for the relationship between the level of HMI and the level of autonomy with the factors of mission and environment complexities. In his framework, the level of autonomy is quantified and shown with correlations with the level of HMI that are measured by the situation awareness of unmanned systems and human. This correlation is described as higher the level of HMI is lower the level of autonomy and vice versa. The level of HMI is controlled to be higher when the complexities are higher.

Adams [22] identified the relationship between unmanned vehicles (UV) and situation awareness(SA) using her interpretation of the definition of SA by Endsley [23]. She explained 3 levels of unmanned system SA with comparison with Endsleys’ 3 levels of human SA such as perception, comprehension, and projection. The perception is sensed by visual, sound, smell, and so on for the human SA. Human also senses the consequence and relationship with the perception instead of only searching to sense for a programmed perception target like UV’s SA does. But UV can overcome the physical limitation that human experience such as diminished focus to cause failure to obtain perceptions and if sensors are available and can be equipped, it can overcome human skills of perception. The perception with the UV systems are commonly obtained visually for human. Particularly the USVs obtain it by live streaming video or data are converted to visual form on the screen. The comprehension is understanding of the obtained data by processing and integrating with the mission and related information. This accuracy rate can vary by the level of experience or training the human operator had.

UV’s SA for the comprehension is controlled by the human comprehension or not being obtained. The projection is obtained by the perception and comprehension. Human could be obtaining the projection under severe stress of the mission environment. The UV’s SA for the projection can be used to support by utilizing its programmed mission planning or the decision making tools. Those UV’s SA skills can be improved by better understanding of human cognitive system. More human cognitive data, more options can be provided by UV.

The vehicles which are larger than Intermediate-Class are often used for military missions and usually have more than a single operator. This is because larger the vehicle is larger the payloads are. These systems can be equipped and can operate multiple missions at the same time and it will be actually cost-efficient to operate multi-missions than a single mission. This is simply because the operation cost for the larger vehicles are more expensive and it will not worth to have a single mission. One the other hand, smaller vehicles are limited to equip several sensors. They also face larger impacts and damages when they are on the water, so their sensors are better to be cheaper. But it does still require a threshold of the optimum balance that vehicle can be useful [24]. This characteristics for the difference in sizes can be improved by the quality, size, and price of sensors. But currently, it is more beneficial to make low-cost single mission small USVs and high-cost multi-complex mission large USVs.

Then, the goal of HMI for both vehicle types are observed to be different. The HMI goal for the smaller vehicles are multi-agent team, ease in communication and operation. The HMI goal for the larger vehicles are full autonomy. This difference is also supported by the types of missions that each groups deals with. The larger vehicles conduct more complex mission, and the smaller vehicles conduct more simple missions. Then simple mission has low level of HMI, so that autonomy for the smaller vehicles can be less problem. On the other hand, the complex mission requires high HMI followed by more problems to be autonomous. These supporting factors are due to payload, and accessibility and tolerance of the environment conditions.

2.3

Human Roles

Although specific human roles for USVs do not necessary appear in each individual lit-erature, but there are 2 trends categories of roles that a majority of platforms fall into. Huang [21] identified the human roles for unmanned systems in general such as Supervisor,

Teammate, Pilot (Operator), Mechanic/Developer, and Bystander. For UAVs, Peschel et al suggested the core human role called Misson Specialist addition to Pilot, and it seems also to apply for some USVs. Hence with literature reviews of USVs and reviews of UASs by Peschel et al. [25], followings are the 2 categories for human roles for USVs: Pilot and Mission Specialist.

2.3.1

Pilot

The Pilot for USVs is a role that is a combination of Operator, Teammate, and the Su-pervisor can be added for some cases such as for smaller and simpler platforms. Huang [21] described the Operator role as the person performing remote control or tele-operation, semi-autonomous operations, or other man-in-the-loop types of operations. The Operator determines the condition of the mission status, and makes decision whether to continue the mission or need to make some changes. The Teammate assist the Operator the overall mis-sion. Finally the Supervisor is a person monitors one or more robots with respect to progress on the mission, can task the robot(s) at the mission level, monitors mission progress, provides mission level directions, coordinates missions, and can assign an operator to assist a robot if needed [21]. If this role separately exists for the platform, this role will receive collected and processed information from the Operator and Teammate and will make a decision and command the mission.

The Pilots for both sub-groups of X-Class and the Intermediate-Class might have to con-duct multi-tasks to control, make a decision, and operate without presence of the Supervisor. Their missions, control system, and mission environment are simple enough for a Pilot to conduct the multi-task. The Lutra [24] has a human operator interface called the Agent that acts as a Teammate to process data and provides necessary information for a Pilot to gain proper SA to conduct and complete their mission. The Hurricane Tracker [26] is auto-piloted and missions can be programmed such that an Operator might only do a few minutes work for a day or two of mission time. In case of change in mission that are announced by human interface, the Operator will make a decision and conduct a change via smartphone text mes-sage. The Smart-phone Charlie utilizes its human computer interface, so that an Operator is allowed to perform fine maneuvering operations needed for instance to deploy the vehicle at sea, docking, taking the control of the robot in case of dangerous situations, etc [13].

Usually smaller classes requires a single person or 2, this number can increase or additional roles will be added when severity of the environmental condition or complexity of the mission

increases.

The X-Class follows Shared Roles Model dominantly, it is a mixture of the Taskable Agent Model and the Remote Tool Model for describing human-robot teaming [27]. Majority of semi-auto platform in this class requires human supervision or remote control if it is necessary in the any moments. The Pilot will control the vehicle through live stream video or LOS while the vehicle by itself is monitoring or collecting data as it was programmed. On the other hand, the operator can be looking for some targets through the monitor and make decision while the vehicle is auto piloted for programmed pathway. For example, Valada [28] calls his interface end-user interface and centralized operator interface that falls into the shared roles model category. Valada [28] states this interface provides a single Pilot with an overview of the boat’s condition or situation and provides high and low level commands for interacting with them. For example, a centralized Pilot provides the highest situation awareness but in the case of lost connection from human operator, the boat will make decision and adjustments depending on the programmed tasks or a priority.

On the other hand, for a larger platform with complex and multi-missions, the Pilot is dedicated to a single complicated task with other human team members. The Pilot is dedicated to the operation of the vehicles and additional human roles such as Teammate and Mission Specialist are added to the human team. Especially for the combat mission, the Pilot controls the vehicle and identify the target, then additional human role such as Mission Specialist either give an order as a Commander or conduct the order on its own. The balance of the Pilot ’s task level can apply, influence, and optimize its mission and purpose.

2.3.2

Mission Specialist

The Mission Specialist role is a combination of a Supervisor and Teammate. Peschel [25] describes this role for UAVs as the team member responsible for visual investigation and recording and, in more advanced vehicle systems, delivery of an on-board pay-load. For this survey, the Mission Specialist is a person who is dedicated to process and analyze data or purely conducting supervision to make decisions without piloting the platform.

This role does not necessary shows up to every platform if this role is conducted by the Pilot. The X-Class and some of Intermediate-Class seems to have a single human role system, so that the Mission Specialist role is included as a part of the Pilot role. The Mariner 560 calls its Mission Specialist as an USV Operator and it monitors the Mariner 560 and its installed payload from the Vehicle Control Station (VCS) which features electronic charts,

engine and navigation info [29]. For the larger classes, this role presence might be a critical. This role also does not necessary has to be at the same site as a Pilot as long as a Mission Specialist has a robust and fast communication system. But usually a Mission Specialist stands next to a Pilot. For Harbor-Class, ivind [30] calls it as Observer and is responsible for fusing sensor data to provide a good estimate of the vessel state, as well as creating an image of the surrounding environment. Then this processed data becomes an input for their Pilot to make an order and conduct the mission. The Mission Specialist appearance depends on the type and complexity of mission and numbers of ongoing tasks at the missions.

Table 2.1: Classifications of Selected Unmanned Surface Vehicles (USVs) Currently in Operation1 Group USV Platform Name Size2 [meters] Weight3 [kilo-grams] Range [kilo- me-ters] Endu -rance [hours] Mission type

Ziphius 0.35 × 0.25 1.5 0.09 1 Recreational use (Shallow/Open

Water)

Micro Lutra 0.81 × 0.47 6.92 2.4 4-8 Environmental/Water Monitoring,

Flood Study (Shallow Water) Pioneer 1.07 × 0.64 7.0 0.3 0.2-1 Environment Cleanup, Infrastructure

Inspection, Filming (Shallow Water) Hurricane

Tracker 1.65 × 0.38 57.0 Satellite 120-240

Hurricane/Sea-Level Research (Open/Choppy Water) Small Smartphone

Charlie 2.4 × 1.7 300 WIFI N/A

Environmental Monitoring (Shallow Water)

Wave Glider

SV3 2.9 × 0.67 122 WIFI 8760

Patrol, Monitoring (Open/Choppy Water)

Wam-V 3.6 × 1.8 68.0 80 N/A Surveillance, Research (Open/Choppy

Water)

Inter-mediate VaCas 4.79 × 2.0 181 WIFI 72

River Traffic/Navigation (Shallow Water)

OASIS 5.48 × 2.4 1,360 Satellite 2160-4320

Weather Faorescasting, Hurricane Study (Open Water)

Harbor Kan-chan 7.99 × 2.8 3,500 N/A N/A Environmental/Ocean Study (Shallow Water)

Viknes 8.52 × 2.97 3,300 N/A Mine Sweeping, Weapon Attack

Training (Open/Choppy Water)

Protector 9.5 × 3.5 4,000 10-20 8 Armed Combat (Open/Choppy

Water)

Seastar 11.0 × 3.5 6,000 555 10 Home Land Security/Naval

Application (Open/Choppy Water)

Fleet Protector 11.0 × 7,800 N/A 12 Armed Combat (Open/Choppy

Water)

SCOAP 11.0 × 5.0 N/A Satellite 720 Oceanographic Observation & Data Collection (Shallow/Open Water) Vigilant 16.0 × 3.6 6,000 2778 720 Surveillance, Search & Rescue

(Open/Choppy Water)

EFG Piranha 16.5 × 3.2 3,630 32 4023 Surveillance (Open/Choppy Water)

Poseidon 20.0 × 5.5 40,000 3704 168 Surveillance (Open/Choppy Water) 1_{Maximum operational parameters are reported and referenced from manufacturer specification sheets - normal} operational parameter values will usually be lower and domain dependent. 2Dimensions given are (length × width) 3_{The maximum payload weight the vehicle can carry are proportional to the vehicle weight}

T able 2.2: Hardw a re Based HMI of Selected Unmanned Surface V ehicles (USVs) Curren tly in Op eration V ehicle Jo ystic k R C -Con troller Key -b oard Buttons T ouc h -screen Laptop or PC7 Smart _-phone T ablet Dislpla y Console Ziphius [7] [7] [7] [7] Lutra [8] [8] [8] Pioneer [9] [9] Hurricane T rac k er [12] [10, 12] [10] Smart-phone Charlie [13] [13] [13] [13] [13] [13] [13] [13] W a v e Gli der SV3 [14] [14] W AM-V [16] [16] V aCas [31, 15] [31, 15] O ASIS [32] [18] [18] Kan-c han [33] Viknes [30] Protector (9m) [34] [34] [34] [35] [34] [35] Seastar [36] [36] [36] [36] Protector (11m) [34] [34] [34] [35] [34] [35] SCO AP [37] [37] Viglan t [38] [38] [38] Piranha _Poseidon [39] [39] [39]

T able 2.3: Soft w are Based HMI of Selected Unmanned Surface V ehicles (USVs) C urren tly in Op eration V ehicle Real Time Video API Customization Syn thetic Ov erla y Men us(Simple) Men us(Compl e x ) Ziphius [7] [7] [7] [7] Lutra [8] [8] [8] [8] Pioneer [9] Hurricane T rac k er [10] [10] Smart-phone Charlie [13] [13] [13] [13] W a v e Glider SV3 [14] [14] [14] W AM-V [16, 17] V aCas [15] O ASIS [32] [40] [40] Kan-c han [41, 33, 42] [41, 33, 42] [41, 33, 42] Viknes [30] [30] Protector (9m) [35, 34] [35, 34] [35, 34] Seastar [36] [36] Protector (11m) [34, 35] [34, 35] [34, 35] SCO AP [37] Vigilan t [38] [38] [38] Piranha [43] P oseidon [39] [39]

CHAPTER 3

HUMAN-MACHINE INTERACTION ANALYSIS

In this section, hardware and software based HMI are identified and analyzed by each classes. Most classes have real time video streaming for human in loop operation. Many platforms throughout the classes fall to the category of semi-autonomous which can be conducted remotely (tele-operated) or have some autonomy. The following terms are de-fined for SUV control systems: Fully autonomous is when USVs accomplish the entire mis-sion without human assistance while adapting to operational and environmental conditions; semi-autonomous is when USVs conduct the mission with various level of human-machine interaction, but it is also have capability of autonomous operation; manual or remote control is when USVs are operating under human supervision and human control, when operating in this state, the HMI is considered to be the maximum level [21, 1].

The relationship between the level of HMI and the level of autonomy has an optimized balance for each environmental situation or the mission type. This is also supported with the relationship for the level of SA for human and machine influencing the level automation [22] as discussed in Chapter 2. Human roles are mainly defined as a Pilot and a Mission Specialist. For the smaller platforms, single or few operators are identified but their actual tasks are not clearly stated. Smaller USVs typically conduct environmental monitoring and surveying missions that are programmable under human operator’s remote vehicle control, or completely programmed operations without human control. Chapter 2 summarized hu-man roles as tasks during the mission for the huhu-man role have direct influences when the automation level is discussed. The X-Class pilots might have to conduct multi-tasks during operations including operating and control the SUV and making mission critical decisions. As the size of the vehicle increases, the complexity of its missions can also increase. The

operation requires a larger number of human roles and typically this additional role is con-sidered to be the mission specialists. Then their control system shifts toward semi-auto and remote full-autonomy while human roles are dedicated to a single complicated task with other human team members. The balance of operator’s task level seems to apply, influence, and optimize its mission and purpose while the size of vehicles and human roles are adjusted depending on the complexity and types of their missions.

In addition to this factor, larger vehicle classes tend to receive a human control from the base station or other vessels. This is because the larger vehicles have an ability to be equipped with a wider range of signal communication systems compare to X-Class. Vehicle controls are conducted through software rather than handset remote controller that are commonly used by the operator for X-Class vehicles. Software used for larger system uses more complex control system due to number of on-going mission at once and this fact shows a correlation to higher number of human roles that are required for larger systems. Complex human tasks require full attention by human (high human SA). In order to complete a multi-mission, the number of human increases. Hence, human roles depends highly on the type of missions, target locations or environmental conditions, size, and complexity of platforms.

3.1

Human-Machine Interaction for X-Class

Types of the human role for the X-Class are not necessary identified or specified due to simplicity and user-friendliness of X-Class as its advantages. With the information, the operator might be the most commonly used term for this class and they tend to be required to conduct multi-tasks during their mission. X-Class follows Shared Roles Model dominantly, it is a mixture of the Taskable Agent Model and the Remote Tool Model for describing human-robot teaming [27]. The Majority of semi-auto platform in this class requires human supervision or remote control if it is necessary in the any moments. The pilot controls the vehicle through live stream video or line of sight while the vehicle by itself is monitoring or collecting data as it was programmed. On the other hand, the operator can be looking for

some targets through the monitor and make decision while the vehicle is auto piloted for programmed pathway. For example, Valada [28] calls his interface end-user interface and centralized operator interface that falls into the shared roles model category. He says this interface provides a single operator with an overview of the boat’s condition or situation and provides high and low level commands for interacting with them. For example, a centralized operator provides the highest situation awareness but in the case of lost connection from human operator, the boat will make decision and adjustments depending on the programmed tasks or a priority.

In X-Class, the majority of vessels are controlled with laptop or computer manually, semi-auto or combination of both. Then rests are either controlled manually with RC remote control system or special consoles made for the vehicle. The majority of system are mixture of remote and semi-auto for simplicity of deployment and recovery. Also depends on the mission, this level of the mixture of control system varies due to the dependence level of humans’ SA for decision making. Osga [44] claimed as ”Human Factors Issues” for HMI challenge: attention management and allocation explains about user’s requirement to ad-just USVs control to their environment condition such as wave speed, surface traffic, and mission tempo; mental model of robot and state explain that users are required to maintain their situation awareness of USVs mission status and USVs condition; and lastly, users are required to perform emergency maneuvers for sudden change in mission or accident during programmed operation of USVs. For example, a complex mission such as military mission and rescue mission requires high level of human situation awareness since the mission is not programmable with their software interface. EMILY [45] is controlled remotely for rescue mission which is the second part of the Osga’s [44] claim but the same platform of EMILY that is used for hurricane research NOAA’s Hurricane Tracker [12] is both semi-control and remotely controlled to collect the data in the eye of the hurricane. Their missions can be operated through cell phone at the base station to make an adjustment on the hardware on-board. On the other hand, a simple mission such as environmental monitoring or sensing does not require high human SA.

Except the time of manual control, most of the vehicles are monitored and controlled through a monitor with live stream with LOS or software utilizing way-point location by GPS or satellite. This class vessels are usually operated by a one person conducting pilot and mission specialist role. Such multi-tasks are enable since their missions are tend to be less complex and in smaller range.

As the software point of view, there is a mixture of complex and simple menus for the operator. Most of platforms use real time video stream for the visual control for human in-loop operation and these software are API customizable. Another observation of this class is that vehicle control is usually human in-loop, but actual missions to obtain and store the data can be conducted fully autonomy such as water sampling, temperature sensor,depth measurements features.

3.1.1

Micro X-Class

The majority of the Micro X-Class includes both RC-controller and laptop or computer as the operation system. Depending on the mission or environmental condition, this operation method can be adjusted that will cause the difference in automation. Others are controlled through laptop or computer system. Hardware control is utilized more often rather than software while their missions are usually software based and programmable.

The Ziphius [7] is the smallest, and the first app-controlled aquatic USV in the industry. Its application on the tablet is utilized as a device by an operator with synthetic overlay on the live stream video. This application creates a simple synthetic overlay console on the touch screen tablet display to provide user-friendly operation. Their application is developed using the YVision of the Unity platform that allows visual based platform based on Natural User Interface (NUI) [46]. On the other hand, the Pioneer [9] uses RC controller on the LOS for direct piloting or through the live video stream. The Lutra [24, 8] is controlled visually with a laptop interfaced with Arduino micro-controller system linked with cell phone and its video on board. It can be controlled autonomy by path planning but since the Lutra is used for flood situation, the obstacle avoidance and decision making become the critical

role. Additionally the Lutra is a team player, the multi-vehicle mission is allowed for this platform. An operator can link up and monitor several platforms on the one shared laptop screen.

3.1.2

Small X-Class

The majority of the Small X-Class also has laptop or computer as their operation system. Addition to the RC-controller, there are unique consoles for each platforms including joy-sticks, keyboard, and buttons. Real time video or GPS positioning are used for the most of platforms to control the vehicles. The Wave Glider [14] is controlled by base station though WIFI. Their operation is web-based open source application and the glider is remotely con-trolled with position on the screen over aerial image. This platform is solar powered and it is more like floating (Glider) than a boat.

For some platforms, smart phones are used to command the decision or pilot the vehicles while the mission. The smart phone allows the wider range of communication due to the development of 3G, 4G network. Some of the cell phone applications are following. The Hurricane Tracker uses an autopilot system connected to line of sight digital radio or Short Burst Iridium Satellite link [26]. It can be controlled through a ground station laptop or Internet software site operated by NOAA. The Hurricane Tracker also can receive the urgent change in mission through text message from the cell phone to control the hardware on the vehicle [12]. The Smart phone charlie [13] can be piloted by using applications on the smart-phone touch screen for ad hock adjustment while the large console stays at base station. It uses Google’s Android open source platform, a touch screen and attitude sensor on the Android’s OS system. Then a smart-phone become a compact controller for the Charlie. There are 2 applications developed for the control system: the first one uses Google map to remote or semi-control by using a smart-phone as a joystick to change the direction by utilizing the attitude sensor though LOS, then touch screen will allows the user to command a decision such as speed; the second one uses geo-referenced map and the user will decide the direction by touching a position on the smart-phone screen and the vehicle will follow to the

route. This human interface improved the operator’s difficulty for the mission control. Their smart-phone operation was tested not to require special training for the operator. Addition to these following platforms, long endurance environmental monitoring glider uses cell phone modem and satellite as the communication tool from base station computers.

3.2

Human-Machine Interaction for Intermediate-Class

The majority of Intermediate-Class has semi-autonomous system and it is remotely con-trolled by special console or laptop at the base station or other following vessels. The consoles are consisting of joystick, buttons, keyboard, and display. Then once is located in target spot, it can conduct the mission following path planning or way-point. For long endurance mission, the mission is thoroughly conducted by non-human and human will be notify if ur-gent situation occurs. This long endurance data collection can be robust due to improvement of communication systems and fast processing time. On the other hand, for short endurance mission, human is always in loop.

The mission type have direct impacts on human roles and the level of HMI. In this class, there are more than one human roles can be expected due to possibilities of the complexity of control and expanded range of missions.

The control system for the WAM-V became the challenge topic for 2014 Martime RobotX competition. The OASIS is semi-autonomous control and it is remotely controlled with portable remote control unit or game controller then once it is deployed to the targeted area, it is controlled by autonomous guidance navigation and control (GNC) for course tracking and station keeping [18]. Additionally OASIS states ”Additionally at the ground control station, a graphical control station application provides an interface for an operator to monitor platform telemetry via strip chart and tabular display as well as view images received from the on-board camera [18]”. The pilot is allowed to process, sort, and make decision with the control station that can also be interfaced with Google Earth. Then the control station ”interfaces with a charting application to facilitate planning and situational awareness[18]”.

The VaCAS [31] group developed the guidance and control system for river-line operation. Its underwater sonar and laser-scanner collect the data and process then create a real-time mapping for path-planning for the mission. The VaCAS experiment succeed fully-autonomous control for their targeted river-line area. From this class, the transition of hardware based control to software based control became more obvious.

In Intermediate-Class, the vehicles are controlled semi-autonomy. As a experimental stage, VaCAS group succeed full-autonomy control for limited region. Due to the size of the vehicles, many platforms are controlled remotely though LOS or though LOS on real time video until target area and then it conduct the missions for long endurance. If the mission is short endurance, it is usually remotely controlled with higher speed by an operator.

3.3

Human-Machine Interaction for Harbor-Class

Human roles for Harbor class is mixtures of dependence in software and hardware based control. Human is still in the center of the circle of for the decision making or remotely operated through live video stream. Most of the system uses laptop or computer at the base station as a control devices. This computer or laptop is operated as a part of console that includes joystick for piloting through live video on the display. Software side of the systems are very complex and utilize synthetic overlay for positioning assistance and sensing data visualization.

Harbor-Class, the vehicles are controlled through RC controller with lines of sight, laptop and computer, or consoles with joysticks, keyboards, and buttons through real time videos. The software systems are complex and majorities of systems are made for the specific mission. Due to the complexity of the control and the accuracy requirement, multiple operators are present at the base station. Each person will have individual task to make decisions. Humans are heavily in loop during the missions.

3.4

Human-Machine Interaction for Fleet-Class

Fleet class is very similar to Harbor class. Complex menus are used for software based control and there are at least two operators to control the vehicle.

In 2014, Office of Naval Research (ONR) announced the retrofitting projects for Fleet-Class Unmanned Surface Vehicles to perform ”Swarm”, multi-vehicle team mission. This system allows existing manned or unmanned boat to perform ”Swarm” mission with low-cost of installation [47]. This projects utilizes the technology developed by ONR called Control Architecture for Robotic Agent Command and Sensing (CARACaS). This technology is still under development but in the future, CARACaS is expected to be retrofitted not only to Fleet-Class USVs, but other sizes. Also it is considered to be adapted for UAVs [48]. The CARACAs will allow existing manned vehicles to be retrofitted to be utilized as unmanned surface vehicles. Advantageously, CARACAs requires low cost of device and installation. Hence, this technology can lead further development and can expand the mission types to be wider ranges and more variety for Fleet-Class vehicles.

In Fleet-Class, all of the platforms have a joystick for its control either with console or the laptop. For the military mission platforms, their software were not customizable but made for the specific complex missions that are interchangeable for each mission. Seastar [49] uses a software called UMAS multi-application command and control system allows the the human team to control and operate from base station and also it allows Seastar’s integration into any C4I network. For the oceanographic monitoring mission of SCOAP [37], the software is customizable. This class is very similar to the Harbor-Class platforms. The level of HMI for this class is considered to be low due the remote-operation control and human team has the full insight on decision making. Two people team is appeared to be a common in this class, and each of the member are dedicated to their operation or decision making roles.

3.5

Human-Machine Interaction for EFG-Class

Human roles for E,F,G-Class are not specifically identified due to the limited mission of warfare. There seemed to be more than one person as an operator. The mission specialist stays besides them to analyze the situation, support and make decision.

In EFG-Class, they are either controlled by computer or special console from the base sta-tion or other vessels. All are controlled remotely through live stream video and have complex menus are missions that require multiple human roles but they are not specifically stated since their missions are all military warfare or surveillance. Each vehicle costs over million US dollars and the amount of damage and risk that could cause to human, environment and politics show the hesitation or impossibility of full automation. This class requires full situation awareness from human.

CHAPTER 4

HUMAN-MACHINE INTERACTION FINDINGS

An analysis of the human-machine interaction for six categories of unmanned surface vehicles was conducted and resulted in three findings. The first finding determined that the research literature does not well characterize the human roles present in unmanned surface systems. The second finding suggested that the domain complexity may necessitate increased automation of the robot platform for the human team. The third finding showed that unmanned surface vehicles likely lay on the human-machine interaction spectrum between unmanned ground vehicles and unmanned aerial vehicles.

4.1

Finding 1

Human roles are not explicitly defined or reported in the research literature; at best an Op-erator role is assumed.

Human roles are not always identified in the unmanned surface system literature. Espe-cially for X-Class, human roles are identified in the context of user-friendly control systems or operations and by the actual names of the roles. Systems presented tend to have simple software based menus and control systems that would usually imply a single operator or at most two or three people. An operator typically conducts piloting and decision making. For smaller platforms, automation and remote controls are usually combined for the ad hoc control. For example: (1) piloting can be automated by following planned path while the operator conducts the core mission; (2) piloting can be conducted under supervision of the human operator while the vehicle is programmed to do missions and collect data; (3) or both piloting and the mission are conducted under supervision of a human. Usually, simpler

roles/tasks in each cases are automated or programmed as machines roles. For larger plat-forms, the cost and risk of the failures tend to be higher than for smaller platforms. Due to these reasons, the number of the human roles may be expected to be higher when compared to the smaller platforms as the size of vehicles increase.

4.2

Finding 2

Domain variability (e.g., open water versus debris-filled waterway) and intended tasks neces-sitate increased automation or other human roles with dedicated interfaces.

Obstacle avoidance can vary based on the level of HMI and also heavily limited by the environmental conditions. The level of HMI should correlate with the level of human situ-ation awareness similar to the relsitu-ationship between the level of autonomy and the level of human machine situation awareness stated in Adams [22]. In a slow stream or open water, in order for the vehicles to be floating and moving can be controlled without human in loop. In this situation, the level of situation awareness is low and it does not require high speed processing time. On the other hand, high level of situation awareness is required for fast stream or choppy water with debris and it does require full supervision by human to avoid the obstacles quickly.

The risks and types of the tasks have also direct impact on the variability of the level of the autonomy. High intensity tasks require the highest level of situation awareness, this causes the increase of the number of human roles and human will be in-loop operation. Low intensity tasks require the lowest level of situation awareness that enable the operator to multi-task with an assist from software based machine’s situation awareness. For this situation, human does not necessary have to be in loop most of the time but they cannot be completely absence. Even though there are nearly or completely full-automation with artificial intelligence (non-human in loop), there is no USS with this level of HMI due to policies, ethics, and risks. Especially in the cases of warfare/combat missions, a potential to take away a human life by artificial intelligence can cause controversy. These human feelings

could hold back this improvement for certain types of missions but also there is no ethical answer for the potential risk of the failure.

4.3

Finding 3

The human-machine interaction for unmanned surface systems falls on the spectrum between unmanned ground systems and unmanned aerial systems.

Given the operational intent of USVs, they typically do not have a appropriate HMI besides pilot-centric. The pilot controls and makes decisions under full supervision by the same pilot especially for smaller platforms. Software interfaces for the pilots for the small platforms are simpler, and prepared for the ad-hock use. For larger platforms, the pilots and mission specialists can either share the one interface or have duplicated systems. Sometime the pilots can be the link between the USV and base station. The operators on the chase vehicle can control and conduct the decision after mission specialist analyze the situation from the base station onshore.

Types of missions also can affect this level of HMI. Rescue or military missions that require high level of situation awareness at the decision making during the mission, their HMI have to me human in loop with current level of automation. But for programmed data collection mission like environmental monitoring can expect less human in loop due to absence of decision making. For time of emergency such as obstacle avoidance or lost in communication, they can either manually operated or pre-programmed to return to the shore.

While other unmanned systems have proper HMI for each systems, HMI for the USVs adjusted to the most fitted version and it seems to be among HMI for UGVs, UAVs, and UUVs.

CHAPTER 5

CONCLUSIONS

This research investigation determined that detailed information for HMI and human roles needed to be documented for unmanned surface systems. Without proper documentation, the problems involving HMI are difficult to be identified for the future improvement or potential mission applications.

There are technologies that allow vehicles to be fully automated without human in loop. Vehicles can be programmed with the moving path, mission, and decision making. Although there exists this level of technology, human situation awareness cannot be avoided due to ethical view of the mission. If the level of HMI indicates the level of autonomy, then this level should be adjustable for USVs in order to make ethical decisions along with the most optimized and safe mission activity.

The pilot role exists for every category of USV, but the actual operation of the entire tasks can be differ by size and especially mission types and domains. Even though human tasks are similar throughout USVs such as piloting, data acquisition and process, and decision making, the level of intensity and number of tasks that can be conducted by each individual at the mission can be different. From an environmental perspective, if the ocean is high traffic or severe condition by the waves, high current, or debris, higher human SA is required and this person should be dedicated to this task rather than multi-tasking to conduct other part of mission such as data collection. As socioeconomic view, a simple environmental data collection can be done without maximum usage of human SA. Then human could also do other task at the same time while programmed software is conducting the mission. But for warfare mission like a high intensity combat mission, the critical decision should not be made by the computer since it has high potential to impact human lives and be against human

ethics. Then this task should be carefully reviewed by human roles and the additional roles like mission specialist or additional number of human roles for this task can be utilized. Especially those complex tasks usually have complex software based controls compare to simple mission vehicles. Additional number of human tasks are expected in order for each human role to conduct their tasks under full supervision.

Moreover, the cost of the platform can influence the level of autonomy. Trade off between level of HMI and the risk regarding cost can also be influenced by type and complexity of mission. Smaller low-cost platforms can be less necessary to be concerned even it is lost or broken. With Huang’s approach [21] of visualization of autonomy and HMI. Additional cost and risk aspects into Huang’s diagram can lead to better understanding of the future USV development.

Currently, site condition and the complexity of the mission controls the level of HMI in order to avoid or minimize the failure. For the future, the ideal platform should be able to minimize the level of HMI with independence of the complexity of the mission or environmental condition.

APPENDIX A

T able A.1: Summary of X-Class Unmanned Surface V ehicle s Curren tly in Op eration V ehicle Dimensions [meters] W eigh t [kilograms] Range [kilometers] Endurance [hours] Design Use Ziphius [7] 0.35 x 0.25 1.5 0.09(WiFi 802.11) 1 Sho e-shap e d shell for recreational use for shallo w to deep w ater. Minno w [50, 51] 0.66 x 0.21 -2 WiFi 802.11 N/A Catmaran for join t action with UA V or other larger USV to monitor and surv ey en vironmen t for shallo w to op en w ater. Esp ec ial ly tolerable to chopp y w ater. Lutra [52, 53, 54, 28, 8] 0.81 x 0.47 6.92 2.4(WiFi, 3G or EDGE) 4 -8 Airb oat for m u lti-b oat team mission for en vironmen tal monitoring, w ater qualit y monitoring, depth buo y v erification, flo o d study for shallo w w ater. MAKARA-02 [55, 56] 0.83 x 0.53 3 WiFi 1 Small W aterplane Area Twin Hull (SW A TH) for Indonesian Go v erme n t defense purp oses for shallo w to op en w ater. Esp ecially tolerable to chopp y w ater.

T able A.1: T able A .1 (Con t.) V ehicle Dimensions [meters] W eigh t [kilograms] Range [kilometers] Endurance [hours] Design Use Pioneer [9] 1.07 x 0.64 7 0.3(Radio) 0.2 -1 Twin p on to on for Sur v ey , film pro duction, en vironmen tal clean up, infrastructure insp ection for shallo w w ater. Mala ysia US V [57] 1.1 x 0.87 14.79 25(RF mo dem) N/A Catamaran for w ater qualit y monitoring for li ttoral. ESM30 [58] 1.15 x 0.75 26 2(Handset) -10(Base st ation) 6 Catamaran for w ater sampling monitoring for shal lo w to op en w ater. SURF 20F [59] 1.28 x 0.45 9 2 1 Monoh ull for Eutroph ication and ater Qualit y Monitoring for shallo w to op en w ater. Kingfisher [60] 1.3 x 0. 94 29 0.25(WiFi 2.4GHz ) 1.5 -3 Catamaran h ul l p on to on for m ulti-b oat mission for en vironmen tal monitoring, mapping underw ater top ograph y, bridge scour and sedimen t loss study for shallo w w ater. Kaizu USV [61, 62] 1.42 x 1.2 22 0.5 6 Airb oat for w ate r qualit y monitoring for shallo w w ater.

T able A.1: T able A .1 (Con t.) V ehicle Dimensions [meters] W eigh t [kilograms] Range [kilometers] Endurance [hours] Design Use EMIL Y [63, 45] 1.37 x 0.4 11.3 1.6 0.5 Buo y for Emergency In tegrated Life Sa ving for shallo w to op en w ater(o cean, riv er, flo o d). Esp ecially tolerable for chopp y w ater. ME70 MM70 [64, 65] 1.47 x 0.9 58 2(Handset) -10(Base st ation) 10 Catamaran for o cean surv ey , p ollution trac king and sup ervision tec hnology fo r op en w ater. CatONE [66, 67] 1.6 x 1.0 12 N/A 8 Catamaran for m ulti-b oat team mission for Hydrograph ic Measuremen t and En vironmen tal Monitorin g for shallo w w ater basin, se epage detection in cannals for shallo w w ater. Hurricane T rac k er [10, 11, 12] 1.65 x 0.38 57 Short Burst Iridium Satellite link 120 -240 Buo y for h urricane researc h, collect se a-lev el data for NO AA for op en w ater. Esp ecially tolerable for chopp y w ater.

T able A.1: T able A .1 (Con t.) V ehicle Dimensions [meters] W eigh t [kilograms] Range [kilometers] Endurance [hours] Design Use Surfsense [68, 69] 1.65 x 1.22 68 Sigh t digital radio, SA TCOM N/A Catamaran for Mappi ng and surv ey m ission for v ery shallo w w ater. AutoCat [70] 1.8 x 1.0 10 Radio N/A Catamaran for Sub -b ottom surv ey , A UV telemetry link, or marine life trac king for op en w ater. GSV ”Ro cky” [71, 72] 1.9 x 1.2 34 3.2(W iFi 802.11G) 2 Catamaran T-shap ed chassis for en vironmen tal Monitoring to ol, and mapping for Op en w ater. Rob o d uc k 2 [73] 2.13 x 0.71 43 N/A 6 -8 Monoh ull for collecting data used b y a team of researc hers, study the effect of harmful algal blo om for shallo w to op en w ater.

T able A.1: T able A .1 (Con t.) V ehicle Dimensions [meters] W eigh t [kilograms] Range [kilometers] Endurance [hours] Design Use AMB/ASMV [74, 75, 76] 2.13 x 0.91 176 32 5 -7.5 NOMAD buo y for Surv eying and monitoring the lago on or coastal en viron m en t with resp ect to geological, biological, ph ysical and chemical phenomenon in a cost-effectiv e manner for Op en w ater (deep w ater) C-Cat2 [77] 2.4 x 1.2 80 28 -37 5.8 Catamaran rugged h ull for w ater qu alit y sampling, en vironmen tal assessmen ts and in-shore surv ey for shallo w w ater. C-Stat [78] 2.4 x 1.2 450 463 96 Buo y for surface to un derw ater comm unications no de, p ort, harb our and ship se curi ty , o ceanographic data collec ti on, subsea asset p ositionin g for Op en w ater.

CNR-ISSIA Smartphone Charlie

[13] 2.4 x 1.7 300 WiFi N/A Catamaran for en vironmen tal monitoring for shallo w w ater.

T able A.1: T able A .1 (Con t.) V ehicle Dimensions [meters] W eigh t [kilograms] Range [kilometers] Endurance [hours] Design Use SESAMO [79, 80] 2.4 x 1.8 360 0.55 20 Catamaran for sea w ater sampling for op en w ater(deep w ater. Shanghai Maritime Univ ersit y USV [81] 2.7 x 1.48 60 1.8 2 Catamaran for harb or surv eillance, w ater qualit y sampling, h ydrologic surv ey , maritime searc h and rescue for op en w ater(o cean, littoral). W a v e G lider SV3 [14] 2.9 x 0.67 122 WiFi 802.11g, Cellphone Mo dem 8760 Glider for P atrol and monitoring for op en w ater(deep w ater). Mala ysia P atrol Mo del [82] 2.96 x 1.07 267 0.25(WiFi 2.4 GHz) N/A Jet Ski for sea patrol and en vironmen tal monitoring for op en w ater. Esp ecially tolerable for chopp y w ater.

T able A.2: Summary of In termediate-Class Unmanned Su rface V ehicles Curren tly in Op eration V ehicle Dimensions [meters] W eigh t [kilograms] Range [kilometers] Endurance [hours] Design Use SCOUT [83, 84] 3.05 x 0.55 81.6 0.25(WiFi 802.11b, Radio 2.4 GHz) 8 Ka y ak for milti-b oat tem mission for o ce an ographic and undersea testing for op en w ater. Stingra y [85, 86] 3.2 x 1.2 800 Wireless LOS 8 Jet ski for Coast Guard application, patrol, clearing shipping lanes and underw ater searc h missions for op en w ater. Esp ecially tolerable for chopp y w ater. Blac kfish [87, 88, 89] 3.22 x 1.2 470 1 1 Jet Ski for Maritime securit y , na v al sp ecial w arfare and tactical in telligence, surv eillance and reconnaissance missions for shallo w to op en w ater. Esp ecially tolerable for chopp y w ater. MESSIN [90, 91] 3.3 x 1.8 350 0.25(WiFi 2.4 GHz) 3 -10 Catamaran for scien tific marine researc h, trac king dolphins for shallo w to op en w ater. Esp ecially tolerable for chopp y w ater.