Cyber-Physical Human Systems: Putting People in the Loop

Abstract

This article outlines the challenge to understand how to integrate people into a new generation of cyber-physical-human systems (CPHSs) and proposes a human service capability description model to help.

Cyber-Physical Human Systems (CPHS) consist of interconnected systems (computers, cyber physical devices, and people) “talking” to each other across space and time, and allowing other systems, devices, and data streams to connect and disconnect.¹

The NASA System Engineering Handbook defines a system as the combination of elements, including personnel, processes, and procedures that function together to produce the capability to meet a need². Thus, to build CPHS effectively with people in the loop, it is important to be able to describe what a person can do, and when and where he or she can do it within the CPHS.

While most of us think about people using systems, many complex systems (such as the smart grid, or smart cities) are actually a combination of computers, machines and people all working together to achieve the goals of the systems.¹ Sometimes the role of people in these systems is passive. For instance, when social networks are used as sensors, people are not active participants, rather their general observations are analyzed for information useful in a specific situation. In other cases, people actively participate. A person may be required to perform maintenance on a piece of equipment, or perhaps respond to a medical emergency. In this case, it is important to understand what capabilities the person who might perform the task has. Many have predicted that human-interactive systems, such as CPHSs, will continue to leverage people’s capabilities. In particular, high levels of situation awareness and adaptability to better meet the goals of advanced complex systems. A big challenge is how to choose the right person for a given task.

People in the CPHS Loop

While most systems are designed to satisfy a person’s requirements as a user, people, in turn, can help a complex system to make intelligent decisions and achieve its goals (which ultimately are people’s goals). Human-in-the-loop and human-in-the-mesh have extensively been used to describe the person as an operator and a part of CPHS.³ To develop a new generation of CPHS that include people in the loop, it’s important to understand what differentiates a person participating in a CPHS from a traditional cyber-physical component:

• Cognition. People have brains, eyes, and ears, while computer systems use CPUs, RAM/ROM, sensors, and actuators. The different ways in which people and computers observe, process, and act present challenges (and opportunities) for people to work together with computers to best achieve a goal.

• Predicatibility. People do not perform the same task the same way every time, they may choose not to follow instructions or may lose focus in the middle of a task. While people may be less reliable than computers when it comes to following instructions, they have the ability to adapt much better to changing situations and can often come up with out of the box solutions.

• Motivation. People, unlike computational systems, require incentives. This motivation takes many forms, from monetary compensation to a pat on the back. Without proper motivation a person may not perform a task even after agreeing to do so.

This is a challenge because it’s difficult to know what a person is capable of doing (his capabilities) —that is, what each individual can and cannot do in a given system, in a given context. As opposed to mechanical sensor devices, people might choose not to do something, or choose to do it in a different way. For instance, an air quality sensor in a CPHS will always report the air pollution values in the same way with the same uncertainties. On the other hand, a person observing might let their mood influence their perceptions of pollution when reporting, or may stop reporting altogether without notice. The systems designer must design the CPHS system with an understanding of what people can do and how they interact with other people, computers and machines.

To help express the information described above, a Human Service Capability Description (HSCD) data model is proposed that shows the general capabilities of a person and his/her capabilities to perform specific tasks (for instance, sensing, actuating, and processing). The capabilities are grouped into elements, so that designers can effectively identify and integrate people into their design process. The HSCD model uses the basics of service capability description⁴, but is tailored to describing people.

Human Service Capability Description Model

In a typical CPHS, people will function together with other components such as sensors, actuators, processors, data stores, and networks.4 The main objectives of the HSCD model, shown in Figure 1, are to represent:

Figure 1. Overview of the human service capability description model.

The model can be used to help understand the capabilities of a person and his/her ability to perform a task.

• the tasks a person can perform;

• the qualifications a person has for performing the tasks;

• the types of interfaces that can be used to interact with the person;

• the identity of the person (from a system perspective).

In the HSCD model, a CPHS is a system that is composed of core components (including the human component) that are delivered as services. These components are networked at every scale, sharable, and configurable. They exist as service components that interact with the other CPHS components through digital communications. People can perform complex tasks based on their ability to sense, act, store, and process data.

The HSCD model describes a person’s capabilites using the following elements (italics is used to represent a class in the HCSD model shown in figure 1): Service, Person Descriptor, Interaction End, Task, Availability, Qualification and Certification, Rating, and Authority.

Service

Service describes the information for a person to interact with other services. Each service has 1) an identifier (uid), which can be referenced when the service is being requested by another component or application; 2) a person who is represented by the service; 3) a set of interaction ends that describe how other services would communicate with the person.

Person Descriptor

Person Descriptor is the person that can perform a service. The person uses one or more applications that provide the connection between the person and other services (through some IT device). The Person is a collection of information on a person’s qualifications and past experience in performing a given task.

Interaction End

The Interaction End is one end of the communication between two service components, which the person accesses through an application. It contains information related to the Data that can be communicated, as well as the way it can be communicated through its Interfaces.

Interface describes how the Interaction End used by the person exchanges information with other components, such as protocols, devices (e.g. mobile), and mechanisms used to communicate. It’s important to understand how information can flow in either direction.

Data describes the format of the data being exchanged between the Person and other CPHS components—it covers data structure, format, and semantics. This is important because humans use different languages to communicate compared to computers.

Task

A Task relates directly to a task the person can perform within the CPHS. The task can be very specific or general depending on the situation. A Person may have the ability to perform more than one task.

Performed Tasks are those a person or device has previously performed in the context of a CPHS. These are particularly important because it can be difficult to determine a confidence level for task performance. Because people can be rated based on the performance of previous tasks, this is one of the better ways to gain confidence in their ability to perform. Under a Rating, the performed task may be rated by the system builder or by other components based on how well they performed the task. As this reflects real world feedback on the person’s performance, it might provide the most relevant information about a person’s ability to perform a given task.

Potential Tasks are those that a human is willing to perform within the context of the CPHS.

Availability

The person must be available to perform and complete a task that is; the person must be in the right place at the right time to perform a task. The location is the place in which a person is available to perform tasks. It might be specific to a Potential Task, or can apply to all the tasks the person is willing to perform. The time type describes the period in which a person is available to perform tasks. It might also be specific to a Potential Task, or can apply to all the tasks he is willing to perform.

Qualification and Certification

A qualification shows the person has some general or specific aptitude. Qualifications are more formal than ratings and usually cover a set of tasks. The qualification types are:

• Training is designed to contain any information related to the formal education or training a person has that is relevant to the tasks the person can perform.

• Testing contains any aptitude test results relevant to the set of tasks the person may perform.

• Clearance contains any security status related to the set of task the person may perform

• Experience contains any job related information, including recommendations from previous or current employers, commendations, and other relevant information.

A Certificate, therefore, represents proof of a Qualification.

Rating

A Rating is informal feedback given on how a person performed a given task. Because of its informal and likely unverified nature, a single rating may not be useful, but a set of ratings should give the system builder some confidence is the person’s ability to perform a task. In the case of participatory sensing, individuals might be rated by their peers (other services), and the results could be used to gauge their ability to perform a task. Combining the Ratings with the Qualifications will hopefully provide a high level of confidence.

Issuing Entity

In the context of a CPHS, an Issuing Entity provides some level of assurance of the rating or certification of the person’s qualifications. This can be an accreditation or standardization body, or an informal entity. The assurance level required will depend on the task at hand.

Evaluation of the Model

This model focuses on the structure of data representing a person’s capability. Further input is needed from the community of experts working on modeling human behavior. One next step is to work with experts in the fields of human behavior, human resources, and other experts involved in understanding how people act. Additionally, a test bed is being developed at the National Institute of Standards and Technology (NIST), where this model could be implemented. A possible test scenario for the test bed is the emulation of a disaster response system. Based on a real-world scenario, a simplified virtual environment will be created with experimetors representing residents and emergency responders in a flood-affected region involved in disaster and recovery operations. The CPHS uses information from physical and social to direct emergency and rescue teams to respond to specific emergencies in the most efficient manner possible. Volunteers on the scene are then directed to help out where needed.

Conclusion

Research and development in ubiquitous and social computing—especially in the fields of the Internet of Things and participatory sensing and crowd sourcing for disaster management5—is driving the need to be more explicit about how people are an integral part of of systems and need to be included in the system development process. The HSCD model is a good starting point for describing the roles people play in CPHSs and should help improve the CPHS design process.

Biographies

Sulayman K. Sowe is a senior researcher in the Information Services Platform Laboratory at the National Institute of Information and Communications Technology (NICT), Japan. His research interests include big data, cloud computing, data-intensive knowledge discovery, open source software development, knowledge sharing, information systems evaluation, social and collaborative networks, and software engineering education. Sowe has numerous publications in international journals and proceedings, and serves on a number of academic, review, and international committees. He received a PhD in informatics from Aristotle University, Greece. Contact him at sowe@nict.go.jp.

Eric Simmon is a senior scientist in the Cyber Infrastructure Group at the US National Institute of Standards and Technology. He leads the NIST effort in cyber-physical cloud computing and is co-chair for the NIST Cyber-Physical Systems Public Working Group use case subgroup. In addition, Simmon is a member of the NIST Cloud Computing program (project editor for ISO/IEC 19086 Service level agreement (SLA) framework and Technology Part 2: Metrics) has worked with the international electronics manufacturing community to develop an international standard for exchanging material content data, and was an original member of the NIST Smart Grid Program (leading the architecture and vehicle-to-grid activities). Contact him at eric.simmon@nist.gov.

Koji Zettsu received Ph.D in Informatics from Kyoto University in 2005. He is a Director of Information Services Platform Laboratory at Universal Communication Research Institute of National Institute of Information and Communications Technology (NICT), Japan. He was a visiting associate professor of Kyoto University, Osaka University from 2008 to 2012, a visiting researcher of Christian-Albrechts-University Kiel, Germany in 2009, and the technical editor of Value-creating Network sub-working group of New Generation Network Forum, Japan from 2009 to 2010. His research interests are information retrieval, databases and software engineering. He is a member of IPSJ, IEICE, DBSJ and ACM. Contact him at zettsu@nict.go.jp.

Frederic de Vaulx is the vice president of Prometheus Computing and a senior software engineer specializing in the analysis, design, and development of software applications contracted to NIST. He is involved in the NIST Cloud Computing Program, leads the Cloud Metrics Public Working Group, and has been involved in cyber-physical system projects at NIST for the past five years. De Vaulx received an MS in computer science from ESIAL//please spell out//, France. Contact him at f.devaulx@prometheuscomputing.com

Irena Bojanova is a computer scientist at NIST. She is the founding chair of the IEEE Computer Society Cloud Computing Special Technical Community, a co-chair of the IEEE Reliability Society Internet of Things Technical Committee, and a founding member of the IEEE Technical Subcommittee on Big Data. Bojanova is the acting editor in chief of IEEE Transactions on Cloud Computing, associate EIC of IT Professional, and an associate editor of the International Journal of Big Data Intelligence. You can read her IoT blog and her cloud computing blog at www.computer.org. Contact her at irena.bojanova@nist.gov.