Assessment of the Psychological and Physiological Effects of Augmented Reality (APPEAR)

Augmented reality (AR) is the use of a computer-based simulation engine to add non-real sensory information to the real sensory world. Essentially, AR directs participants’ attention to either existing information that they would have not been consciously aware of or to new information that changes their perceptual information. Although this co-registered information can be visually projected directly onto real objects, AR information is often presented directly to the recipient by a device attached to the recipient.

APPEAR maps to Defense Health Agency’s (DHA’s) Joint Evacuation and Transport Simulation (JETS) and Point of Injury and Trauma Simulation (POINTS) programs, under the U.S. Army Medical Research and Materiel Command (USAMRMC) Joint Program Committee-1 (JPC-1) Medical Simulation and Information Sciences Research Program (MSISRP) Medical Simulation (MedSim) portfolio. It addresses the capability gaps in the Virtual Patient System (VPS) of JETS and POINTS. The VPS provides intelligent, scalable, modular medical training products, tools, and devices across globally distributed, integrated, and interconnected Live, Virtual, Constructive, and Gaming training environments. Technology using Augmented Reality (AR) is a significant piece of the VPS, at point of injury (POINTS) and point of demand across the complete chain of evacuation (JETS). This research is critical for assessing the limitations of AR that could impact learning effectiveness to ensure optimal development and utilization of AR technology to address the identified capability gaps in military medical simulation training.

 The ultimate goal of this program is to identify psychological and physiological limitations of AR prototypes currently under development or used for medical simulation training in the diverse high pressure and stressful context anticipated in Roles of Care 1-4. This research will assess and inform prototype development and/or refinement of existing AR prototypes for medical simulation, reducing overall developmental risk, 1) enabling efficiency in the design of future AR scenarios, 2) identifying potential safety issues, and 3) identifying risk factors for adverse reactions to AR medical simulations. Assessing the physiological and psychological effects of AR prototypes for military medical simulations is imperative to the technological development and refinement needed to fully address the existing capability gaps identified in the JETS and POINTS programs and deliver effective solutions to the Warfighter. The outcomes of this work will be used to ensure that AR technology is safely and optimally utilized to enhance learning capabilities in medical simulation.

This MTEC program is expected to conduct human studies to address either:

  • Psychological effects of AR on the end user in training behavior (secondary assessments of cognitive load during learning using AR and/or assessment of neuropsychiatric conditions contraindicated for training with AR could be included), or
  • Physiological effects of AR on the end user in training behavior (secondary assessments of cognitive load during learning using AR and/or assessment of physical conditions contraindicated for training with AR could be included).

It is anticipated that the outcome of these awards will achieve at least one of the following (not in rank order):

  • Deliver a human subjects study that assesses the physiological or psychological effects of AR that will be analyzed and recorded in technical reports and, eventually, in the final report.
  •  Enable efficiency in the design of future AR scenarios/prototypes in medical simulations.
  • Identify potential safety issues that should be accounted for in AR prototypes used in medical simulation training.
  • Provide recommendations and/or proposed mitigation strategies regarding any safety issues identified to improve or refine development of or existing AR prototypes.
  • It is anticipated that AR scenarios assessed will be relevant across the military and U.S. Department of Veterans Affairs and potential academic, clinics, rural healthcare settings, private and public hospitals, and international healthcare situations.
  • If applicable, support safety data in investigational device exemption or 510K applications or provide post-mark surveillance data for Food and Drug Administration (FDA)-cleared AR technology.

The research project award recipients were selected from the Offerors who responded to MTEC’s Request for Project Proposals (MTEC-18-09-APPEAR).

Title of Project: Assessment of the Psychological and Physiological Effects of Augmented Reality (APPEAR) – Psychological
Project Team: Chenega Healthcare Services, LLC; The University of California, Irvine School of Medicine; MedCognition
Award Amount: $1,224,864
Project Duration: 24 months
Project Objective:  The overall objective of this project is to examine the psychological effects of augmented reality (AR) medical simulation training. A range of outcomes will be measured with self-reported affective (e.g., emotion) questionnaires in addition to text analyses of qualitative descriptions of the experience. We will also explore whether pre-existing psychological traits (e.g., pre-existing psychological disorder, past stress experiences) influence the AR medical training experience and moderate the emotional responses to the simulation. We will evaluate whether the higher-fidelity, more realistic AR simulation will more successfully illicit emotional stress compared to a standard medical simulator, in a medical student population.

Year One Accomplishments:

  1. Received approval from University of California, Irvine Institutional Review Board; and, US Army Human Research Protection Office
  2. Simulations are created and programmed into the PerSim augmented reality simulators
  3. All Surveys have been extensively piloted
  4. Monitors for electrodermal activity have been tested and are prepared for data collection
  5. All associated researchers have been trained and practiced study protocol
  6.  Recruited and completed study protocol for 48 out of the goal 72 participants.

Title of Project: Assessment of Psychological and Physiological Effects of Augmented Reality: System for Honing Augmented Reality Psychological Suitability (SHARPS) Measures and Models
Project Team: Design Interactive, Inc.
Award Amount:  $1,249,999
Project Duration: 24 months
Project Objective:  SHARPS will quantify the psychological effects of AR on end-user training behavior in combat casualty care. The research is designed to define and evaluate AR Psychological Suitability in terms of AR Training Receptivity and AR Training System Efficacy. AR Training Receptivity can be described as the degree to which the individual to be trained, tasks to be trained, and the environment is conducive and receptive to AR training. AR Training System Efficacy is the degree to which the AR training system promotes skill development and transfer of training to reality. This effort utilizes an existing receptivity model (Jeffrey & Seaton, 2004) to characterize the unique contribution of the individual, the task, and the application environment on human information processing and learning in AR-based medical training. For all three factors receptivity will be characterized by a willingness (of an individual) or inherent characteristic (of a task or environment) as well as an ability (of an individual) or capability (of a task or environment) to absorb, convey, accept, implement, and/or effectively utilize AR training. A process-based empirical evaluation of the AR training system identifies the necessary and efficient factors for improved medical training efficacy. Results support quantitative characterization of the Receptivity construct and facilitate the development of recommendations for increased AR training efficacy and safety.

Title of Project: Assessment of Psychological and Physiological Effects of Augmented Reality: Development of the Dual-Adaptation Protocol for Augmented Reality (DAPAR)
Project Team: Design Interactive, Inc.
Award Amount:  $1,249,999
Project Duration: 24 months
Project Objective:  The DARPA project aims to:

  • Identify and quantify physiological limitations of augmented reality (AR) medical training systems both during and after AR exposure, and characterize, if present, an adverse symptom profile and physiological recovery time course associated with AR exposure.
  • Quantify AR suitability for military medical training via Design Interactive’s psychophysical testing battery and identify effectiveness of AR dual adaptation strategies associated with potential positive physiological effects of AR exposure.
  • Develop AR usage protocols, AR design guidelines, AR usability criteria and AR best practices guide to ensure AR is safe and effective to use in military medical training.