Augmented and virtual reality simulator for professional and educational training

Inventors

Ribeira, Ryan Joseph • Sarma, Karthik Venkataraman • Stadelman, JR., Stanley Thomas • Namperumal, Srihari Kumar • Ribeira, Jason Robert • Miller, Christopher Ryan

Assignees

Simx Inc

Interested in licensing this patent?

MTEC can help explore whether this patent might be available for licensing for your application.

Publication Number

US-10672288-B2

Patent

Publication Date

2020-06-02

Expiration Date


Abstract

A method and apparatus for an augmented reality simulator for professional and educational training is provided. The simulator provides a training environment spanning one or more physical locations in which one or more virtual avatars representing purely virtual objects or persons or real physical objects or persons which are located at a different physical location are projected into the physical space. The avatars are interactive with other avatars and real objects or persons and update over time or in response to actions taken by other real or virtual elements, or based on predefined instructions. Sensors and devices are used to detect the locations of and actions taken by real persons or real objects and this sensed data is used to evolve the state of the simulation and avatars based on predefined instructions and programs and update the view of all participants.

Core Innovation

The invention relates to an augmented or virtual reality simulation training system in which an authoring application receives a set of one or more values that define at least one physical or physiological property of a virtual avatar. In response, the authoring application generates an instruction set for virtual avatar projection into an augmented or virtual reality environment, and the augmented or virtual reality application executes the instruction set so that the virtual avatar projects into real or virtual space with behavior driven by the defined properties.

The instruction set is executed for a plurality of augmented or virtual reality devices so that each device concurrently projects a respective view from a different angle of the virtual avatar based on a physical location of the respective device. The system detects, by a wireless sensor, an interaction between a medical tool that is or virtually present in the real or virtual space and the virtual avatar, and simulates a physiological response within each respective view based at least in part on the type of medical tool being used and the defined physical or physiological properties.

The invention further encompasses instruction-file infrastructure and control capabilities, where generating the instruction set includes selecting predefined instructions, composing new instructions, or uploading instructions from external non-transitory computer-readable media. The system supports time-based changes that create a defined sequence of events over time, and the sequence execution can be navigated by pausing, reversing, accelerating, jumping to a time point, restarting from that point, resuming, or stopping execution for participants sharing the augmented or virtual reality environment.

Claims Coverage

Two independent claims are present. They center on authoring an instruction set that drives projection of a virtual avatar exhibiting property-based behavior across multiple augmented or virtual reality devices, with wireless detection of tool-avatar interactions and simulation of physiological responses.

Authoring property-defining values for virtual avatar

Receiving, by an authoring application running on a computing device, a set of one or more values that define at least one of a physical or a physiological property of a virtual avatar.

Generating instruction set for projection with property-based behavior

Generating, by the authoring application running on the computing device, an instruction set for virtual avatar projection into an augmented or virtual reality environment, wherein the instruction set includes instructions used by an augmented or virtual reality application to project the virtual avatar into real or virtual space such that the virtual avatar exhibits a behavior based, at least in part, on at least one of the physical or the physiological properties defined through the authoring application.

Concurrent multi-device projection from different angles based on physical location

Executing the instruction set by the augmented or virtual reality application for a plurality of augmented or virtual reality devices, wherein each respective augmented or virtual reality device concurrently projects a respective view from a different angle of the virtual avatar exhibiting the behavior based on a physical location of the respective augmented or virtual reality device.

Wireless sensor detection of medical tool interaction with virtual avatar

Detecting, by a wireless sensor of at least one of the augmented or virtual reality devices, an interaction between a medical tool that is or virtually present in the real or virtual space and the virtual avatar.

Physiological response simulation based on medical tool type and defined properties

Simulating, within the respective view for each respective augmented or virtual reality device, a physiological response based at least in part on a type of medical tool being used in the interaction and at least one of the physical or the physiological properties defined through the authoring application.

Instruction-file execution on non-transitory computer-readable media

One or more non-transitory computer-readable media storing instructions which, when executed by one or more computing devices, causes receiving values that define at least one physical or physiological property of a virtual avatar and generating an instruction set for virtual avatar projection into an augmented or virtual reality environment.

Non-transitory media to enable property-based projection and physiological simulation across devices

Executing the instruction set for a plurality of augmented or virtual reality devices, detecting an interaction between a medical tool and the virtual avatar, and simulating a physiological response based on the type of medical tool and the defined physical or physiological properties.

Across both independent claims, the core coverage is an authoring application that defines physical or physiological properties of a virtual avatar, generates an instruction set that drives property-based behavior projection, executes concurrently across multiple AR/VR devices from different angles tied to physical location, detects wireless interactions between a medical tool and the virtual avatar, and simulates physiological responses based on tool type and the defined avatar properties.

Stated Advantages

Not explicitly described in patent.

Documented Applications

Not explicitly described in patent.

JOIN OUR MAILING LIST

Stay Connected with MTEC

Keep up with active and upcoming solicitations, MTEC news and other valuable information.