User interfaces for gesture detection
Inventors
Nie, Yiqiang • Dogrusoz, Kaan E. • MAALOUF, Charles • Ottens, Elizabeth A • QUIST, Brady J.
Assignees
Interested in licensing this patent?
MTEC can help explore whether this patent might be available for licensing for your application.
Publication Number
US-12737050-B2
Publication Date
2026-09-15
Expiration Date
Abstract
A computer system detects a gesture, such as an air gesture, and performs an operation based on the gesture.
Core Innovation
A computer system communicates with a display generation component and one or more input devices, and displays a user interface that includes a plurality of representations corresponding to a plurality of detected external devices. While displaying the user interface, the system detects a first air gesture to display a respective indication corresponding to a respective external device from among the detected external devices, and in response to detecting the first air gesture, displays the respective indication in the user interface.
While displaying the indication corresponding to the respective external device, the computer system detects a respective air gesture and determines whether the respective indication corresponds to a first external device or a second external device different from the first external device. Based on the determination, the system sends a command corresponding to the respective air gesture to the corresponding external device without sending the command to the other external device.
The disclosed interaction is extended with a physiological sensor, including detecting a user's heart rate and displaying the detected heart rate on the display generation component, and with one or more microphones for utterance detection. Additional interaction behaviors include using multiple air gesture types to display and act on device-specific indications, providing tactile feedback, and limiting interaction behaviors based on threshold-based conditions and rate limiting.
Claims Coverage
The independent claims cover three implementation forms—computer system, non-transitory computer-readable storage medium, and method—each using the same core inventive logic for device selection and sending gesture-specific commands only to the externally indicated device. Across the independent claim set, the main inventive features focus on a user interface with representations for multiple detected external devices, selection of an external device by an air gesture that updates an on-screen indication, and sending the gesture-based command only to the determined indicated device, not to other devices.
Device-indication selection via air gesture over a multi-device user interface
Displaying, via the display generation component, a user interface that includes a plurality of representations corresponding to a plurality of detected external devices; while displaying the user interface, detecting a first air gesture to display an indication corresponding to a respective external device from among the detected external devices; and in response to detecting the first air gesture, displaying the indication in the user interface.
Send gesture command only to the indicated external device
While displaying the indication corresponding to the respective external device, detecting a respective air gesture; and in response to detecting the respective air gesture, determining whether the indication corresponds to a first external device or a second external device different from the first external device, and sending a command corresponding to the respective air gesture to the corresponding external device without sending the command to the other external device.
Storage-medium program form of multi-device gesture selection and selective command routing
A non-transitory computer-readable storage medium storing one or more programs configured to be executed by one or more processors, the one or more programs including instructions for displaying a user interface with representations of detected external devices, detecting air gestures to display an indication for a corresponding external device, and in response to detecting a respective air gesture sending a command corresponding to the respective air gesture to only the external device corresponding to the indication.
Method form of multi-device gesture selection and selective command routing
A method comprising, at a computer system in communication with a display generation component and one or more input devices, displaying a user interface with representations corresponding to detected external devices, detecting a first air gesture to display an indication corresponding to a respective external device, displaying the indication, detecting a respective air gesture while displaying the indication, and sending a command corresponding to the respective air gesture to only the external device corresponding to the indication without sending the command to the other external device.
Across the independent claims, the inventive coverage is directed to gesture-driven selection of a device-specific indication within a multi-device user interface and to command routing that transmits gesture-specific commands only to the external device whose indication corresponds to the gesture context, while omitting the same command to other detected external devices.
Stated Advantages
Allows sending a command corresponding to the respective air gesture to an external device indicated by the user gesture without sending the command to a different external device.
Supports selection and indication of one external device among a plurality of detected external devices through air gestures.
Documented Applications
Gesture-detection user interface for selecting among multiple nearby external devices using detected air gestures and device-indication on a display.
Physiological-sensor-integrated interaction where detected heart rate is displayed in conjunction with the gesture-based multi-device interface.
Utterance-based routing using microphones to send information based on an utterance to the external device whose indication corresponds to the utterance.
Interested in licensing this patent?