The current communication systems like Zoom, WebEx, MS Teams, Skype, WhatsApp, etc. lack tactile feedback, limiting the immersive experience to audio and visual cues only. Users cannot physically interact with each other during chats, which reduces the realism and engagement of remote communication. This gap highlights the need for a telepresence system that integrates tactile perception alongside audio and visual feedback, allowing users to feel as well as see and hear each other in virtual environments. This telepresence system should enhance the sense of presence and interaction in remote communication scenarios, making conversations more realistic and engaging across distances.
Figure 1: Overview of single user side of the system showing tactile feedback
Figure 2: The overview of the total system showing two users in the virtual space
Figure 3: Block diagram of the entire system
This patent introduces an advanced 3D virtual reality communication system that goes beyond typical audio and video chat by integrating tactile feedback. Using Kinect sensors, it captures users' movements to animate their virtual avatars in a shared virtual environment. This allows users to not only see and hear each other but also physically interact through touch. The system includes miniature vibrator actuators on users' clothing, which respond to collisions and interactions in the virtual world, enhancing the immersive experience. By combining these technologies, the system aims to create a more realistic and engaging communication platform suitable for multi-party interactions across different geographical locations.
- Tactile Feedback Integration: Enables users to experience tactile sensations through miniature vibrator actuators placed as a wearable device, enhancing immersion and interaction realism in virtual environments.
- 3D Avatar Representation: Instead of traditional video streams, utilizes pose estimation data to render users as interactive 3D avatars, allowing for spatial awareness and enhanced visual engagement. Hierarchical Body Structure of the
- Interacting Body: Establishes a hierarchical bone framework based on joint positions, facilitating accurate and lifelike mesh deformation for avatar animations, enhancing visual fidelity.
- Pose Estimation: Utilizes Kinect equivalent devices to precisely estimate joint coordinates, enabling real-time tracking and synchronization of user movements into the virtual environment, promoting seamless interaction.
- Low Hardware Overhead: Optimizes system efficiency by leveraging commercially available Kinect equivalent sensors and vibrator actuators, ensuring cost-effectiveness and accessibility while maintaining high performance in virtual reality applications.
This technology has reached TRL level 5. It has been implemented and demonstrated and is ready for absorption in any interactive bidirectional communication engine or platform through appropriate adaptation to the platform.
5
- Promotes cultural exchange and global collaboration through realistic avatar interactions and shared virtual environments.
- Facilitates remote learning and professional training with enhanced engagement and realistic simulation capabilities.
- Supports therapeutic applications by enabling tactile feedback and immersive environments for rehabilitation and mental health treatments.
- Encourages innovation in communication technologies, potentially redefining how people interact and collaborate across distances.
Bi/multi-agent communication and interaction, video conferencing applications, messaging applications, communication technology, virtual reality, Information and communications technology (ICT), haptics, Human-computer interaction (HCI)
Communication Systems, Virtual Reality, Human-Computer Interaction, Tactile Feedback, Immersive Environments, Physics Simulation
503/MUM/2015
430440