Posters & Demos

Bhavin Patel, Anthony Foli, Anushree Udhayakumar, Maanas Agrawal, Adam Cross, and Inki Kim

The purpose of our poster/demo is to present the latest development of FlightPath, a Mixed-Reality app originally created to detect concussions. We will highlight the new feature designed to capture and analyze body posture, along with preliminary results from varsity sports players who have experienced concussions.

Isaac Ngui, Courtney McBeth, Grace He, Andre Santos, Luciano Soares, Marco Morales, and Nancy Amato

Many tasks are intuitive for humans but difficult to encode algorithmically when utilizing a robot. Robotic systems often benefit from learning from expert demonstrations, wherein operators move the robot along trajectories. Often, using a physical robot to provide these demonstrations may be difficult or unsafe. Extended reality provides a natural setting for demonstrating trajectories while bypassing safety concerns or modifying existing solution trajectories. We propose the Robot Action Demonstration in Extended Reality (RADER) system for learning from demonstration approaches. We additionally present its application to a state-of-the-art approach and show comparable results between using a physical robot and our system.

Brad Sutton, James Evans, Matt Bramlet, Jenny Amos, Elliot Bethke, Andres Maldonado, Connor Davey, Graham Huesmann

In drug resistant epilepsy patients, there are a two surgical intervention treatments that have been proven to be effective: surgical resection and repetitive neural stimulation. However, intervention requires accurate localization of the seizure onset zone, a task that requires an epileptologist to integrate imaging data across MRI, CT, and EEG electrodes. This task requires significant working memory load to integrate spatiotemporal information across modalities to create a 3D mental model of the patient. In addition, the epileptologist will need to communicate the surgical plan to the neurosurgeon and the rest of the care team. We are creating virtual reality models that integrate this multimodal clinical information into a patient specific 4D model of seizure activity. Our analysis pipeline automatically processes the CT, MRI, and EEG data to produce a model of the brain, recording electrodes, and activity to help visualize the seizure activity in the patient.

Beitong Tian; Duo Wang; Robert Kaufman; Jingcheng Yang; Yuheng Wang; Jionghao Wei; Hongwen Xiao; Shuoning Shi; Junhao Pan; Klara Nahrstedt

Patronus Tech develops AI-powered XR-IoT platforms for research labs. By combining smart safety glasses, modular sensors, and AI agents, we help streamline lab workflows and improve productivity.

Duo Wang, Jerry Ni, Yuhan Zhou, Weiyu Ding, Jackson Song, Mike Yao, Caroline Cao

We present the design of a Mixed Reality (MR) training simulation for procedural medical skills, integrating spatial user interfaces, object tracking, hand tracking, and a large language model (LLM)-powered AI tutor. This work aims to address persistent challenges in manikin-based medical simulations—namely, inconsistent instructional delivery, lack of real-time feedback during practice, and limited access to high-fidelity, repeatable demonstrations. We conducted a formative study with eight medical simulation stakeholders to identify these challenges and inform the design of our system. Using lumbar puncture as a representative high-stakes procedural use case, we developed an MR application with three key features: (1) object-based virtual demonstration using tracked instruments and spatially anchored 3D animations; (2) interactive real-time feedback using hand tracking to guide learner actions; and (3) an AI-powered tutor delivering standardized, multimodal support via voice, text, and animation. This work contributes to the design of hands-free, AI-supported MR interfaces for procedural skill training and offers a framework for applying object-tracking and LLM technologies in medical simulation.

Fernando Sakabe, Grace He, Caroline Stoklosinski, Omosefe Edomwande, Saumya Agarwal, André Santos, Courtney McBeth, Isaac Ngui, Katherine Mimnaugh, Luciano Soares, and Nancy Amato

Understanding how motion planning for robots will be implemented and making changes to those plans is not easy for non-experts. Therefore, providing a way to see how a manipulator or mobile robot will move and allowing the ability to alter those plans in a simple way would make it easier for any user of a robotic system to be in control. Working towards this goal, we built a virtual reality game for presentation at Engineering Open House to teach middle and high school students about motion planning. In the nautically-themed game, players must alter the path of their ship to collect treasures and avoid pirates. This game is the first step towards a future immersive toolkit to facilitate interaction with motion planners.

Sahasra Kotarkonda, Yugank Arora

The GIES Metaverse project is creating a visually immersive, walkable virtual replica of the Business Instructional Facility (BIF) at the University of Illinois. Built in Unity, the environment emphasizes high-quality aesthetics including realistic lighting, textures, and architectural detail to mirror the physical BIF space and enhance user engagement. Users can freely navigate the building in VR, experiencing the layout of Gies in an interactive and intuitive way. The team is in the process of integrating a locally hosted large language model (LLM) into an avatar of Dr. Brunner. This LLM is trained on a general Gies College of Business dataset to enable informative, natural conversations about the school. Current development efforts are focused on refining the virtual environment's look and feel, improving VR movement, and finalizing the LLM integration.

Matthew Bramlet, Bradley P Sutton

This project builds on the AIM Lab’s groundbreaking development of a fully automated 4D heart model derived from retrospectively gated CT scans. Using machine learning techniques trained on a decade of manually segmented cardiac imaging data, the team has created a pipeline that reconstructs the beating heart in motion across the entire cardiac cycle. The approach converts static 3D datasets into dynamic, anatomically accurate models by leveraging ML-based segmentation of myocardial structures, followed by time-sequenced reconstruction and visualization using open-source modeling platforms. This core innovation offers a patient-specific, temporally resolved view of cardiac anatomy—transforming traditional static imaging into an interpretable 4D model. The current project seeks to advance this foundation by refining the generation process, increasing model fidelity, and establishing a seamless pipeline from clinical CT input to diagnostic-grade 4D outputs. By enabling consistent, automated modeling of cardiac function, this technology addresses critical gaps in existing imaging tools, which are often limited to 2D or static 3D slices. The 4D model serves as a platform for future diagnostic tools, including chamber sub-segmentation, functional assessment, and training simulations. Ultimately, this work positions the AIM Lab’s 4D modeling approach as a transformative leap forward in cardiovascular imaging, diagnosis, and medical education.

William Sherman

This demonstration showcases an immersive visualization of a large-scale 3D microscopy volume dataset to a low-cost consumer HMD. The challenge of interactively rendering data on the order of 100's of gigabytes to an all-in-one HMD requires fast rendering, video compression and streaming, and an HMD interface to present the stream. To meet this challenge, several sub-systems are linked to provide the end-user experience. Systems include volume rendering on a high-end GPU, making use of the ANARI rendering standard. The ANARI standard allows applications to choose from a collection of available rendering "backends" -- for this effort, a path-tracing renderer capable of multi-channel volume rendering to an omnidirectional camera is used ("Visionaray"). The rendered image is then compressed and streamed using the FFmpeg library, which streams the video to a Janus WebRTC server. Janus provides a server-based system where clients can connect and request access to various media sources. The next requirement is for the low-cost HMD, such as a Quest headset from Meta, to access and present the streamed data. The nature of the rendering process and image transmission ultimately leads to delays in presenting the visuals to the user, which is a debilitating concern for immersive visualization. To mitigate this issue, the data is rendered with a virtual omnidirectional camera, which is then re-projected onto the inside of a sphere in the HMD. This method allows the user to quickly rotate their head without noticing any lag as a result of presenting the rendering. A browser-based WebXR application connects to the Janus server and presents the video stream to the user on the spherical surface. It also provides a simple user interface for the user to control the rendering. Unsurprisingly, there are trade-offs for generating the images on one system, compressing it streaming it -- the quality of the rendering is generally negatively impacted. The quality is reduced through the addition of compression artifacts to keep network speeds high, but furthermore, it is negatively affected by spreading the pixel resolution over an entire sphere, even when the user can only look at a portion of the sphere. The other limited factor introduced by spherical rendering is that while user head rotations are instantly handled (reducing motion-to-photon latency), user head translation movement will conversely increase the motion-to-photon response. Our current demonstration shows a single channel view of data collected though 3D microscopy. A user interface in the HMD will demonstrate some controls over the rendering parameter. This framework will serve for testing additional features that are important to the task of reviewing 3D microscopy data. In particular, the ability to annotate features within the data will provide pathologists the means to collect specific data that can be analyzed. We also will be adding a collaboration feature allowing users to directly work together in the immersive space. Beyond these additional features, we will be exploring ways to mitigate the latency and resolution issues, such as limiting the rendering and transmission to a portion of the sphere.

Colter Wehmeier

We demonstrate a methodological framework for humanistic inquiry through immersive computing that leverages strategic ambiguity to catalyze interpretation. While conventional heritage visualization approaches prioritize photorealistic accuracy, our methodology employs deliberate abstraction to scaffold participatory knowledge construction. Our application examines Cyprus's Nicosia International Airport (1967-1974)—a symbol of national modernity frozen in time by conflict. The interactive visualization functions not as a definitive reconstruction but as a contestable proposition, creating interpretive gaps that invite visitors to contribute lived memories. "Through recorded testimonials and media contributions, we create an archive of everyday experiences often overshadowed by conflict narratives, revealing new dimensions of modern architecture typically neglected in conventional historiography. Hybridizing digital humanities with learning science, we employ 'divergent collaboration' as our interpretive framework for 'meaningful play'—divergent interpretation followed by collaborative sense-making. Our autonomous year-long museum installation has engaged over 2,100 visitors, collecting 120 audio testimonials and hundreds of media artifacts. This demonstrates how our methodological approach situates virtual environments as productive tools in the constellation of participatory heritage practices that sustain collective memory.

Patrick Naughton, Hyoungju Lim, Kris Hauser

Teleoperation of dexterous robot hands can allow people to transfer advanced manipulation skills to a remote environment that may be difficult to reach or dangerous for them to work in. Current techniques for teleoperating these hands rely on manually specified rules to retarget the operator's hand motion to the robot. While these retargeters can achieve coarse manipulation tasks such as object grasping, fine details of the operator's intention are often lost, preventing them from achieving more complex in-hand manipulation. This project proposes to use human manipulation trajectories to learn a retargeting function that reproduces the operator's functional intent rather than their motion alone, allowing them to achieve much higher levels of dexterity.

Jeff Carpenter, Stuart Levy, Matthew Turk, Bradley Thompson, Donna Cox, Robert Patterson, AJ Christensen, Kalina Borkiwicz, Alex Betts, Gretchen Hall, Lorne Leonard

AVL is committed to communicating science and inspiring audiences to learn about scientific concepts through capturing the thrill of scientific discovery and the wonder of complex systems. Working in close collaboration with domain scientists, AVL creates high-resolution, cinematic, data-driven scientific visualizations from large, complex datasets, serving as a vital tool for scientific understanding. These visualizations transform intricate data into stunning, educational art.

Srikar Annamraju, Yuge Nie, Chad Peterson, Inki Kim, James Allison

Optimal packaging is a necessity in multiple domains such as automotive, healthcare, etc. The objective to minimize the packaging cost is typically realized through analytical formulation, involving weight and volume constraints, penalty for wrong placement etc. In this work, a VR based digital twin is being developed to obtain a real-time interface for the user. The platform allows operators to adjust the objects in a virtual environment, wherein the packaging scores are computed for each particular configuration. Common daily life environments of dining / arranging a room are created in Unity software, and with the integration of a VR headset, optimal packaging can be easily realized.

Aditi Tiwari, Klara Nahrstedt

Effective training and debriefing are critical in high-stakes, mission-critical environments such as disaster response, military simulations, and industrial safety, where precision and minimizing errors are paramount. The traditional post-training analysis relies on manually reviewing 2D videos, a time-consuming process that lacks comprehensive situational awareness. To address these limitations, we introduce ACT360, a system that leverages 360-degree videos and machine learning for automated action detection and structured debriefing. ACT360 integrates 360YOWO, an enhanced You Only Watch Once (YOWO) model with spatial attention and equirectangular-aware convolution (EAC) to mitigate panoramic video distortions. To enable deployment in resource-constrained environments, we apply quantization and model pruning, reducing the model size by 74% while maintaining robust accuracy (mAP drop of only 1.5%, from 0.865 to 0.850) and improving inference speed. We validate our approach on a publicly available dataset of 55 labeled 360-degree videos covering seven key operational actions, recorded across various real-world training sessions and environmental conditions. Additionally, ACT360 integrates 360AIE (Action Insight Explorer), a web-based interface for automatic action detection, retrieval, and textual summarization using large language models (LLMs), significantly enhancing post-incident analysis efficiency. ACT360 serves as a generalized framework for mission-critical debriefing, incorporating EAC, spatial attention, summarization, and model optimization. These innovations apply to any training environment requiring lightweight action detection and structured post-exercise analysis.

Adam Steel, Brenda Garcia, Kala Goyal, Anna Mynick, Caroline E. Robertson

How do the brain's visual and memory systems interface to enable memory-guided visual behaviors, like navigation? We investigated the brain mechanisms that support memory of our visuospatial environment using a combination of VR training and fMRI. These tools enable a direct, controlled test of contextual representation in the brain by causal manipulating the environment's content. Participants (N=17) studied 20 real-world scenes using head-mounted VR comprising three spatial context conditions: Image (45° field-of-view), Panorama (270° field-of-view), and Street (navigable environment of three contiguous photospheres). Using fMRI, we compared neural responses when participants recalled (Exp.1) or perceived (Exp.2) discrete fields-of-view from each place. In both experiments, we found a discrete set of brain areas uniquely processed the visual information out of view. Together, these results show that the PMAs are uniquely sensitive to the amount of spatial context associated with a real-world scene suggesting that they may be involved in providing spatial context to the SPAs to facilitate visually-guided behavior.

Tian Sun, Duo Wang, Kyrian Liang, Victor Lu, Jackson Song, Zyra Sheikh, Jen Whiting, Joshua Neela, Caroline Cao

In the post-COVID-19 era, burnout, mental health and overall wellbeing of healthcare workers have become more prevalent concerns. Empathy has been singled out as a key quality in successful crisis management. This project explores the innovative application of XR (Extended Reality, including virtual reality, augmented reality, and mixed reality) and AI (artificial intelligence) technologies to train empathy in first responders. First responders such as police officers, firefighters, and Emergency Medical Technicians (EMT) regularly encounter high-pressure, emotionally charged situations that demand not only technical expertise but also a high degree of emotional intelligence. In such environments, the ability to demonstrate empathy can be crucial in delivering patient-centered care and improving communication with individuals in distress. We aim to leverage immersive simulations to enhance empathy through a structured training framework. A literature review was conducted to define the construct of empathy and its properties, identifying existing models of empathy across various application domains. The five-factor empathy model by Gerdes et al that led to the development of the Empathy Assessment Index (EAI) was selected to frame the problem space for clinical empathy. With the help of the Police Training Institution, we build a VR demo for law enforcement.

N/A

Mirage is an initiative to gamify chemistry labs in virtual reality to boost student engagement and increase test scores

Mohammed Rashad

Recent studies have noted that relying on advanced AI tools can sometimes impede a deep understanding of coding fundamentals. To foster genuine learning, DuckSight revitalizes the classic rubber duck debugging approach by integrating mixed reality and conversational AI. In a brief demonstration video, the app showcases how a student resolves a C# for-loop error with the reflective guidance of a virtual duck. By prompting users to explain their code aloud, DuckSight helps break down complex problems and encourages self-discovery rather than providing immediate answers. Built using Unity, Meta’s Horizon OS SDK, and OpenAI's LLM API, DuckSight leverages state-of-the-art interaction, voice, and scene capabilities to offer an immersive, distraction-free debugging experience on devices like the Quest 3 / 3S. Despite voice transcription and spatial positioning challenges, the system employs interactive checks and manual controls to maintain reliable functionality. Future enhancements include incorporating a depth API for heightened realism and refining the duck’s placement within real-world environments. This work demonstrates a promising educational tool that bridges immersive technology with metacognitive problem-solving, potentially transforming software education for students.

Raluca Ilie, Nancy Kang

The three-dimensional aspects of electromagnetism concepts do not easily translate to two-dimensional platforms, making it challenging for students to develop an intuition for these phenomena in traditional learning contexts. Virtual Reality (VR), a simulated three-dimensional environment, offers an immersive, realistic, and interactive 3D experience that enables students to visualize complex concepts and actively manipulate these abstractions. This paper discusses the creation, implementation, and evaluation of a self-guided tool designed to teach engineering students the fundamental principles of electromagnetism theory: Maxwell’s equations. VR-based learning is self-directed and flexible, allowing students to adjust the difficulty level and control the pace of their learning, which enables them to engage with increasingly complex topics. This personalized approach has the potential to foster a deeper understanding and sustained interest in the subject. In-game conceptual quizzes assess students’ comprehension to evaluate the tool’s effectiveness. Moreover, headset-tracking technology captures user eye-tracking data, providing valuable insights to inform design decisions for the experience.

Kevin K. D. Tan, Tianyu Wu, Zane R. Thornburg, Zaida Luthey-Schulten, Stephen A. Boppart

There is a substantial opportunity to utilize the vast amount of image data generated in the biosciences to educate and inspire the next generation of scientists. As a first step towards this goal, we built an easy-to-use and general-purpose Python based framework for translating segmented images into the videogame Minecraft. Using data from label-free microscopy, we showed that these Minecraft models can be used to teach how cells and tissues change in disease by comparing normal and cancerous epithelial breast cells. A Minecraft datapack was developed to convert a timelapse of live human neutrophils into Minecraft blocks in real time, demonstrating phorbol myristate acetate activation dynamics. Because these models are entirely within Minecraft, they are accessible without technical expertise and are fully compatible with virtual reality for educational, and engaging visualization of empirical cell biology.

Peter Grzywacz

Much VR research has focused on how creating immersive environments encourages learning. However, this research focuses on how VR can be used to hide information in order to encourage focused conversation and language acquisition. Viewers will be able to see the programming, design, set theory and user testing required to build an interactive multiplayer information gap activity for English and Japanese learners. https://youtu.be/-rS3U9_BOMo?feature=shared

Chris Palaguachi, Nima Suri, Helen Ryding, Navomi Byju, Ellen Min, Jacob Sobel, Robin Rajarathinam, Rodrigo Hidalgo, Jina Kang

How can immersive technologies support engaging, scalable learning experiences in the classroom? In this demo, we present HoloOrbits, an interactive educational simulation that teaches orbital mechanics through immersive visualization and collaborative problem-solving. Developed in both Extended Reality (XR) using Microsoft Hololens 2 and a browser-based WebGL version, HoloOrbits lets students explore a fictional exoplanetary system and calculate orbital eccentricity through hands-on interaction. We showcase the design process across both platforms, highlighting key decisions in spatial interface design, collaboration modes, and feedback-driven iteration. The XR version offers a deeply immersive, embodied experience for student pairs, while the WebGL version broadens access by supporting small-group learning on standard classroom devices. Through two classroom experiments, we collected pre/post-tests, user interviews, and observational data to compare engagement, collaboration, and learning outcomes. Findings reveal unique strengths and trade-offs between the two platforms in terms of immersion, accessibility, and classroom integration. We invite conference attendees to explore both versions of HoloOrbits at our demo station and share feedback on usability, educational value, and potential improvements. We're especially interested in insights around designing immersive experiences that are both impactful and practical for real-world classrooms.

Dan Cermak, John Toenjes, Robbie Sieczkowski, Sepehr Vaez Afshar, the stu/dio Leads, and the Master Dancer student development team

The stu/dio: The stu/dio represents a student-driven effort to manage game development projects, especially in support of education and research, on the University of Illinois campus and beyond. Master Dancer: In Master Dancer, you meet the famous dancer Loïe Fuller, the first to integrate technology into modern dance. As a player, you engage in action games to learn her movement principles and, guided by Virtual Loïe, explore her world. You will use modern VR technology to create your own unique dance, becoming a Master Dancer yourself.

Dan Cermak, Laura Shackelford, Robbie Sieczkowski, Sepehr Vaez Afshar, the stu/dio Leads, the VRchaeology student development team

The stu/dio: The stu/dio represents a student-driven effort to manage game development projects, especially in support of education and research, on the University of Illinois campus and beyond. VRchaeology: Hands-on excavation experience is a critical component of archaeological education, yet financial and logistical barriers often limit student access to field opportunities. To bridge this gap, this project introduces a fully immersive Virtual Reality (VR) archaeology simulation, developed in Unreal Engine and tested on Meta Quest, designed to replicate excavation fieldwork within a scientifically structured and interactive virtual environment.

Dejan Trencevski, Vishal Sachdev, and Aric Rindfleisch

In this poster, we present an innovative way of how we explored the affordances of immersive technologies, specifically augmented reality (AR), to facilitate students’ learning in classrooms by enabling them to visualize their 3D models as digital 3D objects in immersive environments before they print their 3D prototypes as physical products on 3D printers. In this research study, we introduced immersive technology in courses and workshops that require students to conceptualize, design, prototype, manufacture (with 3D printers), and market new products by using 3D computer-aided design (CAD) software and 3D printing and scanning hardware. Although CAD allows students to visualize and manipulate digitally manufactured 3D objects on a 2D computer monitor surface, it does not allow them to visualize and manipulate those 3D objects in 3D space. However, AR does afford visualization and manipulation of digitally manufactured 3D objects in 3D space which is why our research investigates the effects of augmenting 3D modeling with AR facilitated product development. Our research analysis will inform educators and business practitioners about the potential pedagogical application of AR for classroom instruction and about the prototyping capabilities of AR during the product design process.

Vihaan Khare, Helen Ryding, Trustan Price

Neonatal procedures are particularly delicate and error-prone in nature, especially compared to adult and even pediatric procedures. Invasive neonatal procedures do not occur at a rate sufficient enough to allow for adequate skill retention and provide proper training to physicians. Augmented reality (AR) / Mixed reality (MR) platforms specifically using the Microsoft HoloLens device are effective when applied in clinical settings due to their ability to incorporate tangible environmental factors such as mannequins and surgical instruments. Skill degradation can be minimized through structured practice targeting both cognitive and motor skills.We have developed a prototype for the needle thoracentesis (NT) procedure in a mixed-reality environment. The guided simulator allows learners to go at a desired pace and allows for flexibility to learn visually, audibly, or both.

Amber Dewey Schultz, Saad Shehab, Sarita Adve, Eric Shaffer

The Interdisciplinary Certificate in Immersive Experience Design offers undergraduate students a unique opportunity to explore and shape the future of immersive experiences. A collaboration between the Siebel Center for Design and IMMERSE: Center for Immersive Computing, this program empowers students to create emotionally engaging, human-centered immersive experiences across a variety of mediums, including augmented reality, mixed reality, and physical environments. Launched in Fall 2023, the certificate provides students with a foundational understanding of Design Thinking and Human-Centered Design (HCD), teaching them to empathetically connect with stakeholders, define challenges, and develop innovative solutions. Through courses like "Introduction to Design Thinking" and "Innovation Studio," students learn to apply inclusive ideation, ethical prototyping, and meaningful feedback gathering in real-world contexts. The certificate culminates in a collaborative project, allowing students to work with campus units and organizations to tackle real-world problems using immersive experiences. By the end of the program, students will possess the skills to design impactful, sustainable, and evolving immersive experiences that resonate deeply with audiences, preparing them to innovate in the rapidly growing field of immersive design.

Nanzeeba Tabassum (CEE), Yangze Liu (CS), Jenny Amos (BIOE), Eric Shaffer(CS) and Nishant Garg(CEE)

Advancements in virtual reality (VR) have enabled greater immersion, engagement, and retention among students. A well-designed educational game not only sustains engagement but also motivates learning throughout gameplay. In engineering, a robust understanding of materials is essential for making informed decisions when selecting components for innovative designs. However, acquiring such in-depth knowledge often requires complex 3D visualizations at the atomic level, which can be intimidating when relying solely on conventional 2D views in undergraduate classes. By merging VR, game design, and core engineering materials concepts, educational outcomes can be significantly enhanced. We developed “Crystal Vision 2.0,” a video game for undergraduate engineering in which 82 students participated, resulting in improved learning outcomes and increased interest in the subject. Building on this success, we are now developing a VR version to explore the potential of immersive computing for learning fundamentals of crystallography, aiming to further enhance interactive involvement and understanding in materials education.

Roberts Fulss, Seung Yun Song, Patrick Moore, Riku Kanzaki, Linh Dao, and Elizabeth T. Hsiao-Wecksler

Hands-free control of robots is an underexplored solution addressing limitations of traditional control methods (e.g., handheld controllers) while advancing human-robot interaction (HRI). We have developed the Torso-dynamics Estimation System (TES) to estimate the leaning and twisting torso motions of a seated user by capturing two translational and one rotational signals from the torso. In this study, we illustrate how the TES can be used for lean-to-steer navigation of a simulated mobile robot in a virtual environment. The TES signals are mapped to the robot’s command velocities using both linear and non-linear mapping strategies, which also allow for adjustment of control gains, enabling personalized hands-free control of the robot. To determine the effectiveness of hands-free control, we compare its outcomes with traditional handheld controllers (joystick), analyzing differences in navigation performance. This study also examined multiple 3-signal sensor embodiments: inertial measurement unit (IMU) only, IMU (yaw only) + instrumented seat, instrumented seat + instrumented backrest (yaw only), and instrumented backrest only. The instrumented seat used four compact loadcells (FRC4142_0, Phidgets, Canada) to measure resultant reactive moments in the sagittal and frontal planes, estimating user translational positions for control. The instrumented backrest can estimate one to three torso angles. To evaluate the feasibility of this approach, a virtual reality (VR)-based experimental setup was developed. Participants navigated a structured test course while navigational performance metrics (e.g., collision count, completion time, telemetry data) were recorded. The course design adhered to the US Building Code to assess HRI performance over a broad range of realistic scenarios consisting of hallways, restrooms, and static and dynamic obstacles. Future work includes conducting performance comparisons between manual wheelchair users and able-bodied participants to provide insights into the potential of these possible torso-driven control embodiments. This study contributes to the development of human-robot collaboration strategies, which can be used to control the movement of physical or virtual devices or avatars solely using the user’s upper body motion.

Beitong Tian, Lingzhi Zhao, Bo Chen, Haozhen Zheng, Jingcheng Yang, Mingyuan Wu, Deepak Vasisht, Klara Nahrstedt

Underwater activities like scuba diving enable millions annually to explore marine environments for recreation and scientific research. Maintaining situation awareness is essential for diver safety and effective communication. However, traditional underwater communication systems are often bulky and expensive, limiting their accessibility to divers of all levels. While recent systems leverage lightweight smartphones and support text messaging, the messages are predefined and thus restrict informative communication. In this paper, we present AquaVLM, a tap-and-send underwater communication system that automatically generates context-aware messages and transmits them using ubiquitous smartphones. Our system features a mobile vision-language model (VLM) fine-tuned on an auto-generated underwater conversation dataset and employs a hierarchical message generation pipeline. We co-design the VLM and transmission, incorporating error-resilient fine-tuning to improve the system's robustness to transmission errors. We develop a VR simulator to enable users to experience AquaVLM in a realistic underwater environment and create a fully functional prototype on the iOS platform for real-world experiments. Both subjective and objective evaluations validate the effectiveness of AquaVLM and highlight its potential for personal underwater communication as well as broader mobile VLM applications.

Chunliang Yang & Stefan Uddenberg

The visual system not only extracts simple features (e.g., color, shape) but also high-level properties like intentionality. In the wolfpack effect (Gao et al., 2010), observers falsely perceive intentional chasing behavior from randomly moving darts when those darts face them. Yet it remains uncertain whether this phenomenon extends beyond conventional 2D displays of simple shapes into immersive, realistic 3D environments experienced from a first-person perspective and accompanied with richer social cues (such as a person’s demographics, posture, or emotional expression). The current study investigates whether and how the wolfpack effect occurs within the context of virtual reality (VR) scenarios featuring varied levels of human realism. Participants navigate VR environments and are asked to accurately identify a designated "wolf" hidden among randomly moving distractors before it catches them. Across three experiments, character realism is systematically varied (ghost-like, humanoid robots, and hyper-realistic humans) to explore how realism moderates wolfpack effect. Primary outcome measures include identification accuracy under wolfpack versus random orientation conditions. Other implicit measures including walking velocity, speed variability, and posture dynamics will be analyzed with computer vision algorithms to deepen our understanding of the wolfpack effect's downstream impacts. Further analyses will examine how realism and associated social cues (e.g., race, gender) modulate perceptions of intentionality, thereby distinguishing between purely perceptual and broader social cognitive mechanisms. This study will reveal the mental defaults (e.g. facingness, social cues) used by the visual system to infer intentionality in an ecologic environment.

Chunliang Yang & Stefan Uddenberg

The visual system not only extracts simple features (e.g., color, shape) but also high-level properties like intentionality. In the Wolfpack Effect , darts that randomly move about the screen give the impression of chasing an object when they are oriented towards it (Gao et al., 2010). Yet it remains uncertain whether this phenomenon extends beyond conventional 2D displays of simple shapes into immersive, realistic 3D environments experienced from a first-person perspective, much less when accompanied by richer social cues (such as a [virtual] person’s demographics, posture, or emotional expression). The current study will investigate whether (and to what degree) the Wolfpack Effect occurs within the context of virtual reality (VR) scenarios featuring varied levels of social realism and relevance. Participants will navigate VR environments and need to identify a "wolf" (a pursuing agent) hidden among randomly moving distractor agents before it catches them. Across three experiments, character realism will be systematically varied (ghost-like, humanoid robots, and hyper-realistic humans) to explore how social realism augments the Wolfpack Effect. We hypothesize that immersive presentation and socially meaningful agents will lead to greater impressions of agent animacy and stronger behavioral avoidance of the agents on the part of participants (e.g., as measured via distance to the agents as they navigate the environment). Taken together, this work will elucidate the role of ecologically valid social cues in the perception of animacy and intentionality, with implications for social perception and the automaticity of animacy attribution.

Przemyslaw Bosak, Michel Bellini, Bob Pahre, Toni Gist, Julie Munoz-Najar, Kevin Poh Hiong, Vidya Haran, Chiara Vincenzi, and Rana Khoury

Virtual Reality (VR) has emerged as a powerful tool for fostering empathy through immersive experiences. At the University of Illinois, various courses have integrated VR to enhance students' understanding and emotional connection to diverse perspectives. The CITL Innovation Studio hosts numerous course visits each semester where students engage in VR experiences designed to evoke empathy. These visits include immersive: refugee experiences, fashion design projects, interactions around diversity, and storytelling. One notable project is the "Clouds Over Sidra" VR experience, which allows students to step into the life of a 12-year-old Syrian refugee named Sidra. This immersive experience takes students through Sidra's daily life in the Za'atari refugee camp in Jordan, providing a powerful perspective on the challenges faced by refugees. Another impactful VR experience is "Traveling While Black," which explores the history of restricted movement for Black Americans and the ongoing struggle for freedom and equality. Research indicates that empathy can be enhanced through experiential learning—learning by doing rather than merely observing or reading. VR technology takes this concept a step further by placing users in immersive environments where they can actively engage with scenarios that require empathetic responses. These VR experiences not only engage students but also cater to various learning styles, making education accessible and enjoyable for everyone. As the digital landscape continues to evolve, integrating immersive computing, including VR, ensures that students approach the world with an empathetic viewpoint.

Arnav Shah, Kaiyuan Wang, Elahe Soltanaghai, Mohamad Alipour

Hand gesture recognition in virtual reality (VR) enables users to interact with digital environments without controllers, allowing for intuitive mid-air object manipulation. However, the absence of tangible feedback presents a challenge in conveying haptic sensations. Pseudo-haptic techniques, which leverage visual feedback to simulate touch perception, offer a potential solution. Prior studies have induced a sense of stiffness in virtual objects using deformation cues, but none have explored the role of Control-Display (C/D) ratio manipulation in enhancing stiffness perception. Through a series of user studies (N = 14), we show that (1) a range of C/D ratio manipulations remain imperceptible to users, (2) users associate C/D ratio changes with corresponding changes in stiffness, and (3) users can reliably associate the stiffnesses of physical spring with corresponding C/D ratios. We contribute a linear model mapping physical spring stiffness to C/D ratios, offering designers a lightweight, visually-driven method for simulating resistance in mid-air VR interactions. This approach opens new educational, training, and immersive simulation design opportunities.

Smital Lunawat, Lingzhi Zhao

Cybersickness, a form of motion sickness experienced in virtual reality, poses a significant challenge to user comfort and adoption. Symptoms such as nausea, dizziness, and disorientation can limit engagement, particularly during gameplay with artificial locomotion. While technical factors like frame rate and user-related factors such as posture have been implicated in previous work, their combined effects remain underexplored in practical settings. This study investigates how display frame rate (60 FPS vs. 90 FPS) and user posture (sitting vs. standing) influence cybersickness in a locomotion-intensive VR experience. Nineteen participants engaged with a first-person shooter game using a Pico 3 headset, completing the Simulator Sickness Questionnaire after each of four randomized conditions. Results indicate that sitting posture consistently reduced SSQ scores compared to standing, and 60 FPS yielded slightly lower symptoms than 90 FPS. Participant traits such as motion sickness susceptibility and VR experience also played a notable role. These findings suggest that session comfort can be improved through posture recommendations and tailored VR settings, particularly for novice users.

Maryam Jahadakbar, Mike Yao, Rachel Adler

This poster presents an integrative exploration of embodied learning and interactive gaming as tools for enhancing educational, therapeutic, and recreational experiences for children with Autism Spectrum Disorder (ASD). We introduce "The Routine," a virtual learning environment (VLE) that integrates embodied learning theory, AI-driven personalization, and sensor-based feedback to adapt in real time to children's cognitive, social, and behavioral needs. The game combines physical gestures with real-life scenarios to promote independence and daily life skills. To extend this work, we are conducting qualitative research through semi-structured interviews with parents and therapists across school, clinical, and home environments. These interviews explore their experiences supporting autistic children using educational, recreational, and therapeutic games. Thematic analysis of early data reveals key insights into user needs—such as flexibility, sensory adaptability, and cooperative engagement—and identifies appropriate, scalable technologies for community implementation, including gesture-recognition tools and accessible mixed-reality platforms like Apple Vision Pro. By combining interactive system design with qualitative insights, this research offers practical guidelines for creating responsive, inclusive, and context-aware digital interventions. It advances a framework for tailoring embodied virtual experiences to support the developmental goals of autistic children in diverse real-world settings.

Weiyu Ding, Yanyun (Mia) Wang, Mike Yao

This study investigates how augmented reality (AR) can enhance health message framing effects—specifically gain vs. loss frames—on vaping risk perceptions. Using a 2 (gain vs. loss frame) × 2 (self-focused vs. non-self-focused AR) experimental design, we exposed 120 e-cigarette users (ages 18–30) from a large U.S. university to AR video filters visualizing the effects of vaping on lung health. Participants completed a pre-survey and a lab-based AR experience, followed by a post-survey measuring risk perception, psychological distance from lung disease, psychological reactance, and related variables. The four conditions included: (1) loss-frame + self-focused AR, (2) loss-frame + non-self-focused AR, (3) gain-frame + self-focused AR, and (4) gain-frame + non-self-focused AR. Preliminary results are expected to show how immersive AR can modulate the impact of message framing on perceived health risks and behaviors. Findings highlight AR's potential as an innovative tool in persuasive health communication.

Yuming Zhang; Houtan Jebelli

Work-related musculoskeletal disorders (WMSDs) remain a major concern in the construction industry due to physically demanding tasks such as heavy lifting and repetitive motions. While exoskeletons offer a promising solution by reducing physical strain and enhancing worker performance, improper use can lead to inefficiencies or even new safety risks. Effective training is therefore critical. However, traditional training approaches often lack realism and interactivity, limiting their effectiveness in preparing workers for real-world challenges. To bridge this gap, we developed VR-EXOPRO, a human-centric Virtual Reality (VR)-based training platform that promotes safe and effective exoskeleton use in construction. VR-EXOPRO immerses users in realistic construction environments, allowing them to practice key tasks with real-time motion tracking and interactive feedback. The system includes adaptive training modules tailored to individual performance, demonstration animations, and detailed feedback to reinforce ergonomic best practices. A user study was conducted to evaluate the platform’s impact, showing notable improvements in participants’ technical understanding, practical skills, and overall user experience. These results highlight VR-EXOPRO's potential to revolutionize workforce training by enhancing engagement, learning outcomes, and safety compliance in exoskeleton-assisted construction work.

Eliot Bethke, Matthew T. Bramlet MD, Bradley P. Sutton PhD, James L. Evans, Jennifer R. Amos PhD

Pre-surgical planning requires domain-specific knowledge, experience and intuition. Virtual Reality (VR) has attracted attention for its potential to visualize complex 3D anatomy for medical applications. However, many of these prior studies evaluating applications in cardiology rely heavily on subjective self-reported metrics, evaluate non-clinical cases, or evaluate the technology with medical students or trainees. Self-reported data from surveys and metrics based on confidence and task completion alone may not yield sufficiently detailed understanding of the complex decision making and cognitive load experienced by surgeons during VR-based pre-surgical planning. We therefor chose to focus on evaluating how the surgical practitioner’s thought processes in VR affects their mental model of real patient cases. This study aimed to assess the impact of VR on physicians’ mental models by analyzing their perceptions of real patient cases. Our goal was to describe the strategies surgeons employ when navigating complex surgical cases in a VR software (Enduvo). To accomplish this, we deployed an open-ended, think-aloud protocol offering deeper perspectives into physicians' thought processes within the VR environment. Participation was entirely voluntary and physician-initiated, proceeding according to OSF IRB protocol #2021766-5 . We performed qualitative analysis of video recordings from pre-VR, during VR, and post-VR sessions, in addition to collecting quantitative measures from a modified NASA task load index (NASA-TLX) questionnaire. Additional analysis was conducted on the coding results to identify thematic sequences in VR which influenced clinical decision making when reviewing patient anatomy. We found low perceived demand and high perceived performance accomplishment in VR, consistent with other VR studies. Combined with the qualitative analysis, our results indicate that physicians had an easier time understanding the patient anatomy in VR than by reviewing traditional medical imaging. We also found a significant increase in understanding of the patient anatomy from before VR to after (p=0.012), with low cognitive demands reported on the NASA-TLX. In conclusion, we have found that VR improves surgeon understanding of patient anatomy before surgery and that unguided 3D exploration helps build and refine mental models of patient anatomy.

Bradly Alicea

I will give an overview of the PhASiC (Physical Intelligence, Adaptive Systems, Information, and Computation) project, a long-term and multidisciplinary project spanning XR, intelligence augmentation, and dynamical systems. PhASiC provides a framework for a methodological and theoretical synthesis around XR's various application domains. The PhASiC framework allows us to take a broader perspective on the relevance of XR to various science and engineering domains. The project website is here: https://cybernetics-physical-intelligence.weebly.com/

Amber Dewey Schultz

A key driver of engagement and transformation within immersive experiences is presence and focus. However, when participants are overloaded with multiple, simultaneous interactions, cognitive overload occurs, fragmenting attention and diminishing the transformative effect.To explore the relationship between interaction and engagement, I developed the Audience Interaction Model for Immersive Playable Theatre. Building on Salen and Zimmerman's Multivalent Model of Interactivity, my model categorizes interactions into four types: functional (engagement with systems, space, props, costumes), cognitive (emotional engagement, narrative synthesis, meaning-making), explicit (puzzle solving, decision making, exploration), and social (verbal and non-verbal communication). This framework was applied to I Wish: A Theatrical Puzzle Room, produced by the Department of Theatre in collaboration with Game Studies & Design and CU Adventures in Time and Space, where careful sequencing and pacing of interactions reduced cognitive overload and maintained audience engagement, comprehension, and emotional transformation. Qualitative data showed that this sequencing helped participants stay focused, understand the narrative, and achieve flow state, with many reporting emotional transformation in the form of connections to characters and narrative. These findings suggest that structuring interactions in this way enhances both engagement and learning, offering valuable insights for designing immersive experiences across various contexts.

Katherine J. Mimnaugh, Evan G. Center, Markku Suomalainen, Israel Becerra, Eliezer Lozano, Rafael Murrieta-Cid, Timo Ojala, Steven M. LaValle, and Kara D. Federmeier

VR sickness is a significant barrier to comfortable and enjoyable immersive experiences. In addition to causing discomfort, VR sickness may impact how people engage with virtual content and perform in simulated environments, though there is still much that we do not know about these impacts. Therefore, the goal of this study was to better understand how VR sickness affects attention and performance. In the study, brain activity was measured using electroencephalography, and the P3b even-related potential was used to track changes in attention while VR sickness was induced. The results showed that attention decreased as VR sickness increased, and that task performance was negatively impacted by VR sickness as well.

Xinhui Hu, Michael Twidale

Recent research on empathic VR has been marked by notable inconsistencies, with ongoing dissonance in the literature regarding the outcomes and design implications of VR-based empathy experiences. Researchers remain divided on several key questions: What types of empathy can VR elicit? What design dimensions are essential for fostering empathic engagement? Which design choices most effectively enhance empathy? And is VR inherently more effective than other media in eliciting empathy? By identifying divergences in findings and interpretations, our analysis highlights that these inconsistencies stem from a complex interplay of factors. Specifically, they arise from varying conceptualizations of empathy, differing interpretations of VR’s capabilities and limitations, and methodological discrepancies shaped by differences in study scope, objectives, and evaluation frameworks. Our research aims to clarify these inconsistencies, providing a more precise understanding of the factors influencing empathic VR research outcomes.

Bradly Alicea

The characterization of immersion as humans and machine symbiosis requires considering how symbiosis can be characterized as a dynamical system that enables unique opportunities for adaptation. This includes forms of adaptive and maladaptive accommodation, as well as enhancements in performance (learning). Biological systems are typically supervised by social (tutoring) and externalized (symbolic) cues. For immersive environments, however, there are unique aspects of procedural learning that may be overcome with natural supervised learning. The presence of rare or unusual physical relationships or unique world geometry are particularly difficult to learn from instruction. Furthermore, the ability to orient oneself and develop embodied interactions in unusual, simulated physics is something that is not easily taught or acquired without assistance. What is required is a supervisory process that regulates the mapping between sensory inputs and behavioral outputs, with the goal of maintaining isomorphy between the two. In cases where the nervous system cannot adequately serve to regulate this mapping, naturally supervised learning can serve to fill in the gaps of the action-perception loop. We can enhance learning by adding nonlinear or superadditive components to the learning curve. As a dynamical system, cognition in immersive settings must often do this on the fly, being embedded in the interaction itself. Specifically, immersion can trigger latent supervisory processes. Examples from dynamic motor learning and multisensory perception in virtual environments will be used. In the former, two potential mechanisms are switching and resonance. The latter can be induced by the strategic induction of sensory signals, with similarities to resonance in motor learning. These examples are interpreted in terms of insights from Ecological Psychology and Psychophysics, which provide avenues to a broader theory of non-Euclidean immersion. Overall, naturally supervised learning allows us to identify properties of symbiotic systems dynamics and offers a potentially controllable technique for encouraging immersive interactions.

Lingzhi Zhao, Yongqiang Gui, Yanyan Suo, Sandesh Dhawaskar Sathyanarayana, Ruixiao Zhang, Klara Nahrstedt, Shu Shi

In this poster, we present Trinity, a novel and practical solution that enables untethered game streaming from cloud servers to virtual reality (VR) headsets. Achieving the stringent motion-to-photon (MTP) latency requirement of <25ms is a critical challenge for VR game streaming. Existing approaches, such as motion prediction and pre-fetching additional frames, significantly increase server computational overhead and network bandwidth demands. Trinity takes a different approach based on an intuition that player's game interactions exhibit heterogeneous latency requirement. Specifically, we classify user interactions into three distinct motion categories: head motion, hand motion, and body motion. Each motion type is handled based on its specific latency sensitivity, leveraging novel latency adaptation and rendering strategies to meet the corresponding requirements. We evaluate Trinity with a highly latency-sensitive VR first-person shooting game streamed from a remote server to a VR headset. Subjective evaluations demonstrate that Trinity delivers a user experience comparable to local streaming while remaining within the cloud server and network capabilities.

Pasquale Bottalico, Carly Wingfield, Charles Nudelman, Joshua Glasner, Yvonne Gonzalez Redman

Background: In the realm of classical singing, performances of identical repertoires in diverse acoustic and visual settings display significant adaptations. These adjustments in the singer's delivery are influenced by a myriad of factors, encompassing the artist's perception of the acoustic surroundings, the visual aesthetics of the performance venue, and the measured acoustic properties of the space. Objectives: Voice production behaviors were evaluated to explore the effects of room acoustics on five voice parameters: vibrato rate, vibrato extent, vibrato jitter (Jvib), vibrato shimmer, and quality ratio (QR), an estimation of the singer's formant power. Methods: The subjects were ten classically-trained professional singers (five males and five females). Subjects sang the aria da camera “Caro mio ben” by Giordani in their preferred key in three different performance venues with different acoustics and dimensions, with and without a virtual reality headset that simulated the same room. Results: The study revealed significant adaptability in voice production behaviors among ten classically- trained professional singers, both male and female, as they performed the aria da camera "Caro mio ben" by Giordani in three different performance venues. Consistency in the performance was found between the condition in the real room and the condition with VR simulating the same room. Notably, vibrato rate, extent, jitter (Jvib), shimmer, and quality ratio (QR), an estimate of the singer's formant power, remain consistent due to the successful immersion provided by the virtual reality technology. Conclusions: These results emphasize the complex interplay between room acoustics, visual perception, and vocal parameters in influencing the delivery of classical singing performances, shedding light on the multifaceted nature of artistic adaptation.

Charles Nudelman, Pasquale Bottalico, Bella Lopez, Kaliyah House-Henry

Introduction: Student teachers face a high risk of vocal problems. Speech therapy techniques improve vocal efficiency, but ensuring these benefits carry over to real-world communication is challenging. Virtual reality (VR) offers a promising solution by simulating everyday speaking environments to enhance the transfer of improved vocal behaviors. Objective: This study evaluates the feasibility of delivering speech therapy in a VR environment. Specifically, it examines how clinician-mediated feedback in VR impacts voice outcomes, such as vocal intensity and pitch. Methods: Ten student teachers taught a sample lesson within a VR classroom and received real- time clinician-mediated feedback from a speech pathologist within the VR. Their vocal intensity was monitored and if it exceeded the 85th percentile of the participant’s baseline, the clinician's avatar raised its hand as visual feedback. Results: Overall, the results showed that vocal intensity and vocal pitch decreased with clinician feedback in the VR simulation versus instances without feedback. Conclusion: To summarize, from our findings, the clinician feedback during the VR therapy minimized the risk of voice problems by decreasing vocal intensity and pitch. We conclude that VR could be a potential therapy tool to elicit safer speech behaviors in student teachers.

Emre Eraslan, Qiyuan Cheng, George Mois, Wendy A. Rogers, Avinash Gupta

Background: Extended reality (XR)—including virtual, augmented, and mixed reality—offers diverse benefits for different age groups. As XR gains traction in research, its potential to enhance the eight interconnected Dimensions of Wellness (DoWs)—emotional, environmental, intellectual, financial, occupational, physical, social, and spiritual—is increasingly recognized. However, it remains unclear which XR technologies are best suited for each DoW or how these applications vary by age group. Objective: This systematic mapping review explores XR applications that support DoWs for young, middle-aged, and older adults, offering a comparative analysis across age groups and XR types. Methods: Following the PRISMA-ScR framework, 145 studies from 2016–2024 were reviewed across PubMed, Scopus, and Web of Science. Studies were categorized by XR type, age group, study setting, and targeted DoW. Results: Fully immersive VR (FIVR) is the most commonly used XR type, especially for emotional, physical, intellectual, and social wellness. Young adult studies also emphasize occupational wellness. In contrast, environmental, financial, and spiritual DoWs are rarely addressed. Middle-aged and older adult studies focus more on rehabilitation and therapy using NIVR, AR, and MR. Significance: This review identifies research gaps and calls for broader, interdisciplinary XR applications to support neglected DoWs and promote holistic wellness across the lifespan.

Emre Eraslan, Veronica Falcon, Qiyuan Cheng, Jacob Stolker, Sai Meenakshi Hariharan, Pallabi Bhowmick, Tracy Mitzner, Wendy A. Rogers, Avinash Gupta

Virtual reality (VR) applications hold growing potential for supporting the cognitive, emotional, and social well-being of older adults. As aging populations face challenges related to isolation, cognitive decline, and limited physical mobility, well-designed VR experiences can offer stimulating, engaging, and socially connected environments. However, most commercial VR platforms are not tailored to the sensory, motor, and cognitive needs of older users, limiting accessibility and usability. We present two custom-developed VR games designed specifically for older adults, grounded in human factors and ergonomics principles: GenVRse and Serenity Park. GenVRse features familiar mini-games such as checkers and cards within a calm, cozy virtual setting to promote comfort and reduce overstimulation. The game emphasizes intuitive interaction through natural hand movements, a menu-free interface, and simplified one-button controls. Accessibility is enhanced through seated gameplay, minimal navigation, and high-contrast visuals. Serenity Park invites users to a peaceful park environment where they can engage in a variety of relaxing activities, including leisurely boat rides, scenic strolls, and playful golf cart experiences. The environment is enriched with natural elements like fish, birds, and trees to enhance presence and emotional well-being. The design of both games was guided by five core human factors areas: (i) purpose and target users, (ii) design and interaction, (iii) cognitive and physical demands, (iv) social experience, and (v) progressive onboarding. Social features such as customizable avatars and seamless voice chat encourage real-time peer engagement and address digital loneliness. Both applications will be tested with older adults to assess usability, comfort, and affective engagement, offering a model for inclusive, age-appropriate VR design.

Bhavin Patel, Anthony Foli, Anushree Udhayakumar, Maanas Agrawal, Adam Cross, and Inki Kim

Virtual reality (VR) applications hold growing potential for supporting the cognitive, emotional, and social well-being of older adults. As aging populations face challenges related to isolation, cognitive decline, and limited physical mobility, well-designed VR experiences can offer stimulating, engaging, and socially connected environments. However, most commercial VR platforms are not tailored to the sensory, motor, and cognitive needs of older users, limiting accessibility and usability. We present two custom-developed VR games designed specifically for older adults, grounded in human factors and ergonomics principles: GenVRse and Serenity Park. GenVRse features familiar mini-games such as checkers and cards within a calm, cozy virtual setting to promote comfort and reduce overstimulation. The game emphasizes intuitive interaction through natural hand movements, a menu-free interface, and simplified one-button controls. Accessibility is enhanced through seated gameplay, minimal navigation, and high-contrast visuals. Serenity Park invites users to a peaceful park environment where they can engage in a variety of relaxing activities, including leisurely boat rides, scenic strolls, and playful golf cart experiences. The environment is enriched with natural elements like fish, birds, and trees to enhance presence and emotional well-being. The design of both games was guided by five core human factors areas: (i) purpose and target users, (ii) design and interaction, (iii) cognitive and physical demands, (iv) social experience, and (v) progressive onboarding. Social features such as customizable avatars and seamless voice chat encourage real-time peer engagement and address digital loneliness. Both applications will be tested with older adults to assess usability, comfort, and affective engagement, offering a model for inclusive, age-appropriate VR design.

Doug Friedel, Steven Gao, Qinjun Jiang, Jeffrey Liu, Yihan Pang, William Sentosa, Rahul Singh, Boyuan Tian, Brighten Godfrey, Sarita Adve

Augmented, Virtual, and Mixed Reality, collectively known as Extended Reality (XR), are transformative technologies impacting diverse areas from education and science to social interaction and entertainment. However, modern XR systems face significant challenges in performance, power efficiency, and quality, hindering the realization of their full potential. To democratize architecture and systems research in XR, we developed ILLIXR (Illinois Extended Reality Testbed), a fully open-source, end-to-end XR system. ILLIXR includes state-of-the-art XR components, integrated through a modular, multithreaded runtime framework. ILLIXR is compliant with OpenXR, allowing it to support applications developed with popular game engines such as Godot and Unreal. ILLIXR also provides extensive metrics for both component- and system-level performance and quality of experience, facilitating research in all aspects pertaining to XR. On top of that, ILLIXR now features support for remote-assisted XR (e.g., offloading rendering and head tracking), new interactive user interfaces (e.g., hand tracking), 3D scene understanding, and several cutting-edge neural algorithms — all of which we'll be highlighting in our poster and bringing to life through live demos!

Aidan Wefel Joseph Torrellas Arnav Shah Samuel Hurh Joao Lucas Cadorniga Prof. Luciano P Soares Dr. Eric Shaffer Elahe Soltanaghai

We introduce GazeSwipe, a novel circular keyboard interface for gaze-based text entry in extended reality (XR) environments, allow- ing users to type words by swiping their gaze across the keyboard. We discuss the layout and decoder design of GazeSwipe, as well as key constraints and considerations—including the impact of eye tracker accuracy and gaze swiping on key arrangements. To help users learn gaze typing and memorize key locations, we de- veloped and evaluated several learning processes for teaching the new keyboard layout. A preliminary user study with GazeSwipe of- fers insights into implementing and teaching unfamiliar text-entry systems in immersive environments.

Houze Yang, Yufeng Liu, Paul Jeziorczak, Shawn Li, Sarah Sun, Inki Kim

Low-input, high-return simulations offer a cost-effective, scalable solution for delivering highly immersive hands-on learning experiences to patients, families, staff, and caregivers without demanding extensive resources. We developed a touch-based Mixed Reality (MR) app for mobile devices to deliver cost-effective cardiopulmonary resuscitation (CPR) training. To enhance physical immersion, the team incorporated innovative sensing and computing techniques within MR to precisely translate the learners’ interactive forces during CPR into visualized chest compressions on a virtual child.

Chih-Hao Lin, Zian Wang, Ruofan Liang, Yuxuan Zhang, Sanja Fidler, Shenlong Wang, Zan Gojcic

Generating realistic and controllable weather effects in videos is valuable for many applications. Physics-based weather simulation requires precise reconstructions that are hard to scale to in-the-wild videos, while current video editing often lacks realism and control. In this work, we introduce WeatherWeaver, a video diffusion model that synthesizes diverse weather effects---including rain, snow, fog, and clouds---directly into any input video without the need for 3D modeling. Our model provides precise control over weather effect intensity and supports blending various weather types, ensuring both realism and adaptability. To overcome the scarcity of paired training data, we propose a novel data strategy combining synthetic videos, generative image editing, and auto-labeled real-world videos. Extensive evaluations show that our method outperforms state-of-the-art methods in weather simulation and removal, providing high-quality, physically plausible, and scene-identity-preserving results over various real-world videos.

Chih-Hao Lin, Jia-Bin Huang, Zhengqin Li, Zhao Dong, Christian Richardt, Tuotuo Li, Michael Zollhöfer, Johannes Kopf, Shenlong Wang, Changil Kim

Inverse rendering seeks to recover 3D geometry, surface material, and lighting from captured images, enabling advanced applications such as novel-view synthesis, relighting, and virtual object insertion. However, most existing techniques rely on high dynamic range (HDR) images as input, limiting accessibility for general users. In response, we introduce IRIS, an inverse rendering framework that recovers the physically based material, spatially-varying HDR lighting, and camera response functions from multi-view, low-dynamic-range (LDR) images. By eliminating the dependence on HDR input, we make inverse rendering technology more accessible. We evaluate our approach on real-world and synthetic scenes and compare it with state-of-the-art methods. Our results show that IRIS effectively recovers HDR lighting, accurate material, and plausible camera response functions, supporting photorealistic relighting and object insertion.

Steven Gao, Ziyang Xie, Jing Wen, Sarita Adve, Shenlong Wang

We present a real-time immersive teleconferencing system using Gaussian avatars, enabling photorealistic rendering and dynamic adaptation from real-time video input from a consumer-grade webcam. Our system integrates generalizable feed-forward reconstruction with streamable test-time optimization to deliver high-quality avatars with low latency. Leveraging a hybrid cloud-edge architecture, we propose a continuous refinement and streaming scheme of the avatar to enhance realism as the teleconference progresses without disruption to the user experience. A novel cross-view 2D refinement module further propagates high-frequency details and improves fidelity under limited views. Our system achieves competitive rendering quality and responsiveness across varying durations of input, demonstrating effectiveness for real-world deployment in XR communication settings.

Bradley P Sutton

The proposed initiative aims to establish the Center for Advanced Imaging and Modeling in Medicine (AIM), a multi-institutional hub dedicated to transforming clinical imaging data into actionable 3D models for enhanced patient care. Building on a decade of experience at OSF’s AIM Lab—one of the nation's leading sites for virtual reality (VR) in pre-surgical planning—the project seeks to create standardized data pipelines and software infrastructure for medical image processing. The center will develop core technical components to support segmentation, quality control, model generation, and VR visualization, enabling scalable, secure, and clinically relevant outputs. This structured approach will support integration across diverse clinical domains, including cardiology, neurology, and oncology, and will accelerate research in artificial intelligence (AI), digital twins, and medical education. Three prior projects—on aortic arch modeling, myocardial segmentation, and epilepsy mapping—will serve as proof-of-concept pilots within the new framework. The center will facilitate collaboration between engineers and clinicians, leveraging expert-labeled datasets for algorithm development and validation. By uniting technical rigor with clinical utility, AIM positions UIUC and OSF to lead the field in medical imaging innovation and establish the foundation for a federally funded NIH P41 center.

Sirui Xu, Hung Yu Ling, Yu-Xiong Wang, Liang-Yan Gui

Achieving realistic simulations of humans interacting with a wide range of objects has long been a fundamental goal. Extending physics-based motion imitation to complex human-object interactions (HOIs) is challenging due to intricate human-object coupling, variability in object geometries, and artifacts in motion capture data, such as inaccurate contacts and limited hand detail. We introduce InterMimic, a framework that enables a single policy to robustly learn from hours of imperfect MoCap data covering diverse full-body interactions with dynamic and varied objects. Our key insight is to employ a curriculum strategy – perfect first, then scale up. We first train subject-specific teacher policies to mimic, retarget, and refine motion capture data. Next, we distill these teachers into a student policy, with the teachers acting as online experts providing direct supervision, as well as high-quality references. Notably, we incorporate RL fine-tuning on the student policy to surpass mere demonstration replication and achieve higher-quality solutions. Our experiments demonstrate that InterMimic produces realistic and diverse interactions across multiple HOI datasets. The learned policy generalizes in a zero-shot manner and seamlessly integrates with kinematic generators, elevating the framework from mere imitation to generative modeling of complex human-object interactions.

Yanming Zhang, Jun-Kun Chen, Jipeng Lyu, Yu-Xiong Wang

We presents a novel, training-free framework enabling instruction-guided editing of immersive video and 3D scenes. By progressively decomposing complex edits into manageable subtasks, our method finely balances content preservation and editing effectiveness through strategic noise manipulation and attention control. Experiments demonstrate high-quality, 3D-consistent results, significantly enhancing immersive computing applications involving dynamic video modification and sophisticated geometric scene edits.

Ammar Tahir, Yongzhou Chen, Prateesh Goyal, Radhika Mittal

Interactive video applications (e.g. cloud gaming, XR, VR) are highly sensitive to even small amount of queuing and network drops. They can result in persistent high frame delays and stalls, that severely impact the end-user's quality of experience. At the same time, it is almost impossible to avoid queuing delay or packet drops for several reason: encoder bitrate variations, slow reaction of encoder to bandwidth changes and estimating bottleneck bandwidth requires inducing some amount of queuing and drops. In this work, we argue that artificial nature of bottlenecks on today's Internet can be exploited to significantly simplify control loop for interactive applications and gain high improvements in quality of experience. Internet Service Providers (ISPs) routinely use rate limiting mechanisms to throttle per-user bandwidth. We show that if we use traffic policers, which allow for coarser rate limiting that lets occasional overshoots go through without queuing or dropping, we can improve interactive application's QoE on all fronts (high bitrate, low stall and delays).

Mingyuan Wu, Ruifan Ji, Haozhen Zheng, Jiaxi Li, Beitong Tian, Bo Chen, Ruixiao Zhang, Jacob Chakareski, Michael Zink, Ramesh Sitaraman, Klara Nahrstedt

We propose an interactive and intelligent hybrid teleconferencing system compatible with Virtual Reality devices. Our system understands meeting contexts and leverages user interactions to enable better system configurations. By employing interactive scene graphs \cite{sg}, our system extracts and transmits essential meeting context to users while relaying user interactions back to the streaming systems for user-involved adaptive streaming and foveated rendering. We demonstrate the system's real-time performance and compatibility with commercial VR devices such as the Meta Quest 3.

Ben Civjan, Bo Chen, Ruixiao Zhang, Klara Nahrstedt

Firefighters leverage immersive 360° video analytics pipelines for decision-making, post-mission feedback, and developing new training scenarios. However, deploying cameras in outdoor training environments presents two major challenges: limited network connectivity due to the distance from Wi-Fi infrastructure and restricted energy availability as cameras operate on battery power. To investigate network limitations, we conducted field tests at the Illinois Fire Service Institute (IFSI) to analyze connectivity from various locations on the grounds. Additionally, we developed a 360° video streaming framework supporting multiple video codecs (MJPEG, WebP, Tiled MJPEG, and H.264) and systematically evaluated their impact on bandwidth usage and 360° video streaming performance under real-world conditions at IFSI. To address the energy constraint experienced by cameras streaming video we developed a system, EcoLens, that dynamically optimizes processing configurations to minimize energy consumption of the camera while preserving essential video features for deep learning inference. We first conducted an extensive offline evaluation of various configurations consisting of device CPU frequency, frame filtering features, difference thresholds, and video bitrates, to establish apriori knowledge of their impact on energy consumption and inference accuracy. Leveraging this insight, we introduced an online system that employs multi-objective Bayesian optimization to intelligently explore and adapt configurations in real time. Our approach continuously refines processing settings to meet target inference accuracy with minimal edge device energy expenditure. Experimental results on 2D video demonstrated the system’s effectiveness in reducing video processing energy use while maintaining high analytical performance, offering a practical solution for smart devices and edge computing applications.

Yiqiu Sun, Phyllis Wang, Tharindu Patabandi, Saugata Ghose

Processing-using-memory (PUM) architectures offer the potential for large performance improvements and energy savings by eliminating data movement between memory and processors. However, these dramatic benefits are currently difficult to achieve in practice without significant manual programming effort and expert knowledge. In this work, we present MaRIMBA, a novel framework designed to program and manage PUM architectures at scale. MaRIMBA adapts the MapReduce model to scale programs, overcoming critical challenges for PUM that prior MapReduce works did not face. MaRIMBA effectively partitions computational tasks and optimizes communication patterns to support scalability across a wide range of PUM architectures. We use three PUM architectures to demonstrate how MaRIMBA, with its architecture-agnostic code, can unlock significant performance improvements.

Vidhi Rambhia, Vikram Adve (TBC)

As modern applications become increasingly compute-intensive, deploying them efficiently on resource-constrained platforms like Extended Reality (XR) systems, autonomous robots, or mobile devices becomes a major systems challenge. XR workloads, in particular, demand real-time responsiveness, high visual fidelity, and strict energy efficiency, placing significant stress on both hardware and software. A key bottleneck lies in the inflexibility of conventional optimization techniques, which often treat models and pipelines as black boxes and rely on application-agnostic metrics like raw accuracy or latency to guide decisions. ApproxTuner addresses this gap by offering a modular, programmable framework for application-aware approximation tuning, enabling automated exploration of performance-quality trade-offs tailored to the actual downstream task. Rather than optimizing for proxy metrics alone, ApproxTuner evaluates configurations using domain-specific Quality-of-Service (QoS) metrics that reflect real application utility. At its core, ApproxTuner introduces a pluggable architecture composed of knobs (which represent approximation techniques such as pruning, quantization, resolution reduction, etc.), applications, and QoS evaluators. This design allows developers to flexibly define and tune approximation strategies that span across algorithms, system software, and hardware. By incorporating the goals and constraints of the target application directly into the optimization loop, ApproxTuner unlocks more aggressive yet acceptable approximations, discovering configurations that generic frameworks overlook.

Nabila Tasnim, Haoran Liu, Shomik Chatterjee, Divake Kumar, Lorenzo Bujalil Silva, Pratham Jain, Areeba Rashid, Shaloo Rakheja, Amit Ranjan Trivedi, Qing Cao, Saugata Ghose

Edge devices such as autonomous robots, UAVs, and self-driving vehicles need to adapt to dynamic environments in real-time by learning from new data in the field - without forgetting prior knowledge. However, conventional ML training approaches are ill-suited for edge devices due to impracticality of storing all historical data. Continual-Learning offers a promising alternative by enabling models to adapt incrementally without losing old pre-trained information, even in the absence of full training datasets. Yet, current hardware platforms like CPUs and GPUs are ill-equipped for continual-learning on edge because of energy-intensive nature – primarily due to extensive data movements between memory and compute units. While existing in-memory accelerators support inference efficiently, they lack resource-efficient training. This project aims to develop the first continual-learning accelerator for edge systems using ECRAM-CMOS in-memory computing. Specifically, it adapts iMAP framework—an MLP-based continual-learning system that reconstructs 3D scenes for navigation from sparse keyframes - to operate efficiently on our custom in-memory accelerator chip. By performing critical parts of continual-learning computations directly within memory, the system eliminates costly traditional memory to compute-units data transfers and significantly reduces energy consumption. From our preliminary estimation, we aim to make the system over 20x energy-efficient compared to conventional accelerators.

Bakshree Mishra, Ying Jing, Sriram Devata, Luca Carloni's group at Columbia University

Computer systems designed for resource-constrained domains such as Extended Reality (XR) are becoming increasingly heterogeneous with multiple hardware accelerators specialized for various tasks. A traditional approach achieving performance for such domains is to design coarse-grained specialized accelerators that fuse the multiple computations into one monolithic instance. However, such an approach suffers from disadvantages of low flexibility for reuse or extensibility – which are critical for a diverse and fast-moving domain such as XR. In our work, we demonstrate an alternative approach that software-composes fine-grained accelerators through shared memory to address the disadvantages of the monolithic approach. However, fine grained accelerators are not susceptible to control and data acceleration taxes. We propose an SoC architecture, Mozart, that tames acceleration taxes to promote composable acceleration as a performant alternative. We will also introduce ongoing work on accelerator virtualization that leverages a shared memory interface to provide a device-agnostic and transparent interface for supporting heterogeneous parallelism. Our work has great implications for generalized compilers and runtimes for heterogeneous systems, such as those found in XR SoCs.

Q. Jiang, Y. Pang R. Singh, M.K.Vaddiraju, S. Adve

Extended reality (XR) can revolutionize a wide range of industries and human activities. To fully realize this potential, XR devices need to be mobile, lightweight, comfortable, and capable of providing immersive experiences throughout the day. However, there is a fundamental tradeoff between low power, or long battery life, and rich experience which demands heavy computations. This motivates distributed XR, where heavy computations are offloaded to remote servers, allowing low power devices to provide rich, immersive experiences. In this poster, we showcase recent advances in distributed XR features within ILLIXR, including offloading head tracking, offloading rendering, and the introduction of a modular network interface in the runtime. We will also highlight ongoing work on distributed real-time scene provisioning, distributed semantic mapping and querying, and dynamic computation orchestration within the runtime.