Committee

Nicolas Bonnier, Apple
Paul Hubel, Apple
Jerry Jia, Reality Labs
Sophie Triantaphillidou, University of Westminster
Bennett Wilburn, Reality Labs

Imaging for XR: 4 March 2022

ABOUT

Imaging for XR focuses on content creation by both professional and end consumers; imaging technologies and workflows for various applications including VR documentaries, live entertainment, online meetings, or medical uses; and how to improve imaging to improve comfort.
 
Find out about the latest trends in imaging for AR/VR/MR (XR), discuss pain points, learn from power users, and identify opportunities for further research and collaboration! Topics cover:

• XR image quality (camera and display)
• Image and color processing for XR
• Content creation for XR
• Quality of user experience, comfort

FINAL PROGRAM

9:00 CA / 12:00 NY / 18:00 EU

Welcome and Introduction

9:15-10:25 CA / 12:15-13:25 NY /18:15-19:25 EU

Session 1

The State of the VR Immersive Media Ecosystem, Eric Cheng, immersive media lead, Reality Labs
About the talk: This talk discusses the current state of the VR immersive media ecosystem, including current spec targets and best practices for peak-quality production and distribution; where we currently are in the spectrum of immersive experiences between flat video and fully 6DoF narrative storytelling; possible areas of need for technology development across the entire ecosystem; and more.

Surgical Telerobotics, Ian McDowall, Fellow, Vision Systems, Intuitive Surgical
About the talk: One approach for improving surgical outcomes and minimizing patient trauma employs telerobotics and high-quality 3D vision. Telerobotics and the mechanical design of the arms can minimize damage at the body wall, while advanced instruments enable small incisions with excellent control and surgical performance.  Stereo vision technologies enable the surgeon to navigate with confidence and precision. This talk explores how these technologies all come together to create systems that have been used to treat millions of patients.

10:50-12:00 CA / 13:50-15:00 NY / 19:50-21:00 EU

Session 2

Virtual Production for Film and Virtual Reality, Logan Lemmon, head of studio, Supernatural
About the talk: Simulating the appearance of real humans in virtual environments poses a number of challenges. This talk explores some of the techniques filmmakers and game developers employ to create believable scenes both on-screen and in virtual reality. We examine 3D cameras and equipment used to simulate proper inter-pupillary distance of the human eyes, stereo billboarding as a concept, Mixed Reality as a video production tool, and virtual production techniques employed on contemporary film sets to realistically blend actors with computer generated backgrounds. 

Why you shouldn’t ask questions, Jannicke Mikkelsen, film director and cinematographer specialized in movie productions using next-gen technology
About the talk: ‘The one project you should sign up for is the one that starts with an overly ambitious goal presented as a minor challenge’ is a short tale of my naïve aspiration to change the future of entertainment, instigated by my bosses who set the bar, from Sir David Attenborough to the rock band QUEEN. Our speaker, Norwegian explorer, Guinness World Record holder, and filmmaker takes us through the journey of cutting-edge content creation from unexplored territories in the arctic, underwater, aviation, and space.

12:30-13:40 CA / 15:30-16:40 NY / 21:30-22:40 EU

Session 3

Color Appearance in Augmented Reality Imaging Systems, Michael J. Murdoch, associate professor, Rochester Institute of Technology
About the talk: Color appearance is always contextual, especially in imaging, where neighboring colors and scene interpretation can affect perception. Optical see-through augmented reality (OST-AR) systems add another layer (pun intended) to the complexity of context, because they allow real-world background scene objects to remain visible through a transparent AR overlay. Ongoing research is uncovering how transparent AR layers are perceived and to what extent viewers are able to discount, or ignore, background elements that physically distort the AR content. A model of layer interpretation and AR color appearance will be shared, with implications for content creation and the color imaging pipeline in AR system.

Apple Object Capture, Long Quan, senior research manager, Apple Object Capture, Apple
About the talk: It is most natural to capture reality from photos of multiple views with cameras, phones, and drones. This talk discusses how modern computer vision technology reinvents the photogrammetry for a full 3D reconstruction of objects and scenes. Learning visual descriptors is critical for the early stage of structure from motion and camera poses recovery. The multi-view stereo and networks enable efficient depth map and point cloud generations. Surface reconstruction could also be adapted for the scale of the scene or the object. In addition to viewing recent examples in Object Capture of Apple, the talk also presents the limitations of the practical capturing and the constraints of object materials. Looking forward, not only surface geometry, but also material properties and illumination are increasingly maturing for integration into the augmented reality.

13:40-14:30 CA / 16:40-17:30 NY / 22:40-23:30 EU

Panel Discussion with Speakers

14:30-15:00+ CA / 17:30-18:00+ NY / 23:30-24:00+ EU

BYOB Gathering with Speakers

Join the speakers in Gather.town to continue the discussion in an informal setting. Learn more about what they are doing and enjoy the opportunity for some one-on-one conversations.

Supplemental On Demand Short Course

Expand your knowledge by registering for the On Demand version of this 4-hour short course, recorded at EI 2021. Downloadable course notes provided. Special reg package of workshop + course offered. View course as many times as you like until end of July 2022.

Stereoscopic Imaging Fundamentals Instructor: Andrew Woods, Curtin University