Apple Unveils Revolutionary Mixed Reality Headset, Apple Vision Pro, at WWDC 2023
Posted on June 6, 2023 17 minutes 3497 words
Table of contents
- Blending Real and Digital Worlds
- A Revolutionary User Experience
- Revolutionizing Work and Collaboration
- Transforming Home Experiences with Spatial Computing
- Innovative Design for Comfort and Wearability
- Technological Marvel Redefining Spatial Computing
- Empowering Developers and Prioritizing Privacy
- Privacy and Security: Safeguarding Users’ Personal Information
- Conclusion: The Future of Personal Electronics Unveiled
- References
At the highly anticipated WWDC 2023 event, Apple surprised attendees with a thrilling announcement. Tim Cook, CEO of Apple, took the stage and delivered his classic “One more thing” line, revealing the company’s groundbreaking foray into the world of mixed reality. Cook expressed his belief in the transformative power of augmented reality (AR), and how it can revolutionize our experiences. With that, he proudly introduced the crowd to the highly anticipated Apple Vision Pro, a cutting-edge mixed reality headset.
Blending Real and Digital Worlds
The Apple Vision Pro is not just another display; it is an entirely new kind of computer that seamlessly blends the real and digital worlds. Cook emphasized that it is the first Apple product that users look through, rather than at. This innovative headset allows users to see, hear, and interact with digital content as if it were part of their physical environment. Apple achieves this remarkable feat by leveraging the most natural and intuitive tools available: the users’ eyes, hands, and voice.
With the Apple Vision Pro, users are liberated from the constraints of a traditional display. Instead, their surroundings become an infinite canvas, where apps can be used anywhere and resized according to their preference. Users can capture photos and videos, reliving precious memories in a whole new way. They can also enjoy movies, shows, sports, and immersive gaming experiences on a giant screen, all enveloped by Spatial Audio. Furthermore, Vision Pro allows users to connect with others, creating the feeling of sharing the same physical space.
Apple believes that Vision Pro will mark the beginning of spatial computing, similar to how the Mac and iPhone revolutionized personal and mobile computing, respectively. This groundbreaking technology will bring a new dimension to powerful and personal technology, unlocking endless possibilities.
A Revolutionary User Experience
Alan Dye, Apple’s VP of Human Interface, took the stage to shed light on the extraordinary user experience offered by Apple Vision Pro. He emphasized that the interface of Vision Pro is entirely three-dimensional, magically controlled with the users’ eyes, hands, and voice. Upon wearing the headset, users find themselves in the Home View, where all their apps are easily accessible. The interface is both familiar and revolutionary, with each element crafted to possess physicality. Elements respond dynamically to light, have dimension, and even cast shadows to provide users with a clear sense of scale and distance.
Apps on Vision Pro transcend the boundaries of traditional displays and can fill the users’ surrounding space. Users have the freedom to resize apps, making them larger than life if desired. Placing apps in different locations within the space feels natural, mimicking the act of moving real objects. Importantly, launching new apps does not displace existing ones; instead, they occupy a new space centered around the user.
The experience of using Apple Vision Pro is further enhanced by Environments. These virtual landscapes extend and transform the users’ physical space, enabling previously unimaginable experiences. Environments are captured volumetrically, allowing users to control their level of immersion simply by turning the Digital Crown. Whether creating a moment of calm with a Mindfulness session or reducing clutter in a busy environment, Environments provide beautiful and inspiring new views.
Intuitive Interaction with Vision Pro
Apple’s design philosophy for Vision Pro was centered around creating an intuitive input model for spatial computing, one that requires no additional hardware or controllers. The headset relies solely on users’ eyes, hands, and voice, ensuring a seamless and magical experience. Users can effortlessly browse the system simply by looking, with app icons coming to life when gazed upon. Selecting items is as easy as tapping fingers together, while scrolling is achieved by gently flicking. Apple has meticulously crafted each gesture to be subtle and natural, allowing users to keep their hands in comfortable positions such as resting on their laps or the sofa.
Voice input has never been more seamless than with Vision Pro. Users can dictate by looking at a search field, and Siri integration enables quick app navigation, media control, and more. Furthermore, Vision Pro ensures that users remain connected to the people around them.
The headset’s breakthrough innovation called EyeSight not only displays the users’ eyes when others are nearby but also provides vital cues about their focus. When fully immersed in an experience, Vision Pro subtly signals to others that the user’s attention is occupied. However, even in such moments, the headset helps users stay connected by displaying nearby individuals and revealing the users’ eyes simultaneously.
Revolutionizing Work and Collaboration
To demonstrate the incredible capabilities of Apple Vision Pro for work and productivity, Allessandra McGinnis, Product Manager for Apple Vision Pro, took the stage. She highlighted how the headset enhances familiar apps such as Notes, Messages, Keynote, Safari, FaceTime, and more, breaking free from the limitations of traditional displays. Vision Pro seamlessly syncs with users’ iPhone, iPad, and Mac, ensuring that all their notes, messages, photos, and documents stay up to date through iCloud.
Safari on Vision Pro offers an exceptional browsing experience, with large web pages that make watching videos and reading articles a delight. The headset allows users to freely arrange multiple apps in their space, layering them or placing them in proximity to enhance multitasking. Within Messages, users can receive and interact with 3D objects, inspecting them from every angle as if they were physically present. The spatial awareness provided by Vision Pro ensures that users remain connected to their environment and other individuals.
The headset seamlessly integrates with popular Bluetooth accessories like the Magic Trackpad and Magic Keyboard, facilitating tasks such as writing emails or working on spreadsheets in Numbers. Moreover, users can wirelessly incorporate their Mac into Apple Vision Pro by simply looking at it. This integration transforms the Mac into a massive, portable 4K display, allowing users to work within their ideal workspace, whether it’s the office or a remote location.
Apple Vision Pro also excels in facilitating remote collaboration. Users can easily collaborate with their team on the same document, share apps, and utilize SharePlay within FaceTime. Group calls on FaceTime become spatial, enabling life-sized video tiles for participants, while Spatial Audio enhances the sense of presence. Whether reviewing a presentation, sharing photos and videos, or watching a movie together, FaceTime on Vision Pro delivers powerful shared experiences that seamlessly integrate with other Apple devices.
Apple Vision Pro empowers users to connect with others effortlessly, create personalized workspaces, and embrace productivity wherever they may be.
Apple’s foray into mixed reality with the Vision Pro headset demonstrates the company’s relentless pursuit of innovation and its commitment to transforming the way we interact with technology. The combination of seamless integration, intuitive interaction, and immersive experiences paves the way for a new era of spatial computing, unlocking a world of possibilities for both personal and professional use.
Transforming Home Experiences with Spatial Computing
En Kelly, Senior Engineering Program Manager at Apple’s Technology Development Group, took the stage at the WWDC 2023 event to discuss how Apple Vision Pro enhances experiences in the home. The headset offers groundbreaking capabilities that redefine how users relive and interact with their precious moments.
Capturing and Reliving Memories in 3D
Apple Vision Pro introduces a new dimension to photos and videos by incorporating 3D and Spatial Audio. Users can select a photo and watch as the room around them dims, providing an immersive viewing experience. The headset showcases users’ libraries of photos and videos in stunning detail, with brilliant colors. Users have the freedom to make their memories as big as their room or zoom in for a closer look. Additionally, panoramas taken on an iPhone expand and wrap around users in life-size, transporting them back to the moment the photo was taken.
As Apple’s first 3D camera, Vision Pro allows users to capture and relive memories in 3D, with the added enhancement of Spatial Audio. This spatial video feature brings memories to life, making users feel like they are truly present in that moment. Whether it’s reliving a special birthday or capturing a fun family moment, Vision Pro’s spatial photos and videos create a closer connection to cherished memories.
Immersive Entertainment Experiences
Apple Vision Pro offers an extraordinary entertainment experience within the comfort of users’ homes. The headset provides a spatial cinema that can transform any space into a personal movie theater. The screen size can be adjusted to perfection, automatically dimming the surrounding light and casting a beautiful glow. The immersive experience is further enhanced by Spatial Audio, which surrounds users with lifelike sound. By opening an Environment like Mount Hood, users can expand their screen beyond the dimensions of their room, resulting in a breathtaking viewing experience.
Vision Pro also supports 3D movies, allowing users to experience incredible depth and crisp motion. The 3D movie experience provided by Vision Pro is unparalleled, making users feel as if they are peering directly into the movie itself. In addition to movies, Vision Pro enables interactive storytelling with immersive 3D experiences. For example, Encounter: Dinosaurs allows users to witness dinosaurs interacting in front of them, complete with Spatial Audio.
Users can also enjoy their favorite Apple Arcade games on Vision Pro, with the ability to play on a screen as large as desired and with exceptional audio quality. With over 100 Arcade titles available, Vision Pro opens up a new realm of gaming experiences.
Apple and Disney Partnership
Apple CEO Tim Cook welcomed Bob Iger, CEO of Disney, to the stage. Iger expressed his excitement about the partnership between the two companies and how Apple Vision Pro can bring Disney’s storytelling to life in unprecedented ways. Iger highlighted the personal experiences that can be created, allowing fans to get closer to their favorite characters and become fully immersed in Disney’s stories.
Bob Iger shared a glimpse of what Disney has in store for Apple Vision Pro, including experiences that allow users to travel on National Geographic adventures, interact with imaginary characters, and bring Disney World into their own world. Additionally, Disney+ will be available on day one, offering a vast array of Disney content accessible through Vision Pro.
Innovative Design for Comfort and Wearability
Richard Howarth, Apple’s VP of Industrial Design, took the stage to discuss the remarkable design of Vision Pro. The headset is crafted using advanced, lightweight materials to achieve optimal performance, mobility, and wearability. The front of the product is a seamless piece of three-dimensionally formed and laminated glass, acting as a lens for EyeSight and various cameras and sensors. The glass smoothly transitions into a lightweight frame, which houses a button for capturing spatial photos and videos and a Digital Crown for controlling immersion levels in Environments.
The frame, made from a custom aluminum alloy, elegantly curves around the user’s face, serving as a primary structural element. To ensure comfort and a precise fit, Vision Pro features a modular system. The LightSeal, available in different shapes and sizes, flexes to conform to the user’s face. Flexible straps extend from the frame to keep the audio pods close to the ears, providing Spatial Audio while maintaining awareness of the surroundings. The 3D-knitted Head Band offers cushioning, breathability, and stretch. It attaches securely and can be easily adjusted for a personalized fit. Apple has collaborated with Zeiss to create custom optical inserts that magnetically attach to the lenses, accommodating various vision corrections without compromising display performance or eye tracking accuracy.
Apple Vision Pro delivers extended battery life, supporting all-day use when plugged in and up to two hours of use with an external, high-performance battery. The separate battery ensures a lightweight and comfortable experience, with the battery unit easily fitting into a pocket and connecting via a supple woven cable.
With its incredible design and blend of digital content with the real world, Apple Vision Pro emerges as an exceptionally capable and powerful spatial computer that offers a new era of immersive experiences for users at home.
Technological Marvel Redefining Spatial Computing
Mike Rockwell, VP of Apple’s Technology Development Group, took the stage at WWDC 2023 to delve into the technological advancements behind the Apple Vision Pro headset. The creation of their first spatial computer required innovation across every aspect of the system, including displays, audio, sensors, Apple silicon, and the visionOS operating system.
Cutting-Edge Displays for Immersive Experiences
Apple recognized the importance of delivering exceptional resolution and color fidelity to users’ eyes. To achieve this, they invented a display system with an abundance of pixels in a compact form factor. Micro-OLED Apple silicon backplanes, with pixels measuring just 7.5 microns wide, fit 64 pixels within the space of a single iPhone pixel. Vision Pro boasts a staggering total of 23 million pixels across two panels, each the size of a postage stamp. This provides users with more pixels than a 4K TV for each eye. The display is magnified and wrapped around users using a custom three-element lens, resulting in a continuous, smooth image. The true 4K resolution, wide color gamut, and high dynamic range enable incredible video rendering, while fine text remains sharp from any angle.
Immersive Spatial Audio
Audio plays a vital role in shaping users’ perception of the world. Apple Vision Pro features an entirely new Spatial Audio system. Personalized sound is delivered through integrated dual-driver audio pods, creating ambient Spatial Audio that convinces the brain that sounds are originating from the environment around the user. Furthermore, Vision Pro matches the sound to the user’s room by analyzing its features and materials using audio ray tracing. This advanced audio system is just one example of how Vision Pro leverages its powerful sensor array.
Advanced Sensors Powering Real-Time Experiences
Vision Pro incorporates a range of sensors that enable high-resolution video to the displays, precise head and hand tracking, real-time 3D mapping, and more. A high-performance eye tracking system utilizes high-speed cameras and LED rings that project invisible light patterns onto the eyes, providing responsive and intuitive input. This precise eye tracking eliminates the need for external controllers, freeing up users’ hands for a seamless experience. The massive computational power required to process sensor information and deliver ultra-high-resolution experiences is made possible by Apple silicon. The dual-chip design comprises the Apple M2 chip for compute performance and the specialized R1 chip for real-time sensor processing. The R1 chip processes input from 12 cameras, five sensors, and six microphones, virtually eliminating latency between sensors and displays. With a streaming speed eight times faster than the blink of an eye, Vision Pro creates experiences that feel incredibly lifelike.
EyeSight and Digital Persona for Authentic Communication
Apple Vision Pro addresses the challenge of allowing interaction with others while wearing the headset. EyeSight utilizes a curved OLED panel and lenticular lens to project the correct perspective of the user’s eyes, creating a transparent appearance. For digital communication, such as FaceTime, Vision Pro goes beyond conveying only the eyes and generates an authentic representation of the user. Through advanced machine learning techniques and an enrollment process using front sensors, a neural network creates a dynamic digital Persona that matches the user’s facial and hand movements. This Persona enables communication with over a billion FaceTime-capable devices, and when viewed on another Vision Pro headset, it possesses volume and depth far beyond traditional video.
The Power of visionOS: An Operating System for Spatial Computing
visionOS, the operating system powering Vision Pro, builds upon the engineering innovation of macOS, iOS, and iPadOS. Designed from the ground up for spatial computing, visionOS introduces new capabilities to support low-latency requirements, including a real-time execution engine, dynamically foveated rendering pipeline, and a multi-app 3D engine that allows different apps to run simultaneously in the same simulation. Existing application frameworks have been extended to natively support spatial experiences, making visionOS a groundbreaking operating system tailored for spatial computing.
The culmination of these technological advancements makes Apple Vision Pro an unrivaled device, marking the beginning of an entirely new platform. The headset’s groundbreaking innovations open up endless possibilities for developers to create immersive and transformative experiences.
Empowering Developers and Prioritizing Privacy
As Apple unveils the groundbreaking Apple Vision Pro headset, developers are eager to explore the vast possibilities it presents. Susan Prescott, VP of Worldwide Developer Relations, expressed excitement about the potential of Vision Pro, highlighting its ability to reimagine existing apps and inspire the creation of new ones. Spatial experiences, including lifelike 3D objects, can now be brought to life, revolutionizing education, collaboration, productivity, and entertainment.
Early App Previews Showcasing the Power of Vision Pro
Developers who were given early access to Vision Pro demonstrated its immense capabilities through their innovative apps. Complete HeartX, for example, offers an interactive 3D beating heart that enhances students’ understanding of the human body. JigSpace allows designers to collaborate and review concepts, such as the design of the Alfa Romeo F1 car. Businesses can utilize Stages from PTC to evaluate and approve new production lines before manufacturing begins. Music enthusiasts can mix tracks using djay’s reimagined spatial interface, while Sky Guide provides users with their own personal planetarium, offering breathtaking views beyond what the naked eye can perceive. Furthermore, Vision Pro seamlessly integrates with popular Microsoft apps like Excel, Word, and Teams, taking advantage of its expansive canvas and sharp text rendering. Video conferencing apps leverage users’ digital Personas for effortless collaboration. Additionally, developers can utilize familiar tools and frameworks like Xcode, SwiftUI, RealityKit, and ARKit to build new apps specifically for Vision Pro. To streamline the preparation of 3D content, Apple introduces Reality Composer Pro, enabling the assembly of complex scenes with realistic objects. Hundreds of thousands of iPad and iPhone apps will be compatible with Vision Pro at launch, offering a rich and diverse app ecosystem.
Unity Partnership Expands Possibilities for Game Developers
Apple has collaborated with Unity, a leading game development platform, to enable Unity-based games and apps to fully utilize visionOS features. Game developers can leverage visionOS’s pass-through capabilities, high-resolution rendering, and native gestures to create immersive and seamless gaming experiences. Unity apps running natively on Vision Pro can coexist with other visionOS apps simultaneously, opening up endless possibilities for interactive and engaging gameplay. The launch of the Apple Vision Pro App Store will serve as a central hub for users to discover and download apps specifically built for visionOS, as well as compatible iPad and iPhone apps.
Privacy and Security: Safeguarding Users’ Personal Information
Apple’s commitment to privacy and security extends to Apple Vision Pro. The device employs Optic ID, a secure and effortless system that utilizes the uniqueness of users’ iris patterns to instantly unlock Vision Pro. Optic ID data is encrypted, stored only on the device, and accessible solely by the Secure Enclave processor, ensuring the highest level of security. Users’ eye-tracking data remains private, isolated within a separate background process, preventing apps and websites from tracking users’ gaze. Camera data is processed at the system level, safeguarding users’ surroundings from individual app access. Apple’s stringent privacy measures ensure that users can fully enjoy the immersive experiences offered by Vision Pro with peace of mind.
Conclusion: The Future of Personal Electronics Unveiled
Apple Vision Pro emerges as the most advanced personal electronics device ever created. Its advanced displays, revolutionary Spatial Audio system, powerful Apple silicon chips, visionaryOS operating system, and stunning aerospace-grade alloy design redefine what is possible in a standalone spatial computer. With over 5,000 patents filed during its development, Vision Pro offers infinite screen real estate, unparalleled entertainment, enhanced collaboration and communication, and unrivaled memory capture capabilities. Priced starting at $3499, Apple Vision Pro will be available for purchase early next year on apple.com and in select Apple retail stores across the United States, with availability in more countries to follow. The Apple Store will provide personalized demonstrations and guidance to users seeking to experience the full potential of Vision Pro. As developers continue to explore the vast possibilities offered by this groundbreaking platform, Apple Vision Pro heralds the dawn of an entirely new era in spatial computing.
