Scribbler Logo

    You have 4 summaries left

    Breaking Banks

    Episode 499: Apple’s Future Vision

    Thu Jun 22 2023
    Breaking BanksfintechAppleaugmented realityVR headsetspatial computingAImemory storage

    Description

    Breaking Banks is a global fintech radio show and podcast that explores personalities, startups, innovators, and industry players in financial services. The hosts are Brett King, Jason Henrichs, JP Nichols, Katie King (Miss Metaverse), and Brian Solis. The guests on this episode are Robert Scoble and Brian Romley. They discuss Apple's entry into the augmented reality market with their new product Vision Pro. The Apple VR headset has advanced sensor technology, eye and gesture control, and elegant design. It is primarily designed for sitting on the couch and watching movies, but has potential applications in various fields. Apple's entry into spatial computing will have a significant impact on the industry and attract investor interest. The future challenges include integrating computers into glasses and ensuring privacy.

    Insights

    Apple's Vision Pro headset

    Apple's Vision Pro headset is a result of significant engineering efforts and offers high-resolution display, advanced sensor technology, and elegant design.

    Potential applications of the Apple VR headset

    The Apple VR headset has potential applications in delivering immersive experiences, dimensional photography, recording sports events, and remote control of humanoid robots.

    Memory storage and AI capabilities

    Apple's journaling system and AI capabilities allow for storing and organizing memories, reliving past experiences, combating dementia, improving mental health, and providing positive perspective shifts.

    Impact of Apple's entry into spatial computing

    Apple's entry into spatial computing will kick off a new wave of startups in the field, transform society, and potentially replace other devices with glasses-based spatial computing devices.

    Future challenges and developments

    Improving battery technology and dealing with heat generated by GPUs on glasses are some of the challenges Apple faces in integrating computers into glasses. Apple prioritizes privacy and has a significant advantage in AI power. The developer version of Apple's AR glasses represents a convergence of technologies, with a high-end consumer version expected in about a year.

    Chapters

    1. Introduction
    2. Apple's Entry into Augmented Reality
    3. Interface and Technology of the Apple VR Headset
    4. Advanced Sensor Technology and Control of the Apple VR Headset
    5. Control and Future of the Apple VR Headset
    6. Features and Potential Applications of the Apple VR Headset
    7. Camera Technology and Potential Applications
    8. Memory Storage and AI Capabilities
    9. Impact of Apple's Entry into Spatial Computing
    10. Future Challenges and Developments
    11. Conclusion
    Summary
    Transcript

    Introduction

    00:06 - 08:34

    • Breaking Banks is a global fintech radio show and podcast that explores personalities, startups, innovators, and industry players in financial services.
    • The hosts are Brett King, Jason Henrichs, JP Nichols, Katie King (Miss Metaverse), and Brian Solis.
    • The guests on this episode are Robert Scoble and Brian Romley.
    • Robert first appeared on the podcast in August 2015 to discuss artificial intelligence, particularly Siri.
    • Brian first appeared on the podcast in March 2016 to discuss voice interface and voice AI.

    Apple's Entry into Augmented Reality

    08:08 - 15:58

    • They discuss Apple's entry into the augmented reality market with their new product Vision Pro.
    • Apple had to invent several technologies to develop this product.
    • Only Huawei and Apple could have achieved what they did with Vision Pro due to their engineering capabilities.
    • Oculus would not have been able to accomplish this due to hardware limitations.
    • Vision Pro has a high-resolution display that required significant engineering efforts from Apple.
    • Battery technology was a challenge for Apple, but they managed to create a device without an external cord or tethered power supply.
    • Weight balance was also important for comfort during use of the device.
    • The guests had a brief experience with Vision Pro but were mostly discussing it rather than using it.

    Interface and Technology of the Apple VR Headset

    15:33 - 23:26

    • The user interface of the Apple headset is three-dimensional and allows users to see around and above the interface.
    • The device allows users to focus on what they're trying to actuate, making it a magical experience.
    • Users attune to the environment presented by Apple's visual OS within about 10 minutes.
    • The mapping technology in the device rapidly maps the room and creates a detailed 3D environment.
    • The reality seen through the headset is disconcertingly close to real life, making users second guess their surroundings.
    • Watching videos on the device enhances adaptation and makes the digital twin of the world look better.
    • The web browser on the device is particularly enjoyable, while video viewing improves perception of the environment.

    Advanced Sensor Technology and Control of the Apple VR Headset

    22:59 - 29:54

    • The interface of the Apple VR headset becomes the user's reality once they get used to it.
    • The device has advanced sensor technology, including nine cameras and six microphones.
    • The IR scanning system tracks the user's eyes and iris, which can reveal their attention and gaze.
    • There is a half-second delay between the conscious mind and subconscious mind in decision-making.
    • Foveated rendering allows the screen to appear as a full 12k resolution where the user is looking, conserving resources.
    • Apple moves gaze data into a secure enclave to protect privacy and prevent advertisers from accessing it.
    • The device offers both eye control and gesture control, with accurate finger tracking using multiple cameras.
    • Users may need time to adjust to the sensitivity of gestures on the interface.
    • Subtle hand gestures can be used instead of overt movements for nuanced interactions.

    Control and Future of the Apple VR Headset

    29:34 - 37:22

    • Apple's new headset has elegant and simple gestures that allow users to interact with the screen by pulling it towards them.
    • The headset does not require a separate control and can be controlled with gesture movements.
    • Game controllers may become obsolete in this environment as the headset provides a more immersive experience.
    • Apple had debates about including rings and bracelets for tracking, but those ideas were voted down.
    • The headset is a prototype for Apple's upcoming glasses, which will be the mainstream event for augmented reality.
    • The glasses will project data into the user's field of view using lasers or other mechanisms.
    • The first product is not suitable for dangerous jobs or activities like working near heavy machinery or fast-paced sports.
    • The headset is primarily designed for sitting on the couch and watching movies.

    Features and Potential Applications of the Apple VR Headset

    36:52 - 44:32

    • The Apple Vision Pro mixed reality headset is primarily designed for sitting on the couch and watching movies, with some lightweight walking around.
    • The device is not intended for activities like running, mountain biking, or skiing.
    • The digital crown on the headset is similar to the one on the Apple Watch and provides a well-designed control mechanism.
    • The crown has haptic feedback while turning and can be used for various functions like zooming.
    • Eye contact simulation is crucial in creating a social connection and preventing injuries in virtual reality environments.
    • Apple has learned from mistakes made by other companies in VR development, such as Microsoft's HoloLens and Oculus Quest.
    • The Apple headset detects people entering your play space and temporarily stops the environment to show you their presence.
    • The device's visual window extends beyond your normal peripheral vision, giving a sense of infinity to the screen display.

    Camera Technology and Potential Applications

    44:06 - 51:29

    • The cameras in this device can see beyond what the human eye can see, capturing a wide range of surroundings.
    • The keynote presentation did not showcase all the features of the device, including its powerful camera technology.
    • The purpose of this device is to deliver immersive experiences, such as concerts or sports events, to people's homes.
    • The device allows users to embody themselves in someone else's perspective, like watching them cook dinner from their point of view.
    • This technology has the potential to be used with humanoid robots for remote control and interaction.
    • These devices are game changers for dimensional photography and recording sports events without screens.
    • AI capabilities are integrated into the device's camera, allowing real-time adjustments and stabilization.
    • Unity partnership enables generative AI characters in virtual worlds.
    • The demo unit showcased only a fraction of the potential apps and functionalities that will be available on this platform.

    Memory Storage and AI Capabilities

    51:05 - 58:03

    • Apple is creating a three-year process of baby steps to build new features.
    • Moore's Law will lead to increased compute power and generative NPCs.
    • Apple aims to create a social contract with users to protect their context in a hyper-local environment.
    • The journaling system will store and organize users' memories, photos, and videos.
    • AI can construct memories in a way that aligns with our preferences.
    • 360 cameras and immersive technology allow for reliving past memories from different perspectives.
    • Recording memories can help combat dementia and improve mental health.
    • AI can provide a positive perspective shift for individuals struggling with negative self-talk.
    • The ability to watch personal memories like in the movie Minority Report is becoming a reality.

    Impact of Apple's Entry into Spatial Computing

    57:46 - 1:04:58

    • The resistance to spatial computing technologies is high due to its intimate and wearable nature.
    • Resistance to AI is also prevalent, but eventually all technologies will converge into AI.
    • The rapid pace of technological advancements is transforming society.
    • Apple's entry into spatial computing will kick off a new wave of startups in the field.
    • Apple's moves have already attracted investor interest and funding for these startups.
    • Apple stores will be redesigned to accommodate spatial computing demos and experiences.
    • Lines are expected to form outside Apple stores for months after the release of spatial computing devices.
    • By around 2027-2030, most other devices may be abandoned in favor of glasses-based spatial computing devices.

    Future Challenges and Developments

    1:04:37 - 1:11:33

    • Apple plans to integrate computers into glasses by 2030, but improving battery technology and dealing with the heat generated by GPUs on the glasses is challenging.
    • Mark Zuckerberg is also working on a similar idea but relies on cloud computing for AI inferencing, while Apple wants to keep it close to the user's eyes or in their house using ultra wideband and mesh network technology.
    • Apple prioritizes privacy by keeping sensor data in the headset and not sending it to the cloud like Facebook does.
    • Apple has a significant advantage in AI power with its M1 chips that have neural units, which are currently underutilized.
    • Apple prefers using machine learning over large language models due to concerns about errors and hallucinations produced by such models.
    • In the future, Apple may introduce a new terminology for AI without the machine learning attribute.
    • The developer version of Apple's AR glasses is not rushed to market but represents a convergence of various technologies; however, a high-end consumer version is expected in about a year.

    Conclusion

    1:04:58 - 1:11:33

    • The current version of the product is just the beginning and there will be a new iteration in a year.
    • Developers are excited about the possibilities of building on this platform.
    • Apple has partnered with companies like Disney and Unity for their ecosystem.
    • No pictures or leaks of the device or user interface have been released, which is impressive.
    • The consumer version will be lighter and generate less heat, with more consumer-oriented experiences.
    • There may be two devices from Apple in the marketplace by next summer, with some feature enhancements or reductions.
    • The price of the entry-level version may drop by holiday next year.
    • The cost of building the device is not that expensive considering its parts breakdown.
    • The device is exquisitely engineered and should not be blended.
    1