Luma AI with Barkley Dai and Karan Ganesan
AIgraphicstext-to-3Dphotorealistic environmentsVR
Luma AI develops cutting-edge AI and graphics technologies, including a text-to-3D tool and a tool for creating photorealistic environments from photos. Karan Ghansan and Barkley Dai from Luma AI discuss their backgrounds in VR, AR, generative AI, and their roles at the company. The founder of Luma AI, Amit, previously worked at Apple on camera research before starting the company in 2021 to focus on Neural Radiance Fields technology. Gitpod offers cloud development environments to address issues with dev environments breaking and security challenges in organizations. Neural Radiance Fields (NERFs) are explained as a 3D representation with five parameters for querying outputs from different angles. Gaussian splats are an explicit representation compared to NERFs' implicit representation, using spheres or shapes to query color from different angles. NERFs and Gaussian splats originated from academia, specifically UC Berkeley, with Matt Tarnak being involved in the research. Genie technology generates objects from scratch based on user input by starting from noise images and iterating towards the desired output. Genie technology is related to Gaussian splats but converts them into traditional 3D formats for user presentation. The Genie product was developed to make generating 3D objects easier and quicker compared to the company's capture product. Genie allows users to create 3D objects by simply typing text, making it more accessible and less labor-intensive than using a capture product. Genie evolved from a previous product called Imagine, which took longer to generate 3D models, highlighting the advancements in software architecture and computing power. Genie has applications beyond gaming, including e-commerce, VFX, and creating imaginary assets for sandbox games like Roblox. The future vision for Genie includes creating larger scenes with interactive elements and potentially serving as an easier game engine for users to turn their ideas into interactive experiences. Applications of NERFs and Gaussian splats include robotics for unsupervised learning and autonomous driving cars. Digital twins in industry sectors are another use case for NERFs and Gaussian plots, providing context for engineers working on big machines. Capture has attracted the most community attention due to its ability to generate 3D objects from text, leading to natural social media engagement. Users have discovered the ability to simulate drone views by recreating scenes with reshoot in Nerf, sparking a trend of showcasing drone views on social media. Luma AI app features hand-picked captures on its profile page to inspire users. Most growth comes from social media collaborations where users showcase their captures. Genie is a functional product for vertical industries like gaming, advertisement, and e-commerce. Genie's current limitation is creating 3D models that may not provide a complete consumer experience. Future plans for Genie aim to enable the creation of holistic consumer experiences from 3D models. Luma AI is working towards achieving visual creation and multimodality in AI to turn imagination into reality. The company aims to create experiences where 3D characters can move and talk on both spatial and temporal scales. Listeners can learn more about Luma AI at lumalabs.ai or by searching for Luma AI on app stores.