Core Viewpoints
In the future, a lot of content is going to be created with AI tools. Some of the content is going to be creators using the tools to create new content. Some of it may be the content that's either created on the fly for you, or pulled together and synthesized through different things that are out there. (Mark Zuckerberg)
I dream of one day, you can almost imagine all of Facebook or Instagram being like a single AI model that is unified, with all these different content types and systems being together that actually have different objectives over different time frames. (Mark Zuckerberg)
Llama is going to pretty quickly evolve to go away on multiple time frames when you give it an intent. (Mark Zuckerberg)
Our vision is that I don't think that there's just going be like one AI model, but that we want to empower all the people who use our products to basically create AI agents for themselves. In the future, every business is going to have an AI agent that interfaces with their customers, just like every business has an email address, a website, and a social media account or several. (Mark Zuckerberg)
In the PC generation, the open ecosystem won. I am kind of hopeful that in the next generation of computing, we're going to return to a zone where the open ecosystem wins and is the leading one again. (Mark Zuckerberg)
Next-generation smart glasses: mixed reality, headsets, and smart glasses. (Mark Zuckerberg)
It's sort of incredible that we're living through a time where the entire computing stack is being reinvented. As for how we think about software, Andrej proposed the concepts of software 1.0 and software 2.0, and we're basically in software 3.0 now. (Jensen Huang)
We went the other way. Instead of building smaller and smaller devices, we made computers the size of warehouses. (Jensen Huang)
SIGGRAPH
July 29, 2024
-----
Jensen Huang
Ladies and gentlemen. I have a very special guest. I am so impressed by this person. Three reasons.
First of all, there are only a handful of entrepreneurs and founders that started a company that literally touched the lives of billions of people around the world as part of the social fabric and invented services. Our special guest built up a state-of-the-art computing company.
Secondly, very few entrepreneurs and founders founded the company and led it to over $1 trillion of value.
And last, a college dropout.
All three things are simultaneously true. Ladies and gentlemen, please help me welcome Mark Zuckerberg.
Mark, welcome to your first SIGGRAPH. As one of the pioneers of computing and a driver of modern computing, you are ideally suited to SIGGRAPH. So, Mark, sit down. It's great to have you here. Welcome. Thanks for flying down.
Mark Zuckerberg
Yeah. This will be fun.
Jensen Huang
This is SIGGRAPH. There's 90% PhDs. The thing that's really great about SIGGRAPH is that this is the show of computer graphics, image processing, artificial intelligence, and robotics combined. Some of the companies, over the years, have demonstrated and revealed amazing things here, like Disney, Pixar, Adobe, Epic Games, and, of course, NVIDIA. We've done a lot of work here.
This year we introduced 20 papers at the intersection of artificial intelligence and simulation. We're using artificial intelligence to help simulation be way larger scale, way faster. For example, in differentiable physics, we're using simulation to create synthetic data generation environments for artificial intelligence. These two areas are really coming together.
We're really proud of the work that we've done here. At Meta, you guys have done amazing AI work. One of the things that I find amusing is when the press writes about how Meta has jumped into AI this last couple of years, the work that the FAIR has done is as if ignored. We all use PyTorch that comes out of Meta. You have done groundbreaking work in computer vision, language models, and real-time translation.
My first question for you is, how do you see the advances of generative AI at Meta today, and how do you apply it to either enhance your operations or introduce new capabilities that you're offering?
Mark Zuckerberg
Meta has been at SIGGRAPH for eight years, but we're noobs compared to you guys. We have showed some of the early hand-tracking work for our VR and mixed reality headsets.
We've talked a bunch about the progress that we're making on Codec Avatars, the photorealistic avatars that we want to be able to drive from consumer headsets, which we're getting closer and closer to. So, I am pretty excited about that. And also, we've done a lot of work on display systems, including some of the future prototypes and research for getting the mixed reality headsets to be able to be really thin with pretty advanced optical stacks and display systems. We've typically shown these integrated systems here first.
I am excited to be here this year. This year is not just talking about the metaverse stuff, but also, all the AI pieces. As you said, we built the FAIR AI research center even before Reality Labs. We've been at this for a while.
All the stuff around generative AI is an interesting revolution. It's going to end up making all of the different products in interesting ways. You can look at the big product lines that we have already, especially the feed and recommendation systems on Instagram and Facebook. We've been on this journey where that's gone from just being about connecting with your friends, and the ranking was always important. This is because even when you were just following friends, if someone did something really important, like your cousin had a baby or something, you want that at the top. You'd be pretty angry at us if it was buried somewhere down in your feed. So the ranking was important.
Over the last few years, it's gotten to a point where more of that stuff is just different public content that's out there. The recommendation systems are super important because, now, instead of just a few hundred or thousand potential candidate posts from friends, there are millions of pieces of content, and that turns into a really interesting recommendation problem.
With generative AI, we're going to quickly move into the zone where the majority of the content that you see today on Instagram is not only the content recommended to you from the stuff that's out there in the world that matches your interests, whether or not you follow the people. I think in the future, a lot of this stuff is going to be created with these tools, too. Some of the content is going to be creators using the tools to create new content. Some of it may be the content that's either created on the fly for you, or pulled together and synthesized through different things that are out there.
That's just one example of how the core part of what we're doing is going to evolve. And it's been evolving for 20 years already.
Jensen Huang
Well, very few people realize that one of the largest computing systems the world has ever conceived of is a recommender system.
Mark Zuckerberg
It's a whole different path. It's not quite the kind of generative AI hotness that people talk about, but it's all the transformer architectures, a similar thing of just building up more and more general models.
Jensen Huang
Embedding unstructured data into features.
Mark Zuckerberg
One of the big things that drives quality improvements is used to be that you'd have a different model for each type of content. A recent example is, we had one model for ranking and recommending Reels and another model for ranking and recommending more long-form videos. And then take some product work to basically make it so that the system can display anything in line. But the more you create more general recommendation models that can span everything, it just gets better and better.
Part of it is like economics and liquidity of content and the broader of a pool that you can pull from. You're just not having these weird inefficiencies of pulling from different pools. But as the models get bigger and more general, that gets better and better.
I dream of one day, you can almost imagine all of Facebook or Instagram being like a single AI model that is unified. All these different content types and systems together that actually have different objectives over different time frames. Because some of it is just showing you what's the interesting content that you're going to want to see today, but some of it is helping you build out your network over the long term, like people you may know or accounts you might want to follow.
Jensen Huang
These multi-modal models tend to be much better at recognizing patterns, weak signals and such.
It's so interesting that AI has been so deep in your company. You've been building GPU infrastructure, running these large recommender systems for a long time. Now, the thing that's really cool about generative AI is these days when I use WhatsApp, I feel like I'm collaborating with WhatsApp.
Imagine I'm sitting here typing and it's generating the images as I'm going. I go back and I change my words. It's generating other images. So the one that old Chinese guy, enjoying a glass of whiskey at sundown with three dogs: a Golden Retriever, a Goldendoodle and a Bernese Mountain dog: it generates a pretty good-looking picture.
Mark Zuckerberg
We're getting there. Every month.
Jensen Huang
And then now you could actually load my picture in there and it'll actually be me.
Mark Zuckerberg
Yeah. That's as of last week.
Jensen Huang
Super excited about that.
Mark Zuckerberg
Now imagine me. I'm spending a lot of time with my daughters imagining them as mermaids and things over the last week. It's been lot of fun. But that's the other half of it. A lot of the gen AI stuff, on the one hand it's going to be this big upgrade for all of the workflows and products that we've had for a long time. But on the other hand, there's going to be all these completely new things that can now get created.
So Meta AI, the idea of having an AI assistant that can help you with different tasks, in our world is going to be very creatively oriented, like you're saying. But they're very general, so you don't need to just constrain it to that. It'll be able to answer any question.
Over time, when we move from like the Llama 3 class of models to Llama 4 and beyond, it's going to feel less like a chat bot where it's like you give it a prompt and it just responds. Then you give it a prompt and it responds, and it's back and forth. I think it's going to pretty quickly evolve to: you give it an intent and it actually can go away on multiple time frames. And it probably should acknowledge that you give it an intent up front. But some of the stuff I think will end up, you know it'll spin up compute jobs that take weeks or months or something and then just come back to you, like something happens in the world. That's going to be really powerful.
Jensen Huang
Today's AI is turn-based. You say something and it says something back to you. But obviously when we're giving a mission or giving a problem, we'll contemplate multiple options. Or maybe we come up with a tree of options, a decision tree, and we walk down the decision tree simulating in our mind: what are the different outcomes of each decision that we could potentially make. And so we're doing planning. So in the future AI's will do the same.
One of the things that I was super excited about when you talked about your vision of Creator AI. I just think that's a home run idea, frankly. Tell everybody about the Creator AI and AI Studio that's going to enable you to do that.
Mark Zuckerberg
I don't think that there's just going be like one AI model. Some of the other companies in the industry are building like one central agent and we'll have the Meta AI assistant that you can use. Our vision is that we want to empower all the people who use our products to basically create agents for themselves. So whether for many millions of creators that are on the platform or hundreds of millions of small businesses, we eventually want to be able to pull in all your content and very quickly stand up a business agent and be able to interact with your customers and do sales and customer support and all that.
The one that we're just starting to roll out more now is AI Studio. It basically is a set of tools that eventually is going to make it so that every creator can build an AI version of themselves, as an agent or an assistant that their community can interact with. There's a fundamental issue here where there's just not enough hours on the day. It's like if you're a creator, you want to engage more with your community, but you're constrained on time. Similarly, your community wants to engage with you, but it's tough. There's just limited time to do that. So the next best thing is allowing people to basically create these artifacts. It's an agent, but it's you train it on your material to represent you in the way that you want. I think it's a creative endeavor, like a piece of art or content that you're putting out there.
It's not engaging with creators themselves. But it'll be another interesting way, just like how creators put out content on these social systems, to be able to have agents that do that. Similarly, I think that there's going to be a thing where people basically create their own agents for all different kinds of uses. Some will be customized utility, or things that they're trying to get done that they want to fine tune and train an agent for, and some of them will be entertainment. And some of the things that people create are just funny and silly in different ways, or kind of have a funny attitude about things that we probably wouldn't build into Meta AI as an assistant, but I think people are pretty interested to see, and interact with.
And then one of the interesting use cases that we're seeing is people using these agents for support. This was a little bit surprising to me: one of the top use cases for Meta AI already is people basically using it to role play difficult social situations that they're going to be in. So whether it's a professional situation, it's like, I want to ask my manager: how do I get a promotion or raise? Or I'm having this fight with my friend, or I'm having this difficult situation with my girlfriend. And basically having a completely judgment-free zone where you can basically role play that and see how the conversation will go and get feedback on it. But a lot of people, they don't just want to interact with the same agent, whether it's Meta AI or ChatGPT or whatever it is that everyone else is using, they want to create their own thing. So that's roughly where we're going with AI Studio. But it's all part of this bigger view that we have, that there shouldn't just be one big AI that people interact with. We think that the world will be better and more interesting if there's a diversity of these different things.
Jensen Huang
I think it's so cool that if you're an artist and you have a style. You could take your style, all of your body of work. You could fine tune one of your models. And now this becomes an AI model that you can come and you could prompt it. You could ask me to create something along the lines of the art style that I have, and you might even give me a piece of art as a drawing, a sketch, as an inspiration. And I can generate something for you. And you come to my bot or my AI for that. It could be, every single restaurant, every single website will probably in the future have these AIs.
Mark Zuckerberg
Yeah I mean, I kind of think that in the future, just like every business has an email address and a website and a social media account or several, I think in the future, every business is going to have an AI agent that interfaces with their customers. And some of these things, I think have been pretty hard to do historically. If you think about any company, it's like you probably have customer support as just a separate organization from sales, and that's not really how you'd want it to work as CEO.
When you're CEO, you have to do all this stuff. But, I mean, then when you build the abstraction in your organization, a lot of times, like in general, the organizations are separate because they're kind of optimized for different things. But I think, like the platonic ideal of this would be that it's kind of one thing, right? As a customer, you don't really care. You don't want to have a different route when you're trying to buy something versus if you're having an issue with something that you bought, you just want to have a place that you can go and get your questions answered and be able to engage with the business in different ways. And I think that that applies for creators, too.
Jensen Huang
And all that engagement with your customers, especially their complaints, is going to make your company better.
The fact that is all engaging with this AI is going to capture the institutional knowledge and all of that can go into analytics which improves the AI and so on, so forth.
Mark Zuckerberg
So the business version of this has a little more integration and we're still in a pretty early alpha with that. But the AI Studio making it so that people can kind of create their UGC agents and different things, and getting started on this flywheel of having creators create them. I'm pretty excited about that.
Jensen Huang
Can I use AI Studio to fine tune with my collection of images?
Mark Zuckerberg
Yeah, we're going to get there.
Jensen Huang
And then could I load it with all the things that I've written and use it as my RAG?
Mark Zuckerberg
Yeah. Basically.
Jensen Huang
Okay. And then every time I come back to it, it loads up its memory again, so it remembers where it left off last time. And we carry on our conversation as, though nothing ever happened.
Mark Zuckerberg
Yeah, and like any product, it'll get better over time. The tools for training, it will get better. It's not just about what you want it to say. I think generally creators and businesses have topics that they want to stay away from too. So it's just getting better at all this stuff. I think the platonic version of this is not just text, right?
You almost want to just be able to have an interaction with some of the codec avatar work that we're doing over time. You want to basically be able to have almost a video chat with the agent. And I think we'll get there over time. I don't think that this stuff is that far off, but the flywheel is spinning really quickly, so it's exciting. There is a lot of new stuff to build. And I think even if the progress on the foundation models kind of stopped now, which I don't think it will, I think we'd have like five years of product innovation for the industry to basically figure out how to most effectively use all the stuff that's gotten built so far. But I actually just think the kind of foundation models and the progress on the fundamental research is accelerating. So, it's a pretty wild time.
Jensen Huang
You know, CEOs, we're delicate flowers. We need a lot of back. Yeah.
Mark Zuckerberg
We're pretty grizzled at this point. I think we're the two kinds of longest standing founders in the industry, right?
Jensen Huang
It's true.
Mark Zuckerberg
And your hair has gotten gray. Mine has just gotten longer.
Jensen Huang
Mine's gotten gray. Yours has gone curly, what's up?
Mark Zuckerberg
It was always curly. That's why I kept it short.
Jensen Huang
If I'd known it was going to take so long to succeed, I would have dropped out of college, just like you. Get a head start.
Mark Zuckerberg
Well, there's a good difference between our personalities.
Jensen Huang
You got a 12-year head start. That's pretty good.
Mark Zuckerberg
You're doing pretty well.
Jensen Huang
I'm going to be able to carry on. Let me just put it that way.
The thing that I love about your vision that, everybody can have an AI, that every business can have an AI. In our company, I want every engineer and every software developer to have an AI, or many AIs. The thing that I love about your vision is that you also believe that everybody and every company should be able to make their own AI. So you actually open-sourced, when you open-sourced Llama I thought that was great. Llama 2.1, by the way, I thought Llama 2 was probably the biggest event in AI last year. And the reason for that.
Mark Zuckerberg
I thought it was the H100
Jensen Huang
That's a chicken or the egg question. Which came first?
Mark Zuckerberg
The H100. Well, Llama 2, it was actually not the H100.
Jensen Huang
Yeah, it was A100. Thank you.
The reason why I said it was the biggest event was because when that came out, it activated every company, every enterprise and every industry. All of a sudden, every health care company was building AI. Every company was building AI. Every large company, small company, and startup were building AIs. It made it possible for every researcher to be able to reengage AI again, because they have a starting point to do something with. Now, 3.1 is out and just so you know, we work together to deploy 3.1. We're taking it out to the world's enterprises. The excitement is just off the charts. I think it's going to enable all kinds of applications.
But tell me about your open-source philosophy. Where did that come from? You open-sourced PyTorch. It is now the framework by which AI is done. Now you've open-sourced Llama 3.1. There's a whole ecosystem built around it. I think it's terrific. But where did that all come from?
Mark Zuckerberg
So there's a bunch of history on a lot of this. I mean, we've done a lot of open-source work over time. I think part of it, just bluntly, is we got started after some of the other tech companies in building out stuff like the distributed computing infrastructure and the data centers. Because of that, by the time that we built that stuff, it wasn't a competitive advantage. We're like, all right, we might as well make this open and then we'll benefit from the ecosystem around that. So we had a bunch of projects like that. I think the biggest one was probably Open Compute, where we took our server designs, the network designs, and eventually the data center designs and published all of that. By having that become somewhat of an industry standard, all the supply chains basically got organized around it, which had this benefit of saving money for everyone. So by making it public, and open, we basically have saved billions of dollars from doing that.
Jensen Huang
Open Compute was also what made it possible for NVIDIA HGXs, that we designed for one data center, all of a sudden, works in every data center.
Mark Zuckerberg
So that was an awesome experience. And then, we've done it with a bunch of our infrastructure tools, things like React, PyTorch. So I'd say by the time that Llama came around, we were sort of positively predisposed towards doing this, for AI models specifically. I guess there's a few ways that I look at this. I mean, it's been really fun building stuff over the last 20 years at the company.
One of the things that has been sort of the most difficult is having to navigate the fact that we ship our apps through our competitor's mobile platforms. So in the one hand, the mobile platforms have been this huge boon to the industry. That's been awesome. On the other hand, having to deliver your products through your competitors is challenging, right? You know, I grew up in a time where, the first version of Facebook was on the web and that was open. And then, as a transition to mobile, the plus side of that was, now everyone has a computer in their pocket. So that's great. The downside is, okay, we're a lot more restricted in what we can do.
So, when you look at these generations of computing there's this big recency bias where everyone just looks at mobile and thinks, okay, because the closed ecosystem, because Apple basically won and set the terms of that. And like yeah, I know that there's more Android phones out there technically, but Apple basically has the whole market and all the profits. Basically Android is kind of following Apple in terms of the development of it. So I think Apple pretty clearly won this generation. But it's not always like that where if you go back a generation. Apple was doing their kind of closed thing. But Microsoft, which as you know, it obviously wasn't like this perfectly open company, but, you know, compared to Apple with Windows running on all the different OEMs and different hardware, it was a much more open ecosystem and Windows was the leading ecosystem.
Basically in the kind of PC generation of things, the open ecosystem won. I am kind of hopeful that in the next generation of computing, we're going to return to a zone where the open ecosystem wins and is the leading one again. There will always be a closed one and an open one. I think that there's reasons to do both. There are benefits to both. I'm not like a zealot on this. I mean, we do closed source stuff and not everything that we publish is open. But I think in general for the computing platforms that the whole industry is building on, there's a lot of value for that if the software especially is open. So that's really shaped my philosophy on this.
And, for both AI with Llama and with the work that we're doing in AR and VR, where we are basically making the Horizon OS that we're building for mixed reality, an open operating system in the sense of, what Android or Windows was, and basically making it so that we're going to be able to work with lots of different hardware companies to make all different kinds of devices. We basically just want to return the ecosystem to that level where that's going to be the open one. And I'm pretty optimistic that in the next generation, the open ones are going to win. For us specifically, I just want to make sure that we have access to, I mean, this is sort of selfish, but, you know, after building this company for a while, one of my things for the next 10 or 15 years is like, I just want to make sure that we can build the fundamental technology that we're going to be building social experiences on, because there have just been too many things that I've tried to build and then have just been told, nah, you can't really build that by the platform provider, that like, we're going to go build all the way down and, and make sure that.
Jensen Huang
I think it's a great world where there are people who are dedicated to build the best possible AIs, however they build it, and they offer it to the world, as a service. But if you want to build your own AI, you could still also build your own AI. So the ability to use an AI. There's a lot of stuff, I prefer not to make this jacket myself. I prefer to have this jacket made for me. So the fact that leather could be open source is not a useful concept for me, but I think the idea that you could have great services, incredible services as well as open service. Then we basically have the entire spectrum.
But the thing that you did with 3.1 that was really great was you have 405B, you have 70B, you have 8B you could, use it for synthetic data generation, use the larger models to essentially teach the smaller models. And although the larger models will be more general, it's less brittle, you could still build a smaller model that fits in, whatever operating domain or operating costs that you would like to have. Llama Guard for guard railing. Fantastic. And so now and the way that you built the model, it's built in a transparent way. You've got a world class safety team. World class ethics team. You could build it in such a way that everybody knows it's built properly. And so I really love that part of it.
Mark Zuckerberg
Just to finish the thought from before I got sidetracked there for a detour. I do think there's this alignment where we're building it because we want the thing to exist, and we want to not get cut off from some closed model. But this isn't just like a piece of software that you can build. You need an ecosystem around it. And so it's almost like it wouldn't even work that well if we didn't open source it. It's not we're not doing this because we're kind of altruistic people. Even though I think that this is going to be helpful for the ecosystem, and we're doing it because we think that this is going to make the thing that we're building the best by having a robust ecosystem.
Jensen Huang
Look how many people contributed to PyTorch ecosystem. Mountains of engineering?
Mark Zuckerberg
Yeah.
Jensen Huang
I mean, NVIDIA alone, we probably have a couple of hundred people just dedicated to making PyTorch better and scalable and more performant and so on and so forth.
Mark Zuckerberg
It's also just when something becomes something of an industry standard, other folks do work around it. So like all of the silicon in the systems will end up being optimized to run this thing really well, which will benefit everyone, but it will also work well with the system that we're building. And that's, I think, just one example of how this ends up being really effective.
I think that the open-source strategy is going to be a good one as a business strategy. I think people still don't quite get it.
Jensen Huang
We love it so much. We built an ecosystem around it. We build this thing called AI Foundry.
Mark Zuckerberg
You guys have been awesome. Every time we're shipping something, you guys are the first to release this and optimize it and make it work. I appreciate that.
Jensen Huang
We have good engineers.
Mark Zuckerberg
Well you always just jump on this stuff quickly too.
Jensen Huang
I'm a senior citizen, but I'm agile. That's what CEOs have to do.
I recognize an important thing. I think that Llama is genuinely important. We built this concept called AI Foundry around it so that we can help everybody build, take a lot of people, they have a desire to build AI. And it's very important for them to own the AI because once they put that into their data flywheel, that's how their company's institutional knowledge is encoded and embedded into an AI. So they can't afford to have the AI flywheel, the data flywheel that experience flywheel somewhere else.
Open source allows them to do that. But they don't really know how to turn this whole thing into an AI and so we created this thing called AI Foundry. We provide the tooling, we provide the expertise, Llama technology, we have the ability to help them turn this whole thing, into an AI service. When we're done with that, they take it, they own it. The output of it is what we call a NIM. And this NIM, this neuro micro NVIDIA Inference Microservice, they just download it, they take it, they run it anywhere they like, including on-prem. And we have a whole ecosystem of partners, from OEMs that can run the NIMs to, GSIs like Accenture that we train and work with to create Llama-based NIMs and pipelines and now we're off helping enterprises all over the world do this. It's really quite an exciting thing. It's really all triggered off of the Llama open-sourcing.
Mark Zuckerberg
I think especially the ability to help people distil their own models from the big model is going to be a really valuable new thing because there's this, just like we talked about on the product side, how at least I don't think that there's going to like one major AI agent that everyone talks to. At the same level, I don't think that there's going to necessarily be one model that everyone uses.
Jensen Huang
We have a chip AI, chip design AI, we have a software coding AI, and our software coding AI understands USD because we code in USD for Omniverse stuff. We have software AI that understands Verilog, our Verilog. We have software AI that understands our bugs database and knows how to help us triage bugs and sends it to the right engineers. And so each one of these AIs is fine tuned off of Llama and so we fine tune them, we guardrail them, if we have an AI design, for chip design, we're not interested in asking it about politics, and religion and things like that. So we guardrail it. So I think every company will essentially have for every single function that they have, they will likely have AIs that are built for that. And they need help to do that.
Mark Zuckerberg
I think it's one of the big questions is going to be in the future, to what extent are people just using the kind of the bigger, more sophisticated models versus just training their own models for the uses that they have? And at least I would bet that they're going to be just a vast proliferation of different models.
Jensen Huang
We use the largest ones. And the reason for that is because our engineers, their time is so valuable. Right now we're getting 405B, optimized for performance. And 405B doesn't fit in any GPU, no matter how big. And so that's why the NVLink performance is so important. We have every one of our GPUs connected by this, non-blocking switch called NVLink switch. And in the HGX for example, there are two of those switches and we make it possible for all these, all these GPUs to work and, and run the 405Bs really performant. The reason why we do it is because the engineers' times are so valuable to us. We want to use the best possible model, the fact that it's cost effective by a few pennies, who cares? And so we just want to make sure that the best quality of results is presented to them.
Mark Zuckerberg
The 405 I think is about half the cost to inference of the GPT 4 model. So at that level, it's already pretty good. But I think people are doing stuff on devices or want smaller models. They're just going to distil it down. So that's like a whole different set of services.
Jensen Huang
That AI is running, and let's pretend for a second that we're hiring that AI, that AI for chip design is probably $10 an hour. If you're using it constantly and you're sharing that AI across a whole bunch of engineers. So each engineer probably has an AI that's sitting with them. And it doesn't cost very much. And we pay the engineers a lot of money. And so to us, a few dollars an hour, amplifies the capabilities of somebody that's really valuable.
Let's talk about the next wave, one of the things that I really love about the work that you guys do, computer vision, one of the models that we use a lot internally, is Segment Everything, and that we're now training AI models on video so that we can understand the world model. Our use case is for robotics and industrial digitalization and connecting these AI models into Omniverse so that we can model and represent the physical world better, have robots that operate in these Omniverse worlds better. Your application, the Ray-Ban Meta glass, your vision for bringing AI into the virtual world, is really interesting. Tell us about that.
Mark Zuckerberg
Yeah. Well, okay, a lot to unpack in there. The Segment Anything model that you're talking about—we're actually presenting the next version of that here at SIGGRAPH, that is Segment Anything 2. Oh, here we go. It works in video now as well. I think these are actually cattle from my ranch in Kauai.
Jensen Huang
By the way, they are called Delicious Mark's Cows. Next time we do... Mark came over to my house and we made Philly cheese steak together. Next time you're bringing the cow.
Mark Zuckerberg
I'd say you did. I was more of a sous-chef. But, it was really good. And then at the end of the night that you were like, "hey, so you ate enough, right?" And I was like: "I don't know. I could eat another one." You're like: "Really?"
Jensen Huang
You know, usually when you say something to your guest: "Did you get enough to eat?", usually your guest says: "Oh yeah, I'm fine."
Mark Zuckerberg
"Make me another cheese steak, Jensen."
Jensen Huang
Just to let you know how OCD he is. I turn around. I'm prepping the cheese steak and I said: "Mark, cut the tomatoes." And I handed him a knife.
Mark Zuckerberg
Yeah, I'm a precision cutter.
Jensen Huang
And so he cuts the tomatoes. Every single one of them is perfectly to the exact millimeter. But the really interesting thing is, I was expecting all the tomatoes to be sliced and kind of stacked up, kind of like a deck of cards. But when I turned around, he said he needed another plate. And the reason for that was because all of the tomatoes he cut, none of them touched each other. Once he separates one slice of tomato from the other tomato, they shall not touch again.
Mark Zuckerberg
Yeah. Look, man, if you wanted them to touch, you needed to tell me that. That's why I'm just a sous-chef, Okay?
Jensen Huang
That's why he needs an AI that doesn't judge.
This (Segment Anything 2) is super cool, Okay? It's recognizing the cows' tracks. It's recognizing and tracking the cows.
Mark Zuckerberg
Yeah. So, a lot of fun effects will be able to be made with this. And because it'll be open a lot of more serious applications across the industry, too. Scientists use this stuff to study like coral reefs, natural habitats, kind of evolution of landscapes, and things like that. But, being able to do this in video and having it be a zero-shot, and being able to kind of interact with it and tell it what you want to track is—It's pretty cool research.
Jensen Huang
For example, you have a warehouse and they've got a whole bunch of cameras and the warehouse AI is watching everything that's going on. And let's say, a stack of boxes fell, or somebody spilled water on the ground, or, whatever accident is about to happen. The AI recognizes it, generates the text, and sends it to somebody. And help will come along the way. That's one way of using it, instead of recording everything. If there's an accident, instead of recording every nanosecond of a video and then going back and retrieving that moment, it just records the important stuff because it knows what it's looking at. So having a video understanding model, a video language model, is really, really powerful for all of these interesting applications.
Now what else are you guys going to work on beyond Ray-Ban. Talk to me about it.
Mark Zuckerberg
There are all the smart glasses. When we think about the next computing platform, we kind of break it down into mixed reality, the headsets, and the smart glasses. And for the smart glasses, I think it's easier for people to wrap their head around that and wear it because pretty much everyone who's wearing a pair of glasses today will end up getting upgraded to smart glasses. And that's like more than a billion people in the world. So that's going to be a pretty big thing.
The VR MR headsets, I think, some people find it interesting for gaming or different uses, some don't yet. Yet my view is that they're going to be both in the world. I think the smart glasses are going to be sort of the mobile phone, kind of always-on version of the next computing platform. And the mixed reality headsets are going to be more like your workstation or your game console where you're sitting down for a more immersive session and you want access to more computing power.
I mean, look, the glasses have a very small form factor. There are going to be a lot of constraints on that. Just like you can't do the same level of computing on a phone.
Jensen Huang
It came at exactly the time when all of these breakthroughs in generative AI happened.
Mark Zuckerberg
Yeah. So for smart glasses, we've been going at the problem from two different directions. On the one hand, we've been building what we think is sort of the technology that you need for the kind of ideal holographic AR glasses. We're doing all the custom silicon work and all the custom display stack work. Like all the stuff that you need to do to make that work—in their glasses, right? It's not a headset. It's not like a VR or MR headset. They look like glasses. But, they're still quite a bit far off from the glasses that you're wearing now. I mean, those are very thin. But even on the Ray-Bans that we make, you couldn't quite fit all the tech that you need into that yet for kind of full holographic AR.
We're getting close. And over the next few years, I think we'll basically get closer. It'll still be pretty expensive, but I think that will start to be a product. The other angle that we've come at this is... Let's start with good-looking glasses by partnering with the best glasses maker in the world, Essilor Luxottica. They basically have all the big brands that you use. You know, it's Ray-Ban or Oakley or Oliver Peoples or just like a handful of others. It's kind of all Essilor Luxottica.
Jensen Huang
The NVIDIA of glasses.
Mark Zuckerberg
I think they would probably like that analogy, I mean, who wouldn't at this point?
So we've been working with them on the Ray-Bans. We're on the second generation and the goal there has been that let's constrain the form factor to just something that looks great. And within that, let's put in as much technology as we can, understanding that we're not going to get to the kind of ideal of what we want to fit into it technically. But in the end, it'll be like great-looking glasses. And at this point, we have camera sensors, so you can take photos and videos. You can actually livestream to Instagram. You can take video calls on WhatsApp and stream to the other person what you're seeing.
It has a microphone and speakers. I mean, the speaker is actually really, really good. It's open ear so a lot of people find it more comfortable than earbuds. You can listen to music and it's just like this private experience. That's pretty neat. People love that. You take phone calls on it. But then it just turned out that the sensor package was exactly what you needed to be able to talk to AI, too.
So that was sort of an accident. If you'd asked me five years ago: "Were we going to get holographic AR before AI?", I would have said: "Yeah, probably." Right? I mean, it just seems like kind of the graphics progression and the display progression on all the virtual and mixed reality stuff are building up the new display stack. We're just making continual progress towards that.
And then this breakthrough happened with LLMs. It turned out that we have sort of really high-quality AI now and getting better at a really fast rate before you have holographic AR. It's sort of this inversion that I didn't really expect. I mean, we're fortunately well positioned because we were working on all these different products. But I think what you're going to end up with is just a whole series of different potential glasses products at different price points with different levels of technology in them.
Based on what we're seeing now with the Ray-Ban Metas, I would guess that display-less AI glasses at like a US$300 price point are going to be a really big product that tens of millions of people or hundreds of millions of people eventually are going to have.
Jensen Huang
So, you're going to have super interactive AI that you're talking to. You have visual language understanding and real-time translation that you just showed. You could talk to me in one language, and I hear in another language.
Mark Zuckerberg
Then the display is obviously going to be great too, but it's going to add a little bit of weight to the glasses and it's going to make them more expensive. So, I think there will be a lot of people who want the kind of full holographic display. But there are also going to be a lot of people who want something that eventually is going to be like really thin glasses.
Jensen Huang
Well, for industrial applications and some work applications, we need that.
Mark Zuckerberg
I think, for consumer stuff too.
Jensen Huang
You think so?
Mark Zuckerberg
Yeah. I was thinking about this a lot during the Covid when everyone kind of went remote for a bit. It's like you're spending all this time on Zoom. That's okay. This is like it's great that we have this, but in the future we're like, not that many years away from being able to have a virtual meeting where I'm not here physically. It's just my hologram. It just feels like we're there and we're physically present. We can work on something and collaborate on something together. But I think this is going to be especially important with AI.
Jensen Huang
With that application, I could live with a device that I'm not wearing all the time.
Mark Zuckerberg
Oh yeah. But I think we're going to get to the point where it actually is. Within glasses, there are like thinner frames, thicker frames, and all these styles. But I think we're a while away from having full holographic glasses in the form factor of your glasses, but I think having it in a pair of stylish, kind of chunkier framed glasses is not that far off.
Jensen Huang
Sunglasses are of face size these days. I could see that.
Mark Zuckerberg
Yeah. That's a very helpful style.
Jensen Huang
Yeah, sure. That's very helpful.
Mark Zuckerberg
It's like I'm trying to make my way into becoming a style influencer. So, I can influence this before the glasses come to the market.
Jensen Huang
Well, I can see you attempting it. How's your style influencing working out for you?
Mark Zuckerberg
It's early.
I feel like if a big part of the future of the business is going to be building stylish glasses that people wear, this is something I should probably start paying a little more attention to. We're going to have to retire the version of me that wore the same thing every day. But that's the thing about glasses, too. I think, unlike even the watch or phones, people really do not want to all look the same. Right? I do think that it's a platform that is going to lend itself. It's not like everyone is going to want to put the one pair of glasses whoever else designs. I don't think that's going to fly for this.
Jensen Huang
It's sort of incredible that we're living through a time where the entire computing stack is being reinvented. As for how we think about software, Andrej proposed the concepts of software 1.0 and software 2.0, and we're basically in software 3.0 now. The way we compute evolves from general purpose computing to this generative neural network processing way of doing computing. The capabilities and the applications we could develop now are unthinkable in the past. And, this technology, generative AI, I don't remember another technology that in such a fast rate, influenced consumers, enterprise, industries, and science. And to be able to, to cut across all these different fields of science from climate tech to biotech and physical sciences. In every single field that we encountered, generative AI is right in the middle of that fundamental transition.
In addition to that, Generative AI is going to make a profound impact on society, the things that you're talking about. You know, the products that we're making. And one of the things that I'm super excited about, and somebody asked me earlier, is there going to be a Jensen AI? Well, that's exactly the creative AI you were talking about, where we just build our own AIs, I load it up with all of the things that I've written, and I fine tune it with the way I answer questions. Hopefully, with the accumulation of use over time, it becomes a really great assistant and companion for a whole lot of people who just want to ask questions or, bounce ideas off. And it'll be the version of Jensen that's not judgmental, as you were saying earlier. You're not afraid of being judged, so you could come and interact with it all the time. I think that those are really incredible things.
We write a lot of things all the time. How incredible it is just to give it three or four topics. Now, these are the basic themes of what I want to write about and write in my voice, and just use that as a starting point. So, there are just so many things that we can do now. It's really terrific working with you. I know that it's not easy building a company, and you pivoted yours from desktop to mobile, to VR, to AI, all these devices. It's really extraordinary to watch. NVIDIA's pivoted many times ourselves, and I know exactly how hard it is doing that. Both of us have gotten kicked in our teeth a lot, plenty over the years. But that's what it takes to be a pioneer and to innovate. So, it's really great watching you.
Mark Zuckerberg
And likewise, I'm not sure if it's a pivot if you keep doing the thing you were doing before. But, as well, you add to it. I mean there are more chapters to all of this. It's been fun watching the journey that you guys have been on. We went through this period when everyone was like. Everything is going to kind of move to these devices and, it's just going to get super kind of cheap compute. And you guys just kept on plugging away at this, saying "no, actually you're going to want these big systems that can parallel".
Jensen Huang
We went the other way. Instead of building smaller and smaller devices, we made computers the size of warehouses.
Mark Zuckerberg
A little unfashionable.
Jensen Huang
Super unfashionable. But now, it's cool. We started building a graphics chip, a GPU, and now when you're deploying a GPU, you still call it Hopper H100. But so you guys know, when Zuck calls it H100, his data center of H100s, I think you're coming up on 600,000.
Mark Zuckerberg
We're good customers. That's how you get the Jensen Q&A at SIGGRAPH.
Jensen Huang
Wow. Hang on. I was getting the Mark Zuckerberg Q&A. You were my guest.
Mark Zuckerberg
You just called me one day. You're like, hey, you know, in like a couple of weeks, we're doing this thing at SIGGRAPH. I'm like, yeah, I don't think I'm doing anything that day. I'll fly to Denver. It sounds fun.
Jensen Huang
Exactly. I'm not doing anything that afternoon, and you just showed up. That's just incredible. These systems that you guys build are giant systems, incredibly hard to orchestrate and incredibly hard to run. You said that, you got into the GPU journey later than the most. But you're operating larger than just about anybody, and it's incredible to watch.
Congratulations on everything that you've done. You are quite the style icon now. Check out this guy.
Mark Zuckerberg
Early stage, working on it.
---End---
Reposted from Andy730
Source video: https://www.youtube.com/watch?v=w-cmMcMZoZ4
