Lex Fridman - 2020-02-05
Jim Keller is a legendary microprocessor engineer, having worked at AMD, Apple, Tesla, and now Intel. He's known for his work on the AMD K7, K8, K12 and Zen microarchitectures, Apple A4, A5 processors, and co-author of the specifications for the x86-64 instruction set and HyperTransport interconnect. This episode is presented by Cash App. Download it & use code "LexPodcast": Cash App (App Store): https://apple.co/2sPrUHe Cash App (Google Play): https://bit.ly/2MlvP5w PODCAST INFO: Podcast website: https://lexfridman.com/podcast Apple Podcasts: https://apple.co/2lwqZIr Spotify: https://spoti.fi/2nEwCF8 RSS: https://lexfridman.com/feed/podcast/ Full episodes playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOdP_8GztsuKi9nrraNbKKp4 Clips playlist: https://www.youtube.com/playlist?list=PLrAXtmErZgOeciFP3CBCIEElOJeitOr41 OUTLINE: 0:00 - Introduction 2:12 - Difference between a computer and a human brain 3:43 - Computer abstraction layers and parallelism 17:53 - If you run a program multiple times, do you always get the same answer? 20:43 - Building computers and teams of people 22:41 - Start from scratch every 5 years 30:05 - Moore's law is not dead 55:47 - Is superintelligence the next layer of abstraction? 1:00:02 - Is the universe a computer? 1:03:00 - Ray Kurzweil and exponential improvement in technology 1:04:33 - Elon Musk and Tesla Autopilot 1:20:51 - Lessons from working with Elon Musk 1:28:33 - Existential threats from AI 1:32:38 - Happiness and the meaning of life CONNECT: - Subscribe to this YouTube channel - Twitter: https://twitter.com/lexfridman - LinkedIn: https://www.linkedin.com/in/lexfridman - Facebook: https://www.facebook.com/LexFridmanPage - Instagram: https://www.instagram.com/lexfridman - Medium: https://medium.com/@lexfridman - Support on Patreon: https://www.patreon.com/lexfridman
Lex: question
Jim: answer
Lex: “you avoided the question”
Jim: “no, your question is wrong”
Savage 😂😂
Genius😂🤣
Minute ?
@erick sanchez 16:37
He didn’t actually say that.
Jim's super cool and also very respected in the industry. He was a big inspiration for me early on in my career. I was a fairly junior CPU performance engineer at AMD when Jim joined AMD in 2012 (he'd worked at AMD earlier in his career too). I worked on the Zen program, which was being led by Jim. I saw him at my gym one morning, deadlifting 275 lbs (or maybe more) and I went up to him and introduced myself. He was super friendly and continued to be so whenever I'd run into him. We'd exchange our personal bests in lifting. IIRC, he was in his mid 50's then. Given the similarity in our backgrounds (CPU design) and his professional achievements and his amazing discipline, it was a no-brainer for me to aspire to be like him.
He was at AMD from age 53 to 57.
Lol, I was like man this guy is really cool. But this just made him even better. Thanks for the story dude!
thank you for sharing this. i really enjoy listening to him. i make a study out of people who speak so calmly and clearly. I speak excitedly and wildly.
@stincity in response to : "I hope you continue to push yourself and do some great things like he has. You can do it." It's true. Like Jim said, "you have to be passionate about something" That will push you to solving some of the mysteries of things. That will help you to evolve the industry, but you're doing it for you! For your understanding and development.
@Sommyarup Roy It's a common name. I have that name, I changed it to prude, crude prude.
I love how Lex forces his guests to define technical vocabulary whenever they use one. I was going to skip this podcast since I don't know much about hardware, but I understood most of it thanks to Lex's questions. Keep it up!
@William Wood i think lex was making a point that designing a computer to drive a car in the real world is a harder task than designing a computer to assemble a vehicle.
@ed case I actually think building something is way harder then driving. Handicap and old senile people drive every single day. They wouldn't make it 5 minutes in a factory at all!!! They would get killed
@ed case My grandmother drives every single day and she can't touch a tennis ball tossed across the kitchen table😅😅😅😅
@ed case People are slow and stupid always living almost a whole 2nd behind reality only reacting to things. We never Preemptively react to anything not in real Time. Our Intelligence is extremely overrated
lex does that because lex had NO CLUE what those terms are which makes it sooo annoying that he always plays it off as if he knew all along and yet even after having term explained lex just doesn’t grasp it.
IMO, this is definitely one of the best interviews by Lex. I found Jim Keller to be a version of Elon Musk with improved communication skills and more grounded/effective approach to projects. Thank you Jim for the insights and Lex for facilitating the interview, it was truly fascinating.
I still find it... unimaginable even with his communication skills to work with marketing... I dont know are they smart or not in tech, IT, but it sure is challenge.
Oh my god Silicon Ronin himself.
Silicon Ronin has no home. He goes from impossible task to next impossible task and builds his miracles.
Jim is a man of focus, commitment, sheer will... something you know very little about. I once saw him design a FPGA in a bar... with a pencil, with a fucking pencil.
Then suddenly one day he asked to leave. So I made a deal with him. I gave him an impossible task. A job no one could have pulled off.
To make AMD competitive again and he succeeded . The architecture choices he made that day laid the foundation of what we are now.
@Joe Duke Well, machines can get buggy on occasion, and that's about as much chance you have to convince an organized stooge no to harm you.
So he is the John Wick of microprocessor engineering?
😂😂😂😂😂😂😂 legendary comment.... wow
@Freek Hoekstra @Freek Hoekstra Sounds like Jim is satisfied his work was done at Tesla. And he is confident and enthusiastic about Tesla's technology. Or as haters call it. A Fan Boy.
THIS MADE ME LAUGH!!!....
These Lexures are nothing short of amazing. Don’t change anything. Your voice is yours.
Lexures lol
Sam Stone nice coinage lol
New video is out. I've never heard of the guest. His field is microprocessors, which I've never thought of as something interesting.
Turns out to be one of the wisest humans I've listened to and possibly the best hour and a half of listening experience I've had in my life. Jim is incredibly insightful about technology, science, and just the human condition in general. Thanks Lex for making stuff like this possible.
This is a man at the cutting edge of his field.
@Robert idonotsharemyfullname This is dumb. People are not just "gaining more intelligence" intelligence has pretty much been the same since however long. But id assume if there is ANY change its just all the stimuli may be having a small effect on pattern recognition, etc. also possibly people eating healthier, that's all. uh, as i said, there is not much of an "intuition advantage" with regards to this.. its machines and programmed algorithms.. what are you even saying. also, your statement could have been said 5 years ago about another 10 year old. yea, and i still don't know why i have to tell you a 10 year old isnt goin to surpass him in 5 years.
@Robert idonotsharemyfullname The key word in that TED talk title is "our"; it's referring to the average IQ of an entire population of people (not individuals). As has been stated, this is due to a variety of factors (i.e., more widespread education, better nutrition, etc.). So, there may be a larger number of individuals whose IQ is significantly above the average, but it's not like the biological limits of human intelligence have suddenly disappeared (meaning this isn't a cyclic phenomenon that will continue ad infinitum; rather, we'll just have slightly more really smart people).
@Robert idonotsharemyfullname Also, I feel that you are using the word intuition wrong. The word refers to being able to know a thing or apply some unit of knowledge without having to explicitly think through it. To build an intuition for a subject (especially a hard one) generally requires not only years of studying that subject, but also several years of applying that studied knowledge to solve hard problems. As such, there is no way that a 15 year old is going to have the intuition about computer architecture that Jim's crystallized knowledge of the field merits him.
Robert idonotsharemyfullname Listen to the Kahneman episode for some perspective on that idea of intuition ;)
He is modern times Einstein. Many things wouldnt be possible or were harder if people like him werent around. Moore's Law is taken for granted, but not many can do things required for that. Yet science , research many fields advance just by having better computers around.
Best podcast I have heard in a while. A very refreshing guest!
One thing that's incredible is that Jim doesn't "err" or "umm" at all, or almost at all. It's crazy how present all the stuff he's taking about is in his mind.
@Dual Face do you know how funny it is that while calling other people uneducated you wrote "there" instead of "their"?
He's a T-1000. Their architecture allows for a very low memory latency when communicating.
His tick is saying 'right ' but its more of a statement than a question.
I guess you just glazed over the diarrhea of You know you know you know you know
@Honey Badger Offroad also called autism
Could watch this one on a loop indefinitely. So many good tidbits in here.
"Physics itself has been a shitshow for thousands of years." Brilliant!
@2K Yep true.
I would agree with hundreds, but the civilizations that seemed to have understanding of physics thousands of years ago seemed to take their knowledge and understanding with them when their empires collapsed.
@Mark Compton oh my Lord you are 100% correct, they took that and certain medicinal cures with them too. Maybe even a sense of spirituality as well. I think of the Mayans ability to track time with such precisions, the Egyptians building pyramids with remarkable engineering and precision. It's like there were aliens we don't know about that somehow got them by (or lived among them).
@chuimon slp The Gobleki Teppe is another anomaly. A 12 000 year old hunter gatherer society should not have been able to produce the biggest monolithic site in history. Even more incredibly, they did not live there. Instead they used the site for a religious pilgrimage.
Ancient aliens came here to steal our gold
This is the guy behind the Ryzen series of processors which brought AMD back from the dead.
He was the onlooker, and mentor to that. The actual work was done by Suzanne Plummer and Michael Clark.
@Jake Rosendahl Or, as they used to call it, EPIC.
Where’s the intel guy 🤧
@smiechu60 why in the hell do you cpu nerds go that deep into something which has no bearing on your life
@Kyle Stewart why not?
Jim's work has changed so many peoples lives, including mine.
1:06:13
Lex: "Everything you said is correct."
Jim: "Yay"
"I've read a couple books a week for 50 years." WOW. 1:24:00
But does he have a Lambo in his garage full of books?
@Niemals im Geiste I'm guessing he made a fortune on AMD stock options.
Charlie Munger is another
I'd imagine those books are dense in his case, It's all well and good comparing him to Bill Gates and the like, but when you dig deep you find that those 'books' are often incommensurable with the kind of books that actually teach you anything substantive. I really doubt there are many people reading entire textbooks, thousands of pages per week. Let alone two in one week.
@Samuel Hauptmann van Dam Elon too
Thanks Lex! I am a big fan of Jim Keller. I somewhere heard that the microprocessor industry is just this guy competing with himself. He is hired by different companies to beat the processor he had built previously at other company.
Looks like he is uniquely qualified to do so.
Well wherever he works, it's not just him. Ryzen for example, he made the data fabric to get the MCM and then chiplet architecture to mesh together, infinity fabric. The rest of the Ryzen architecture was mostly built by other AMD engineers lead by Jim.
I think it's fair to say he always adds something significant to everything he works on, but it's not just him.
@Jake Rosendahl a bunch of the very best engineers seem to gravitate around him and each other too whenever he moves. Or so I heard.
@Jake Rosendahl you're taking the joke a bit overly literally there
I think you can take it literally because he as much said that companies want to build a new architecture every 10 years but he wants to do it every 3-5 years. The only way to do that is to switch companies effectively.
This is such an approachable talk on the most complex things humans manufacture. Thanks Lex.
I was shocked by the accuracy of the prediction of branching as well. I learned it when taking a computer architecture class that's required by my CS major. The explanation that I came up with is this: Intuitively when we think of a branch instruction, we think of it as an if-else structure in C. Then, the accuracy should be around 50%. Achieving 85% without doing anything fancy doesn't make any sense. However, the blind-spot is the loop structure. For example, when using a for loop in C, the complier translates the loop into a branch structure where if the condition is true, it branches back to the top of the loop; if it's not true, then it branches out of the loop. Imagining having a for loop that loops 50 times. Predicting that the branch will jump back to the loop will have an accuracy of 49/50 = 98%. Just imagine how many loops a normal program has. Having an 85% accuracy doesn't sound impossible now.
Also, that's why I subjective believe that trying to prefect the prediction algorithm is the wrong way to go. The initial high accuracy is not merit. It's just a nice feature due to programs having many loops. There's no reason to think that we are able to prefect the prediction.
Hope this helps.
@Darwi S Cool! Thanks!
@Darwi S yeah idk how so many people get stuck in their ivory towers. do they just forget everything they learn in computer architecture class?
CRAZY🤯No Clue- Thank you
So in the case of tight loops with unconditional bodies that's true. The equation changes for example when you have top level control-flow for/foreach/while loops that execute large subtrees of code. Depending on what kind of program you have, the statistical taken/not-taken weights on each branch can depend on the current data input, the previous data input, the number of iterations you have already done, or other factors. It's in those cases where modern branch predictions probably do a lot better than the older naive ones that were pretty good at loops. And there is of course also things like recursion and indirect recursion which essentially compiles down to either a simple loop or a more complex goto-graph. I'd guess that new branch predictors would probably be a lot better at complex recursive code than old ones, but depending on the function could still be fairly low.
This is cool ..and you would get "dumps" with 48/50 % and then mesh those dumps to get instructions
Lex, as a former computer engineer, this has to be one of my favorite interviews. Jim Keller is an uber engineer and someone of enormous knowledge, experience, craftsmanship, and an earned humility from the school of hard knocks. What a great podcast. Congrats.
Jim Keller reminds me of an older Gilfoyle from Silicon Valley.
Yeah, a brilliant jerk
From the thumbnail i thought it was him lol
Now I cannot unsee it 😅
this podcast blows my mind every time. thank you, lex.
Jim taking Lex’s case every few minutes 🤣 Jim was trying to save a then sinking AMD when I was there back in 2012-13 time frame, thereafter life forced me towards Deep Learning😊
If Warner Bro reading this, please hire Jim Keller as the next iterations of The Matrix Engineer in The Matrix 4
"when someone cuts you off in traffic your brain has theories about why they did it" I'm dying lmao
This is the one. THIS is the interview I was waiting for.
Great interview. The difference in perspective (academic vs engineer) makes the conversation fascinating.
"let me ask Jim Keller: 'what is conciousness?'" - 🤣
i've listened to many of these while at work in headphones, but this one in particular happened to resonate with me in a special way... thanks for being awesome
As far as the recipe analogy is concerned, I've always thought about it like this: some people know a lot of recipes and are great at executing them while much fewer people know how to actually come up with recipes because they have an intimate grasp on and awareness of the problem space and all of its dimensionality.
Yes, there’s absolutely recognition deserved for the recipe makers.
But how about the expert craftsmen/chefs who trained for thousands of hours that execute the recipe at a much higher level than the average one?
Both are equally impressive
Great Podcast, one of my favorite episodes. Love the analogy between chip architecture and team and corporate structures. So cool to have the opportunity to see Jim explain his work at this level.
this is one of your most interesting interviews ever! AWESOME
I've watched this whole interview 3 times already; this man has me mesmerized.
What truly wondrous intellect and wisdom.
This guy's really well spoken on things and he stands on what he says, great interview👌🏽
That mic would be like: I am becoming intellectually inclined.
Hi Lex, you're GREAT. I'd love to see you talk to some people about the BIG DATA problems with modern astronomy. This conversation with Jim Keller made me think alot about this topic. I wish you had asked him a few things about it. Thanks for doing what you do. Keep it going for all of us who love to learn.
Soo looking forward to the drive to work so I can listen to this on your podcast awesome work
What a fantastic conversation that was! It boggles my mind that people like Jim Keller exist; people who are able to absorb thousands of deeply abstract technical concepts and then as a team use that collective knowledge and intuition to create new, effectively magic objects such as microprocessors. It must take quite a special group of minds to accomplish that. I wish that I was smart enough to do something like that, but in the end I'm just glad we all get to come along for the ride and benefit from their brilliance. I can't wait to see what the next 20 years of exponential growth will do to society. I'm ready for a whole lot of change.
Jim is an absolute legend. Cool, thoughtful, smart and humble.
I need more Keller in my life.
Good call on the ads, Lex! You are absolutely right about maintaining the flow, and I can say for at least myself that it is appreciated.
Would love to see you talk with Linus Torvalds
+1
+10011010
Agree
flips bird not me.
big brrr🥶
A great review of Electronic Calculators university course, and beyond. Great, great video. Thank you so much for the content you are providing us.
Jim looks like a carpenter or trades men just by looks alone id never suspect he is a very talented engineer. 😂
The Harrison Ford of Comp. E.
ironman tooltime i dident once mention money
@ACogloc not everyone that works with tech has the nerd appearance
I was going to say he comes off as a prototypical engineer. He's exactly like the hardware guys who worked at Commodore and Texas Instruments in the 80's. To them it's very much like being a carpenter, they're just doing it on the highest level.
Yeah but how many carpenters and tradesmen do the bare minimum to complete their task and don't improve themselves. You can be an ordinary worker or an innovator, you can't tell just by looking at someone surface level. Also though, look up photos of Jim Keller young, he did look like a nerd.
It is so enjoyable to learn how creators think. I’ll spend another hour, thirty listening to this podcast again. Thank you!
This is F'n Oustanding! I wish I could have watched/listened to this podcast 10 years ago! 👏👍
This is such a great interview with a great mind. He comes off as the dude who should be as appreciated as Steve Jobs, he accomplished as much, but he was always relegated to the background in his professional life, possibly on account of his poor interpersonal skills and so ended up a rich jaded asshole-genius. There is an Oscar winning movie waiting to be written about an amalgamation of these kinds of people.
"rich jaded asshole-genius" lol
I totally get where you coming from. For the most part I agree with you. But if you really strip out any prejudice and listen to Jim....he is just a really really really smart dude being as honest as can be with a good bit of humor.
Yeah but it's so hard for me to see Jim suffering from that problem though...god his sheer mental/will power.....plus it looks like he's having a lot of fun in life. Leading teams of thousands of people...working with the smartest of us all...wow. amazing
@Charles Wang possibly not, as he found suitable team for him. for general public, totally different thing, Steve Jobs was good in that interface of general public, product users and technology. Tho he was biased towards sales and public, to detriment of engineers. If Jim had to explain things and his thoughts to wider public, he would loose his edge and still hard to understand. It is a tradeoff.
Wow. This was a tremendous episode. My new favorite guest!!
that insight on given and found parallelism (and branch prediction) is mind-blowing
One of the best conversations I’ve seen you have. I loved how he pushed back when you try to romanticize things, and I love how he (not meanly) dismantled some of your beliefs about your own research. Amazing guest, thank you for uploading it!
(to be clear, I think your tendency to think Deep Tragic Russian Thoughts is not a flaw, merely a personality difference between you and your guest. There are possibly certain truths which may appear to him but not to you and vice versa)
((One of the reasons I watch as many of your interviews is specifically because of your pathos, so don’t change too much!))
Lex Fridman - 2020-02-05
I really enjoyed this conversation with Jim. Here's the outline:
0:00 - Introduction
2:12 - Difference between a computer and a human brain
3:43 - Computer abstraction layers and parallelism
17:53 - If you run a program multiple times, do you always get the same answer?
20:43 - Building computers and teams of people
22:41 - Start from scratch every 5 years
30:05 - Moore's law is not dead
55:47 - Is superintelligence the next layer of abstraction?
1:00:02 - Is the universe a computer?
1:03:00 - Ray Kurzweil and exponential improvement in technology
1:04:33 - Elon Musk and Tesla Autopilot
1:20:51 - Lessons from working with Elon Musk
1:28:33 - Existential threats from AI
1:32:38 - Happiness and the meaning of life
James Gibson - 2021-02-24
@Heath Sims I think that Jim's analysis of the problem is apt considering that humans can simply be interpreted as slow moving planar objects with given trajectories, things for a car not to hit. At the same time, I can see a little bit of merit in feeling that reduction of humans to this domain is simplistic and even dangerous or that involving something as complex as humans in such simplified domain could be problematic. A car may not be able to interpret someone waving another person on or into traffic, and a car may not be able to predict how a person might react to something without having an analysis of human behavior when even if these cars are perfect physics machines that react immediately to changes in general trajectories, they absolutely can not always react in time to stop without modelling how humans and the environment will act. Their amazing reaction times do not tell them that someone is making a mistake that they cannot fix in time, like stepping out into traffic in too short a distance to brake. If a given driver on the road is swerving and apparently texting, only an AI that can interpret human behavior would take extra safety precautions to give such a driver extra space. A computer could easily wander into harm's way trusting that this driver will behave like 99.999% of the other drivers it has encountered. If some idiot is driving on an old donut, and it looks like the lug nuts are about to come out, you and I could exercise greater caution, but AI wouldn't see it without being able to interpret context and make judgments that went beyond the ballistic space. You know sometimes you can tell someone is not paying attention, and you can see the mistake they are going to make before they even make it. This sort of exclusively physical/geometrical approach can't do that. It can't tell when someone is being stupid or is about to make an attention blunder like we can. If understanding of human behavior is coupled with the geometric approach so that a vehicle's AI could predict where a person will be based on behavior modeling like knowing they will stop at the red light, or the pedestrians will likely walk when their light turns on, and they will walk in a particular way, and that old person with the walker will probably move a little slower and even react a little slower... when that happens, you will have perfect driving machine. Until then, until the system can interpret its environment in human ways, human behavior will create areas where these ballistic systems can still harm them. Eventually, we will have something that does both of these at the same time. Obviously, Jim's approach is the first and most important part of that, and the human element is more icing on the cake. But I really think that Lex is making a valuable point, there is in fact more to it. It is not optimal to stop after solving the problem in the ballistics domain, it is better if possible to and clearly more difficult to model human behavior to allow for 'defensive driving.' If we can do it then we save lives. And of course things are much more complicated when taken into the domain of humans, you are agreeing with Lex in this case, while Jim is arguing that the problem can remain exclusively in the domain of ballistic space, which is I don't think quite right. And it is in fact complicated without question, and the human modelling based on the queues you are referring to is monumentally more complicated than feeding pixels into neural nets. They aren't that smart. Movement modelling can be done, and an AI can learn to expect the people at the crosswalk will probably cross, etc. But that crazy guy who just crossed the street yelling at cars, AI doesn't know what to do except to try to not hit him. It might not see that he has a knife or is pointing a gun at the passenger trying to car jack him, etc. The AI would get that situation wrong. There are cases where knowing what is going on in the human world of meaning is important for driving, and that does not exist inside of the ballistics domain.
Lul M - 2021-03-20
1:13:27 - something flies down from his beard to his shirt
anthony stewart - 2021-03-23
Makita and
Tom Nikolaisen - 2021-04-08
"Start from scratch every 5 years"
Sound like he'll fit right into the church of Muskology and it's gigantic scam on robotaxies.
(wasn't is supposed to be here 2 years ago? Like the semi and the roadster?)
Jon Doyle - 2021-04-14
L