Two Minute Papers - 2023-02-20
❤️ Check out Weights & Biases and sign up for a free demo here: https://wandb.com/papers 📝 The paper "Human-Timescale Adaptation in an Open-Ended Task Space" is available here: https://sites.google.com/view/adaptive-agent/ My latest paper on simulations that look almost like reality is available for free here: https://rdcu.be/cWPfD Or this is the orig. Nature Physics link with clickable citations: https://www.nature.com/articles/s41567-022-01788-5 🙏 We would like to thank our generous Patreon supporters who make Two Minute Papers possible: Aleksandr Mashrabov, Alex Balfanz, Alex Haro, Andrew Melnychuk, Benji Rabhan, Bryan Learn, B Shang, Christian Ahlin, Edward Unthank, Eric Martel, Geronimo Moralez, Gordon Child, Jace O'Brien, Jack Lukic, John Le, Jonas, Jonathan, Kenneth Davis, Klaus Busse, Kyle Davis, Lorin Atzberger, Lukas Biewald, Matthew Allen Fisher, Matthew Valle, Michael Albrecht, Michael Tedder, Nevin Spoljaric, Nikhil Velpanur, Owen Campbell-Moore, Owen Skarpness, Rajarshi Nigam, Ramsey Elbasheer, Richard Sundvall, Steef, Taras Bobrovytsky, Ted Johnson, Thomas Krcmar, Timothy Sum Hon Mun, Torsten Reil, Tybie Fitzhugh, Ueli Gallizzi. If you wish to appear here or pick up other perks, click here: https://www.patreon.com/TwoMinutePapers Thumbnail background design: Felícia Zsolnai-Fehér - http://felicia.hu Károly Zsolnai-Fehér's links: Twitter: https://twitter.com/twominutepapers Web: https://cg.tuwien.ac.at/~zsolnai/
What was not mentioned in the commentary is that this model was pretrained on a number of similar type tasks. The pretrained model is then capable of few-shot learning new tasks it hasn't seen before.
thank you, I was wondering how the AI new that the objects interacted in the first place
Step (1) in the abstract 0:01 already mentioned "meta-reinforcement learning", pretty much a pretraining step as you said. Funny how Károly always omit crucial steps to make the research sounds more awesome than it (already) is.
@@deep.space.12 give him a break, he is just trying to engage people in educational content. If the papers are, as you said, "already" awesome, then omitting something that to you wouldn't change the awesomeness of the paper, but for someone less academically inclined would make the paper seem boring... i think that its worth omitting. At least for a youtube video, its not like you can't just read the paper yourself!
@@nathanielbartholomew5091 It's quite understandable to make an occasional omission to facilitate communication or by pure accident. But it's been a trend for him to misrepresent results and credit the wrong researchers for views. It's appalling, frankly.
@@deep.space.12 well shit man… I thought he wasn’t scummy. Am I wrong? I really don’t want to believe you, but I’m pretty used to being failed by public figures haha. So what you’re telling me (and correct me if I’m wrong) is that “Two Minute Papers” purposefully omits and/or twists the information to mislead people for the benefit of his YouTube channel to get more views and make more money and his transgressions can’t be seen in any way as a wilfully ignorant AI enthusiast trying to share developments in important fields.
"research two more papers down the line; full video 4k; hdr; unreal engine; realistic; accurate; inspiring"
Hahahhahahaa
No one asked
I wish more commentary was provided on this one. It seems like this was likely a fundamental breakthrough that's going to utterly revolutionize dataset size requirements and compute resources required to train NNs.
I cant really believe it. There had to be more than 5 tries for learning something like this
Yep, felt like a narration of the video clips instead of any sort of explanation of how this method was different or innovative, or how it was accomplishing the speed up in training, no insight or analysis, just a "wow, amazing!" react video that could have been made by a vtuber. A rare L on this channel.
@@msytdc1577 In fairness, any explanation of these would amount to 10 minutes of noise indistinguishable from "They did a thing with the thing before, but now they did this other thing and it's better!"
There has to have happend a training process beforehand, where it learned how to play these sort of games. During the game its not so much learning as it is reacting to a changed environment.
@@loneIyboy15 depends on who these videos are targeted towards, I recall watching a video on this channel that was long, detailed, and had a good breakdown of the paper, how things worked, how it compared to previous techniques, what it was still not fully successful at, etc., a solid video in line with most of the other videos on the channel made thus far.
Then I watched a different shorter video on this channel released a month or two later that gave me deja vu watching it, but it was more like this one, just showing some examples and saying it was amazing, but completely surface level and to me not really interesting.
Turns out the deja vu was because both videos were about the exact same paper! The first long one would probably have been "boring" and too technical for the masses and garner fewer views, and the "dumbed down" one was something a hundred channels could have produced, but likely would have been financially more successful and popular for this channel. Positively it would also have been a better introduction to what can be a complicated topic than the information dense version, and at least when produced by this channel even a surface level video is likely to highlight the important parts and not get something horrendously wrong, which is more than you could guarantee the other 99 channels would manage, if they even cared to make the effort.
So, like most things, it's a trade off. Channel produces only high level videos, small audience, low revenue, fewer people exposed to the wonders of current advancement; produce only superficial react videos and the channel loses the special unique attributes that have historically set it apart from less knowledgeable content creators, and provides little reason for the initial audience who subscribed for that more advanced insight to keep coming back.
It is a VERY cool paper, but DeepMind still trains this agent on 25 Billion games of this general type first (5 weeks on 64 TPUv3), before the model becomes "smart" as we see here, and able to generalize to new variants of the same kind of game as rapidly as a human would. Great result, but much more work is required to make this more general purpose.
To see how general it is, the actual task would need to be something like play tetris with the available blocks in the room, using the pre-training that it currently had.
Humans are also pretrained on 20 years of similar experiences.
Something that would have blown us away 2 years ago is now 'okay, I guess'. "Much more work" also means, what, two more whole years?
how did the AI know about geometry, possible actions and that combining objects is necessary to solve the task? it couldn't have guessed that. i mean, the solution could have been anything (for example "trace all walls" or "touch al tiles")
it must have had some knowledge before the game started
I was thinking the same. It feels like this video was more about enjoying what the ai is doing rather than explaining the functioning of it. While admirable, it doesn't help me reason about how great of an achievement this is nor how transferable any of it is.
Yeah, I agree with you. I think the search space is already heavily confined. Not comparable with a real life scenario or a real escape room.
Still impressive though!
It was pretrained on 200 million and 25 billion similar tasks which is missing from the video.
@@Peter-ik9fz So the actual task was figuring out which objects to interact with, not to learn how to move in the world or how to push or grab the objects. Still, great work with very little repeation to make a correct guess from very little data.
I would argue that this type of learning is more specific and therefore less complex than learning to move and fight with multiple limbs.
I would love it if you could explain the key principles as to how these papers achieve their results! That would make your videos twice more interesting than they already are if you ask me!
It'll be revolutionary when minimal shot learning is applied for everything!
I don't really want to talk the achievement down, what they made is still pretty amazing, but since the video fails to do so it feels like I have to set it a bit in perspective. The AI was pretrained on a more generalized version of the puzzle, so it became an expert in the domain, and the only "ad hoc learning" it had to do was figuring out what the concrete problem was it was dealing with this iteration.
Putting it into perspective of what the equaivalent human thing would be ... its like giving a math expert (who trained math for years) a specific math problem to solve and of course he's going to figure it out quickly if he has seen this kind of problem before lots and lots of times. Or someone who is trained as an expert at reparing computers ... he has to figure out what concretely is broken with a computer he is given.
So this AI basically was trained as being an expert of figuring out what the hidden rules are in this game world, to achieve their goal. Thats no small achievement but its totally unrelated to how the video presents it, and it doesn't feel as revolutionary as its hyped up to be, of course if an AI is trained to be an expert in a whole domain it will solve individual problems of that domain very well. I'm disappointed in the video giving no context at all.
@@gavaldor it is true that the video could have provided more information. But, it's still useful this way because you could just train a dozen ai models in somewhat niche categories and then quickly train them for specific applications.
That's an overstatement. Minimal shot learning has its place, but it's not a revolutionary solution for everything. There are still many challenges and limitations that need to be addressed before it can be applied effectively to a wide range of tasks.
@@gavaldor Thanks for your insightful remarks!
@@gavaldor Unfortunately Google is very good at marketing, most of their archievements are not what they claim. I learned that when they showed their Dota2 AI, and as someone that understands that game, I immediately became dissillusioned by Deepmind, they claimed their AI learned to play DOtA, but their presentation was just one big fraud, their AI did not learn to play the game at all, they alterted the game a ton, had many restrictions(like no fog of war, their AI always had to know everyone position) It did not even come remotely close to playing the game like humans would, but they marketed it as if it could.
These next tests require cooperation. Consequently, they have never been solved by a human. That's where you come in. You don't know pride; you don't know fear. You don't know anything. You'll be perfect. - some dude with combustible lemons
Atlas and PBody in real life be like "Beep boop"
DeepMind should train a model to play Portal 2 co-op mode. That would be sick
@@NotASpyReally everyday we stray further from god.... and further toward ApErtUre SCieNce
@@jackb3493 That's the prime place to be.
It would be interesting to put an AI like this through Portal 2 co-op. In fact, it almost seems too perfect, given the themes and aesthetics!
They should but it will fail. The ai is pretrained on similar games, it probably does not generalize that much.
@@JT-hg7mj Mayb if they pre-train it on P2 first. There's no shortage of community created test chambers, workshop support and all that.
@@nixel1324 that would probably work, but it shows that this ai does not really generalize.
I always look forward to your videos. Staying on the frontier of machine learning. Exciting time we live in. Exciting time.
Aw it's so cute when it holds the cube up in the air and jitters around like it's happy
Can't wait for AI like this to show up in games and show real growth and simulated 'personality' in a sense
This is the first time where I've seen an ai solve these games faster than I would. Substantially faster. And that's sorta terrifying...
This is general intelligence. It was trained to learn how to learn. Now it uses realtime observation of its environment plus its accrued realtime past memories to solve problems. This is the beginning of the end. Scale this up and you can replace all employees. This algo and anything similar are going to be the most important inventions ever created by humans. I'm a AI scientist, and have been working on this basic concept for the last few years. These guys nailed it. Can't believe how well it worked, would love to see it scaled across more domains and problems.
This is mind blowing progress. Wow!
Thank you! About halfway through it struck me, one of my earlier jobs an executive had a sign on his door: "Life is the only game in which the goal is to learn the rules." Such a great example of this quote from decades ago. Really neat explorations, thank you again!
Hi, although most of the viewers won't understand much it would be cool if you could explain some things about the architecture of the A.I. You could also do that in a separate video.
We finally got there. This is what comes two papers down the line. Amazing
Yeah, it really sounds amazing (which it is, but not as much as what this video makes it look it is) but they were trained on a looot of these types of games beforehand. What you see in the video is the AI figuring out what the hidden rules are. Not the game itself.
im a layman on this subject, but i'm wondering if it's legit to compare the years learning model to this seconds learning? even though there arent intermediary rewards, this still seems much more simple than learning how to move a body with many moving parts and then learning football or fighting.
You are right, this won't be a fair comparison per se: the task of learning football is more difficult, and, if we read the paper, we will note that the model presented in this video was first trained on a large amount of similar games. So, what we see here is the ability of AI to figure new rules for a game that is somewhat similar to the previous one
However, I'd say that the difference in the amount of data needed for learning a task is so huge that it is indeed impressive, and the casual comparison is totally understandable =)
Thanks again for making these videos. It's incredibly tough to keep up with AI as there's so many different ones coming out all the time. Your format is great because, despite being concise, it offers plenty of information. Very insightful!
I love this kind of AI way more than the "blank canvas" kind of AI, which starts not even knowing how to move or even the fact that it can do so.
As a psychology student one learns that animals dont come to the world as a completely blank canvas but quite the opposite in fact, with many basic patterns of behavior hard coded into our brains by default.
By giving an AI some basic structure one would expect to get a more realistic result and even new non pre-coded behavior that resembles that which one would expect to happen in a real organism.
Thank you so much for your videos, they are absolutely priceless!
this will be amazing for costumer level AI to learn to do some special routine the user wants it to do for him. learning fast like that it wouldn't be too much trouble to teach it.
This is very cool to watch, now I want to see a couple AIs play Portal 2 together!
Seeing AI performing such tests reminds me of Portal games
My background revolves around physics, chemistry, and engineering. This could be huge for simulation studies to help inform better designed experiments. This could save us loads of time, energy, and valuable resources!
Unbelievable. This is amazing.
What does this mean in the long term? Are we getting these kinds of results in other real-world problems soon? This is amazing.
Yes
if i wuz these ai characters, i'd be pissed about someone messing with me everytime i figured it out . . .
you have to sleep sometime - buahhahahaHA =P
They were trained on a loooot of these types of games beforehand. What is impressive is that they can figure out the hidden rules quickly, but saying they didn't have any training is just wrong.
How is it even possible without knowledge or training to just guess from some pixels that they can be interpreted as a 3d system and that you have to move to and grab specific colored objects?
This is truly inspiring! Love it when AI play games
Gotta do more videos about ADA. Please. Amazing. Mindblowing. Revolutionary.
I declare today that video games are no longer about who can do a thing the fastest, but rather who can do something unexpected. Playing the matchmaker in a small town of NPCs, competing in poetry contests, art will become(stay?) the most compelling aspect of games.
You're basically ignoring more than half of all videogame genres in making that claim, how is a platformer game about the art? action, fps, roguelike, there are lots of genres where the art is secondary at best
Wait a second, I'm having a hard time processing this. Did we really just watch the training process in real time? Did this thing really solve those levels that quickly? That is absolutely insane. If I interpreted this video correctly then this is astonishing.
It was pre-trained, if you look at other comments. A lot of pre-training, on other and similar tasks. So it already knew that it was supposed to interact by touching the objects together.
But, that pre-training can be done very quickly, in our relative time anyway.
@@ShidaPenns Yes, after I posted the comment I realized it wasn't quite as incredible as I originally thought. However, I still maintain that if it truly progressed and solved these problems within a couple iterations that is still astounding
This is incredible. What a world we are approaching.
My guess is a test play ai could be on the horizon. AI’s could be able to play through entire games soon
Bug testing AI woahhh
Hmm, this could be helpful in doing specific tasks in control systems. Can train it to "fix" common breakdowns in a plant. I wonder is it pretrained, if so how? and how well can you adjust the training to do other tasks.
woah, this is going to completely change making TAS's and glitch hunting for video game speedruns!
this is awesome!!
Speechless
It'd be great if you also mention the technical details a bit. For example; a short explanation of the diagram at the end.
I like the little random level designs.
Crazy things are gonna happen when we have AI making discoveries, innovating and inventing with brutal, mechanized efficiency. The speed at which the world will start changing will be completely unprecedented. In fact, you could probably consider all of human history before that date as "the before times".
Stunning
Wow, definitely one of the most startling AI videos I've seen so far.
Have you seen the videos having to do with text-to-speech synthesis? Those have startled me the most, personally.
Imagine the AI going through thousands of years of trial and error just for your amusement, "I have no mouth and I must scream"
I would love to use Ada for custom game AIs. Because the training is so fast, you don't need much time to get different level of opponents. In fact, I could see directly training it on the player's machine for a creative game like mario maker.
"I failed so I'll do it right next time"
This seems too good to be true.
I want to watch an AI speedrun Elden Ring 100% completion with no glitches perfectly
Mayhaps it will be used for persuasion? Once the desired goal is known, it could be used for anything really. Efficiency optimization of reaching a goal.
What will the "win" be, that is the better question.
Maybe taking CO2 out of the air, and constraints could be to generate no heat and reaction waste being a useful substance.
The possibilities are endless.
Hopefully only used for good.
Stamp collector AI
@@JorgetePanete Where do u sign up?
@@ntwadumela_jadu9747 It's a concept shown by Robert Miles on his channel and Computerphile
i feel like someone realizing fire is almost beyond stopping while all others are still laughing on that cute accident with candle and gasoline...
The video title sparked an idea: I think it'd be great if you did some comparations of total progress for different types of problems
@Ken1171Designs - 2023-02-20
I have a long history of creating puzzle web games, so this video was especially rewarding to watch for me. Really awesome and inspiring. Made my day! :D
@AaliDGr8 - 2023-02-21
teach me plz