I don’t really like GPT-5.5…
I don’t really like GPT-5.5…
Discover other moments and creators
- Sup nerds. You might notice a couple things different about m...
- wanted to cover it and I think my coverage might surprise you...
- this is a video about GPT 5.5, the latest model released by O...
- I am. But honestly, overall, I'm a little bit disappointed. T...
- that I want to really talk about here because as powerful and...
- also comes with a pretty massive price hike at $5 per million...
- higher than Opus 4.7. While it is much more token efficient t...
- hopefully better than I'm about to justify it by switching to...
- much. It's too much work to get everything set up, to get Xco...
- fun. I've just not been able to enjoy building mobile apps. B...
- ROR. I've never had this good of an experience building for i...
- impossible. These guys made it literally three clicks. Once I...
- Not only that, but you can run the apps on your real device,...
- how crazy is it that I could vibe code with a single prompt a...
- other things you need to build good iOS apps. In Swift, they'...
- local device without having to set up an Apple developer acco...
- this good for getting an app from Idea to reality to your fri...
- results speak for themselves. If you want to build real apps...
- Okay, let's dive into GPT 5.5. See what OpenAI has to say abo...
- Starting with their blog post, they say it's the smartest and...
- it's not the best in the world. Might be that Mythos is bette...
- things. So, we'll keep scrolling to the parts that matter. We...
- We evaluated the model across our full suite of safety and pr...
- biology capabilities and collected feedback on real use cases...
- it was secure. They also call it the fact that this model is...
- to fix that. A lot of this appears to have come from a partne...
- They trained and shipped this on the GB200, I believe. Yeah,...
- state-of-the-art from Nvidia. That's what they shipped this a...
- the workarounds in the near future. They will support it soon...
- to have some early access to test things there and uh have so...
- Benchmark dominance
- compared to the 75.1 they had before
- They've been leading there for a bit now. They don't evaluate...
- very different thoughts about the new Pro model, which we'll...
- that one kind of cooked. Their expert SWE bench, which is an...
- I got before. If you're wondering why they didn't test this o...
- you're wondering why they didn't test 3.1 Pro on it, I don't... [eo5a2]
- Not my words. Okay, those weren't my words. Then we have GDP... [eo5a2]
- versus 83.0. I would go as far as to say that's a little dish...
- less often than 5.4 did. It just tied enough to balance out t...
- GDP valve is a lame benchmark. I It's delayed more releases t...
- beating out Opus 47, but still not by much, just 7%. Toolatho...
- been benched on it. And in Google models, as always, kind of...
- did pretty well, but it did lose to 5.5 Pro. And 5.5 Pro also...
- but it is worth calling out that this is the Pro models numbe...
- While I have been impressed with the browsing capabilities of...
- graphical, all that stuff is still far ahead of everybody els...
- this browse comp test comes down to being better at tool call...
- again, Google still cooks there. I still use Google models fo...
- The Pro model in particular seems to be very good at this stu...
- That's a lie. I want to do one more bench. Flashbang warning...
- the best benchmark, it is a pretty good cohesive cover a lot...
- interesting though is when you turn on the 5.5 versions that...
- even low, you'll see some very interesting numbers. 5.5 mediu...
- While the price per token is higher, the token efficiency gai...
- intelligence, you do have to spend more money. and the standa...
- to the tokens used to run the benchmark section, you'll see s...
- the highest possible run version, use 75 million tokens, whic...
- Opus 47 is not quite double, but still quite a lot more token...
- even medium, you have to go really far down the end here. To...
- medium used only 22 million tokens. That is actually a very i...
- I could do the math here, but I'm lazy
- So instead, I'm going to go to the intelligence versus output...
- light. Yeah, we can delete a lot of these. Now I've removed a...
- has pretty tight dominance on the intelligence versus output...
- The x- axis is how many tokens were used. The y-axis is the i...
- well for the token cost compared to other things. It is actua... [eo5a2]
- while also performing that intelligently and 5.5 low absolute...
- Obviously, these tokens cost more per token and I wish I coul...
- yet. OpenAI visualizes in a very appley chart that's hard to...
- comparable to like Opus 47 was done in roughly half as many t...
- meh it's expensive and they're trying to justify it by showin...
- feel way faster because it takes half as many tokens to do a...
- reasoning levels. This is the first time I've seen them recom...
- instead. And now we get to the frontend capabilities. They sh...
- still full of cards like everywhere. Pretty sure almost all o...
- somewhere. But there is something also quite impressive about...
- When I saw how well these models supposedly did 3D game stuff...
- certainly loves to do that. One of the first things I started...
- an idea for a game I had forever ago called Fish Slop. It's a...
- Insane Aquarium. And I was mostly testing the capabilities of...
- and try to just flush out an idea. I end up getting way furth...
- chaos for a while now, as I'm sure you all have noticed. So,...
- So I took the existing source code, pointed models at it, and...
- "Modernize this, clean it up, make a good, minimal starting p...
- For some reason, it did not want to touch the assets that I h...
- but it worked. I will say actually the movement feels slightl...
- And it's impressive that it was able to generate these assets...
- For some reason, it didn't want to use Phaser, though. It act...
- It also seems like it really up the timings for like when the...
- Yeah, I I thought this was better than it was. Now that I'm a...
- I'm realizing, oh yeah, this this kind of ass. And now I have...
- I don't know what it is, but it is bad about this. It didn't...
- very quickly if you don't feed them instantaneously
- There's a lot of little things I don't love about this versio...
- And now we have the version that we got out of 5.5. Immediate...
- assets from the original for the fish and the like submarine...
- Little things like the coral on the bottom, while not perfect...
- going across. I like the wavy lines here. I don't love it, bu...
- It's got something I would call almost taste, but also someth...