in Common Problems

Apple M3 Max MLX beats RTX4090m

999 Views 50 Comments



Try Paperlike here:

Apple MacBook Pro with the M3 Max chip is even more capable in Machine Learning workflows now that MLX Framework is out. Here I test it against the nVidia RTX 4090 laptop version in one of my typical workflows – speech to text.

Run Windows on a Mac: https://prf.hn/click/camref:1100libNI (affiliate)
Use COUPON: ZISKIND10

๐Ÿ›’ Gear Links ๐Ÿ›’
๐Ÿ๐Ÿ’ฅ New MacBook Air M1 Deal: https://amzn.to/3S59ID8
๐Ÿ’ป๐Ÿ”„ Refurb MacBook Air M1 Deal: https://amzn.to/45K1Gmk
๐ŸŽงโšก Great 40Gbps T4 enclosure: https://amzn.to/3JNwBGW
๐Ÿ› ๏ธ๐Ÿš€ My nvme ssd: https://amzn.to/3YLEySo
๐Ÿ“ฆ๐ŸŽฎ My gear: https://www.amazon.com/shop/alexziskind

๐ŸŽฅ Related Videos ๐ŸŽฅ
* ๐Ÿค– REALITY vs Appleโ€™s Memory Claims | vs RTX4090m – https://youtu.be/fdvzQAWXU7A
* ๐Ÿ‘จโ€๐Ÿ’ป Cheap vs Expensive MacBook for ML | M3 Max – https://youtu.be/snRdjD0w-hw
* ๐Ÿค– INSANE Machine Learning on Neural Engine – https://youtu.be/Y2FOUg_jo7k
* ๐Ÿ‘จโ€๐Ÿ’ป M1 DESTROYS a RTX card for ML – https://youtu.be/u9ECps9b664
* ๐ŸŒ— RAM torture test on Mac – https://youtu.be/l3zIwPgan7M
* ๐Ÿ‘จโ€๐Ÿ’ป M1 Max VS RTX3070 – https://youtu.be/B7CNMHeZ4Ys

๐Ÿ› ๏ธCode๐Ÿ› ๏ธ
https://github.com/TristanBilot/mlx-benchmark
https://github.com/ggerganov/whisper.cpp

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

โค๏ธ SUBSCRIBE TO MY YOUTUBE CHANNEL ๐Ÿ“บ

Click here to subscribe: https://www.youtube.com/@azisk?sub_confirmation=1

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

๐Ÿ“ฑLET’S CONNECT ON SOCIAL MEDIA

ALEX ON TWITTER: https://twitter.com/digitalix

โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€” โ€”

#m3max #m2max #machinelearning

Share this:

50 Comments

  5. Two MacBook Pros died after 14 months. If I could buy. a new one every year, that would be just GREAT.
    8GB of RAM is not enough but Apple figures that profits are better than selling a computer with enough memory to do the job. "Job" – does that remind you of someone??? Too bad we are Cooked.

  6. Serious question: why would anyone buy a windows pc when you can buy a Mac that not only can run windows on it but runs windows BETTER than a windows pc??? In buying a computer soon and would appreciate the feedback. Thanks.

  14. Hey Alex, I was wondering if you have a video planned for your EDC as a software engineer. Iโ€™ve been looking for a light case that I can carry around for my 16 MacBook with the 12.9 iPad. Trying to get ideas of what you utilize

  15. You forget something, when you tried to make a benchmark you faced the same issue, you couldn't use the whole performance of the GPU/CPU when you used Windows or WSL, and you achieved that when moved to Linux. please do it and tell me the results.
    I love your videos.

  17. Part of Apple's long game here is to absolutely dominate the mobile market in every way, and part of that domination is going to require robust machine learning capabilities and speed even for small models that are better suited for mobile uses of machine learning applications. They make their machines able to run small models insanely fast and that's where they're going to have a huge edge in the future

  18. I have no idea why the hack I am watching this now, but everything you say sounds cool. :))
    Ps: no idea how to code at all, wish I could.

  20. Thank you! What is the correct way of comparing my current AMD Radeon Pro 5300M 4 GB (MacBook Pro 2019) to an Apple M silicons? In terms of a MacBook gaming experience. I am playing a game from time to time and would like to make sure that a M chip won't take it away from me ๐Ÿ™‚

  23. What is the purpose of this computing power? Do you need it every moment of your day? And if you don't have it, is it a serious issue? I have a Mac Mini M2 at home. I also have 2 Windows PCs. I have no affection for these two machines that heat up, blow, scream, make a loud noise to obtain the power you're talking about. Not to mention the poor quality of plastics that crack and the miserable battery life of the laptop (whose power supply is larger and heavier than my Mac Mini M2). The production of PCs should be stopped.

  24. My apologies if I am being dumb, by why wouldn't you use an NPU for this machine learning process, as I thought this is the sought of task NPUs were designed for, and maybe even better at than a GPU? And if you could, how would the performance compare when running on an Apple Silicon NPU (on paper M3 NPU is 18 TOPS for FP16)? And as every processor manufacturer is now getting on the AI bandwagon, you could even extend it to compare the performance of AMD 7000 series with AI NPU (10 TOPS, 8000 series NPUs 16 TOPS) or Intel's Meteor Lake core Ultra with NPU (10 TOPS)? Of course, the processor I would really like to see would be Qualcomm's Snapdragon X Elite with its 45 TOPS NPU, but that's yet to be released.

  29. At 9:50 you mention that MLX is more difficult to run because of Conda. You can just use direnv to activate environments automatically: make a softlink in the directory to .venv/bin/activate named .envrc and everything runs automatically.

  30. Watched tens of your videos before upgrading from my old i9 MacBook Pro to my M3 Max MacBook Pro.

    Nowadays I still watch your videos (even if I already have an M3 MacBook) because I like the way you make your content โ€“ pragmatism, tone of voice, length and cuts.

    ๐Ÿ‘

  33. Great video, Alex! You have some really enjoyable content on your channel.
    Are you able to send me one of your old M-series Macs; Iโ€™m a student and Iโ€™m trying to learn some ML/AI stuff.

  36. Anybody have a roadmap for me to learn on what about a language or framework performs better on one arch or another. How clever can tensor operations get? Python I get. But what is it b/w mlx, cpp and ggml, jax and mojo?

  37. Alex, I found your channel when researching for my M3 max laptop purchase. I love your benchmark methodology, but also wish I could copy some of your workflows. If you added a code repository to your membership, I would join!

  42. appleinsider already did a test with whisper. You can find it via google with "Apple Silicon M3 Pro competes with Nvidia RTX 4090 GPU in AI benchmark" because i cant post links here.
    They basically did the sam tests and found that if you ran an optimized version the 4090 is faster. But the 4090 in the article is the desktop one and not the mobile one so the results cant be directly translated. The only thing thats clear is that the 4090m isnt being used to its full potential. This is actually an good example why for AI software is often times more important than the hardware, which is why all these marketing about tops and flops is misleading.

  44. WSL & even Windows itself has a lot of overhead. If you wanted a more "Apples to Apples" comparison, you should've compared it with the 4090 laptop running something like Clear Linux or Ubuntu. It likely would've not closed the gap but the results would be a lot better.

  48. WSL uses hyperv, there is no way around it.

    MSI laptops are always noisy. If you need a powerful and less noisy windows laptop then Lenovo Legion 9i is a better choice

  49. Awesome video! I would love to see more LLM or other DL architecures benchmarked between the M3 Max and the RTX 4090m laptop. A definitive video saying the M3 Max is X% better/worse than the 4090m for RNN, CNN, or transformer architecutres would be a gold mine for other AI/ML devs like me!

  50. Wow, exciting results! I was always optimistic that Apple's unified memory architecture would pay dividends in certain workloads, and MLX appears to be effectively exploiting that paradigm shift.

    Keep up the good work! Love the channel!