I still get excited by the sheer power of current and upcoming tech

It may be the same stuff we had 20 years ago, but the specs!

What was a supercomputer before is now just a component in a datacenter, one of 72 per rack.

I used to be like that. I would pore over feature matrices in Computer Shopper ads and pick out stuff I couldn't afford and wasn't going to buy.

Then, about 7 years ago I realized that for the first time, I was more or less happy with the tech in my life. My computer was good, I liked my phone, my tablet, and my ereader. I had a good TV and stereo I used all the time. The appliances in my home are reliable and work well. The tools I use in my garage and yard are all fine. With Android Auto or Car Play (I've used both) even my car felt like it had crossed some threshold of being good enough that I didn't really have the itch for something better.

I had been working full time for 30ish years and had accumulated a lot of stuff that makes my life better and there really isn't much more that I want and certainly nothing that I need. When Christmas rolls around and people want gift ideas for me, it's hard for me to think of things. My favorite thing in the world is to have a few days off and no plans or commitments and I can sort drill bits, change my guitar strings, watch a movie, or take a nap.

What used to be a super computing center now fits in my pocket and runs on batteries. I haven't tried, but I'm reasonably sure my phone can emulate a Cray-1 at faster than the original.

A Cray-1 had 180M flops. Your phone can emulate it as speeds many times faster.

Makes sense to me.

We will see, since if true then it is likely the other makers of small, dense models will copy it and include high reasoning by default.

If that also makes the other dense open source models better, then you are probably correct.

The first thing I thought was: this might make Tesla's cameras also not see your funky-patterned Yaris on the road as a car. Might want to test that first!

I'd be totally fine with that. I try to keep as much distance between myself and Teslas as possible anyway.

How vigilant do you want to need to be at all times? If an autonomous vehicle smashes into you at high speed and causes your car to flip a median because it didn't see you, it might be at fault for killing you but you'll still be dead.

It wouldn't change how vigilant I need to be. Any vehicle, autonomous or not, could do that at any time regardless of how my car is painted.

An electric ute? I'm interested. Seems like a good move!

Just like everything else: to flood the channel

The current environment comes to mind

I second this. Anyone considering running their own hardware would also consider quantization since they would have full control over what quant to run.

The value proposition changes a lot if you can get 90% of the quality for 50% the price with a quant due to halving your hardware requirement.

Thank you for the article, though! Very informative.

Hmm. Pennywise operated on a 27 year cycle. Maybe Cthulhu does the same, but at 1MM the scale.

Eggs don't lay eggs

Very informative article. I was hazy on where NVLink sat in the local inference stack. This makes it very clear: to adapt consumer hardware that lacks multiple full x16 PCIe lanes.