I’ve found Sol performance to be incredibly spiky. It has tremendous IQ and can fix very difficult bugs. But it is horrible at design (both visual and system design), anything that involves thinking about users or UX, and massively overcomplicates almost all work.
I noticed the same. I wanted a simple crud webapp and suggested an insane techstack involving C#, Razor Pages, MSSQL and more. I went with my planned setup of python flask with an sqlite db which served me well for years.
It's still incredibly important to have a human in the loop correcting design decisions and having good taste.
Was your prompt just "I want a simple crud webapp" and that's the extent of it? There's absolutely no way you included the words "python", "flask", or "sqlite" and it still went with a Microsoft stack.
I vastly prefer Sol. It does what I tell it to almost exactly, pretty much every time.
I work on very low level stuff (think RTL/FPGA, firmware, software where optimising for nanoseconds is just normal).
For me Sol is the only cost effective model available. Fable 5.1 is indeed good and vastly better than original Fable (which refused to work on most of my stuff for 'safety' reasons).
It's very good at this sort of low level stuff to the point that I really can't understand/relate to people having a good time with Opus (which comparatively performs extremely poorly on my particular workload).
I also just don't like how lazy Anthropic models are. They will do 10% of what is asked and then summarily declare victory.
Sol on the other hand is more like "one of us", slight touch of the 'tism, extremely pedantic, will go to the edge of the known universe if that is what it takes to prove/fix/build what you asked for or run out out of credits trying.
It's a personal and workload dependent thing. For me right now Sol for 99% of stuff because Fable 5.1 still burns through $5k in credits a day.
Agree 100%. And I also work a lot on lower level / systems stuff (including RTL here and there, too). Opus is sloppy, and leaves negative cases all over. The GPT models in Codex have a more pedantic and detail oriented "personality." Often to a fault.
Sol will leave a mess of excessive redundant tests and isn't so great at abstraction ; but it produces more reliable working systems.
It's kind of nice to have access to both, but I don't have the $$ for that right now, so I just keep the Codex sub
Can confirm this as well, mostly VHDL and HLS. Sol and Fable can reason about performance and designs consistently. Whereas Opus and others seem to just throw generic optimisation techniques at the wall unprovoked (while hallucinating a justification + expected improvement) until the synth reports improve.
We're not asking the model to simplify something, we're asking it to perform a task. Its subtle preferences show up as an overcomplicated path to the goal.
In some cases, there are also nuances that we don't pick up on. Here it's our preference for simplification that's showing up. We set the lossy compression factor higher than it does.
I doubt it. It’s not just America working on these breakthroughs anymore. Now we have two powers working at break neck speed to get to that point and the Chinese are making a lot of progress.
Looks interesting. What's your plan on the legal front? Bet the company on the compounding lawsuits breaking in your favor, or do you have a backup plan?
BTW: when you say you manufacture the drugs, I assume that means you're buying from an API supplier, but do you actually mean you're synthesizing it yourself?
We are co-manufacturing the API, and then we are bringing it in-house in the next 9 months.
The legal cutout that we have is on the personalization front, which is very well protected. The trick is that most compounders are not actually personalizing, and we are. Our feedback loop allows us to do real personalization on a week-by-week basis.
^ Zach is the founder of Uberduck, which was massively popular. He got into YC with it.
More story:
ElevenLabs launched a few years after FakeYou and Uberduck. ElevenLabs got a lot of attention for their remarkable zero shot model, but their pricing and enterprise focus created an enduring niche.
Companies like Musicfy.lol were what ultimately drew the ire of the music labels and created a lot of friction for consumer UGC audio tools. They ruffled feathers and that created headwinds.
I distinctly remember talking to their founder on a Monday at Founders, Inc. in SF. They were in talks with a16z to raise $10M or so after their first few weeks of wild success - viral marketing and clever positioning for the first RVC music models. By that same Friday the music labels had them shut down the majority of their offering and the funding offers fell apart.
A year or so later, Suno and Udio ultimately took that side of the market. They were more consumer-friendly than raw vocal models and appealed to a broader set of users. Both companies launched to great success despite the risk of lawsuits.
15.ai, FakeYou, UberDuck, and Weights.gg (a latecomer) all gradually wound down due to the founders being pulled in different directions. Weights.gg joined OpenAI, Fifteen.ai became anti-AI and built an art community website, I went into movies/film, and Zach (Uberduck) forged a new path too.
ElevenLabs kind of lost the plot with audio. They've been building a bigger, more comprehensive offering that has scaled their revenue well beyond just TTS and audio. But it also left something of a void to fill.
Last year, Fish Audio launched in our old segment (consumer / UGC TTS fine tunes and zero shot) and built a massive user base and revenue stream. The market was wide open for it. They raised a $40M seed round.
FYI: If you have a company that you are shutting down, you too can sell your data to the labs. Companies will help you do this. If you have a company with a handful of people and you wrote code, collaborated in Slack, and used task management tools for a few years, you can probably sell this data for $50k or so.
You can also do this if you're not shutting down, but it's probably not worth it.
Before you spend 20 minutes reading this article, it's worth understanding that the writer has been posting popular but consistently wrong takes for 2+ years (e.g. https://www.wheresyoured.at/peakai/ from March 2024) arguing that AI is failing, is a waste of money, is bad, will never work, etc.
Not sure where I heard this, but I'm reminded of a story about someone predicting the dotcom crash early, circa 1998. For 2 years they were demonstrably crazy, and missed out on massive stock market gains. Then they were right. (And yes, tech slowly bounced back after that.)
Predicting the timing of such a thing is notoriously difficult. I don't think being wrong about timing 2 years ago means there won't be a correction.
I'm also reminded of all the HN posts from 2007-2009 that predicted that the adoption of social networking would be a terrible thing for privacy, that it would destroy society, that people would lose their jobs over crazy shit they said on the Internet, that it would lead to the decline of trust and in-person interactions, that people would forget how to socialize, etc.
They were right about all of that but it took 15-20 years and the companies involved grew 100x in that timefold, eventually reaching trillion-dollar valuations that would've seemed insane in 2007.
There is a tremendous amount of money to be made in destroying society.
Eh, you can find HN posts predicting that literally everything will destroy privacy/society/trust/etc. Predicting doom is a popular pasttime.
What I remember from that time period is people predicting that we were in a tech bubble driven by social media, that obviously Facebook and LinkedIn were overvalued because social media was a trivial fad, and so on. Example article pulled at random:
They weren't wrong. We were in a tech bubble driven by social media. Digg, StumbleUpon, Kongregate, MySpace, Orkut, Slide, Meebo, Mahalo, Bebo, Justin.TV, etc. aren't exactly around anymore. Facebook and YouTube are the winners.
I was definitely around when that video was current, but I don't remember it. It's pretty amusing.
Ironically I feel like it captures the spirit of the then-coming 2010s boom more than the climate in 2007, though some of the language it's using is decidedly pre-mobile and more "web 2.0"-ish.
Not related to AI but, I recently rewatched "The Big Short" and your comment reminded me of it. I can't testify the accuracy of the movie, but for over year, Michael Burry was viewed as in the same manner for shorting the market, while the economy was was in a hype cycle.
I'm open-minded to arguments about AI being a financial bubble and a bad business.
I'm not open-minded to arguments about utility, given that I personally witnessed LLMs evolve from interesting but useless toys to insanely helpful tools I use every day.
I guess one of Zitron's arguments is that the utility you see today is based on subsidized costs, that if you had to pay more it might not be worth the tradeoff to you.
So the claim is the cost isn't coming down enough to make it make sense for a lot of uses in the long term. When I hear that next to the most wild claims, some by influential people, that the entire white collar workforce is going to be replaced very shortly, it's a bit of a useful reality check.
Exactly. The question is not "are people using it to do stuff?" because we know right now they are. Given free or heavily-subsidised access to powerful tools, people will use them.
If I had someone giving me free access to cranes and excavators, I'd be raving about how easy it was to build houses now. But tomorrow when I have to pay full price for them, I'm going to be making very different calculations about return on investment.
The question we need to be asking is "what is the likely full-price cost we'll have to pay for these tools, and is that cost likely to be worth paying?"
What Ed's pointing to is that the full-price cost will have to cover the capital expenditures that have been invested, or the companies which risked that capital will go bust. That gives us a floor for what the full price cost will be, and that floor seems higher than the value being offered by the tools.
Can you point to anything specific from the article that you'd describe as consistently wrong? Not disagreeing with you, but nothing popped out to me after skimming the article.
I didn't read the posted article (I don't read this author anymore because I think it's basically anti-AI ideological propaganda).
But from the article I linked back in March 2024:
"Generative AI models are expensive and compute-intensive without providing obvious, tangible mass-market use cases. Murati and Altman's futures depend heavily on keeping the world believing that development and improvement of their models' capabilities will continue a rapacious pace of progress that has unquestionably slowed, with OpenAI admitting that GPT-4 may be worse on some tasks.
As I've written before, hallucinations are a feature not a bug. These models do not "know" anything. They are mathematical behemoths generating a best guess based on training data and labeling, and thus do not "know" what you are asking it to do. You simply cannot fix them. Hallucinations are not going away."
Since then:
- hallucinations are dramatically less of a problem
- several mass market use cases have emerged, most notably coding
I think the points you raise are reasonable signals to consider, but I don't think they show the author being "consistently wrong". The overall thesis still remains plausible even though we have seen LLMs continue to improve.
> - hallucinations are dramatically less of a problem
Sure, but it remains a big enough problem that human intervention and review is still necessary for any serious work across all use cases and industries.
> - several mass market use cases have emerged, most notably coding
Coding seems to be the only one, but there are still a lot of open questions about how the market can sustain the costs, and that's without considering the market dynamics that could emerge once costs are lowered enough that open source models start to become an attractive option.
> Sure, but it remains a big enough problem that human intervention and review is still necessary for any serious work across all use cases and industries.
Another important consideration: Hallucinations getting less common/severe but not (as-good-as) solved makes them worse.
LLMs used to very obviously get things wrong. And people wouldn't trust them. Now they're good enough that people blindly trust them.
Now people just directly PR AI output with little to no manual review. We even have clowns calling for the complete abolition of directly human-authored code.
Whatever gains were had in better AI code output over the past two years I lose in having to review much more thoroughly.
> several mass market use cases have emerged, most notably coding
Most notably? This is not a mass market use case in the way the author is describing. They are asserting that the amount of spend they need to get this off the ground necessitates the entire world coming in on it, and I would say that opinion has aged pretty well. There are a lot of coders, but there are more people scratching their heads as AI is shoved into every part of their lives.
Has rate of progress increased? How does one measure that? Genuinely curious - would be very interesting to map out the "effectiveness" of each AI model vs how long it took to train/release.
From my perspective, the model gains are mostly incremental now and a lot of the gains are just from things like improving the agent harnesses. I could be wrong though.
Hallucinations are still a problem. I recently asked one to give me a quote from a book, figuring that since these AI companies have pirated all books in existence surely it can just recite a specific passage no? It hallucinated the quote, I had even told it what chapter it was in. Had I not read the book recently maybe I would've believed the hallucinated quote.
And it got me thinking, they sell these AIs as assistants, but it couldn't even look up a passage from a book. This is basic, elementary stuff, it should get it right. I would have fired this assistant right away if it were a person. Not only did it get it totally wrong, it came to me with utmost confidence that this is the quote from the book. Unreliable assistants? That's the product they're trying to sell? Get out of here with that trash. I can't trust it.
> hallucinations are dramatically less of a problem
No they aren't. The models still hallucinate just like they always did. You cannot trust them, ever, to get something right.
> several mass market use cases have emerged, most notably coding
They aren't really useful for coding based upon the above. Since you can't trust them, you have to carefully review everything they make, which in turn destroys any productivity they could've given you.
> rate of progress has increased
I have yet to see any progress. Opus 4.8 that you get today is no more effective than GPT-3.5 was. Much less would I agree that the rate of progress has increased. Only hype has increased, but there has yet to be a drop of substance.
His point is that coding is only a “market” because it is being sold at a loss. Businesses have to pay per-token prices and are saying that the cost is not justified.
Nevertheless, it all misses the point if we get to AI post-scarcity utopia. But thats a big if.
It doesn't miss the point because if we get to some AI post-scarcity utopia then the companies pouring trillions into it now are never going to make their money back on that investment.
The only way they make their money back is if everyone pays them tons of money for it.
> I believe that artificial intelligence has three quarters to prove itself before the apocalypse comes, and when it does, it will be that much worse, savaging the revenues of the biggest companies in tech. Once usage drops, so will the remarkable amounts of revenue that have flowed into big tech, and so will acres of data centers sit unused, the cloud equivalent of the massive overhiring we saw in post-lockdown Silicon Valley.
We have seen 8 quarters since. Has any of that come to pass?
if you can't predict when it will pop then you should really not predict anything. I can also predict that Google will pop. I won't tell you when but I'll tell you that it will. I'll remain thoroughly unfalsifiable and I'll keep pushing the dates.
The quality of AI doomerism takes is matched only by the quality of AI boosterism takes. Ed's kind of interesting as a temperature sensor but I don't feel like you can really take anything he writes seriously.
When asked whether it will be possible to fix Sora's videos after they've been generated, Murati said "eventually," and then couched that by saying "that's what we're trying to figure out...how to use this technology as a tool that people can edit and create with." She promised that there would "eventually" be "more steerability, control and accuracy...and reflecting of intent of what you want."
You'll "eventually" be able to add audio to Sora videos, and when asked when Sora's generative videos will be available to the public, she once again said "eventually," and when pushed said that Sora's launch would "definitely be this year, but could be a few months."
Murati, living in a world of "eventuallies," provided no technical insights, no specifics, and very few details.
What if you phrase the question from "will AI ever be useful" (a term as utterly vague as "IT") to "will it ever be able to promise the financial gains these companies are hoping? Especially with local models eating their lunch :shrug:
Yeah they seem clickable because anything Anti-AI is a bit soothing right now, but he is constantly wrong and usually is pushing the angle of "these businesses aren't even profitable!"
Instantly close the tab as soon as the popup to subscribe to his newsletter pops up.
They ain’t profitable yet. Most of the model maker’s will be gone soon. It’s unsustainable unless you’re Google who has other income coming in to support their hobby, and the Chinese model makers are spending a fraction to be six months behind and many of them will be there for the long-term because they have backup support (government) who is in the race for the long-term.
One other thing that’s working against the model makers is the hardware is getting better and the models are getting smaller and more capable. I don’t think we’re going back to the mainframe days. Local will be the endgame.
Is Ed right? Probably because in the end it’s unsustainable the companies left will be the companies that have income coming from somewhere else and there’s one large tech company that isn’t even participating in the boondoggle unless you count $1 billion dollars a year as participating ultimately there is no moat in AI model making.
Nvidia and Microsoft trying to introduce another Arm processor in a laptop of all things won’t change the tide either.
> Culture is destroyed, truth is destroyed, democracy is destroyed.
Just because those things happened in Dune doesn't mean they'll come true. It should be obvious what massive potential the thinking machines have for humanity as a whole.
> which is an extremely rationale judgement to make.
So it's "rational" to take bias into reading? Why even read? If you know what you think and refuse to accept new information then what purpose is there in consuming anything?
You should just read the comments and get a warm fuzzy that the crowd, for the time being, agrees with your intentionally static ideology.
Comments like these obviously hope they can sway the crowd before they can take an unbiased reading of the article. If the author is that wrong then the crowd here should be able to discover that on their own. If the author convinces the crowd then I'd think you'd want to present a better argument than "well, he was wrong _before_." Post hoc, ergo propter hoc, in action.
That's the exact opposite of rational. It is, in fact, a formal logical fallacy (ad hominem). His argument can be correct even if he himself is not typically correct.
On the surface, that's quite fair. However, there's one problem: it is much easier to make statements than to verify them, and that asymmetry is part of why the internet has been slowly eroding society.
It's useful/necessary to use past writing/arguments from an author to say whether they should actually receive any further critical evaluation, or be dismissed. We shouldn't say definitively "they're always wrong, so they're wrong now". However, it's reasonable to say: the author has a demonstrated lack of credibility, so we can probably assume they're wrong here, particularly if they have been wrong in this domain so many times before. Or if they happen to be correct, it's probably not strongly demonstrated by their work.
What's the point of reading someone's writings on a subject where you know they're not typically correct? How would we know what we 'learn' from Ed is right?
He's a Gary Marcus-level contrarian with none of the credentials or contributions to the industry. The "AI bubble" cope narrative is getting stale but will still appeal to luddite autists years after it has ceased to be relevant.
I disagree I think nutrition guidence is extremely important and in the precense of horrible examples nations get really unhealthy. The only country 1st world country not to have really obese people is Japan (~5% obese ~20% overweight). (~35% obsese ~70% overweight US) and I'd wager a large part of that is the fact that kids cook for themselves in school so they learn early what a reasonable meal is. They also learn how to cook not that they do that forever but setting reasonable food expectations is extremely important.
Being obese as a kid is almost causal for being obese later in life[1] as becoming obese screws up a lot of your bodies biology permenantly. You can of course change and become healthier but many lingering symptoms linger regardless of you losing weight. While still 70% obese adults were not obese as children 80% of obese children end up being obese.
Open to other ideas but school meals and peoples relationship with food is extremely important to maintaining weight in my experience.
> The only country 1st world country not to have really obese people is Japan (~5% obese ~20% overweight). (~35% obsese ~70% overweight US) and I'd wager a large part of that is the fact that kids cook for themselves in school so they learn early what a reasonable meal is.
There might also be a genetic factor, why japanese are less obese or overweight, because the difference for diabetes patients between US and japan is a lot smaller.
There is no genetic factor because when Japanese people move to the States they are as obese as america's within 2 generation. I want to find the study but I think they end up being physically lighter because of other factors but are just as obese or overweight as americas[1]. The reasoning from the paper is that Japanese 2nd generation adopted western cultures eating habits
That's clearly true, given people by and large know what's good and bad for them but their consumption choices need to factor in a much larger set of pressing constraints like price, availability, and readiness and more abstract constraints like "am I able to be at home with my child and cook for them or do I need to work a second job to make ends meet?" I will not trust a single word from RFK's mouth until he has something to say about food deserts and prices and a plan to do something about it. Until then, he's done the easiest part which bureaucrats specialize in, which is publishing an updated set of guidelines.
Long but totally heartbreaking.
reply