This is only the case because streaming services today are so cheap and convenient. Jack up the price, or decrease the convenience, and see people sail the high seas en masse again..
Indeed, the origin stories of these streaming services are deeply intertwined with combating piracy if you go back 20+ years. And I agree, if they outlaw open weight models, then only outlaws will have open weight models. But good luck finding and prosecuting the outlaws beyond a few whales.
> There are other reasons to dislike LLMs and fear or hate what AI is doing to the world. But people who feel something was "stolen" from them, who now close their blogs and turns repos private because LLMs - they're just showing they had ulterior motives for their work - which again, is fine if they were up-front about it.
100% this. Why should I be bothered that some of my work that I released freely to the world is now potentially used by millions of people as a small part of the knowledge in a state of the art AI system? I am honored!
But I guess some people just have different motivations for releasing their code, and require explicit attribution to feel honored enough to make it all worth it for them. Not me, though.
> Is a social contract being broken? Yeah, kind of, but the problems that that contract existed to solve are no longer a thing. Creation is now easy and commodified.
Exactly. The reason why we have rules and laws is because they make society better. Once the law or rule no longer makes society better, it should be repealed or ignored (or perhaps new, updated rules put in its place).
If, in the future, AI can easily create any digital content on par with the best human creators (and I am not talking just about software), then IP laws should be mostly repealed, since information has truly become free. I can imagine some justified, temporary IP restrictions even in such future (altrough enforcement will be very problematic), but current extensive IP laws will not survive.
We are not there yet, but we might get there sooner than many people think...
I don't think it will be ever-increasing. It might last a long time, but sooner or later supply will increase to meet the demand (or the demand will drop, or both) and hardware will be cheap again.
> AIs are outright terrible explainers even when they do have a watertight logical argument
I think this only applies to cutting edge mathematics (novel proofs of hard problems). I have seen it reported more than once that such AI proofs are cumbersome to follow.
But in my experience, when it comes to explaining well-established math that is already in the training data, AIs can be very good teachers (at least with recent models). Especially if you use it along with a textbook and ask it about anything that might not be explained well in the textbook.
Eh? Such houses are now common, at least here in Europe. I live in one! Entire walls were manufactured in factory, then brought on site and assembled. But the house is not from bricks, the structural material is wood and oriented strand boards (OSB).
> Jail-broken Fable level LLMs can mean constant hacking leading to public desertification of the web (best case).
A jailbroken model that can hack a website can also easily fix the vulnerability that allowed it to hack it. There is a limited number of such vulnerabilities. Over time, all vulnerabilities that are discoverable by the model would be patched and status quo would be restored (until a more powerful model is released: rinse and repeat).
And in the meantime, every site in the world gets hacked. It takes far longer to fix every vulnerability than it does to find one unpatched one. Mythos has led to huge numbers of reported vulnerabilities on every kind of software. If those were all dropped as zero days (which is effectively what an open weight model would do), attackers can choose an unpatched on at their leisure while defenders race to fix them all and update everything.
Hence, the best case. Yet, the damage is permanent. Your favourite trail walking app patching things after the attack does not undo your location getting leaked.
In your scenario, during the disruptive phase, I can imagine apps advertising 'we have Mythos backed security company defending our data'. Reminiscent of showing how much your gun is bigger than your neighbour or gangs on the street, it's truly distopian.
Yes. And for most applications it's more important that the open models get better in absolute terms and perhaps that they are competitive on a per-Watt basis.
- I think competitive open models are just an artifact of the AI race we're witnessing right now. What's the incentive for a company to spend billions researching, developing, and training a model, only to release it for free? Leading to the next point.
- Even if open models are good enough to be competitive, how are we going to run them? Doing so locally is next to impossible and I don't see that changing. The capabilities of models that you can run locally will always get better, of course, but the level of quality that is considered essential for work will always stay pinned at "near-frontier". Datacenters will always have better optimization and economies of scale, the industry will consolidate over time and eventually we'll end up with a handful of companies that operate the hardware serving 95% of all inference needs.
I'm not convinced that we can reach the fantasy world that they're trying to sell without killing the entire planet, but if we somehow do I don't see how we can avoid the world turning into a dystopian hellscape. We would need extremely radical interventions to avoid that scenario, such that these models and the hardware to run them would be owned and governed democratically, i.e. the end of capitalism.
So far we haven't seen much of that consolidation.
And a lot of people are using trailing edge models just fine already.
> [...] but the level of quality that is considered essential for work will always stay pinned at "near-frontier".
Why? When we'll finally all write our software in Lean and prove it correct and prove it fast, it won't matter that a slightly more clever model could have found a slightly nicer proof or whatever.
Just like today people happily use Python for many programs, even though rewriting in C might give you a performance boost. Good enough is often good enough.
And you are assuming they don't remain in control. Which of you is right? We don't know yet, but I don't find the arguments of card-carrying doomers any more convincing that those of card-carrying singularitarian utopians.
Historically though, technological improvement has lead to large increases of living standards for the overwhelming majority of people, so I think that past trends support the utopian view more than the doomer view.
Awfully convenient isn't it? To invent a whole class of arguments that by definition can't be falsified. You can argue for basically anything if you then tack on the excuse of "Well the world would have ended already if it came true, so by definition I won't have evidence for it"
The anthropic principle isn't providing evidence for the argument, nor is it a universal counterargument. It's just stating that the specific counterargument "well, the world has never ended before" doesn't work.
Its not a counterargument to basically anything, except as to avoid having to deal with actual evidence.
It is an argument that seems almost tailor made to have to ignore mountains of evidence against you.
In any other contexts the supposed "rationalists" would be fully in agreement that having evidence matters, and that not having any works against you.
So, in order to fight against this severe issue with their arguments, they have to invent a reason as for why the entire concept of evidence itself doesn't apply to them and they get to ignore normal evidentiary requirements.
Evidence is critically important. There is no evidence against, and plenty of evidence for. The point of the anthropic counterargument is merely that "it's never happened before" is not evidence against.
> "humanity can't be destroyed by anything, because I said so"
No, the argument is instead that the person claiming that humanity is going to be destroyed is making a fairly extraordinary claim and that requires fairly extraordinary evidence.
Or, in other words, we have tons of evidence already as for why the world ended is a fairly far out there prediction, given all the crazy people making these predictions keep turning out to be wrong.
So, you can make your extraordinary claim if you want, but really the burden is entirely on you to prove your extraordinary claim, and everyone else is free to remain on the default and completely normal end of the prediction spectrum, of believing that the world isn't going to end.
And when people do tricks like this, they are running away from the fact that they are making a wild completely out-there prediction, and hiding behind that by trying to come up with reasons as for why evidence doesn't matter and actually the burden of proof is shifted to those who have the default and boring prediction of the world not ending.
> The anthropic principle applies here: anyone warning about an existential risk will by necessity never have precedent to point to.
We have precedents of people warning about existential risks in the past, when the warnings turned out to be false, or overblown. In some cases, such overblown warnings led to serious negatives for society (demonization of nuclear power).
Yes, that is precisely my point. Any timeline with humans flourishing will never have a past history of a correct prediction of existential risk, unless you're willing to pay attention to the counterfactuals.
Those counterfactuals are important, though. We do have a history of averted disasters, albeit not as large. Ozone hole, Y2K, think about things that seemed overblown at the time, and consider whether they were actually overblown or whether there was a concerted effort to successfully avert them.
reply