What happened to the "reverse compiler" LLM restrictors?
The last step of an LLM is to take a softmax of the predictions and then generating a token from that. But there was tooling that would just generate all allowed next tokens from a grammar (e.g. restrict to valid JSON), zeroing all the ones not allowed and then picking the best among the allowed tokens.
This seems to taking an approach from the pre-transformer days. Seq-to-seq is hard and we don't always need it. So let's do seq-to-1 because it's often way easier to get it training properly and so you can often get it optimized way better. And, more generally, make sure to pick the best option out of the possibilities: 1-to-1, 1-to-seq, seq-to-1 and seq-to-seq. Where seq-to-seq requires far more resources than any other option and so it's a case of "please don't".
Also note that "1" only means the input is fixed. It does not mean 1 number or ... it just means fixed. The best image description models remained 1-to-seq models 4 years or so after transformers were introduced. Even ASR models remained 1-to-seq + CTC to stitch overlapping parts together to a final prediction ... I'm not sure if they lasted all the way to whisper release.
Even today training transformers remains expensive. So this should at least be a way to be a lot cheaper than any LLM can hope to be.
And I really like the doom demo. Obviously a pretty stupid model which is really cheap to run can still get a robot walking, if you run it quickly enough. That's how we get insects and mice and ...
And one might even add that biologically, humans aren't smart, or at least, most of the human nervous system isn't smart, compared to the whole, and does work independently if needed (and possible). The human mind is a LOOOOOOOONG chain of fast-but-stupid-and-totally-blind -> slightly-slower-but-smarter-and-not-entirely-blind -> slower-smarter-and-actually-senses-things -> all-information-you-could-want-but-at-most-1-signal-per-minute. We have "neural circuits" (using Bishop's definition) that can run at >2khz (2000+ tok/s, say, but you probably can't teach anything more than averaging) and on the other end up to our frontal lobe that takes one decision per week if it feels like working hard, and seems to decide on it's prediction of the future weeks to months out. Months or years if you're 40 or older.
I think you mean "Because the UK wants them, only 2% or so of illegal immigrants arrive by boats".
Boris Johnson did Brexit "to stop the immigrants" ... and opened the floodgates. The very opposite of what his campaign promised. The very opposite of WHY the British public wanted Brexit.
Sssshhhh stating this for his whole life was one reason Warren Buffet got villified.
His point was he's an old guy. And Reagan was better than Nixon who was better than Bush junior who was better Trump ... but they all sucked and all of them were the worst thing ever to happen to America.
So yeah, Trump is the worst that has happened to America ... so far.
(and if you point out that "The American age ending" what so many people seem to want will obviously lead to democratic presidents sucking up to Putin as well ...)
Yeah, Buffet is right, the trend has been downward, although I think Bush Sr. might have been the exception. And I do agree that the American age is ending.
With trump tho, it just seems worse. Course the only thing I have to go on is evidence and "walk like a duck-isms" but with Bush jr and before, it seemed like these guys at least appeared patrotic and they didn't look so malignantly treasonous and I think they put forth the idea that while they were conservative, at least the pretense was that they honored the office and the constitution. so since that is gone the progress toward national decline now seems to be a deliberate operation on behalf of people who are no longer interested in democracy. This is new to me.
Yes but if Bush Sr listened to his wife, and additionally, his wife loved him enough to not leave him, thats something right? Or do you think someone with Barbara Bush's elevated love of america lost that love long enough to marry Bush Sr given that there's a theory that he was a bad person too?
I still offer the alternate theory that Bush Sr. was indeed a pause in the GOP's descent into the current realm of evil and contempt for the population of our country. Hell, I read trump doesn't even like his own voters and that rings so true haha
The GOP got so devoid of patriotism so fast it put the Democrats to shame.
I haven't been able to trust any political parties for a while.
After Obama, the GOP couldn't elevate anybody, all they could do is sink so low that only the worthless attention-getting blowhards were left standing.
I was in Texas before during and after Bush Sr was Governor, and distinctly remember the way he was not good enough for Texas before he became not good enough for America. But who is? The further back you go, a political figure is more like a normal person, usually.
The parties have always been the bad influence over the man until they got so bad that Trump made the boldest example as the party dropped the ball and let the man be an even worse influence on the party.
I watched both parties prove their incompetence in my lifetime and witnessed the outcome on a daily basis in real time. Confirmation all took place in the last part of the 20th century. Well first a warning shot from offshore when the Communist Party of Cuba proved their incompetence. Mao was an echo in the distance. Then after a number of years comes the GOP, China picks up the ball the first time, and not that many years later the Democrats. In there somewhere the Soviet Communists gave up the ghost, and where substitutes exist they're still pretty sketchy.
The Chinese Communists are incompetent as ever, that hasn't changed for the longest time, but now they look like a bastion of stability compared to the chaos other parties seek out or tolerate.
The thing in common with the shithole approach is when there's anything like a dictator no matter what they call themselves.
We may already have the hindsight needed to pinpoint the extent of the "American Century" as we knew it. Hard to drag it into the 21st century very far when the parties which built it haven't been with us for so long.
One thing nobody can deny from any party today, I've said it before and that's a big thing that makes people talk about the death of the American Age not just outside the borders or overseas but domestically. Every single day Trump comes on TV and hammers home the fact that Obama was the final President to be recognized as the "leader of the free world". And the party has packed the government with minions in lockstep who can only iterate "what's a Free World?" You can't paint a pretty picture here.
The American Age either likely paused, perhaps quite a bit before Obama, or definitely right afterward, and whether this was a near-fatal injury or a morbid conclusion, it's been quite a while and it's starting to smell too far decomposed for as many people to ignore as there were.
> The Chinese Communists are incompetent as ever, that hasn't changed for the longest time, but now they look like a bastion of stability compared to the chaos other parties seek out or tolerate.
Chinese communists are only looking competent for the same reason every other communist looked competent (but you need to be like 50+ years old to remember what that reason is):
1) they lie
2) they're actually pretty good at preventing people from talking (even the China CCP is now threatening the families of Chinese abroad)
Chinese economic figures (which they openly "partly" falsify, openly as in the government says as much, from oil supplies to youth unemployment) now look like China ever exceeding US economic output is a tossup, 50% odds at best. And that's the falsified figures. Maybe I'm old but I remember the Soviet Union when I was a teenager and I do feel a bit like "I've seen this movie before ...". At least they managed to stay out of a shooting war, but worryingly they keep announcing they're about to start one.
> Every single day Trump comes on TV and hammers home the fact that Obama was the final President to be recognized as the "leader of the free world". And ...
The problem here is the basic assumption that US becoming the leader of the free world had anything to do with the quality of US government. That's just not what happened. I love Obama, but ... he doesn't deserve the credit there. The US became leader of the free world because the rest of the world tried to fight to the death over debt, Aryan and Japanese racial superiority and near-completely destroyed itself in a 20-year series of atrocities that were far worse in reality than even the WW2 movies show, followed by a 40 year "cold war" where communists tore themselves and anyone they touched to shreds, hundreds of millions of victims across almost all of Asia and half of Africa.
In the US, we could elect Andrew Breitbart as the next president and we'd still be nowhere near that.
And, of course, the US indeed kind of stopped being the leader of the free world, and let's see. Open war in Europe, destroying about thousand to 1500 lives per day, as well as about 100 million in economic output daily. War in the middle east, I'm not even going to count since that's been a constant for about 1500 years now. Asia has several open conflicts for the moment somewhat smaller than the Ukraine war (except in area, and total number of people affected) but Yemen and Myanmar are in open warfare right now. Africa has about a dozen open wars, most of which nobody seems to care about.
Frankly, unless this stops, is there really any doubt the US will come out ahead again? The world is getting ready to tear itself to shreds again. EVEN with the total and corrupt disaster called Trump at the helm all anyone needs to do is just stay out of the conflicts for ... perhaps the first year once they really kick off?
And are these countries trying to stop that? No, needless to say, if anything it looks like it'll intensify. The EU is now spending about half a trillion per year to get ready for war ... That's half a trillion per year NOT counting the Ukraine war AND drafting laws have been reintroduced in Germany, the Netherlands, Belgium, Spain and Greece (if I can give anyone reading this one bit of advice: if you're male, GET THE FUCK OUT OF THOSE COUNTRIES BEFORE THOSE LAWS ARE "activated". And if you're female, at least you'll live but let's just not pretend it's going to go well for you). Japan "self-defense" spending (it's still called that) is skyrocketing, and really half of Asia is doing the same, including India. Yesterday der Spiegel said open war between Germany and Russia is "a matter of time", and as I said before China is essentially announcing a military attack on Taiwan, for now without giving a date.
The US never became the world's superpower because it did anything "moral" or had "character" or anything remotely like that. The US became the world's superpower because Europe and Japan decided to do WW2 and destroy everything, while committing atrocity after atrocity after atrocity.
It was never anything about the US per se that made it a superpower. It was to a much greater extent incredible moral, economic and military failings on the part of everyone else. From Albania to Zimbabwe, Morocco to China.
The US lost superpower status and boom. There are big shooting wars again. In Europe. In Asia. I was kind of hoping it to take at least a decade or two after the US lost control. Then again, maybe it has.
Yep. Simple, ~20m2-30m2 square boxes, densily packed (with ideally thick walls) is by far the easiest kind of space to use. Then maybe one 50m2 for having a big living room.
> In fact, at least in my (small) city, fully renovated old buildings ... are where the most expensive flats per m2 are. Considerably more than in new buildings.
Isn't that because of location? These places tend to be right next to the rail/bus station in the middle of the city.
A specific example. The very same building developer completed these two buildings and offered the flats for sale more or less simultaneously, in the second half of last year IIRC.
Building 1. Renovated early 20th century building (originally built in 1923).
Google Maps coordinates (the renovation is not yet visible in Google Maps): https://www.google.com/maps/@43.3621801,-8.4070271,11a,75y,2...
Photos of the renovated building: https://ammapromocion.com/plaza-de-vigo/
Flats have 117 useful sqm and the middle flats were being offered at between 1.1M-1.2M €, and sold fast (you can still buy the 1st floor flat for 950K. But it's a 1st floor with a bar directly below, which reduces its value rather sharply. This is Spain, bars are noisy and open until late).
The distance between both buildings according to Google Maps is 290 m, a 4-minute walk, so they are basically in the same neighborhood. You can argue that the new one is in a worse location because it has more traffic and the street is uglier, and it's true (although the other is in an area with many bars, so I think not everyone would prefer its location). Anyway, that kind of difference could be worth at most 50K in this market, being generous - and most likely more than fully offset by fact that the newly built flats include garage while the renovated ones don't have it! Anyhow, it definitely doesn't explain a 400K+ price difference.
It's just one concrete example, but I've seen many (I was recently looking for flats... finally I settled from one from the 1940s, building not renovated :)). Of course I don't know the trends outside my city, but at least here, fully renovated old buildings are the apex of the housing market.
I've also actually renovated my old place. I did it because I didn't want to rebuild the place entirely (there was no need, it was woven brick, more solid than most bunkers), and because I would have had to agree with the neigbors, which was ... let's politely call it "not guaranteed".
But very quickly it became clear why that was the right decision: building codes. People legislate building codes, but of course they're unwilling to DO anything. So old buildings, built against old building codes, don't have to change, and that means they effectively get to ignore building codes. If I rebuilt the place I would have had to move the house (as in physically move it, just 4 meters, but it's a house, 4 meters might as well be 400 kilometers, you have to rebuild it). I would have to give up at least 40% of the surface area (the "ground surface") AND one of the floors. Net result: the new house would have been about half the living surface of the old one.em.
So obviously the renovated house is worth a lot more than a new house: for starters it's double the size!
It's even self defeating: the isolation of the old house is better. Yes, people legislate modern isolation materials but if the house MUST be a lot smaller there must be a lot more contact area with the cold air, because it wouldn't be built against the neighbor's house anymore and a LOT less thermal mass (thermal mass = free heating, more or less) so you're starting from a very bad starting position.
Here's a thought. Imagine roughly 5% of available apartments are in renovated old buildings. I don't know what the real figure is, doesn't matter I just assume it's significantly lower than newly built apartments.
Now also imagine that 20% of people prefer those old buildings.
Per apartment then, there's 4 times as many interested buyers than per modern apartment, and the buyers of the old apartments are looking at a much slimmer market so they're willing to pay much more for the right one because they're unlikely to see a comparable one soon.
Also I imagine bringing an old building up to modern standards is quite expensive so that also explains some of the price.
I don't really know anything, I just think what I'm writing makes sense.
> The ideal scenario is everyone suspects it's you but can't prove it.
Why? EU countries have shown time and time again they won't do anything. I mean for God's sake, Russia was caught sending THREE remote controlled flying bomb on a German airfield, with another one found by a passerby before it was launched.
They contained professionally made Russian explosives. There is no doubt who did it. This was not the first, or the 10th time they got caught. I mean Russia murdered someone in the UK with polonium, for God's sake.
The idea that the Kremlin is worried in the slightest that you can prove it's them is absurd. They leave their fingerprints knowing there will be no reaction.
> The idea that the Kremlin is worried in the slightest that you can prove it's them is absurd.
I mean they are. That is the reason they haven’t sent balistic missiles to the same airfield. They could have sent the same agents who placed those drones to the fence of the airport with a shoulder mounted missile and shot down the planes as they were departing. Or even shoot them down on the ground.
Or similarly they could send a submarine and disrupt the railway services of the Netherlands with missiles from that.
Those would be clearly attributable actions which trigger an immediate military response. That is the reason why they don’t do them.
You look at the actions they take and conclude that they don’t have any restraint or worries. But you should also be looking at the actions they could have but did not take to see that they are in fact restrained and probably quite a bit worried. If they werent they would play the game differently.
I think it's to slowly increase offenses _that won't lead to military response_. The response to each individual incident, worse than the one before, kinetic action can be delayed because "this was only a little worse than the last one".
This compounds because NATO needs to act collectively to effectively act militarily (maybe Poland could go it alone).
It's European countries that are taking this strategy, any Russia sabotage is aimed at deterring this widening of the conflict - not to encourage it.
Originally Europe didn't want to supply tanks, then it was only short range missiles, then it was longer range missiles as long as they weren't used to attack targets inside Russia - now it's actively supporting doing military strikes deep into Russia, and openly talking about European boots on the ground in Ukraine ( not that they likely didn't have deniable special forces there from the beginning ).
Strategically it's not in Russia's interests at all to have a wider escalation right now - they have their hands full. Any Russia sabotage is a direct response to missiles and drones using Western technology blowing up warehouses and oil installations deep inside Russia - widening the war is not in their interest or their aim here - they are trying to deter that.
There is the wider question of whether Russia's long term goal is to take back the old soviet states bit by bit in a way that doesn't trigger a full response - but I'd argue there is already a clear red line there - and that's the Nato border.
Really? Unless they are complete morons they realize something. If you take EU governments, any random country, as a general rule government expenditures go like this:
35% pensions (usually also has some health care here)
35% social security and health care
15% education
15% everything else, roads, bridges, government buildings, ... (army is in here @ maybe 5%)
They're also drowning in debt, of the non-tiny, only Germany does better than the US. So they can't realistically borrow to finance this for 5 years. They're trying to cheat, and treat the EU like a separate country (meaning get investors to treat the EU as a separate state that is not on the hook for the debt of member countries but levies it's own taxes, then borrow against that, but EU countries are cheating, for example using money the EU borrowed for military purposes to fund general expenses like fixing roads)
If the EU countries mobilize the army will go to 20-25% of government expenditures easily.
That means either pensions or social security HAVE TO go down 15% (or 7.5% each). That will cause a revolt since a great many people can't survive like that. At that point there will be no choice for a lot of people but to tear the government down.
part of the answer is the peace time dividend. the decrease in european spending has mostly been used for lowering or abolishing taxes on the corporations and the wealthy. if this was to be reversed (because we're not in a peace time anymore), the added tax income could be used to finance the re-armament.
i agree with this. the kremlin sees this as a win-win strategy. there are two options:
1. either their actions are ignored by the victim states, which makes this very cheap and effective sabotage to weaken EU members.
2. or alternatively, if there's military retaliation, putin can finally do the mobilization he promised not to do for his war on ukraine, because he can now assert that the country is under attack by foreign nations.
the EU / the member states are now in a pickle: they know it's russia but can't say it out loud because then they'd have to take action. they also can't do nothing because then russian sabotage could wreak havoc with the population and economy. so what can they do?
1. military support of ukraine
2. further sanctions of the russian economy, which includes getting rid of dependencies on russian goods
3. re-arming and modernizing their own military (i.e. introducing modern drone warfare capability, launching their own satelite intelligence, ...)
4. start opposing/controlling russian influence by local malicious actors, i.e. russia friendly right wing political parties (which is somewhat complicated but not impossible in a democratic setting)
Because being too blatant will galvanize the public and create support for open war.
The goal is to ensure all reasonable people know, while keeping enough deniability that Russian-funded right-wing groups can sow doubt about attribution and response policy’s
They don’t care whether it can be proven in court or by people paying close attention. They care whether it can’t be denied even by their aligned agents in western countries.
Maybe it depends on the country, but here the public automatically assumes everything like this done by Russia and these increase support for open war. I don't think Russia wants deeper European involvement in the war, so I'm having trouble understanding why Russia would be doing this. They're already having difficulties in Ukraine, how would angering the European public and increasing the support for further involvement help Russia achieve its goals?
In many countries, russia’s psyop tactics do work frighteningly well, and we see this resulting in a renaissance of the far-right, who are explicitly against supporting Ukraine, and explicitly seek the rehabilitation of russia.
Exactly. If anything, the country that would benefit most from heightened Europe-Russia tensions (as a result of these sorts of infrastructure "accidents") isn't even on the continent. I'm not ruling out Russia completely, but it would sure be stupid for them to effect this strategy. Additionally, Russians weren't the ones that blew up a pipeline at the start of the Russia-Ukraine war.
Look up how successful the US has been in getting right wing crazy people elected as the heads of state in nearly every Central and South American country by funding right-wing groups, with the exception of Mexico and Brazil (but top minds are working on this).
Russia has been trying to get locals to elect governments on their side and even revolt and destroy their own countries to do so ... since 1917. They used the red army to make this happen in most of Eastern Europe, half of Asia, Africa, South America, Middle America ... and they're still at it in quite a few places (like Africa). They fought until their state collapsed to change the government in Afghanistan.
Just because the US does something bad, does not mean others don't do 100x worse. In both meanings of the word.
But people want laws to not have to face this, and yet systematically refuse to actually do anything when those laws get violated.
A law is fundamentally what's enforced, not what's on the paper. Which means if countries vote against any kind of enforcement, like happened here, those laws (whether rights or something else) don't exist. And they don't exist because regular people may shout about them (most don't), but they won't actually do anything, which, for most people, means pay for it.
What's worrying is that human rights are definitely on the same train. When you critique this you get an angry retort that for example China commits large scale human rights violations against it's own population, and how unreasonable it is to expect anyone to do anything about it. Okay, but then we don't have human rights, please don't claim otherwise.
We’ve seen this doesn’t work nearly as well as the homogeneous culture people would like us to believe. Ambrose Bierce nailed it (paraphrasing) the bible is an excellent book for your neighbour.
Laws change morals and ethics over time. They affect each other. Laws change what people think is moral (just look at how many people think it is moral to sell alcohol but immoral to sell cannabis. That of course is changing now as laws are changing).
I think the point is more, "you can't legislate holding those values".
Laws are informed by the interests (which may include morals) of the law-makers, and many people do base their morals in part on what laws exist, but most people believe that at least some laws are immoral, and some immoralities are legal.
Many law-makers want people to view illegalities as immoralities, but that's not how most humans operate.
The goal is the society by and large keeps those values as a whole. The values inform the laws and lead to a certain result that, in theory, reflects the value (politicians decide murder isn’t ok -> murder illegal -> fewer murders -> society now, in effect, broadly reflects a people that don’t think murder is ok). Whether people “buy in” is a different matter and is less material, though given enough time laws can cause people to shift culturally (morally, ethically). A society with lax rules on dumping trash will likely lead to more people not caring about the ethics of it over time as legal permissibility can lead to shifts in what we think is morally ok.
Much of Western Europe/Scandinavia has a stronger renewables culture than the US, for instance, but a lot of that was originally (and still is) driven by necessity. Fuel is more expensive, they don’t have as much space to dump things or build, etc. so they need to be more efficient which has also led to a “green culture.”
Laws and culture/morality/etc. aren’t as independent as you think IMO
A lot of messaging is much more propaganda than reality. For instance it's trivial to turn things around against the US where we have the highest prisoner population in the world, by an overwhelmingly wide margin, disproportionately made up of a single ethnic minority, who are then are pushed into forced labor. Is that because we're engaged in large scale human rights violations against our own population and enabling defacto slavery for profit generation?
Certainly not, but it's easy to frame it in that way. And I think it's something similar with China. They've had major problems with separatist movements and terrorism from the Xinjiang/Uyghur region and are similarly awkwardly trying to resolve it in a way that is extremely easy to frame as being the greatest crime against humanity since Justin Bieber. In the US we have nothing really comparable because the only violent/separatist movements have been extremely small scale. I expect that if we had something on an even remotely comparable scale that our reaction certainly wouldn't be more 'humane.'
It's all just geopolitical poo flinging.
----
Space is going to be weaponized, simply because it's a prisoner's dilemma. If everybody chooses not to weaponize space except one person, that one person now has a massive military advantage over everybody else. No country is going to accept this, and consequently I expect that space has probably been weaponized for decades. The benefit treaties, and of the charade of pretending everybody is abiding them, is that it discourages use of such weapons except as a last resort. Normalizing them is dangerous simply because it opens the door to their deployment in more mundane contexts, like a geopolitical proxy war, rather than as a means of last resort.
> Is that because we're engaged in large scale human rights violations against our own population and enabling defacto slavery for profit generation?
Yes, that is absolutely correct: the United States is engaged in large-scale human rights violations against our own population and enabling slavery for profit generation. You nailed it. It's not even "de-facto" slavery, it's de-jure slavery on the actual books. Slavery is not (fully) illegal in the United States.
> I think it's something similar with China
China also commits wholesale human rights violations against its own people, although the scale is much worse. I strongly doubt the U.S. actually has more prisoners than China, it's just that we actually count them all.
> They've had major problems with separatist movements and terrorism from the Xinjiang/Uyghur region and are similarly awkwardly trying to resolve it
This is assuming good faith to the point way beyond naivete. The Chinese government is not "trying to resolve terrorism", they are committing mass murder and ethnic genocide.
> extremely easy to frame as being the greatest crime against humanity since Justin Bieber
Are we doing Whataboutism here? Let criticisms stand. I can say "it's wrong to round up ethnic minorities and put them in concentration camps" without it having to be the greatest crime against humanity ever. Are we really going to be forgiving all actions because someone in history did something worse?
> It's all just geopolitical poo flinging
Not everything is always the same. While criticisms can be levied against every country, scale and impact matter. I'm sure someone could come in and point out problems with the government of The Netherlands, and those problems should matter to the Dutch people and they should call them out and work on fixing them. But that doesn't mean it's the same as what China is doing or what for-profit prisons in the U.S. are doing, nor would it be hypocritical or "geopolitical poo flinging" for The Netherlands to be critical of those practices.
> A lot of messaging is much more propaganda than reality.
So cloudflare sees your plaintext. Btw: tailscale does not (but ssl errors and warnings are unavoidable)
reply