Your personal vision. I also dont like to be lied.
Is there any way we can meet in the middle without referencing individuals with their own ideas of the world? If the answer is yes, we're in the good path even if we disagree.
Interesting perspective. I think you have a right, whats yours is yours to do as you please with - you know?
But can I give you another "viewpoint"? I guess it's like, "Wow, my code, my work, what came from my brain, my fingers" - it essentially lives forever, if you think about it - it becomes embedded and compressed inside weights/tokens. Like - part of you is there.
I guess it's cool. For me it's just to know that like this super intelligent things deep down actually knows who I am - my code is in it's architecture, it gives me feeling of honor in some way. You know?
you comment did no such thing, in a way that would be recognizable by anyone else. it probably would have been a good comment if you did expand on that though, in some meaningful way.
I use ai daily but reciting the talking points that killed what open source used to mean and using them to further separate original authors from the impact of their work
That’s ai pilled
I write code under the mit license
I know the risk
Helping humans still makes it worth it
And technically these AI companies should have a /licenses route that lists every MIT piece of code their model was trained on.
That’s literally the only expectation I have from anyone as an active author using the MIT license, getting cited.
I think the legal AI defense is that the models themselves are a bastardized form of dynamic linking. I say the models are statically linked though, so they need to spill their sources.
That’d be my question to the person I disrespected:
Why should I, as someone that’s not hypothetically giving back in code, continue to do so, when the social contract has been broken, where the always minimal expectation has been: Say my name?
I literally came up with a unique sdk for all “my elves” such that I can in fact see people in court for mishandling the software supply chain.
There’s a lot of software licensing misinformation out there and including my name and email with the rest of the license text is such a simple thing to misunderstand.
I’m sorry for any and all infractions you’ve committed across all MIT authors to date.
I’m not really planning to take anyone to court, but if you really believe what you’ve said to me here and you’ve been writing code that follows those beliefs
I’m not a lawyer, but you should probably consult a lawyer.
That’s about the inputs going in and even in the first paragraph disclaims other courts may side differently
We’re still not in the legal territory of the outputs on the other side, which is what’s actually more interesting to me.
The technical political maneuvering of to two sides of the accelerationist movement is the fair use bits, which is not even the point I’m debating.
Even there, they are discussing complete bodies of work as source material, which is exactly my point. My stuff can go in, but they still have to cite that I’m in there, which does matter on the other side.
I don’t think anyone deserves to be just weights and algorithms in a dark and shuddered library.
They want this to be a legal laundering device and that’s the bit I’m hung up on.
I literally re-wrote how I wrote software post-ai to ai pill the ai, such that, when these models produce ASTs that match my signature, I do have a legal defense.
Can I give you a compliment? Like you are a super talented terminal designer. I like the style, it's cool. Very terminal...but it like has great UX. main-dash.png - I love it.
Well, Mac chips are badass for training / inference - super underrated. I mean, I've literally run epochs on cloud Nvidia GPU Servers...compared to running them locally (M chip) - and look, not trying to burn any houses down but...eh...Apple does really really well.
The good news for you, you can chain like a bunch / couple of them together and run the largest open source models around. But extremely expensive route - but probably the easiest and smoothest way.
If you're planning on running this on Apple - you can do some stuff with Metal directly...in PyTorch it's 'mcu' if I remember?
I think your llama.cpp route is good - I wouldn't go the Ollama route - I mean great to start, but IMHO: get the models directly, learn the layers and how the heads work as best as you can, make an effort to understand what's going on - well you don't have to, but, I think the models appreciate the effort - respect goes far.
You asked us...well, first tell us what's your real driver? You have three years on local infrastructure? What does that even mean - you're running Ollama Llama_70b for 3 years?
Whats your stack?
And none of that hardware can run larger models, smaller tiny ones, or highly quantized versions of larger ones sure. Or do you have something important to say?
Our main driver and hypothesis was to work with regulated industry. We worked with a few large enterprise clients in defence and industry for R&D and IP use cases mostly.
Our stack changes per project, adapting to client needs and infra: Llama 70B on a Mac Studio M1 with Ollama in 2024, vLLM on 4xH100 private cloud for larger deployments. Most recently, we've been working on a custom workstation with 2x RTX PRO 6000 Blackwell Max-Q + 1.1TB DDR5 to run larger models locally using SGLang and KTransformers.
The question isn't rhetorical, I'm trying to understand if the demand we see in regulated sectors is the whole market or if there's broader adoption I'm missing.
Cool, so you are basically doing local onsite deployments? The H100's are nice. I'm not that rich, so I have some 4xV100 32GB SXM2....server, dual socket - it's OK for inference. You can get when with V100s, RAM, etc for $10-$12k all in used stuff.
I run largest models I can, DeepSeek, adding a few more soon. The fact that I can have a premier high end model run locally is main interest, a 70B model is pointless unless it's a specific task based special model or whatever Text to speech, etc.
I am more interested in ditching Nvidia for AMD Chips+GPUs, but not even ROCm - just run with OpenGL / Vulkan weights in shaders. Faster, more control, better performance for MY architecture, etc. This is the goal.
I don't think many people are running models, maybe outside of a company? I guess you are company/industry focused, I am just a programmer / personal.
People don't see a need I guess? It's complicated. Well - actually it's NOT if you have lots of money to buy all the right stuff, brand new, etc.
For regular guys like me, we have to be creative to get shit to run in the best way, it's all we can afford.
Of course the model is lobotomized - you're giving 0 collaboration, your asking it to even make all the files and folders for you ('mkdir'), all cold instructions/context overlayed by these INSERT_TOOL/IDE_HERE.
It gets inputs and gives outputs. Quality varies - I suppose?
Why don't you just talk to the model in the chat and make the fucking files yourself you lazy fucks? Like collaborate...I don't think thats easy in a 250px wide terminal. I think you would just get good results if you just talk to the model, you know? Like work with the model in the chat window.
"The 'bit-level magic' here is misleading. They're using integer representations and calling them bits. A true bit-level approach would encode positions as pure binary streams. For example, their promotion string '00000034' is 8 bytes (64 bits), not the claimed 9 bits. Has anyone implemented this with actual bitwise operations instead of integer packing?
TLDR: You stupi...lovely folks, need learn what a bit is. What is a byte, and why integers are NOT BITS.
And no one called this post out from years ago? Crazy.
Someone ask, and I will show you how to do it in 24 BYTES IN BINARY ENCODING FOR REAL - no fake "integers are bits". And can we use compression techniques? We can get this to 10-12 bytes maybe.
So it beats the fake "26 bytes" and my version is actually real binary bits.
You're proving my point. Yes, 495 possibilities CAN be stored in 9 bits. But the article shows STRING '00000034' (64 bits) as an example, not the actual 9-bit binary encoding. That's exactly the problem - claiming bit-level compression while showing byte-level examples.
And if you look at article, nothing is binary encoded, they are all integer representations all the way down.
Please someone show me a BIT implementation of this - THESE ARE BITS 0 1 0 1 1 0 - It's called BINARY. There are no 9, or 5, or 3 or 4.....That isn't how logic gates work.
A 3 / INT is 8 BITS...1 BYTE.
HINT: I'm right.
And you never answered my question:
"Has anyone implemented this with actual bitwise operations instead of integer packing?"
Still waiting to see these "9-bit" "bytes"."00000034".
Again, show me. There is no such thing as a 9-bit byte, that isn't how CPUs or computation work. ITS 8 BITS 1 BYTE, that is transistor / gate design architecture.
Yes 9 bits is 2 bytes. The article confusingly says 18 bits = ~2 bytes. It is the "about 2 bytes" that is confusing. They probably mean that an extra bit won't matter too much since we are bit packing the games in a contiguous stream.
BUT
In the article they don't mean that 00000034 is a bit or a byte. It is one of the possibilities and there are 495 of them and if you index each possibly in a 2 byte integer, you can decode it back to that string and get a representation of the promotions that happened in any game.
You understand me, this is most important. And thank you for explaining this exercise - but to be honest, if the article says "How to store a chess position in 26 bytes"
And you actually cannot store this in 26 bytes based on your implementation, and then you show integer bits and bytes that aren't even binary...eh.
And to be honest, like how about we store the chess position in 1 bit.
I will execute some chess position program in 1 bit, ON / 1. How about that for ultra compression? Lets just pretend those other random bits and bytes don't exist I mean (...but they do...) - it's stored somewhere else, but "HOW TO STORE A CHESS POSITION IN 1 BIT" - but ok, fine I will play "pretend" |How to store a chess position in 26 bytes (2022)|
