You say this as if it is a bad thing (or am I misinterpreting here?), but compatibility is enormously valuable. That's why the strategy of choosing compatibility over cleanliness of architecture is so widespread in successful complex systems - ISAs, OSs, the Web, programming languages, etc etc.
It's hard to love the resulting complexities, but remaining compatible really is almost always the right thing to do.
If we were in the habit of recompiling from source with each CPU generation (this presumes source is available), that would allow much more innovation in machine language. And then the design of that machine language would more closely resemble the actual design of the CPU.
But there are such significant advantage to remaining binary compatible, that I'm not surprised with how things actually turned out.
This is an interesting design question - which layer should be backwards compatible and under which significant freedom of design is allowed.
I'm not at all sure that the best answer is "source code". For x86 CPUs the compatible layer has been the ISA and that has turned out pretty damn well, suggesting that source compatibility is not the only and may not be the most effective choice.
Particularly if the source is C, which is rather ambiguous compared to most machine languages.
I'm not at all sure that the best answer is "source code". For x86 CPUs the compatible layer has been the ISA and that has turned out pretty damn well, suggesting that source compatibility is not the only and may not be the most effective choice.
I disagree, though I see your point.
If the x86 instruction set had been designed to be implemented in various ways depending on the available semiconductor technology, then that would be a much better situation. However, the basics of x86 were designed in the 1980s to be directly implemented in the hardware of the time.
If you were designing an ISA expressly to insulate software from the hardware, I'm sure there are dozens of design choices that would be made differently. Ditto for amd-64.
I think most of the opportunity for "invisible" performance improvement today is below the ISA -- new branch predictors, new prefetchers, smarter cache hierarchy eviction/replacement heuristics. These are problems that are fundamental to the code: predicting dynamic behavior and having the right data nearby. (At least, this is what I saw again and again while staring at pipeline traces while working on a commercial microarchitecture.) A new ISA won't fix that.
Or maybe said in a different way, even if we recompile from C to a new ISA, we still have to execute the code that (by virtue of its algorithm) has unpredictable branches and lots of cache misses.
I do think there's room for innovation at a higher semantic level, e.g. maybe a really fast thread message-passing/synchronization mechanism with a lot of hardware thread contexts. Or lots of other ideas, e.g. "helper threads" (user-provided speculative prefetch code) or "informing memory operations" (load-and-branch-on-cache-miss). (All of these come out of comparch research in the 90's and 00's.) But all of those would require programmer cooperation, since they introduce new semantics.
You say this as if it is a bad thing (or am I misinterpreting here?), but compatibility is enormously valuable. That's why the strategy of choosing compatibility over cleanliness of architecture is so widespread in successful complex systems - ISAs, OSs, the Web, programming languages, etc etc.
It's hard to love the resulting complexities, but remaining compatible really is almost always the right thing to do.