I am not an expert, but I do understand the distinction that is being made here. It makes sense to describe the result of pre-training as a ‘next token’ predictor as that’s what it’s been trained to do, not because it’s an autoregressive architecture that produces tokens one at a time.

If this base is then trained using RL towards a different objective (maths and coding), the model becomes fundamentally a different thing and the recent models are clear evidence of that, regardless of they fact they remain autoregressive.