In a new paper, Apple’s research team describes a creative framework that improves LLM responses to mathematical reasoning, code generation, and more. Here are the details.
Distribution and autoregression, unified
In a newly published study titled LaDiR: Latent Diffusion Enhances LLMs for Text Reasoning, Apple researchers, along with researchers from the University of California, San Diego, detail an interesting way to improve the quality of answers produced by large-scale linguistic models (LLMs) in certain domains.
In the past, we have discussed distribution models, which generate text by multiplying multiple tokens with each pass, as opposed to automatic models, which work by counting and predicting tokens one by one.
Apple has also looked at distribution models used in the prediction of protein folding and coding, which is endlessly interesting.
What LaDiR does, in short, is combine both methods: it takes the distribution during the thinking process, and then produces the final result automatically.
In addition, it actually works with multiple thinking methods in parallel, each using its own diffusion process, with the machine pushing them to explore different possibilities, thus generating a different set of candidate answers.
They explain that during inference, when the model comes up with what and how it will respond to the user’s input, LaDiR generates a series of hidden reasoning blocks, each of which starts as a random pattern (or, sound) and is gradually refined into a coherent step.
When the model decides it has made enough assumptions, it switches to generating the final answer automatically, one token at a time.
An important detail is that LaDiR can use several of these ways of thinking in parallel, with a machine that encourages it to explore different possibilities to avoid all of them converging on the same idea early on, which defeats the purpose of the whole thing.
Importantly, LaDiR is not a new model per se, but rather a framework that builds on existing language models. It changes the way they think about the problem, rather than changing it entirely.
How LaDiR works
For the study, the researchers used LaDiR on Meta’s LLaMA 3.1 8B for mathematical reasoning and puzzle planning, and Qwen3-8B-Base for coding.
In statistical benchmarks, LaDiR achieved higher accuracy than existing methods and showed strong performance even on the most difficult, non-distributive tasks.
In code generation benchmarks such as HumanEval, LaDiR produced more reliable results, outperforming standard deviations by a noticeable edge, especially on difficult problems.
And in puzzle-style programming tasks, such as the countdown game, LaDiR tested a wider range of valid answers than any base model, and found correct solutions reliably above all general-purpose bases. However, it has fallen short of a special, task-specific model of single-trial accuracy.
Although some aspects of the LaDiR paper can be very technical, it is worth reading if you are interested in the inner workings of large language models, and new ways to improve performance in text generation.
To read the full paper, follow this link.
It’s worth checking out on Amazon
FTC: We use auto affiliate links to earn income. More.