Some LLMs are not trained with the right amount of parameters or the right learning rate. But the static nature of LLMs is the biggest problem. You need neuromorphic hardware and spiking neural networks to address the issue. In the meantime I think quick fixes will be attempted such as forward 2x passes. My friend Fred says that just adding small random Gaussian noise to the parameters can also help. Obviously human brains are very noisy but somehow very efficient too.