You've asked AI for a training plan, or at least thought about it. The real question is how you can tell whether the result is any good. A new study had three experts rate plans like these. The result: the three barely agreed.
AI Training Plans and Expert Review: Why Three Experts Barely Agreed
Christopher Klenk5 min readAt a glance
Researchers had Gemini 2.5 write training plans for three fictional, complex patient cases and asked three experts to rate them. Each expert rated consistently by their own standards, but they were far apart from each other. One reason, according to the study: they weighed safety and progression differently. That applies to high-risk cases first. Whether experts would disagree as much about healthy athletes, the study doesn't show. More structure in the prompt helped up to a point and then stopped helping reliably.
Stay on top of AI training & analysis? One click — and Google shows you my articles first.
Add as a preferred source on GoogleThree experts, 45 plans, little agreement
You know the situation: one coach says more volume, the next says more intensity, and both are convinced they're right. I know that from 17 years of coaching. This study measured it once under identical conditions. Three experts with PhDs in sports medicine or clinical exercise science rated the same 45 AI plans, and their scores were often far apart. Some focused on strict safety limits, others put more weight on realistic progression and everyday practicality.
Each of them applied their rating scheme consistently. One of them, though, scored lower than the other two across the board, and according to the study that explains a good part of the disagreement. On the direction, all three agreed: the prompt with guidelines scored better than the minimal prompt. The authors call this the "inherently expert-dependent nature of exercise prescription." My reading: in training planning there is often more than one right answer. With only three raters, though, that's a hint and nothing more. The authors themselves urge caution.
"AI needs expert oversight": which expert do you follow?
The standard answer to AI training plans is: have an expert check it. That's reasonable, but it only gets you halfway when three experts rate the same plan differently. You still have to decide whose advice to follow.
I always frame it like this: there is a clear wrong and dangerous, and in my experience experts mostly agree on that. And there is an optimum that nobody actually knows. The goal is to get as close to it as possible, and there are often several good ways to do that. Most of the arguing happens in this middle ground. To be honest, this study doesn't quite fit that picture: I checked, and even on safety the three agreed only a little.
That doesn't mean expert knowledge is worthless. It means there is no neutral authority that can tell you whether your AI plan is good. Even an experienced coach brings their own school of thought and their own priorities. You only notice the difference if you know enough yourself to judge the answer.
More prompt structure helps up to a point
The study tested three prompt levels: a minimal prompt, a prompt with guidelines and contraindications, and a detailed schema with fixed output formats. From level 1 to level 2, average scores rose for safety (3.7 to 4.1 out of 5) and for following the guidelines (3.8 to 4.2). The authors didn't test this statistically. They read it more as the plans fitting the rating scheme better than as the AI planning better. Level 3 gave a mixed picture: the best score in the diabetes case, slightly lower in the knee case and clearly lower in the case of recovery after colon cancer surgery. On safety, level 3 dropped back to the level of the minimal prompt. That more context helps was also shown by an earlier study on ChatGPT plans. Whether the benefit levels off at some point wasn't part of that study.
The authors suspect that a prompt that's too tight makes the model stick rigidly to the schema, and the individual context gets lost. They didn't measure that. What I'd take from this: provide context, yes, but don't force the model into a template. And my guess from practice: if you don't know what progressive overload or a deload is, you won't ask for it, and the LLM won't fill in missing training knowledge on its own.
What remains when you strip away the clinical framing
The study uses three fictional patient cases: type 2 diabetes with obesity, knee osteoarthritis with fall risk, and recovery after colon cancer surgery. In groups like these, a bad plan can be dangerous. The authors expect experts to disagree more in complex cases like these, so the finding can't simply be transferred to healthy athletes. My guess: in the middle ground, meaning volume, intensity and progression, experts wouldn't agree there either. What I take from the study: good context in the prompt helps up to a point, after that your own judgement counts for more. If you want to approach this systematically, with a structured method instead of trial and error, the Claude training plan skill is a good starting point.
What interests me about this study: the question "Is this AI plan good?" often has no clear answer, not even from experts. That's why I don't reject AI tools. I use them with my own judgement instead of blind trust. If you know the fundamentals, you can tell when a plan makes sense and when it doesn't. That applies to AI output just as much as to a coach's advice.
The study: Choi M, Park J, Lee M, Beom J, Jung SY, Lee K. "AI-Generated Exercise Prescriptions for At-Risk Populations: Safety and Feasibility of a Large Language Model Assessed by Expert Evaluation", Journal of Clinical Medicine, 2026. It is available open access at MDPI.
