AI-assisted learning in exercise physiology: a quasi-experimental study using PhysioExercise GPT

What the study measured

Arms

Findings

A finding is one outcome contrast: two arms, one measure, one set of conditions. A paper is never "positive" or "negative" here; each finding carries its own direction and its own conditions. The badge that matters most is the first one: whether the AI was still available when the outcome was measured. Why that gates everything.

Limitations

Stated by the authors: quasi-experimental design without randomization; possible selection bias; no a priori sample-size calculation (power analysis was post hoc); instrument of only ten items; reduced measurement sensitivity; confounders not controlled: baseline performance, attendance/participation, extracurricular study time, digital literacy, external AI use, motivation, interaction between groups; shared classroom: complete separation not feasible, risk of contamination; control students told not to use AI tools, but compliance could not be controlled; gains cannot be attributed exclusively to PhysioExercise GPT; results are associations, not causal; outcome predominantly theoretical, which the authors say may underestimate effects on applied domains. Observed by review: allocation is self-contradictory: feasibility-based (2.1), lottery “simple randomization” (2.3), and “without randomization” (abstract, limitations) in the same paper; reported to STROBE; the outcome test was written and content-validated by the experts who built the tool's knowledge base, and its domains are stated to correspond to that knowledge base: alignment risk favouring the AI arm; the I-CVI of 1.00 comes from the same three developers, not independent judges; conditions of T0/T1 administration (proctored or not, in class or online, whether PhysioExercise GPT or other AI/internet was accessible) are not reported, so AI availability at assessment cannot be established; T1 date is not reported; tool access ended 27 October 2024 while the follow-up is described as about six months / 26 weeks, so the interval between end of exposure and T1 is unknown; T0 was administered after the 7-day familiarization with the tool, so the AI group's baseline may already include tool exposure; it was also 0.79 points higher, untested; time-on-task and instructor-role confounding: the AI arm had prescribed extra practice (20-30 min, at least 3x/week) and a different instructor role; the paper still claims the only difference was the tool; no participant flow, attrition or exclusion counts; exclusion of students who missed assessments or minimum participation implies a per-protocol sample; number of AI-arm students who met the 39-use adherence threshold is not reported (only a mean usage rate of 84.25%); dose figures inconsistent: instruction was at least three uses a week, but the planned dose assumes two a week over 26 weeks; the access window is about 14 weeks; Table 3's 93 mean days with interactions exceeds its 43.81 mean interactions per participant; “Mean usage time*” has an undefined asterisk and no unit; knowledge base described as 58 documents published 1990-2025 although the tool version tested was used July-October 2024; base model stated as GPT-4.5 during July-October 2024; to the reviewer's knowledge (not from the source) OpenAI first released GPT-4.5 in February 2025, so the stated model/date pairing cannot both be correct; abstract claims “enhancing the retention of complex concepts” and a “robust” tool, but there is no delayed or retention test and the discussion says causal inference cannot be established; ANCOVA adjusted means, adjusted difference and its CI are not reported, only F, p and partial eta squared; double-blind scoring with averaging of two evaluators is described for dichotomously scored multiple-choice items, which would not ordinarily need judgment; the procedure is unexplained; consent statements conflict: “All participants provided informed consent” vs written consent from “legal guardians/next of kin” for an adult sample (mean age 21.3); a digital-literacy (ICILS-type) assessment is reported in Results with no description in Methods of when or how it was given, for whole sample only, with an implausibly narrow SD (12.1) for a scale whose proficiency bands are 84-104 points wide; in-text citations of Yildiz/Nisanci (2026) appear swapped against the reference list; one named peer reviewer (Onur Seçgin Nişanci) is an author of the works discussed at length in the Discussion; single course, single instructor, single institution; findings are for a custom GPT built and evaluated by the same team. Integrity note (IN-025, carried into limitations per the owner's ruling): both passes independently found the allocation described three ways, a baseline taken after the AI arm's tool familiarization, an access window shorter than the stated duration, no participant flow, a guardians/next-of-kin consent statement for adults, and a retention claim in the abstract with no delayed measure. Pass B adds, from outside knowledge, that the stated base model (GPT-4.5) was first released after the stated intervention dates, so the base model or the dates are misreported. Neither pass found anything showing fabrication; the pattern is careless reporting plus an anachronism that makes the intervention description unreliable.

Who was studied

Level

university

Years

5th-semester undergraduate Physical Education students

Ages

mean 21.3 years (SD 3.21); range not reported

Country

BR

Setting

Faculty of Physical Education (FAEF), State University of Rio Grande do Norte (UERN), Mossoro, Brazil; regular Exercise Physiology classes; one instructor and one shared classroom for both groups. 71% female.

Prior knowledge

T0 means 3.88 (SD 1.41) AI vs 3.09 (SD 1.67) control on the 0-10 test; the 0.79-point baseline gap (about 0.5 SD) was not tested or discussed. T0 was given after the 7-day tool familiarization period, so it is not a clean pre-exposure baseline for the AI group. Self-reported daily study time 15.2 (SD 4.6) minutes and a digital-literacy score (581.4, SD 12.1, described as ICILS Level 3) are reported for the whole sample only, not by group.

Methodological notes

Extracted from: https://doi.org/10.3389/fphys.2026.1894046 Access: full text (Frontiers, CC BY), read from data/fulltext-cache/bundles/ribeirodasilva-2026.txt (PDF text first, then XML full text); Figure 1 is an image and the Supplementary Material (the T0/T1 instruments) is not in the bundle, so item content is not coded Allocation: Records the allocation level only, not a claim that allocation was random: individual students were assigned to arms (32/32) within a single class cohort. Whether that assignment was random is contradicted within the paper (see design_note). Analysis was student-level (paired and independent t-tests; ANCOVA), which matches the allocation unit, but the shared classroom means outcomes are not independent across arms. IN-025 rulings: randomization_level student records the allocation unit only (every account the paper gives allocates individual students; it does not imply randomness, per the syromyatnikov-2026 precedent). F1 timing immediate_post is a reading of the paper's “post-intervention” label, not a reported date. The ANCOVA is the same contrast re-analysed with no adjusted estimate, so it rides F1's estimate and p-value fields rather than a second finding.