Fancy coding agent harnesses don't improve results, just costs

a study by researchers at shandong university examined the effect of different harnesses on coding agents, which pair language models with systems, tools, and context management. they tested five models - qwen3.6, qwen3.8, deepseek-v4-flash, glm-5.3-flash, and hy4-preview - with three harnesses: claude code, opencode, and mini-swe-agent. the study used 492 tasks from swe-bench verified, finding that swapping harnesses has a minimal impact on results, similar to the variability of rerunning the same setup. the researchers also found that harnesses consume two to three times as many tokens as the leanest option, but pass rates remain within measurement uncertainty. this study affects developers who use coding agents and challenges the claims of harness makers.