I don’t think this detracts from any of your excellent points here, but three things worth pointing out about Hall’s “paper” are that it wouldn’t actually get published anywhere, it used existing data in known repositories, and the instructions to Claude Code were longer than the paper itself.
That last part is critical. Because Hall could write a screenplay for a replication study, the study was instantly automatable. That this is mechanically true now is both remarkable and mundane.
One can wonder of the GenAI tools are going to influence what we do research/publish and what we do not (because GenAI can't easily assist with it). A bit like the drunk searching for their keys under the lantern, because that is where the light is.
I don’t think this detracts from any of your excellent points here, but three things worth pointing out about Hall’s “paper” are that it wouldn’t actually get published anywhere, it used existing data in known repositories, and the instructions to Claude Code were longer than the paper itself.
That last part is critical. Because Hall could write a screenplay for a replication study, the study was instantly automatable. That this is mechanically true now is both remarkable and mundane.
Your comment puts a critical note on mine. Easy is indeed not to be assumed.
One can wonder of the GenAI tools are going to influence what we do research/publish and what we do not (because GenAI can't easily assist with it). A bit like the drunk searching for their keys under the lantern, because that is where the light is.
The purveyors/profiteers of AI are pushing it like it's ready for prime time, and quite frankly, I don't believe that it really is.