As I’m discussing paper writing with more folks in the age of Claude Code and other coding assistants, we’re having more discussions about how to use these tools in writing.
Per my earlier post, I don't want AI tools to "take away the good part" of my job, I want them to help with the bad parts. I'm finding it transformative that coding agents can help me iterate quickly through data munging tasks, navigate cluster scheduling challenges, and prototype experiments, just to name a few examples. They can also be very valuable for manuscript preparation, including finding typos, helping with formatting and reproducibility, and code review.
Agents are especially useful for Rmarkdown or Quarto-based reproducible writing. Writing reproducible papers is a great way to avoid typos and to quickly update your manuscript with new analyses. I teach this content and find it extremely helpful in my work. But getting a reproducible paper to compile can be very annoying, and I would like agents to help me (and my students and collaborators, who struggle with this step sometimes!).
Here's my simple rule: agents should not directly touch the text of the paper. Paper text should be written by a human.* If agents write text, it's still far too easy for them to make things up (especially references) and to write text that is superficially correct but unmoored to the actual human research activities.
So, how do you collaborate with an agent on a paper and guarantee it doesn't touch the text? Here is one workflow that's been working for me:
- The paper is written in a git repository. You commit all changes you make to the paper.
- You ask the agent to do the work on a branch and make a pull request, following strict rules not to change text.
- You review merge the PR and have clear visibility (and an audit trail) for any changes. That way, if the agent does touch the text, you know where and when that happened.
Here's a Claude Code skill file that instantiates this workflow.
---
*Norms may shift in the future but for now, that's where I'll draw the hard line.
No comments:
Post a Comment