Our department just had its first ever town hall event. The goal was to have an open discussion of issues surrounding reproducibility and other methodological challenges. Here's the announcement:
Please join us for a special Psychology Colloquium event: Town Hall on Contemporary Methodological Issues in Psychological Science.I gave a talk on my evolving views on reproducibility, many summarized here, specifically focusing on the issue that individual studies tend not to be definitive. I advocated for a series of changes to our default practice, including:
Professors Lee Ross, Mike Frank, and Russ Poldrack will each give a ten-minute talk, sharing their perspectives on contemporary methodological issues within their respective fields. There will be opportunities for both small and large group discussion.
- Larger Ns
- Multiple internal replications
- Measurement and estimation, rather than statistical significance
- Experimental “debugging” tools (e.g., manipulation checks, negative/positive controls)
- Preregistration where appropriate
- Everything open – materials, data, code – by default
I then illustrated this with a couple of recent examples of work I've been involved in. If you're interested in seeing the presentation, my slides are available here. Overall, the town hall was a real success, with a lot of lively discussion and plenty of student voices discussing their concerns.