Datapages for reusable (and pretty!) data sharing

[This post is joint with Mika Braginsky, a long-time collaborator on data-sharing and data-viz.]

Data sharing is both a critical scientific need and, increasingly, a mandate by many research funders. The FAIR principles – that data should be findable, accessible, interoperable, and reusable – are a critical guide to how data are shared. Yet even FAIR-compliant datasets in approved repositories are often shared in ad-hoc formats that are hard to reuse or to integrate with other data. In contrast, the most impactful datasets tend to be disseminated thoughtfully through dataset-specific or community-specific platforms. These “domain-specific data repositories” (this was our term from a previous blogpost!) create opportunities for creating data standards and ontologies that fit the needs of a particular community, research problem, or instrument type. They also allow opportunities for engagement through interactive visualizations. But custom repositories and pretty websites with nice visualizations are costly and complicated to create.

We are introducing a set of open-source tools and templates for easily creating datapages, interactive websites to disseminate data for broad reuse. Datapages are easy to deploy for a single project, but extensible enough to host large collections of related datasets. You can learn more and get started at https://datapages.github.io/.

Two summer book recommendations

After a long stint primarily reading fiction, I've been on a non-fiction kick recently and just read two books that I would definitely recommend!

Persuasion in Parallel (2022) by Alexander Coppock, a political scientist at Yale, is a scholarly monograph on how political persuasion works. It's a delightful combination of large-scale replications, strong emphasis on effect estimation and causal inference, and really thoughtful discussion of mechanisms. It starts from a replication and re-analysis of Lord, Ross, and Lepper (1979), the seminal work on political persuasion, and goes on to replicate a whole host of more recent studies. Across all of them, the key take-home is that arguments about controversial topics (e.g., gun control, abortion, etc.) operate very similarly across people with very different views: they cause small changes in attitude in the direction of the arguments, regardless of the recipient's initial views. 

In statistical terms, there's very limited heterogeneity across groups in the effect of persuasive arguments. I really appreciated evidence on this heterogeneity question because often students' intuition in psychology is that everything differs based on sociodemographic characteristics; yet this intuition is rarely quantified or challenged. Coppock's analyses take a really important step in this direction. 

The book is short and quite readable (especially given how data-rich it is). It's also very up front about the limitations of the work. There's also a thought-provoking final chapter on Bayesian inference and rational models of belief change that makes a number of connections to computational cognitive science that I enjoyed. Despite being an academic, I am not the sort of person who will sit down on a weekend with a monograph from another discipline for fun; this book was an exception for me because of how interesting, important, and thorough the work is.

On a heavier note, Doctored (2025), by Charles Piller (an investigative reporter with Science), is a screed about scientific misconduct in Alzheimer's research. I'm intimately familiar with replication issues in psychology, but I was still totally horrified to read about the impacts of scientific fraud in the Alzheimer's field. Piller makes a very well-researched and thorough case, working with experts on fraud and scientific reviewers. While a critique of the book by an Alzheimer's authority questions how central the fraudulent work was to the field (Lancet review), I was convinced by the later chapters of Doctored that show how pervasive image falsification has been within the Alzheimer's research enterprise. It's just awful to think that many people have been in dangerous clinical trials due to research misconduct. 

The book was clearly written very fast as there is some redundancy between chapters and a bit of unnecessary stage-setting around various researchers' grandparents (perhaps reflecting a pivot from an earlier vision of the book where certain people were more important to the narrative). But the substance of the scientific critique is so compelling – and honestly terrifying – that I was more than happy to overlook a few minor weaknesses in the prose. Definitely recommend.

Domain-specific data repositories for better data sharing in psychology!

Data sharing is a critical part of ensuring a reproducible and robust research literature. It's also increasingly the law of the land, with new federal mandates taking effect in the US this year. How should psychologists and other behavioral scientists share their data? 

Repositories should clearly be FAIR - findable, accessible, interoperable, and reusable. But here's the thing - most data on a FAIR repository like the Open Science Framework (which is great, btw), will never be reused. It's findable and accessible, but it's not really interoperable or reusable. The problem is that most psychological data are measurements of some stuff in some experimental context. The measures we use are all over the place. We do not standardize our measures, let alone our manipulations. The metadata are comprehensible but not machine readable. And there is no universal ontology that lets someone say "I want all the measurements of self-regulation on children that are posted on OSF." 

What makes a dataset reusable really depends on the particular constructs that it measures, which in turn depends on the subfield and community those data are being collected for. When I want to reuse data, I don't want data in general. I want data about a specific construct, from a specific instrument, with metadata particular to my use field. Such should be stored in repositories specific to that measure, construct, or instrument. Let's call these Domain Specific Data Repositories (DSDRs). DSDRs are a way to make sure data actually are interoperable and actually do get reused by the target community.

Methodological reforms, or, If we all want the same things, why can't we be friends?

 (tl;dr: "Ugh, can't we just get along?!" OR "aspirational reform meet actual policy?" OR "whither metascience?")

This post started out as a thread about the tribes of methodological reform in psychology, all of whom I respect and admire. Then it got too long, so it became a blogpost. 

As folks might know, I think methodological reform in psychology is critical (some of my views have been formed by my work with the ManyBabies consortium). For the last ~2 years, I've been watching two loose groups of methodological reformers get mad at each other. It has made me very sad to see these conflicts because I like all of the folks involved. I've actually felt like I've had to take a twitter holiday several times because I can't stand to see some of my favorite folks on the platform yelling at each other. 

This post is my - perhaps misguided - attempt to express appreciation for everyone involved and try to spell out some common ground.

Transparency and openness is an ethical duty, for individuals and institutions

(tl;dr: I wrote an opinion piece a couple of years ago - now rejected - on the connection between ethics and open science. Rather than letting it just get even staler than it was, here it is as a blog post.)

In the past few years, journals, societies, and funders have increasingly oriented themselves towards open science reforms, which are intended to improve reproducibility and replicability. Typically, transparency policies focus on open access to publications and the sharing of data, analytic code, and other research products. 

Many working scientists have a general sense that transparency is a positive value, but also have concerns about specific initiatives. For example, sharing data often carries confidentiality risks that can only be mitigated via substantial additional effort. Further, many scientists worry about personal or career consequences from being “scooped” or having errors discovered. And transparency policies sometimes require resources that are not be available to researchers outside of rich institutions. 

I argue below that despite these worries, scientists have an ethical duty to be open. Further, where this duty is in conflict with scientists' other responsibilities, we need to lobby our institutions – universities, journals, and funders – to mitigate the costs and risks of openness.

Against reference limits

Many academic conferences and journals have limits on the number of references you can cite. I want to argue here that these limits make no sense and should be universally abolished. 

To be honest, I kind of feel like I should be able to end this post here, since the idea seems so eminently sensible to me. But here's the positive case: If you are doing academic research of any type, you are not starting from scratch. It's critical to acknowledge antecedents and background so that readers can check assumptions. Some research has less antecedent work in its area, other research has more, and so a single limit for all articles doesn't make sense. More references allow readers to understand better where an article falls in the broader literature.