Scientific progress depends on self-correction; it rests on the assumption that supportive evidence will bolster people’s belief, whereas contradictory evidence will undermine it. Despite the importance of belief updating, people do not always update their beliefs in a rational way. We conducted an in vivo test of the degree to which psychological science is self-correcting by tracking psychologists’ beliefs in psychological theories before and after the release of evidence from four rigorous, large-scale replication projects. In doing so, we examined three broad questions: 1) How much do scientists update their beliefs in response to new empirical evidence? 2) Do scientists show evidence of motivated reasoning when evaluating replications? and 3) What moderates the extent of scientists’ belief updating?