Adding Significance to the Implicit Association Test

The Implicit Association Test has become one of the most widely used tools in psychology and related research areas. The IAT’s validity and reliability, however, are still debated. We argue that the IAT’s reliability, and thus its validity, strongly depends on the particular application (i.e., which attitudes are measured, which stimuli are used, and the sample). Thus, whether a given application for a given sample will achieve sufficient reliability cannot be answered a priori. Using extensive simulations, we demonstrate an easily calculated post-hoc method based on standard significance tests that enables researchers to test whether a given application reached sufficient reliability levels. Applying this straightforward method can thus enhance confidence in the results of a given IAT. In an empirical test, we manipulate the sources of error in a given IAT experimentally and show that our method is sensitive to otherwise unobservable sources of error.