Phishing attacks exploit users’ inability to distinguish legitimate websites from fake ones. Strategies for combating phishing include: prevention and detection of phishing scams, tools to help users identify phishing web sites, and training users not to fall for phish. While a great deal of effort has been devoted to the first two approaches, little research has been done in the area of training users. Some research even suggests that users cannot be educated. However, previous studies have not evaluated the quality of the training materials used in their user studies or considered ways of designing more effective training materials. In this paper we present the results of a user study we conducted to test the effectiveness of existing online training materials that teach people how to protect themselves from phishing attacks. We found that these training materials are surprisingly effective when users actually read them. We then analyze the training materials using principles from learning sciences, and provide some suggestions on how to improve training materials based on those principles.