Medical schools universally accept the idea that bioethics courses are essential components of education, but few studies which measure outcomes (i.e., knowledge or retention) have demonstrated their educational value in the literature. The goal of this study was to examine whether core concepts of a pre-clinical bioethics course were learned and retained. Over the course of 2 years, a pre-test comprising 25 multiple-choice questions was administered to two classes (2008–2010) of first-year medical students prior to the start of a 15-week ethics course, and an identical post-test was administered at the end of the course. A total of 189 students participated. Paired t tests showed a significant difference between pre-test scores and post-test scores. The pre-test average score was 69.8 %, and the post-test average was 82.6 %, an increase of 12.9 % after the ethics course. The pre- and post-test results also suggested a shift in difficulty level of the questions, with students finding identical questions easier after the intervention. Given the increase in post-test scores after the 15-week intervention, the study suggests that core concepts in medical ethics were learned and retained. These results demonstrate that an introductory bioethics course can improve short-term outcomes in knowledge and comprehension, and should provide impetus to educators to demonstrate improved educational outcomes in ethics at higher levels of B.S. Bloom’s Taxonomy of Learning.