To improve the effectiveness of higher education, Dutch universities implemented the binding study advice at medical faculties. Accordingly, medicine students of Radboud University need to gain ≥ 42 out of 60 European Credit Transfer System (ECTS) credits to obtain a positive binding study advice and to continue their study programme. In case of a negative advice, the student is obliged to terminate the study, and he/she cannot register for the same study programme in the Netherlands within the next three years. The purpose of this manuscript is to evaluate the effect of implementation of the binding study advice on study outcomes. First, the binding study advice did not impact on student performance, as the average ECTS credits were comparable before and after its introduction. Second, study progress improved 8 % with 93 % of the students obtaining access to the second year of the study programme after binding study advice implementation. Third, the binding study advice did not impact propaedeutic graduation rates. These data demonstrate that the implementation of the binding study advice in medical faculties has only a small impact on study outcomes. The high performance levels of medical students compared with peers at other faculties are likely to contribute to these findings and suggest a ‘ceiling effect’ in the potential improvement of study outcomes at medical faculties.