Physicians need to stay up-to-date with new developments in their field of expertise. This expectation has been made explicit by competency-based educational outcomes in the domain of scholar in the Dutch blueprint. There is a great diversity in teaching methods that aim to achieve a better understanding of scientific knowledge. Applying a constructivist approach to learning in acquiring research competencies we wonder how a research-intensive course is evaluated early in the curriculum and what learning gain students perceive. In a collaborative research-intensive course, the class of 300s-year students rated the quality of 150 preselected randomized controlled trials (RCT) using JAMA Users’ Guides, and the pharmaceutical advertisements in which they were referenced. Each student rated two RCTs. Data were analyzed to answer a relevant research question. After the course students completed an evaluation survey. We did this in five consecutive years to capture student experience in relation to fostering a scientific mindset (n = 1,500). In addition we studied outcome of this scientific mindset as scientific output (publications) in journals. Survey data indicate that it is feasible to successfully implement a research-intensive course based on a large cohort using a constructivist paradigm early in the curriculum. Students consider it challenging and report high learning gain in several domains.