Critical thinking, which can be defined as the evidence-based ways in which people decide what to trust and what to do, is an important competency included in many undergraduate science, technology, engineering, and mathematics (STEM) courses. To help instructors effectively measure critical thinking, we developed the Biology Lab Inventory of Critical Thinking in Ecology (Eco-BLIC), a freely available, closed-response assessment of undergraduate students critical thinking in ecology. The Eco-BLIC includes ecology-based experimental scenarios followed by questions that measure how students decide on what to trust and what to do next. Here, we present the development of the Eco-BLIC using tests of validity and reliability. Using student responses to questions and think-aloud interviews, we demonstrate the effectiveness of the Eco-BLIC at measuring students critical thinking skills. We find that while students generally think like experts while evaluating what to trust, students responses are less expert-like when deciding on what to do next.