Insightful problem solving is a vital part of human thinking, yet very difficult to grasp. The “Aha! experience” is often regarded as the defining characteristic of insight. Traditionally, insight has been investigated by using a set of established “insight tasks”, assuming that insight has taken place if these problems are solved. However, the debate about which problems actually trigger insight is still not resolved, since there is no clear behavioural marker for the occurrence of insight. In the present work, we therefore aimed at testing the validity of three classical insight problems by directly asking participants about their solution experiences. Our results suggest that participants solve insight problems also without any Aha! experience, casting doubt on the common approach of using a priori defined insight problems. Consequently, we advocate the use of direct insight ratings by participants, determining for each problem individually whether it was solved with insight or not.