The crisis caused by the COVID-19 virus has far-reaching effects in the field of education, as schools were closed in March 2020 in many countries around the world. In this article, we present and discuss the School Barometer, a fast survey (in terms of reaction time, time to answer and dissemination time) that was conducted in Germany, Austria and Switzerland during the early weeks of the school lockdown to assess and evaluate the current school situation caused by COVID-19. Later, the School Barometer was extended to an international survey, and some countries conducted the survey in their own languages. In Germany, Austria and Switzerland, 7116 persons participated in the German language version: 2222 parents, 2152 students, 1949 school staff, 655 school leaders, 58 school authority and 80 members of the school support system. The aim was to gather, analyse and present data in an exploratory way to inform policy, practice and further research. In this article, we present some exemplary first results and possible implications for policy, practice and research. Furthermore, we reflect on the strengths and limitations of the School Barometer and fast surveys as well as the methodological options for data collection and analysis when using a short monitoring survey approach. Specifically, we discuss the methodological challenges associated with survey data of this kind, including challenges related to hypothesis testing, the testing of causal effects and approaches to ensure reliability and validity. By doing this, we reflect on issues of assessment, evaluation and accountability in times of crisis.