Instant Feedback for SQL Assignments using Clause-Driven Automated Grading

Automated grading tools can save time and effort while ensuring consistency in evaluating assignments. At our university, we have implemented SOCOLES for evaluating students' SQL answers using parse tree comparisons between student statements and one or more correct statements. It can grade Data Query Language (SELECT), Data Definition Language (DDL), and Data Manipulation Language (DML) statements. Furthermore, it can generate automated and constructive feedback with actionable guidance students can use to correct mistakes. Initially, answers were uploaded in Brightspace and downloaded into the tool for grading, reducing interaction and delaying feedback. To address this, we have enhanced SOCOLES with a web interface for students, instructors and teaching assistants. Instructors can configure questions, and students can submit answers directly and receive instant feedback. Our evaluation of the tool demonstrates its effectiveness in providing timely feedback and improving students' learning experiences. Survey results indicate that students perceive the feedback as helpful in fixing errors and enhancing understanding. However, as SQL feature complexity increases, perceived helpfulness of feedback decreases, highlighting areas for further improvement of our tool. Additionally, analysis of time gaps between attempts and grade changes reveals no significant correlation, suggesting that immediate feedback alone may not impact performance improvements. Finally, usage data shows a linear decline in students reaching successive attempts. This pattern suggests there is no clear cutoff attempt number beyond which additional attempts become unproductive. These findings underscore the potential of SOCOLES to enhance SQL learning while also identifying opportunities for refining feedback mechanisms to better support complex tasks.