Teaching Innovation Awards Winner

Rationale and context 

Several years ago, the assessment system for the chemistry laboratories for our foundation year was a booklet which the students completed, and which was summatively marked over the Christmas vacation. Formative feedback was given informally during the lab sessions. However, as student numbers grew to nearly 200, it was obvious to me as module leader that the assessment method had to change.

Goals 

The initial goals for the new system were that students should get timely formative feedback and that marking should be consistent and sustainable i.e., should not burden staff with more marking than they had done previously, despite the increased student numbers. 

Innovation and implementation 

Fortunately, we had already organised a visit from LearnSci and I realised that we could use Smart Worksheets to fit our already implemented practicals. I decided to reduce the number of experiments, but to pair them up by technique. For the first experiment of a pair, the students perform the experiment and then enter their results and calculations for formative feedback. Used like this, the Smart Worksheets are designed to provide immediate feedback and support, and the students get an indicative mark at the end. Since the worksheets contain the “correct” values, marks and feedback are also provided on the accuracy of the raw data.

The second experiment of the pair is similar, but not identical, and the assessment is summative, with the feedback / support turned off for these experiments. In this way, students get to experience both the required calculations and the worksheets in a “safe”, supported environment, before attempting a “real” assessment. The first pair of experiments were gravimetric analyses and the second titrimetric analyses. This “formative first / summative second” implementation is an innovative implementation of Smart Worksheets - I am assured by LearnSci that no-one else is using Smart Worksheets in this way. 

Impact 

Over the last six years, over 2000 assessments (formative and summative) have been marked using Smart Worksheets. It is fortunate for statistical purposes that approximately half of the students engaged with the formative assessments, generating a robust dataset for comparisons between those who did and those who didn’t. Analysis of this dataset by Emily Coyte at LearnSci demonstrates that those students who fully engage with the formative assessments achieve a summative mark which is 10% higher than those who do not and that that this difference is highly statistically significant (p < 0.001.) Interestingly, those who only interact with the first formative see a smaller uplift, demonstrating that the feedback is enhancing scientific understanding and not just providing experience of the on-line system.

I presented these initial results at European Variety in Chemistry Education conference and at the Variety in Chemistry Education / Physics Higher Education Conference in Durham this summer. These findings are also at the centre of a paper to be submitted to J Chem Ed by Emily and I.

‍

‍