Assessments can be very revealing and tell you much more than just what the student knows.
The value and importance of providing feedback to students has been well documented and researched, so in this blog post I want to look at the formative value of assessments to instructors (lecturers/teachers/trainers/ anyone else running assessments!).
How frequently do you evaluate your assessments?
Once you have designed your assessment (formative or summative) and it has been deemed fit for purpose by whichever “power that be” in your environment, it is very tempting (and common) to see the process as finished and the same assessment may run, without change, for years. So if you’re not regularly evaluating your assessments, you might find this blog post useful.
Assessments provide an opportunity for you to evaluate three main things:
1. The students – Can the students do what you want them to be able to do?
2. The assessment – does your assessment actually assess what it is supposed to?
3. The teaching and learning process – Do your teaching and learning materials align with your assessment criteria?
The first point is the most common use of assessment – establishing that the student can or can’t do what you have identified as key criteria.
The other two are less common uses of assessments (but no less important!), so today I’m looking at the ways we can use assessment data to help to develop fairer and more valid assessments.
The data you get from students completing assessments gives you a very powerful insight into how well the assessment itself performed. Have a look over your results holistically, do they look like you expected them to? Have more people passed or failed? Did you get a lot of missed answers or common incorrect ones? Investigating this regularly can help you to determine if your assessment is actually assessing what you wanted it to assess.
Electronic assessments can make the evaluation of an assessment easier to do, especially in the world of selected response questions (MCQ, true/false, select a blank etc). If you are working with project work, essays or other more variable student submissions then it’s always a good idea to look out for trends in either the feedback you are providing or the grades you are giving.
How has the group as a whole performed on your assignment? Take a look at your standard deviation of grades. Has your assessment resulted in normal distribution or are the scores generally low or high – what factors could be causing this? Are your usually strong students struggling with this assignment. Using any unusual trends or patterns in your assessment data acts as a flag for you to review the assessment.
I’m not getting the results I expected?
One of the first things to check is that the assessment design matches your assessment criteria – i.e. you are genuinely assessing what you wanted to assess. Have a look to see if there could be any subtle influences that could be a factor, for example the use of unfamiliar software or students not having access to the necessary resources.
Secondly, check that the assessment criteria are actually aligned to your learning outcomes i.e. what you are teaching the students in your sessions actually matches what they are being assessed on (this is a very common mismatch which can result in invalid and unfair assessments). For example, if you are assessing students on a presentation, are presentation skills an actual learning outcome for your course? If not, you may be assessing your students unfairly.
If you are using computer software to run quizzes and tests, then many of these come with extremely useful statistical reports that help you to review how valid your questions are. Have a look at the statistical report from Moodle for more information. This is a particularly strong benefit of using e-assessments. You can have access to extremely valuable and insightful statistical information in seconds, that would have taken weeks or months to compile from paper based assessments.
The teaching and learning process
Assessment data can also give you a huge amount of insight into how the students are responding to your teaching and learning materials. If you find that you are being presented with the same incorrect answers or skipped answers, as well as addressing this in feedback with the group, use it to think about how you teach these concepts. It might be that there is not enough time dedicated to the area in classes or that the learning outcome, which is aligned to the assessment criteria, is not covered in the same way in the learning materials. This can also result into a subtle, but sometimes significant, misalignment of assessments.