Assessments.Not exactly a small topic, but working on the old adage ‘write about what you know’, it seems like a good starting point. Although this is my first role where I am exclusively supporting technology enhanced learning in HE, I have been working with learning technologies for many years. In particular, I have been working with assessment technologies for many years.
I first cut my learning technology teeth whilst working for a vocational awarding body back in 2006. It was here that I was exposed to the complicated and occasionally painful world of assessment theory and practice. I had been brought in to help make the transition from paper-based portfolios and exams, to e-portfolios and e-assessments. Whilst there I worked with examiners, moderators, colleges and students and quickly got a sense of where the challenges were (in both the paper-based and online worlds). From there I moved onto working for a company that created e-assessment software. I worked in a consultative capacity with clients all over the world, assisting them make the transition from paper based to online assessments. It was during this time that I completed my PGCert in Online and Distance Education and created some online courses of my own.
To start off my first blog post I thought I would share a few good practice pointers that I have picked up over the years (NB these will probably not be a revelation, but could help if you’re starting out).
Considering eAssessments? Three questions to ask yourself:
If you are considering changing from paper-based to online assessment, thinking about the following three questions can be a good starting point.
Of course, and this goes without saying, before you make any changes to how your assessment is delivered, make sure you have it approved by following your faculty processes for changing assessment methods.
Why do you want to use e-assessment?
This is really a rule for introducing any learning technology, make sure you have a clear understanding of why you are considering moving to e-assessment. Ideally you should be looking at ways in which it will improve the student experience, e.g. you will have access to reports that’ll help you ensure your questions are valid and reliable, or the students will be able to receive their marks and feedback more quickly. Whatever it is, make sure you have a good clear understanding of the benefits you are looking to achieve. It is also very helpful to explore the potential risks involved in the change too, so you can factor them into your assessment design and processes for delivery.
Will your e-assessment still meet your assessment objectives?
This is a tricky one, and easily overlooked. When you are deciding on which technology to use, consider how the technology might be affecting your assessment objectives and criteria. For example, are you changing from constructed response questions (essay/short answer) to selected response questions (multiple choice, select a blank, true false etc). This may be still be an entirely valid way of assessing your students, but make sure you consider the impact this might be having on your assessment objectives – are you still assessing the same things? If not, is this OK? Also be careful that being able to use the technology has not become an inadvertent assessment criteria. This can happen if the technology is not intuitive or familiar to the students, thus placing them at a disadvantage to peers who are comfortable with the technology.
One way around the last point is to make sure that students have ample access to practice tests using the software you have picked. This gives them a chance to familiarise themselves with the software and helps you to identify any potential issues with the technology.
Why are you replicating your paper-based processes?
When you move to e-assessments it is natural to try and replicate as closely as possible your paper based process – these are the processes you are familiar with and have probably been using for years. However, you could be missing out on the opportunity to introduce some real benefits to staff and students.
When you first try and transpose your paper-based assessment into an online version, examine the functionality of the software and consider what is appropriate for your assessment. For example, if you do not currently give students their results for 4 weeks, to give markers a chance to grade the scripts, consider if you really need to impose the same delay for an e-assessment? Conversely, there may also be good reasons to restrict e-assessments to the conditions of paper-based exams – for example, if you don’t want the students to collaborate on their assessments then you will still need to conduct the assessment in exam conditions (even if technically they could take the assessment on their mobile). It is important to establish which of the processes are a core part of the assessment to ensure it is still valid to it’s design, and which are simply the result of the paper-based process.It all depends on what your assessment objectives are!
I have worked with many, many people who have spent a lot of time, effort and money trying to replicate what are essentially unnecessary elements of paper-based assessments, simply to produce an assessment that is almost identical (including all the flaws) to their current paper-based assessment.