Focus On Assessments

Assessments.

Not exactly a small topic, but working on the old adage ‘write about what you know’, it seems like a good starting point.  Although this is my first role where I am exclusively supporting technology enhanced learning in HE, I have been working with learning technologies for many years. In particular, I have been working with assessment technologies for many years.

I first cut my learning technology teeth whilst working for a vocational awarding body back in 2006. It was here that I was exposed to the complicated and occasionally painful world of assessment theory and practice. I had been brought in to help make the transition from paper-based portfolios and exams, to e-portfolios and e-assessments. Whilst there I worked with examiners, moderators, colleges and students and quickly got a sense of where the challenges were (in both the paper-based and online worlds). From there I moved onto working for a company that created e-assessment software. I worked in a consultative capacity with clients all over the world, assisting them make the transition from paper based to online assessments. It was during this time that I completed my PGCert in Online and Distance Education and created some online courses of my own.

To start off my first blog post I thought I would share a few good practice pointers that I have picked up over the years (NB these will probably not be a revelation, but could help if you’re starting out).

Considering eAssessments? Three questions to ask yourself:

If you are considering changing from paper-based to online assessment, thinking about the following three questions can be a good starting point.

Of course, and this goes without saying, before you make any changes to how your assessment is delivered, make sure you have it approved by following your faculty processes for changing assessment methods.

Why do you want to use e-assessment?

This is really a rule for introducing any learning technology, make sure you have a clear understanding of why you are considering moving to e-assessment. Ideally you should be looking at ways in which it will improve the student experience, e.g. you will have access to reports that’ll help you ensure your questions are valid and reliable, or the students will be able to receive their marks and feedback more quickly. Whatever it is, make sure you have a good clear understanding of the benefits you are looking to achieve. It is also very helpful to explore the potential risks involved in the change too, so you can factor them into your assessment design and processes for delivery.

Will your e-assessment still meet your assessment objectives?

This is a tricky one, and easily overlooked. When you are deciding on which technology to use, consider how the technology might be affecting your assessment objectives and criteria. For example, are you changing from constructed response questions (essay/short answer) to selected response questions (multiple choice, select a blank, true false etc). This may be still be an entirely valid way of assessing your students, but make sure you consider the impact this might be having on your assessment objectives – are you still assessing the same things? If not, is this OK? Also be careful that being able to use the technology has not become an inadvertent assessment criteria. This can happen if the technology is not intuitive or familiar to the students, thus placing them at a disadvantage to peers who are comfortable with the technology.

One way around the last point is to make sure that students have ample access to practice tests using the software you have picked. This gives them a chance to familiarise themselves with the software and helps you to identify any potential issues with the technology.

Why are you replicating your paper-based processes? 

When you move to e-assessments it is natural to try and replicate as closely as possible your paper based process – these are the processes you are familiar with and have probably been using for years. However, you could be missing out on the opportunity to introduce some real benefits to staff and students.

When you first try and transpose your paper-based assessment into an online version, examine the functionality of the software and consider what is appropriate for your assessment. For example, if you do not currently give students their results for 4 weeks, to give markers a chance to grade the scripts, consider if you really need to impose the same delay for an e-assessment? Conversely,  there may also be good reasons to restrict e-assessments to the conditions of paper-based exams – for example, if you don’t want the students to collaborate on their assessments then you will still need to conduct the assessment in exam conditions (even if technically they could take the assessment on their mobile). It is important to establish which of the processes are a core part of the assessment to ensure it is still valid to it’s design, and which are simply the result of the paper-based process.It all depends on what your assessment objectives are!

I have worked with many, many people who have spent a lot of time, effort and money trying to replicate what are essentially unnecessary elements of paper-based assessments, simply to produce an assessment that is almost identical (including all the flaws) to their current paper-based assessment.

Advertisements

3 thoughts on “Focus On Assessments

  1. I think the last of these is important for establishments and publishers to consider. If we replicate paper based assessments then we fail to capitalise fully on the potential for e-assessment to revolutionise the way we assess and the relationship with learning. I’ve been pursuing (unsuccessfully to date) the e-assessment of primary science. I feel like there was a move in the right direction a few years ago but that it has stalled and we have returned to outdated and ineffective methods. I want to see technology being used to support the assessment of those difficult to capture elements of performance subjects like science. How difficult, for example, is it to see if pupils are measuring accurately when they are taking readings from equipment? How easy would it be to do this in a simulated environment, using e-assessment. I could go on about the potential advantages, but at the moment I am only frustrated by the lack of will and vision from educators and policy makers!

    Liked by 1 person

    • Hi Juliet, thank you for your comment- very interesting! Why do you think that progress has stalled in your area? It can be very difficult to get people to think about exactly what it is they are trying to assess and what it is in essence that makes that assessment valid, as they are so used to a certain way of doing things. But certainly with a careful and well thought out approach, and utilising the right technology for the job, e-assessments can have real benefits for students (and teachers!).

      Liked by 1 person

  2. Hi Kate
    I think there are several reasons. There have been failures in e-assessment, for example the NILA and NINA in Northern Ireland. This is generally where e-assessment is only replicating paper based methods and the difficulties with infrastructure, and cost, outweigh the benefits. I think educators and policy makers can not relinquish old-fashioned, top-down systems in which pupils or students all follow the same course at the same time. E-assessment has the potential to give some ownership to the pupils and that’s probably a step too far for some.

    Like

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s