The power of voice – Providing audio feedback to students


Tone, intonation, emphasis, expression –  these are all extremely important elements of communication that can be lost in the written word. In fact some have argued that the way that you say something expresses more meaning than the words that you use (Mehrabian 1972). If how we say something adds extra meaning, then why is so much of our feedback to students in written form?

I have been working with colleagues this week who are using recorded audio feedback for students, and the students love it! They express that they feel more connected to the lecturer, that they understand the feedback better and that they feel more supported. In summary, that they value the feedback more. This anecdotal evidence is supported by research done in several studies (Attenborough et al 2012, Merry and Osmond 2008, Brearley and Cullen 2012).

Sounds great, but I don’t have the time!

Time is commonly cited as a reason for not using audio feedback, however users have fed back in studies that it can actually speed the process up.  Rotherham (2007) and Cullen (2011) both found that audio feedback could be as efficient or even more efficient than written feedback. Anecdotally, lecturers I have worked with have also claimed that it saves them time and that they are able to provide students with much richer feedback.

Give it a go!

If you’d like to have a go at providing audio feedback, have a look at these tips:

  1. Decide why you want to use audio feedback – when introducing any new technology it’s always a good idea to spend some time thinking about why you are doing it. What are you hoping to achieve and how will you measure the impact/success?
  2. Choose your tech – make sure the technology is easy to use and produces a file type that is accessible on multiple devices, a mp3 or mp4 can be played on most devices.
  3. Practice using the technology – once you have selected the technology that you want to use, have a play with it and make sure you are comfortable using it. This will help to build your confidence when using it with your students.
  4. Don’t bother to edit –  speak to the student as if they are in the room, like it is a conversation you are having with them. It doesn’t need to be perfect so don’t worry about editing it out, it’s not going to be broadcast anywhere.
  5. Name your files- if you are recording the audio files outside of a built-in coursework tool, make sure you name each file consistently and use the student name or ID. This will make it much easier when you come to distribute feedback to students.
  6. Consider file sizes  – you want to make sure you’ll be able to easily upload and store the feedback files and you also want to make sure the files are accessible to students, particularly if they are likely to be trying to access the files over mobile internet. Try to limit your recordings to around 3-4 minutes to keep the file size small.
  7. Don’t forget to tell the students  – make sure they know how to access the feedback in the new format, what they can expect and why you are doing it.

Technology:

Turnitin: If you are using Turnitin for your assignments, then you can easily record audio feedback using the built-in tool. Feedback can be recorded using Turnitin’s Grademark App for iPads (reviewed by David Hopkins in his excellent blog, Don’t waste your time).  If you are going to be recording using your PC or laptop, consider investing in a headset with mic, as this will greatly improve the audio quality of your recording.

If you are not using Turnitin, you can still easily record audio feedback using your iPhone or iPad. Have a look on the app store for recorders. My new favourite app for audio feedback is voice record Pro, freely available on iTunes and compatible with iPhones and iPads. It’s really easy to use and the recordings can be emailed or saved to various locations, including Google Drive, DropBox and OneDrive – have a look at this review by Dave Yearwood on Faculty Focus.

Your recordings could be shared with the students by email or uploaded as feedback files in your VLE.

References

Attenborough, J., Gulati, S. & Abbott, S. (2012). Audio feedback on student assignments: boon or burden?. Learning at City Journal [online] Vol. 2, Available at http://openaccess.city.ac.uk/1638/ [Accessed 18 December 2014]

Brearley,F. Q. & Cullen, W.R. (2012) Providing Students with Formative Audio Feedback. Bioscience Education, Vol. 20, 22-36. DOI: 10.11120/beej.2012.20000022

Mehrabian, A. (1971). Silent messages, Wadsworth, California: Belmont

Merry, S. & Osmond, P. (2008). Students’ attitudes to and usage of Academic Feedback Provided Via Audio Files. Bioeducation eJournal [online]. Vol. 11, Available at www.bioscience.heacademy.ac.uk/journal/vol11/beej-11-3.pdf [Accessed 18 December 2014]

Rotherham, B. (2007) ‘Using an MP3 recorder to give feedback on student assignments’, Educational Developments, Vol. 8, No. 2, pp.7–10.

Learning from your assessments


Assessments can be very revealing and tell you much more than just what the student knows.

The value and importance of providing feedback to students has been well documented and researched, so in this blog post I want to look at the formative value of assessments to instructors (lecturers/teachers/trainers/ anyone else running assessments!).

How frequently do you evaluate your assessments? 

Once you have designed your assessment (formative or summative) and it has been deemed fit for purpose by whichever “power that be” in your environment, it is very tempting (and common) to see the process as finished and the same assessment may run, without change, for years. So if you’re not regularly evaluating your assessments, you might find this blog post useful.

Assessments provide an opportunity for you to evaluate three main things:

1. The students – Can the students do what you want them to be able to do?

2. The assessment – does your assessment actually assess what it is supposed to?

3. The teaching and learning process – Do your teaching and learning materials align with your assessment criteria?

The first point is the most common use of assessment – establishing that the student can or can’t do what you have identified as key criteria.

The other two are less common uses of assessments (but no less important!), so today I’m looking at the ways we can use assessment data to help to develop fairer and more valid assessments.

The assessment: 

The data you get from students completing assessments gives you a very powerful insight into how well the assessment itself performed. Have a look over your results holistically, do they look like you expected them to? Have more people passed or failed? Did you get a lot of missed answers or common incorrect ones? Investigating this regularly can help you to determine if your assessment is actually assessing what you wanted it to assess.

Electronic assessments can make the evaluation of an assessment easier to do, especially in the world of selected response questions (MCQ, true/false, select a blank etc).  If you are working with project work, essays or other more variable student submissions then it’s always a good idea to look out for trends in either the feedback you are providing or the grades you are giving.

How has the group as a whole performed on your assignment? Take a look at your standard deviation of grades. Has your assessment resulted in normal distribution or are the scores generally low or high – what factors could be causing this? Are your usually strong students struggling with this assignment. Using any unusual trends or patterns in your assessment data acts as a flag for you to review the assessment.

I’m not getting the results I expected?

One of the first things to check is that the assessment design matches your assessment criteria – i.e. you are genuinely assessing what you wanted to assess. Have a look to see if there could be any subtle influences that could be a factor, for example the use of unfamiliar software or students not having access to the necessary resources.

Secondly, check that the assessment criteria are actually aligned to your learning outcomes i.e. what you are teaching the students in your sessions actually matches what they are being assessed on (this is a very common mismatch which can result in invalid and unfair assessments). For example, if you are assessing students on a presentation, are presentation skills an actual learning outcome for your course? If not, you may be assessing your students unfairly.

If you are using computer software to run quizzes and tests, then many of these come with extremely useful statistical reports that help you to review how valid your questions are. Have a look at the statistical report from Moodle for more information. This is a particularly strong benefit of using e-assessments. You can have access to extremely valuable and insightful statistical information in seconds, that would have taken weeks or months to compile from paper based assessments.

The teaching and learning process

Assessment data can also give you a huge amount of insight into how the students are responding to your teaching and learning materials. If you find that you are being presented with the same incorrect answers or skipped answers, as well as addressing this in feedback with the group, use it to think about how you teach these concepts. It might be that there is not enough time dedicated to the area in classes or that the learning outcome, which is aligned to the assessment criteria, is not covered in the same way in the learning materials. This can also result into a subtle, but sometimes significant, misalignment of assessments.

Focus On Assessments

Assessments.

Not exactly a small topic, but working on the old adage ‘write about what you know’, it seems like a good starting point.  Although this is my first role where I am exclusively supporting technology enhanced learning in HE, I have been working with learning technologies for many years. In particular, I have been working with assessment technologies for many years.

I first cut my learning technology teeth whilst working for a vocational awarding body back in 2006. It was here that I was exposed to the complicated and occasionally painful world of assessment theory and practice. I had been brought in to help make the transition from paper-based portfolios and exams, to e-portfolios and e-assessments. Whilst there I worked with examiners, moderators, colleges and students and quickly got a sense of where the challenges were (in both the paper-based and online worlds). From there I moved onto working for a company that created e-assessment software. I worked in a consultative capacity with clients all over the world, assisting them make the transition from paper based to online assessments. It was during this time that I completed my PGCert in Online and Distance Education and created some online courses of my own.

To start off my first blog post I thought I would share a few good practice pointers that I have picked up over the years (NB these will probably not be a revelation, but could help if you’re starting out).

Considering eAssessments? Three questions to ask yourself:

If you are considering changing from paper-based to online assessment, thinking about the following three questions can be a good starting point.

Of course, and this goes without saying, before you make any changes to how your assessment is delivered, make sure you have it approved by following your faculty processes for changing assessment methods.

Why do you want to use e-assessment?

This is really a rule for introducing any learning technology, make sure you have a clear understanding of why you are considering moving to e-assessment. Ideally you should be looking at ways in which it will improve the student experience, e.g. you will have access to reports that’ll help you ensure your questions are valid and reliable, or the students will be able to receive their marks and feedback more quickly. Whatever it is, make sure you have a good clear understanding of the benefits you are looking to achieve. It is also very helpful to explore the potential risks involved in the change too, so you can factor them into your assessment design and processes for delivery.

Will your e-assessment still meet your assessment objectives?

This is a tricky one, and easily overlooked. When you are deciding on which technology to use, consider how the technology might be affecting your assessment objectives and criteria. For example, are you changing from constructed response questions (essay/short answer) to selected response questions (multiple choice, select a blank, true false etc). This may be still be an entirely valid way of assessing your students, but make sure you consider the impact this might be having on your assessment objectives – are you still assessing the same things? If not, is this OK? Also be careful that being able to use the technology has not become an inadvertent assessment criteria. This can happen if the technology is not intuitive or familiar to the students, thus placing them at a disadvantage to peers who are comfortable with the technology.

One way around the last point is to make sure that students have ample access to practice tests using the software you have picked. This gives them a chance to familiarise themselves with the software and helps you to identify any potential issues with the technology.

Why are you replicating your paper-based processes? 

When you move to e-assessments it is natural to try and replicate as closely as possible your paper based process – these are the processes you are familiar with and have probably been using for years. However, you could be missing out on the opportunity to introduce some real benefits to staff and students.

When you first try and transpose your paper-based assessment into an online version, examine the functionality of the software and consider what is appropriate for your assessment. For example, if you do not currently give students their results for 4 weeks, to give markers a chance to grade the scripts, consider if you really need to impose the same delay for an e-assessment? Conversely,  there may also be good reasons to restrict e-assessments to the conditions of paper-based exams – for example, if you don’t want the students to collaborate on their assessments then you will still need to conduct the assessment in exam conditions (even if technically they could take the assessment on their mobile). It is important to establish which of the processes are a core part of the assessment to ensure it is still valid to it’s design, and which are simply the result of the paper-based process.It all depends on what your assessment objectives are!

I have worked with many, many people who have spent a lot of time, effort and money trying to replicate what are essentially unnecessary elements of paper-based assessments, simply to produce an assessment that is almost identical (including all the flaws) to their current paper-based assessment.