Richard – Turningpoint clickers, one of the systems available for capturing classroom response, has been adopted by SBCS for summative assessment. Rachel o’Callaghan, a lecturer in Physiology and Biomedical Science, has kindly agreed to talk to me to discuss her use of the technology.

Richard – Hi Rachel.

Rachel – Hi Richard.

Richard – Thanks for meeting with me today.

Rachel – No problem at all.

Richard – So clicker use has proven popular with some Schools at QMUL, particularly so with SBCS. What prompted you to start using this technology, rather than carry out online tests or written tests?

Rachel – I’ve been using clicker technology in my lectures for a number of years. I’ve used it mostly formatively, but over the last year or 18 months, I’ve started using it summatively. I have quite large groups that I teach, so about 240-250 students and partly because it was a technology that I was familiar with having used it formatively before, so it seemed like a natural progression in terms of familiarity for the students, etc. I also feel that over and above let’s say written assessments, as you can obviously do MCQ assessments using a paper format, there’s a much quicker turnaround – you can actually post results in less than 24 hours. Probably mostly the reasons being, my familiarity with it, increasing the students familiarity with it and the speed which I can get the results back to them.

Richard – And is it a case that you use it in addition to written tests and online tests?

Rachel – I use online tests in a formative way, so I use online quizzes, often students comment that those are useful for revision purposes but also for them to stay on top of their knowledge in things that I’ve taught them. Written tests – I don’t use them in my modules now, I do have some forms of written assignments still, so most of my modules have practical elements and the students have to submit a piece of writing for that, so that still exists but it exists alongside the clicker assessments  which I use the Turningpoint software for.

Richard – How do you ensure the successful use of clickers?

Rachel – In terms of troubleshooting against technology not working? That’s definitely been a steep learning curve really. So the first time I used clickers, I definitely identified that there were issues. I think mainly, students are nervous their responses won’t be recorded, that was probably one of the bigger problems. Secondly, that maybe that their device wasn’t working on the day and may not work throughout the whole assessment. I decided, with Stuart Cadby who supported me trying out the use of these clickers summatively, to basically give a lot of time to the set-up, so at the start we would count the number of the students in the room and we had a practice question so we asked all of the students to answer a question which was not assessed and we made sure that tallied with our tally of the number of students. This also reassures the students that, sitting there, their responses are being recorded. Other problems that we’ve encountered is students coming along and maybe not being that familiar with their clicker, not having their battery and not having the right channel turned on. I suppose originally I gave them a lot of support in terms of that and I would remind them, and we would spend even more time at the start, making sure that it was ready set-up to go. This year, I posted information online, prior to the modules, reminded them that the clicker tests were coming up and I put it on them, giving links to resources, for them to make sure to check the battery and that the right channel was on. So shifting that responsibility seemed to work quite well. Now, I did bring some back up batteries, but moving towards reminding them, right from the start, that it was their responsibility to make sure the clickers were working, that they were on the right channel and just making the rules very clear in terms of whether they can click more than once during the time that’s allocated for the question or that their first click is the one which recorded – so making all of that crystal clear at the start and reinforcing it, I think has helped alleviate any concerns about the technology and any blips that might occur.

Richard – Definitely, that’s really good. In terms of late comers, or people that forget their clickers?

Rachel – In terms of people who forget their clickers, I’ve become really hard-nosed about that, if they forget. Unless they have a very reasonable excuse (I then allow them to use a paper version and then I mark that myself). I think that if that was a repeated activity, if they then didn’t bring their clicker to the next test – that’s never happened actually – I would not think it would be fair to allow them to do that again. In terms of late comers, I very much specify – and we experienced this when you supported me this year – that I say to them that if you are not there by this time your responses won’t count. Although this year, you’ll recall we did have a couple of students who did come in late, but we were able to easily identify who they were as they didn’t answer the early (test) questions, so I just removed their marks. They knew that those were the rules, they were explained right at the start of the module and were reinforced throughout, close to the test – although they could stay and attend, if that helped them, I didn’t give them the marks.

Richard – And do you think a School policy could help you with this or do you believe that it should be flexible?

Rachel – I don’t know how many people are using clicker tests in this way. I know that we are very much encouraging clickers to be used in SBCS, primarily to engage large groups of students in a more formative sense, but I would feel potentially we could think about generating a policy that’s more universal for those people who are using the technology in this way. For now, I feel we are still learning too, so I would want to hold off and be making notes.

Richard – And do you feel that there are any access arrangements or reasonable adjustments that you need to make for disability and dyslexia students?

Rachel – Yes, so in terms of the timings which are allocated – I just made sure I added on sufficient time as recommended by the university, to allow for students who had any disabilities or dyslexia, to be able to answer the questions comfortably in the time frame which was allocated. Although I would imagine for most of the students that felt too long, they knew from the start that this was the time allocated for each question, and why, so it was just accepted.

Richard – Yes, and do you read out the questions at all for the students?

Rachel – Yes, I do also. Yes, I read them out at the start and they typically have 75 seconds to answer the questions.

Richard – That’s really good. So, how many assessments do you tend to find you are carrying out in this way, each year?

Rachel – One to two per module. So, on some modules I have a mini test which is styled using Turningpoint clicker software software, which is summative, so I’d have that test in the middle and then I’d have two assessments at maybe a third and two thirds through the year and with those ones especially I thought what was important, and in fact students commented on it in terms of feedback, was that it was cumulative knowledge so they were assessed for the first test which was knowledge covered up until that point, but the second test included that early content or could include that early content, with the idea being that their learning was continuous and they weren’t learning in sections or by chunks of information.

Richard – So did you encounter any technical or administrative challenges have you found in using this technology in this way?

Rachel – The biggest technical challenge for me, when you’ve got 250 students in front of you and you press ‘poll’ and clearly something is wrong, then that can be quite difficult – because the students are quite stressed in that they want to make sure that they do well and managing that in real time can be quite difficult. Another difficulty can be when students will, absolutely hands down, claim that they were there but that their clicker, for whatever reason, hasn’t shown that they were there. That can be something which is quite challenging.

Something I did this year, for the first time, on the advice of a colleague, was to double check that all students’ clickers were registered prior to the test assessment and if they weren’t, they received a personal email reminding them to register their clicker, so that removed that potential problem. There were a few genuine cases, one being that two students had accidentally switched their clickers and they were both quite traumatised by the fact that they did not have the right clicker but they figured it out before the test, so that was fine. The only other thing that was problematic this year was I experienced students coming in and maybe not having only one clicker – so having more than one clicker. This year it was somewhat easy to manage because in the situation presented, the student was late anyway, so I just removed both of the results and they just didn’t get a mark at all and I did contact Tony Michael how to address this and both students were contacted in relation to this.

Richard – And I suppose through the initial test question at the start, you can identify if you have more clickers than you do students.

Rachel – Exactly, exactly. Yes and little things I’ve found, with doing it this way in the summative way, whereby it counts for something, is saying to students, for example, that ‘your clicker needs to be on the table – nothing else on the table other than your clicker.’ If you have someone supporting you, which is highly valuable, is just having someone to note that there is one clicker per person and that there is no extra hand activity in the students’ pockets, because you just cannot keep an eye on that yourself.

Richard– So you would definitely use the technology with an exam invigilator of some sort?

Rachel – I think so – absolutely. Especially when you’ve got large groups. It’s one thing when you’ve got 20 students, but when you have 200-250 students it’s absolutely impossible to know that they’re all being honest and not checking answers on their phones. I mean, that’s going to be a problem if you have 250 in front of you and they’re doing a written test anyway, but having another person in the room – absolutely, is important and valuable.

Richard – Oh good. So, Turningpoint is the clicker technology which we use at QMUL, but did you consider any alternatives?

Rachel – I didn’t really, mostly because I’d used Turningpoint before and I was familiar with it. I know that in the future Turningpoint might be tricky, so I am very conscious of other AR systems out there – I haven’t had a chance to play around with them or find out if any of them will be potential players for the future, but I think the idea behind them is pretty similar so I’m sure if it comes to a point when Turningpoint isn’t something which we can use here, then there are options for us.

Richard – And do you feel at the moment the current system meets your needs?

Rachel – Yes, I think so – absolutely. Yes, as I said, we’re focusing on summative stuff here, but I feel definitely formatively it’s brilliant whereas summatively there are more challenges, but for now it does meet all of my own needs.

Richard – OK, good. So, you have received some feedback from the students – have you found that it’s been positive or negative?

Rachel – I think positive, yes. I was actually looking through some quotes which students had written in the open text answers from the module evaluation and most of them do like it. They don’t all comment on the use of the clicker mini tests, but most of them do like it. I think what they also like, is that I go through the answers with them straight away afterwards and they do like that and they can identify how they’ve done. They do appreciate that they get their marks really quickly; students like getting any kind of feedback and that speed of return. I think as well, the question styles that I use (I use MCQs 5 option answers) and for one of my modules that’s exactly the same style of an MCQ that they will see on their exam, so I highlight and signpost that to them and that is also valued. They know and are familiar with this kind of format and this is what we’re going to be faced with in the exam, which takes the edge off it.

Richard – So, with that feedback that you’ve received from them and with the help that you’ve received from other members of staff, are there things that you think you may tweak, changing things slightly, for next year?

Rachel – I think maybe I will stick to my guns more in terms of the rules, but otherwise I think no. It seems to be working quite well for where I’m using it. I’m someone who always reflects on feedback and wants to try to change, but from what I can see most students do like it. Of course some don’t, but that’s always going to be the case – some prefer the written tests, but I have definitely found the majority prefer it, so I feel confident that it’s something they like.

My main bit of advice would be, taking time in terms of the environment when you set it up, the students feeling confident in the technology, you feeling confident in the technology, matching the number of them to the number of responses in the practice question, setting the rules very clearly from the start – that it’s there responsibility that their clickers are working, that they have batteries and are on the right channels. All of that. I don’t think you can’t do enough in the preparation stages, because once you’re there, the challenge is that it’s real time so if things go wrong you’ve got this number of students in front of you and you have to figure it out then and there.

Richard – Thank you. This is all useful information and I’m sure a lot of members of staff will find this invaluable in deciding upon using this technology. Thanks very much.