Electronic Voting Systems: Exploring the Research

I'll be doing my presentation at the SOLSTICE conference about the use of these, so I'll write a few posts to record my journey through the research in this area.

We were talking to the team from the Blended Learning Unit about the use of EVS when they were visiting yesterday, and they recommended Steve Draper's pages as a starting point. I'll make it easy for myself for now and just explore what he recommends and links to from there.

1. A video of using voting equipment in class:
This video wasn't working for me but the associated questions show how well designed questions might look. For example:

Question 3 A sample of 500 GU students was asked two questions:
a) Should the UK deploy nucelar weapons?
b) Should the UK have a death penalty?
The results of a Chi-squared Test of Association are shown below [table given]
What is the appropriate formal conclusion about the population of GU students from this test?

1. There is a very strong association between answers to the two questions.
2. There is a very significant association between answers to the two questions.
3. There is a no significant association between answers to the two questions.
4. There is a strong positive association between answers to the two questions.
5. Don't know.

I'd like to explore ideas and research about good practice in creating multiple choice questions. However the team from BLU were saying that simple questions are often effective in displaying if the basics are understood.

2. Interactive Lectures:
This article talks about why this is a good approach. It puts forward EVS as one way of going this way and presents 3 benefits of the approach:

Directly for the learners e.g. by eliciting (re)processing of the content, which deepens understanding and lengthens retention; and by getting feedback that shows them what they do and do not understand to guide study later.

Directly for the teacher: getting feedback that allows them to improve what they do. This may be explicit ("Do you want me to go slower?") or implicit by asking content questions, and inferring from the answers what needs more attention.

True interaction. Independently of private benefits to the teacher and of private benefits to the learners, there are the benefits of establishing real iterative interaction. The defining difference is that the teacher doesn't just get information from the learners' actions, but changes her own actions because of it; and then learners change theirs and so on. This iterative (to and fro) process:

• Achieves improved learning by converging on understanding even if initial attempts fall short
• Makes the learners feel much better, as they perceive their actions making a difference
• Truly adapts the teaching to the particular set of learners
• Improves the teaching much faster (at least from week to week, often from minute to minute) than the standard course feedback (once a year) or a textbook (once per edition i.e. every few years).
• Achieves true interaction, where what happens is fundamentally and constructively contingent on the other parties.

...this all makes sense to me at this point. I puts the benefits better than I had.

3. EVS: A Catalyst for Lecture Reform by A. Bruce.
This short article starts off by saying:

this technology addresses -if imperfectly- significant deficiencies of the teaching-by-lecturing model that dominates mass HE

It puts forward a model used at Harvard and then Strathclyde where the lecture is divided into sections (perhaps about 15 minutes) covering a core topic. Students are given time to answer a multiple choice question (MCQ) on the topic after it has been briefly covered to uncover misconceptions. Students see the results (but not the correct answer) and then Students can then get into small groups to discuss the answer before voting again. The lecturer can then draw the subject to a close depending on what they've seen.

I like that idea - it could have several effects... students taking part in the lecture means that being late adversely affects their experience... Powerpoint Slides on the VLE, or even a recording of the lecture aren't going to be anything near the experience and value of the lecture.

4. CH Crouch and E Mazur, American Journal of Physics 69, 970 (2001) : Peer Instruction: Ten Years of Experience and Results.

This comes out of a physics course and starts of by saying little is learned in lectures. It talks about using a test in the lecture to probe students understanding. Students get a certain amount of credit for participating in tests over the semester, but also saw student performance on exams improve.

Issues brought up involved student expectations ("When can we do some REAL physics?"), so it might be important to explain to students why the discussion and voting is incorporated.

It is also mentioned that open-ended questions can take a place in this sort of lecture. From there a set of "best-fit" responses could be chosen by the students. These sort of questions might be more valuable if they fit in with something the students have been reading before the lecture. I suppose they could fit nicely in a Development Studies type module where you could ask what the best solution to a political/social problem might be... where there isn't a right answer. The fact responses can be anonymous means people can be honest. Then discussions can take place and we can see if people opinions change.

This paper talks about peer-instruction and active-learning. I'd like to explore the research around these areas as they seem to fit in with using EVS.


5. Draper,S.W., Cargill,J., & Cutts,Q. (2002) Electronically enhanced classroom interaction Australian journal of educational technology vol.18 no.1 pp.13-23.


...and then move onto those at http://www.psy.gla.ac.uk/~steve/ilig/main.html#Papers

Notes on Audio: 'ELI In Conversation: Second Life and Virtual Worlds - An Approach to Active Learning'

[listen at]

Chris Johnson - Experience in Second Life - first thing he built was an auditorium, but he realised this was against everything else he believed in. It helped him to rethink his teaching methods and his desire to create an environment where students learn. Second Life doesn't work as a content delivery model.

Sarah Smith Robbins - You see clearly with this that students 'slump over' if you teach at them in Second Life. If this happened in class we might wake up to the limitations. People use this for student created content. My teaching sometimes never really worked because I was not the only expert in the classroom, but was at the front and so was the focus.

Teshia Roby - Active learning rubric can help student teachers think about how this works both when creating presentations and in an active environment like Second Life.

In SL we can create a community of educators who can observe each others work and learn from each other and see the effects of certain things. In fact many teachers invite people to observe.

Chris Johnson - Classroom roles change in SL compared to the classroom. You need to let people know that it is OK to make mistakes, in fact that it is expected that we'll all have problems.

Flashforward 2007: Real World Flash Design Recipes - Part 1

This presentation by Colin Smith looks at fun things you can do in CS3.

Firstly it looks at video effects. You can add effects, filters and blend modes to a video clip when you convert it to a movie clip.

Flashforward 2007: Fun with Flash Media Server - Part 1

In this talk Lisa Larson-Kelly talks about Flash Media Server.

It communicates using RTMP which opens a two-way socket, which you can use to send video, audio [streams] and data [remote shared objects].

There are client side media classes (e.g. Camera, microphone, SharedObject, ect) and we can use ActionScript 1, 2 or 3. There are more on the server side (in ActionScript 1) including the ability to write to files on the server, the ability to deal with data.


It looks interesting for more complex applications. I suppose we'd need to ask whether it is worth learning to use Flash in this sort of detail for the amount of work we would do in it.

Flashforward 2007: What's This Button Do? Communicating Web 2.0 Through Design - Part 1

Moving on from Instructional Design to Instructive Design, this presentation by Robert Hoekman, Jr. starts by pointing out that people will not instinctively know how to use and navigate web sites that are using new technologies. It look at how we can make it easier through simple and unobtrusive instructive design features. These could include brief step-by-step instructions and default values in forms.

What is Instructional Design?

There is a debate about whether ID is necessary. To be honest I cannot really take part in this as I've not got a clear definition of what ID is. In this post I'm trying to develop an understanding of both the term and the debate, and then seeing if it affects us as Learning Technologists in HE. Most of the debate around this term seems to be in industry, but things may apply to us.


Starting my journey in Wikipedia I notice the claim:
"the [ID] process consists broadly of determining the current state of learner understanding, defining the end goal of instruction, and creating some media-based "intervention" to assist in the transition." The book Preparing instructional text is recommended.

Reading this article I'm thinking that the term ID covers what most teachers do.


Learning Circuit's big question for Feb 2008 is Instructional Design - If, When and How Much?

In a post related to this debate - The Death of the Instructional Designer - it seems like the ID term refers to people experienced in doing this online. They might have worked with a subject expert traditionally, but now rapid elearning development tools mean that Instruction Designers are seen as adding little value or are just cut out of the loop anyway. This post says that they should become Solutions Architects helping companies increase productivity at a higher level. Relating this to HE, the subject matter experts (SMEs) are also teachers and therefore should be doing ID at some level.

Vaughan Waller's article Is Instructional Design Dead (Again)? argues that it is the 'death' of the Instructional Designer that is causing poor quality e-learning experiences and loss of reputation for e-learning in general. This includes companies saying that all 500 training sessions they want to run should be online without discussing if this is the best method of delivery.

Is Instructional Design Dead? notes that the issue with ID is when it just focusses on developing a classroom based (or online classroom based) materials. It argues that on the job training, simulations are much better and Instructional Designers need to focus on these approaches. Potential competencies that ID practitioners would make use of are listed including project management (planning, scope, communication, budget), modern instructional design (blended, experiential, longitudinal, on-the-job) and broad format competencies (classroom, e-learning, simulations, podcasting, etc).

Instructional Designer Competencies by Manish Mohan gives another list of skills that the new ID practitioner needs to be effective. These include ability to write stories, dialogues, scenarios, narration scripts, understand basic instructional design principles (this is the foundation- write correct objectives, structure the content etc).

elearning technology: significant work needed to help instructional designers by Tony Karrer, brings together a discussion about ID models like ISD, ADDIE and HPT. It is agreed that they form a good foundation if they evolve, but people are unsure how they could change. I like the quote Russ Crumley:

The challenge we now face is not the creation of more tools, but better use of the tools we have, while applying the fundamentals of human behavior and performance, including the role of emotion, curiosity, discovery, and the desire to improve.

From all this I've got a clearer concept of how a Learning Technologist could include Instructional Design in their list of abilities - separating it from where we do other things. So you could define the role as:
-Giving advice on good practice in the use of technologies.
-Providing training in how to use specific new technologies.
-Research into the use of technologies in Teaching and Learning.
-Involvement in Instructional Design as part of an academic team.
-Technical development of more complex resources.

Notes: Alt-J 15:3

"What constitutes good research in e-learning - are there lessons we can learn from the Research Assessment Excercise?"

-The RAE is used to review HE Research about every 5 years.
-There are 67 units of assessment. The most important are publications.
-4 Stars would denote world leading research and 1 would be nationally recognised research.
-There are some ways of working out if research is worth publishing. For example:
--Is it an important or topical area?
--Will it be cited a lot?
--Does it define an area?
--Will it have impact on decision makers?
--Does it develop a new theory or model?
--Does it clarify an established problem?
-Elearning is multi-disciplinary and cross-disciplinary and doesn't neatly fit in the RAE.

--------------------------------

What does this mean for me?

If I want to try and publish work we've done, I'll probably get institutional support.

I'll do well to look at books like Contemporary Perspectives in E-learning Research to get a better idea of how to approach the area of elearning research.

Identify which areas that I'm looking at and compare with the list of things that make research worth publishing.