ECEL Conference and Poster: A learner-centred induction to Moodle

The “teaching pod” in the University of Groningen’s main hall

Back in October 2012 I attended the 11th European Conference on E-Learning (ECEL) held in Groningen, the Netherlands. This was the first time I’d attended this conference. I presented a poster on our work with the Department of Psychology on their induction programme for the BSc course in September.

The conference was interesting, mostly because it gave some insight into how City’s educational technology perspective is much more teaching-and-learning focused than many other universities’. Many of the presentations I attended were very technology-orientated and I heard people commenting that they would have preferred them to be more so!

I attended some useful sessions: methods for sending students notifications on Moodle updates via Facebook and Twitter; the changing role of the academic in the Web 2.0 world; and an interesting case study on use of blogging for portfolio development.

My poster summarised the word I had done with the BSc Psychology Director of Undergraduate Studies Marie Poirier, to redesign elements of the induction programme for undergraduate Psychology students. There was agreement between the department and our team that induction would be improved by being more learner-centred, less “information overload”, and by giving students more opportunities to get to know staff and each other. Accordingly, we redesigned many of the activities the students take part in during the week. My involvement was mainly in the first day “orientation activities” and in the Moodle induction on the fourth and final day.

Throughout the whole induction week the main principles for the activities were:

1. To reduce information overload
2. To manage students’ expectations and help them understand what is expected of them
3. To start building a sense of cohort community
4. To build a sense of subject-specific identity

The students’ first task on their first day at university in September was to get into their tutorial groups and meet the other students who had been allocated the same personal tutor as them. After a welcome and a brief introductory talk from Marie they were divided into small groups of four or five. Each group was loaned an iPad which they used to go off and make short videos about each other and about their personal tutor. Many of the students were quite excited and impressed to be given these devices to work with on their first day – however, the main reason we used the iPads was because they allow quick and easy shooting, editing and uploading of video (via iMovie and pre-created private Vimeo accounts), and because students can also use them to research their personal tutor. We had run a similar activity the previous year with Flip cameras and laptops: the iPads made the whole process quicker and easier.

The approach we took with the Moodle induction was to redesign it as a task-based fact-finding activity requiring students to work in the same groups as they had been in on the first day. The groups were given access to a tailor-made induction module which contained activities such as quizzes, choices, questionnaires and practice assignment submission points. (They could also watch the videos they had shot on Monday). The idea behind this approach was to have students simultaneously learn about and use Moodle: to learn how to use its tools and functionality by finding out something about it. We also wanted to address many of the concerns and questions students have about their new course by giving them the chance to find out the answers to some common questions (How do I find out my timetable? How do I submit my assignments? How do I connect to the wifi?). Finally, we wanted the induction activity to be clearly and explicitly tailored for Psychology students: we included links to commonly used Psychology resources and included contact details for key members of staff in the department. Without too much work, similar approaches could be taken for other departments in the School.

For details of how the project was evaluated, click on the link below to view a copy of the poster. If, as a member of staff in the School of Arts and Social Sciences, you’d like to try something similar for your induction, please get in touch.

Poster Final

Constructing effective online assessment

I have worked alongside my academic colleagues Isabelle Marcoul and Svenja Erich of the Centre for Language Studies at City University London for the last two years to help develop effective online assessment. This project has now been written up for the recently published Learning at City Journal Vol 2 (2). You can download a full copy of our article here for free.

I’m providing a summary of the article here, focusing on the way the technology was used and how we measured the effectiveness of a multiple choice Moodle quiz.

Background

City University London runs a programme of language modules, some for course credit, some are extra curriculum. The languages taught are French, Spanish, Mandarin, Arabic and German. Before they can join a class the students need to be assessed and assigned to the language course appropriate to their level of linguistic competence, ranging from beginner to advanced levels. In 2011 more than 1000 students took a diagnostic test.

Prior to 2011, the language tests were handed out in a printed format and marked by language lecturers. The administrative burden for this was heavy with very tight marking deadlines, a lot of administrative work to assign students to the correct course, communicate this to students etc. It was concluded that an online system would help automate this, ensure the students received immediate feedback about which level and class was appropriate to them and would speed up the administrative process.

Practicalities

Each year the university runs a Language Fair during Freshers week. Traditionally this was when students took the written test and completed questionnaire (to gather basic information e.g. degree course etc). In September 2011 this assessment was done via multiple choice quiz on Moodle, the questionnaire was also online in a googleform. This meant that

• a computer room was needed for the language fair
• an audio/visual component was deemed to be difficult to manage as a large number of headphones would be required so listening was not part of the test

Design of the test

The languages team wanted to assess different types of language ability while being restricted to using a multiple choice online system. Each language had a test comprising of 100 questions. Please see the article for a full description of the choice of question type and what was assessed.

As a learning technologist I was very interested in how the languages department wrote their multiple choice questions in order to assess different types of language ability. For example, students were asked to read a generic text in the source language and were given comprehension questions to see how much they had understood. Some of the questions also asked that the students not only understand the words but also the cultural context and concept in order to get the answer right.

e.g.

What would you like as a main course?
A sorbet with strawberries
Six oysters
Steak and kidney pie with chips

To answer this question students needed to demonstrate understanding of it and the choices and to pick the correct answer from their own knowledge.

In the article Isabelle writes about how we construct language and how we can assess higher order thinking skills using online assessment methods so please do access the article if you are interested in this.

Use of Moodle and googleforms

City University London uses Moodle as it’s virtual learning environment. This was seen to be the perfect platform for the language testing. I met with the lecturers that would be preparing the questions for the test and explained how the Moodle quiz tool worked. This was to help them understand the types of question that would and would not be appropriate.

Once the questions had been written we had a two hour hands-on training session where the staff were trained in using Moodle quiz and then used it to add their questions with my support. I would recommend this approach. It meant that I could immediately troubleshoot any problems and the staff involved have been successfully using Moodle quiz ever since.

We also needed to collect some personal data from the students e.g. name, degree course etc. We used a googleform for this as they are very easy to set up and the data can be exported in excel format which the administrator requested.

Effectiveness of the language diagnostic multiple choice test

Effectiveness of the test was measured by the number of students that stayed in the group/level they were identified as during testing i.e. the language level of the course matched the language level that the student tested at. We were very pleased to see that the test proved very accurate in determining level for French, German and Spanish (small numbers of students took Mandarin and Arabic so the data was not conclusive).

This shows that an online test can effectively measure language ability in the majority of cases with very little movement of students between levels.

You can download a copy of the full article here

Lecture capture pilot project report

With the help of the Learning Development Centre and Information Services, the Education Support Team piloted a small-scale lecture capture project in the Spring term 2012. The project report is  available to read here, including case studies of use, analysis of staff and student evaluations, and usage statistics.

The lecture capture pilot reached approximately 1000 students on 19 modules in the Schools of Arts and Social Sciences, Informatics, Health Sciences and the Learning Development Centre in the Spring term 2012. There were 4,195 separate views of the recordings during this period, and usage reports show that the majority of use was around the exam revision period.

The student evaluation showed that 91% of students reported using the lecture capture recordings. The most popular uses of the recordings were for revision and to review areas that students did not understand the first time. 93% of students reported that the recordings helped their learning. A very small minority of students (3%) stated that they did not find the recordings necessary.

EDEN conference poster presentation

We have a poster presentation at the EDEN conference in Porto, Portugal next week. The conference theme is ‘Closing the gap from Generation Y to the mature lifelong learner’. Our poster is a case study of a distance learning course, the PGCert in the Principles and Practices of Translation. The vast majority of courses at City University are blended learning courses (using a mix of face to face and online resources) so we were interested to see how the students on a distance learning course utilsed the tools made available to them in Moodle. The poster outlines how mature students from two cohorts on this course, with different levels of technical experience, have utilised the online resources, focussing on the use of discussion forums and Adobe Connect.

Discussion forums

Evelyn Reisinger, Course Director, set up a news forum and discussion forums in Moodle. These were designed to encourage the students to raise and discuss their own issues as they felt appropriate with minimal interference from university staff. I was interested in whether the students utilised these discussion forums to create a community of practice (or communities of practice as there are a number of language combinations available within the programme). This draws from the work of Etienne Wenger (2006).

I completed a content analysis of the discussion forum postings for each year group. I did this by reading through each post and categorising it in terms of its content.

The contributions on the discussion forums specifically relate to some of the criteria for communities of practice as outlined by Wenger (2006). These include problem solving, requests for information, coordination and synergy and discussing developments (see the percentage interaction for each cohort below).

 

 

 

 

 

 

 

 

Adobe Connect

The academic staff were keen to have some face to face tutorial time with the students. This was done using Adobe Connect (AC) web conference software already in use at the university. The sessions were specifically designed to answer student questions just after they had received assessment feedback. The students sent in their questions before the session and the course lecturers answered them in text form on Moodle and through AC.

Results

• AC live sessions were not well attended
• Some students suggested that the time of the session wasn’t convenient
• Some students are studying from different countries so the time zone may have been a factor
• The sessions were held in the afternoons when many of the distance learning students may have been at work or had childcare issues
• Students that didn’t attend did access and view the tutorial recordings so the sessions were perceived as worthwhile

Conclusions

From analysis of the usage of the tools, feedback from students, lecturers and the administrator we concluded that

• Students did use the discussion forums to communicate on many levels and did create communities of practice. In 2009/10 they were focused on the course but in 2011/12 a German to English mothers forum was set up and this includes personal interactions about their lives and similarities. The staff on the course are actively encouraging use of the discussion forums in this way
• Students were willing to use the online resources made available to them. They had signed up for a distance learning course and were made aware that resources were shared online so this could have led to a self selected IT-confident group
• Students used discussion forums for a number of different interactions, mostly related to the course but including some personal interaction
• Age of student was no predictor of their use of the technology
• Adobe Connect recorded tutorials were accessed if students could not virtually attend at the time of the tutorial so proved a valuable resource type

Interview with Course Director Evelyn Reisinger on using Moodle (recorded during the first year of the programme)

Conference papers accepted for Europlat

Kate and I were pleased to learn today that we have had two papers accepted for the twelfth European Congress of Psychology convening in Istanbul in July this year. We attended the conference in 2010 and were impressed at the breadth of work being discussed.
The conference attracts delegates from across Europe and we will be speaking to the teaching and learning strand of the conference (Europlat). The papers that were accepted are

‘Using technology to innovate in Psychology Higher Education in a UK university: a case study’
This paper looks at the move from a traditional face to face approach to teaching to the use of a VLE (Moodle), virtual classrooms and online tools for assessment, grading and feedback. This move to a more blended approach has been prompted by the changing situation in UK higher education funding – with students needing to work part time in order to fund their education and consequently needing a more flexible approach to their learning. This paper addresses how City University London has approached this and some of the challenges we have faced.

The second paper is entitled ‘Student feedback and online marking in an undergraduate psychology program: a case study’  At City University London, we are gong to be taking a feed forward approach to feedback in the undergraduate psychology program. We will be working with the PhD students to produce a detailed guide for students.  This guide will be designed to help students understand what they are expected to do with their feedback, and how they can apply it to their next assignment.

We are looking forward to spending some time learning from our colleagues at this inspiring conference. We will, of course, blog whilst we are at the conference.

European Mobile Learning Conference Bremen 2011

I’m very pleased to announce that Dr Sian Lindsay, Ajmal Sultany and myself recently had our paper on the role of Mobile Learning in HE accepted for presentation at the 2011 European Mobile Learning Conference in Bremen. The abstract for our paper is below:

Just because they own them, doesn’t mean they use them: Exploring the potential for mobile learning in Higher Education

Abstract

This paper explores a two year study conducted into current trends in student mobile device ownership and attitudes in a UK HE Institution. In February 2010 after the first year of our study, we reported on the finding that 99.8% of City University London students owned a mobile device, however our student body had clear ideas as to how they would like to utilise these devices for their Education.

This paper presents the findings of our 2010 and 2011 student mobile surveys. The survey results overwhelmingly indicate that even a year on from our original survey, our students still want to use their mobile devices for accessing teaching-related activities, learning content, and administrative tools. However after implementing a new wireless infrastructure at the University, based on the results from last year’s survey, the majority of students are still not positive about using their personal devices for interacting in class. The paper explores what CUL are doing to respond to the request for more access to information via mobile devices, while questioning why students are not willing to use their mobile devices in a more formal classroom setting, and examines what happened when we piloted in class use of mobile devices.

1. The Mobile Learning Survey at City University

The aims of the student mobile device survey was to discover the types of devices students own, what they currently use them for, their attitudes towards using these devices for formal and informal learning, and any issues surrounding the use of these devices on campus, and more importantly whether these attitudes are changing over time. The explosion in smartphones and more recently tablets has been attributed to advancements in wireless technology and 3G mobile networks, as well as the production of extremely sophisticated hardware. The 2009 Horizon Report suggested that mobile devices would be widely adopted for learning within a year (Johnson et al. 2009), and it is important for institutions to consider both the pedagogic potential, and the degree to which students are both willing and able to put their gadgets to this purpose.

1.1 Methodology

The Mobile Device Survey at CUL ran in January 2010, and then again in January 2011. Several changes were made to the survey the second time it ran, the most significant improvement to the design has been to the sampling. During the first student mobile survey in January 2010, the research team used a ‘self selection’ sampling technique in which the students were made aware, through the use of advertisement, of the study survey and could choose whether or not to complete the survey. The researchers had no control over who would take the survey and how many times, and this method does not allow results to be generalised to the target population as there are many biases involved.

During the second survey conducted in January 2011, based on methods outlined by Dr R. Sapsford, stratified random sampling was applied (Sapsford 2007), this was in order to give a research sample that would be more representative of the population studied as a whole and therefore more likely to satisfy mathematical assumptions underlying many of the statistical tests we wished to use during analysis of the data. To produce a random sample we obtained an accurate list of student email addresses at the University. The sample was then stratified according to School so that students from all Schools were fairly represented, the researchers then used a program to randomly assign a number to each member of the population from which the sample was taken, these students were then send personalised email invitations asking them to complete the survey.

In order to ascertain whether the questions on the survey were clear, we used a pilot group of colleagues within the University, but not necessarily working with technology to pilot the survey and produce feedback, before releasing the survey to the students.

2. Results and Conclusions

Although the methodology used in disseminating the survey changed in 2011, the results have been consistent across both years of the survey despite avoiding a ‘self selecting’ sample the second time round. The survey showed that only one student each year did not own a mobile phone, and many students own Smartphones and tablets. In 2010 69% of students had used their mobile devices for web browsing during the last ten days, in 2011 initial results show that this has risen to 72.3%, In both 2010 and 2011 65% of students use their mobile phone to access their email, and that 53% of students had accessed Facebook on their mobile phone within the last 72 hours. These results illustrate that our students not only own sophisticated devices, but also use them in a sophisticated way.

In the 2010 survey the majority of students complained about their inability to access the wireless network on campus via their mobile devices, at that time you could only access campus Wifi with a laptop. The evidence collected during the survey was used to build a business case to improve the network infrastructure on Campus, and this new network was then implemented in September 2010. However while students now use the improved WiFi network on their mobile devices, the way in which they want to use their mobile devices on campus has not significantly changed.

We asked the students “if accessibility was improved at CUL, what would you like to use your mobile device(s) for? (please select all that apply)”. In both 2010 and 2011 the results were very similar. The top three answers that students gave were: Viewing timetable information (2010 & 11), Receiving Grades and Feedback (2010 & 11), Receiving Txt Alerts from Tutors and administrators (2011), Accessing the VLE (2010). The answers that the fewest number of students opted for were: Asking questions in class by txt message (2010 & 11), in class voting (in place of PRS/Clickers) (2010 & 11), Subscribing to RSS and Podcasts (2010 & 11).

It was clear from these results that while students are extremely competent at using the complex features their phones offer, and would like greater mobile access to institutional services such as grades and feedback, they are not keen to use these devices in a formal classroom setting. In both the 2010 and 2011 surveys the majority of students ticked a box that said they “did not want to use my mobile device in class as part of my education”, many students left additional comments stating that this could be distracting, and they didn’t think it was appropriate.

In his 2010 article on student mobile devices John Traxler said that “student devices unlock the dreams of agency and control and choice amongst students… Universities cannot afford, procure, provide nor control these devices, but they cannot ignore them” (Traxler, 2010). With the current funding cuts in Higher Education across the UK, coupled with the pressure from employers to provide graduates with high level computer literacy skills, institutions may expect student owned mobile devices to play a larger role in students formal education. As institutions become less able to afford up-to-date technologies, our paper examines how student attitudes may influence our use of mobile devices for teaching and learning.

References

Johnson, L., Levine, A., & Smith, R. (2009). The 2009 Horizon Report. Austin, Texas: The New Media Consortium.

Traxler, J. (2010), Student Mobile Devices, ALT-J Volume 18. No.2

Dr. R, Sapsford (2006), Survey Research, Sage Publications Ltd; Second Edition

“Mobile engagement or miss-dial?” A Multi-institution survey interrogating student attitudes to mobile learning

This paper was presented at the Association for Learning Technologies Conference in 2010.

Kate Reader, Mike Cameron, Sian Lindsay, Hilary Griffiths, Ajmal Sultany

City University London, Durham University, City University London, University of Bristol, City University London

The 2009 Horizon Report suggests that mobile devices will be widely adopted for learning in the next year. This study conducted at three UK HE Institutions highlights that a high percentage of our students now carry smart devices (like iPods and Blackberrys) with features like web browsing, running of diverse applications and location awareness becoming standard. There are already numerous commercial and institutional e-Learning packages and ‘apps’ for students to install and access on the go. Just as educators have been keen to exploit the ‘sea-change’ that saw social networking mushroom, institutions are now co-opting students’ mobile devices. As with Web 2.0 tools, we need to consider both the pedagogic potential, and the degree to which students are both willing and able to put their gadgets to this purpose.

Mobiles were previously reserved for personal and social use only. This study investigates whether students are willing to compromise by combining social use of their mobiles with formal education and draws together online surveys of students at three UK universities with different educational strategies, together with a review of existing literature. The paper seeks to establish the willingness of students to use their mobile devices for learning and in which contexts. We explore what devices, contracts and skills students have and consider how we might use them effectively for blended learning and more interactive face to face teaching. Are students willing to use their mobile devices to access learning materials or send and receive texts with classmates and lecturers? Will students use their credits to vote in class and bandwidth to view educational multimedia? Or will students view their mobile devices, like some Web 2.0 sites, as ’their environment, not ours”? Many students counter, “if they go on to Facebook, I’m moving out” (Salmon, quoted in Swaine, 2007). Will students also switch off their mobiles to education?

Understanding ownership, skills and attitudes to mobile learning can guide institutional approaches to adoption. Can we make productive and effective use of mobiles to meet student expectations and provide more interactive, time and location-sensitive learning? Or are we dialling a wrong number?

Click here for a full copy of the study

Click here for our presentation:
http://prezi.com/3835/view