Report on the WebPA Student Surveys

• Introduction
• Survey Approach Adopted by Each Institution
  • Loughborough University
  • The University of Hull
  • The University of Leeds
• Students’ Attitudes Towards Group Work
  • Loughborough University
  • The University of Leeds
• Students’ Attitudes Towards Peer Assessment of Group Work
• Student’ experience of using WebPA
• General Conclusions

Introduction

As part of the evaluation activity undertaken by the WebPA Project or pilot implementations, student surveys were conducted at Loughborough University, the University of Hull, and the University of Leeds.

These surveys were conducted independently of each other with each survey being developed separately at each institution to address individual needs, and thus each having a different focus. This makes cross-institutional comparisons challenging.

This report includes:

• a summary of the survey approaches adopted by each institution
• key messages arising regarding students’ attitudes towards group work
• key messages arising regarding students’ attitudes towards peer assessment of group work
• the students’ experiences of using WebPA
• general conclusions derived from these surveys

Survey Approach Adopted by Each Institution

Loughborough University

Two surveys were conducted at Loughborough by the WebPA team.

The first survey involved a paper-based questionnaire issued to a single cohort of students from a single engineering department. This resulted in 51 responses (a 74% response rate). The main focus of the survey was on capturing the students’ experiences and attitudes towards both peer assessment in general, and WebPA in particular. A brief report was produced in January 2008 summarising the results arising (not available publically).

The second survey involved an online questionnaire which was circulated to all students across the University who had used WebPA during the 2007-2008 academic year. The majority of questions were the same as those used in the initial survey, thus allowing some comparative analysis to be made carried out between the two. This second survey generated 386 responses (an 8.5% response rate) from students across 12 different academic departments. A brief report was produced in August 2008 summarising the results arising (not available publically).

The University of Hull

Two separate student cohorts (of Biology Students and Chemistry Students) were surveyed. Prior to using WebPA both sets of students were questioned about their experience of group work, their preferences with regard to group work, and their confidence of using IT systems.

After using WebPA the same cohorts were surveyed to measure any impact or change in attitude resulting from the experience.

In total 177 students responded to the pre-survey and 110 students responded to the post-survey. A brief report was produced summarising the findings arising from both surveys.

The University of Leeds

At Leeds a small cohort (29 Design students) were was surveyed following a group work exercise involving peer assessment through the use of WebPA. The students were asked a series of questions designed to measure their response to the experience. A summary of the results arising was produced in December 2008.

Students’ Attitudes Towards Group Work

The surveys conducted at the University of Hull, and to a lesser extent Loughborough University, generated data on students’ attitudes towards group work as a learning medium/experience.

In general students had very positive feelings with regard to group work activity.

At Hull the vast majority (93%) of students responding had had previous experience of group work. A large proportion of respondents (78.2%) stated that they “enjoyed working in groups”.

The benefits of group work that the students identified included:

• it enabled them to help and support each other
• it allowed them to share knowledge and information
• it provided insights into alternative perspectives and methods of working
• it allowed for the workload to be shared
• it enabled them to tap into different strengths and competencies
• it helped to generate more ideas
• the social aspect provided a more enjoyable working/learning experience.

An interesting aside note relating to the survey at Hull demonstrated a dichotomy between those who favoured the use of web technology as a means of enhancing group work interaction (through e-mail, discussion boards etc.) as a means of enhancing group work interaction and a sizeable proportion who declared a strong preference for face to face engagement.

At Loughborough a significant majority of students (74.3%) expressed a preference for modules to include a mixture of group work and individual coursework.

A sizeable proportion of the students (45.8%) expressed a preference for groups to be ‘“self-selected’” over other means of selection (e.g. seeded groups).

With reference to the benefits of group work for students, a significant proportion of the respondents at Loughborough identified group work as contributing to the improvement of their communication skills (75.5%), team working skills (78.4%) and problem solving skills (59.6%).

Students’ Attitudes Towards Peer Assessment of Group Work

The surveys conducted at Loughborough University and the University of Leeds generated data on students’ attitudes towards peer assessment of group work.

Loughborough University

At Loughborough both surveys generated similar responses. Students expressed a clear preference for carrying out group work with a peer assessment element (77% through the online survey, & and 72.5% through the paper-based survey). By comparison only 14.3% (online survey) and 7.8% (paper-based survey) of students expressed a preference for group work without a peer assessment element.

Through the online survey similarly large proportions of students stated that they were:

• comfortable with assessing their own performance (79.8%)
• comfortable with assessing the performance of team members (87.1%)
• comfortable with team members assessing their own performance (70.4%)
• confident that their assessment would remain anonymous (73.3%)
• satisfied that the assessment was fair (63.1%)

The results arising from the paper-based survey showed similarly positive responses.

Other interesting responses (received through the online survey) included:

• a small majority of students who stated that self and peer assessment had increased their motivation to carry out group work tasks (51.3%)
• a majority who expressed that relationships they had built up with team members had influenced the scores that they gave (59.2%)
• and a significant minority of students who stated that they did more work because they knew they were being assessed by their peers (38.2%)

Again, these response rates were mirrored in the paper-based survey.

Additionally, a significant majority of students stated that the experience of peer assessment had improved or much improved their skills in peer appraisal (77.7%) and self reflection/appraisal (72.6%) skills.

In measuring the impact of peer assessment a small majority expressed the opinion that it had had a positive impact upon overall performance (54%) and the standard of the work produced (53.4%). By comparison only 3.1% of respondents felt that it had had a negative impact. (The question of impact had not been included on the paper-based survey).

The University of Leeds

The generally positive attitudes towards peer assessment found at Loughborough University students were more or less reflected in the survey conducted at Leeds, although this survey used a different set of questions.

Here a significant majority (90%) of students described the use of peer assessment as “fairer”, with 92% of respondents of the opinion that they had assessed their peers’ work fairly, and 80% feeling that their own contribution had been fairly marked.

Other notable results included a majority of students (63%) who felt that they learned more when team work was peer assessed, and a significant minority (46%) who felt they contributed more when work was peer assessed.

In response to the question “to what extent do you think that peer assessment should affect a student’s mark for the peer assessment assignment?” it was interesting that, despite the overwhelmingly positive responses noted above, that a significant majority of students (70%) felt that peer assessment should not contribute more than 40% of the total marks allocated, with half of these feeling of the opinion that the contribution should be less than 10%.

This survey invited free comment which allowed students to touch upon a number of issues, with the issues of feedback, anonymity, and fairness being readily discussed. Notable comments included:

• “There is no need to see who said what, but it is helpful to know how the others have viewed your contribution so that you know what to work on for next time.
• “I would be interested to know how I was marked but making it anonymous makes people answer more truthfully.”
• “I think that for the work to be peer assessed is good, but I do not think that these scores should affect the final marks in a huge way.”

Students’ experience of using WebPA

The surveys at each of the institutions involved touched upon – to a greater or lesser extent – the students’ experience of using WebPA.

Students commented favourably upon the ease of using WebPA to record their self and peer assessment. The larger of the Loughborough surveys resulted in an overwhelming majority of respondents agreeing with the statements:

• “It was easy to access WebPA to submit my marks” (91.1%)
• “I felt that WebPA was secure” (90.9%)
• “I was provided with enough information about the assessment within the software” (80.7%)
• “The marking scheme/criterion were easy to understand (80.4%)
• “The WebPA software was easy to navigate” (92.2%)

In addition, only a relatively small proportion of respondents (25.7%) felt they needed to use the ‘“Help’” option within WebPA when completing their assessment.

The majority of students responding to the Loughborough survey (54.3%) remarked that, given the choice, they would like to use WebPA in other modules on their course.

Similarly positive responses were given by those surveyed at the University of Leeds and the University of Hull.

At Hull, the 2 two cohorts of students surveyed after using WebPA commented that it was easy to understand and apply the marking scheme (92% and 97%). Similarly positive responses were provided as to the fairness of the marking scheme applied (84% and 91%). A good positive conclusion from the survey at Hull was that there was “no negative impact on the students’ enjoyment of group work” arising from using WebPA.

At Leeds, students commenting on the use of WebPA to record self and peer assessment noted that it was “easy” to provide assessment of their peers (80%), and that the prompts provided allowed them to comment on all relevant aspects of their peers’ contributions (84%).

General Conclusions

In reviewing the surveys conducted it should be noted that, as different surveys were conducted at each of the three institutions involved in the evaluation exercise, that comparisons are difficult.

Nevertheless, the relatively large sample sizes involved at both Loughborough and Hull, and the commonality and strength of responses received, allow us to draw key conclusions from the data arising with a high degree of confidence in their reliability.

These conclusions are that:

• Students enjoy the opportunity to engage in group work
• Students recognise a number of learning opportunities and benefits arising from engaging in group work
• Students believe that group work helps them to develop key transferable skills such as communication, problem solving, and team work
• Students have a preference for an element of self and peer assessment to be part of the assessment process
• Students do not feel that marks derived from peer assessment should exceed 40% of the total marks allocated for a group work exercise
• Students feel that peer assessment of group work increases their motivation to work well
• Students consider peer assessment to be a more “fair” method of assessment
• Students on the whole believe that peer assessment has a positive impact upon group performance
• Students feel that feedback, anonymity, and fairness are key issues in relation to peer assessment
• Students like WebPA and find it to be straightforward and easy to navigate/use
• Students would like to use WebPA more for peer assessment of group work when appropriate
• There was no negative impact on students’ enjoyment of group work arising from the use of WebPA