Supplementary Information About Self and Peer Assessment

• Introduction
• Measure Group Work
• Why Self and Peer Assessment?
• Benefits for Students
• Benefits for Staff
• Some Risks…
  • Reactions of Colleagues and External Examiners
  • Demands on Time
  • Robustness of Assessment
  • Reliability
  • Validity
  • Explicitness
• Student Response
• Assessment Criteria
• Follow up Reading
• Twelve Tips on Self and Peer Assessment
• Appendix 1 – Example Assessment Criteria
• Appendix 2 – Student Briefing Document

Introduction

This document has been developed originally by Jan Tennant (Professional Development, Loughborough University) and subsequently Adam Crawford (engCETL, Loughborough) as supplementary information to the professional development ‘self and peer assessment’ course. The information has been adapted and supplemented to support the web-pa peer assessment system.

The rationale for using self and peer assessment methods within individual modules, and as part of the overall assessment strategy for programs of study, have been well documented. It is possible to find a vast amount of information about why both self and peer assessment used, some of these sources of information can be found in the appendices.

Appendices are attached to provide exemplars of Assessment Criteria, and briefings provided to students.

Measure Group Work

Group work is often assessed on more than one level. Some of the most common are as follows:

• Group Process:
  • How successfully has the group worked together?
  • How effectively the students have managed their work?
• Group Product:
  • What is the quality of the final outcome (the presentation or report)
• Individual Contribution:
  • Students may be asked to submit a report focusing on their contribution to the group activity as a whole.
• Self and peer Assessment:
  • Sometimes students are asked to assess their own performance and/ or the contributions of others, this may or may not contribute to the final group mark

Why Self and Peer Assessment?

In general the reasons for why self and peer assessment are used is related to the meanings of the terms. Generally the assessment forms are under stood to be;

Self assessment being about … making judgments about the standard of the students own work.

Peer assessment being about … making judgments about the standard of other students work.

Benefits for Students

The direct benefits to student are often over looked when considering the use of peer and self assessment. These benefits are normally grouped into four categories and seldom expanded. To try and assist in understanding these benefits the main ones are listed below, these include:

• Improving student learning by:
  • Increased ownership of learning
  • Promoting “deep” learning based on understanding and reflection
  • Developing students’ independent learning skills
• Supporting generic skill development in a number of areas e.g.:
  • Team working
  • Communication
  • Interpersonal
  • Organisational
• Enhancing motivation and enthusiasm by:
  • Encouraging active participation in the learning process
  • Making assessment a shared, collaborative activity
• Increasing the level of feedback from:
  • Improved understanding of assessment requirements
  • Strengthening future performance

Benefits for Staff

There are also benefits for staff in incorporating an increased variety of assessment methods into their assessment practice. The benefits noted below should be read in relation to self/peer assessment though are not exclusive to this assessment method.

Benefits include:

• Improving professional practice in relation to assessment

The range of assessment methods used in higher education has traditionally been narrow. There is still a high dependence on the use of unseen/time-constrained written examinations and tutor marked essays/reports. Such methods are not necessarily best suited to measure certain kinds of learning outcomes. Additionally, any assessment method if used excessively may disadvantage some candidates. It is important to draw on a broad range of assessment methods to provide the maximum opportunity for all students to demonstrate their full potential.

• Increasing opportunities to monitor student progress and identify potential problems.

Self and peer assessment provides an increased basis for dialogue and reflection on progress between students and lecturers

• Strengthening assessment criteria and marking schemes

Successful self and peer assessment is critically dependent on sharply defined assessment criteria and well-developed marking schemes. The utilisation of such assessment methods invites a re-visiting of how achievements are to be evidenced and judged.

• Promoting student-centered learning

Self and peer assessment necessitates that assessment is part of the learning process and that students are active participants in their learning.

Some Risks…

As with any form of assessment that is used there are associated risks. The main areas of risks identified in research are covered in the following sections below.

Reactions of Colleagues and External Examiners

Self and peer assessment challenges well-established beliefs about who should rightfully assess students work and the respective roles/responsibilities of lecturers and students. The methods can be contentious and invite rigorous debate about the maintenance of standards.

As noted by Jarvis et al (1998), traditional views presume that the student (as the recipient of knowledge imparted by the lecturer) cannot be the right person to judge whether knowledge has been correctly learnt since they are not “experts” and therefore lack mastery of the content. If, by contrast, the student is viewed as an active participant in a learning process where learning is constructed by the learner rather than simply received, with students playing a role in both what they learn and deciding whether they have learnt it, then self and peer assessment becomes a legitimate activity for students.

Prior to implementation, it is advisable to involve colleagues and external examiners in discussions about the introduction of self/peer assessment and to explore issues related to design, implementation and review.

Demands on Time

Time is needed in setting up the self/ peer assessment processes. Areas to be addressed include:

• Assessment briefings and supporting papers
• Developing skills in students to undertake what may be new and threatening roles
• Tracking and monitoring progress
• Constructing assessment criteria and ensuring a good appreciation of their meaning
• Debriefing following completion of the assessment process

Robustness of Assessment

Like all assessment methods, if self and peer assessment methods are to have credibility then it is crucial that issues of validity, reliability and explicitness are addressed (see Section of the Code of Practice on Assessment – QAA).
https://dera.ioe.ac.uk/id/eprint/9713/2/COP_AOS.pdf

Reliability

The degree to which the assessment procedure produces consistent results. Reliability is about striving for inter- and intra- assessment reliability ie the assessment should be both independent of the assessor involved and of when/where the assessment is carried out. Well-articulated assessment criteria are most important in promoting reliability. Having analysed numerous studies of self assessment, Boud and Falchikov (1995) conclude that there is no general tendency for students to either over-estimate or under-estimate their own performance, although both have occurred in different studies.

Validity

The degree to which the assessment procedure actually measures what it is supposed to measure. Important to determine whether self/peer assessment supports the evidencing of achievement of the stated module/program learning outcomes.

Explicitness

What is required should be clearly defined and transparent to all parties. Assessment criteria must be explicit – this is particularly important in self/peer assessment.

Student Response

Students may feel anxious when embarking on self and/or peer assessment. These concerns may reflect such things as:

• Deep-seated beliefs about where the responsibility for assessment lies i.e. with the lecturer not students
• Fears about the impartiality of students when undertaking the assessment of other students
• Worries about the impact on relationships with other students where assessment outcomes may not have been favourable

Such concerns should not be minimised and opportunities must be made for these anxieties to be aired and considered.

Assessment Criteria

The need for, and importance of, clearly defined assessment criteria when using self/peer assessment cannot be overstated.

The more students are involved in generating the criteria the greater their understanding of the meaning of each criterion. This is likely to assist in increasing both the validity of the assessment and its overall reliability.

Opportunities for students to develop and practice their assessment skills proves an effective means of improving reliability, as does student involvement in the setting of criteria (Toohey 1999).

An example of an approach whereby students are actively involved in the generation of assessment criteria is outlined below:

• Generating Assessment Criteria with Students
• Each student to list independently at least six things they would expect of a good seminar presentation/report/essay etc
• In groups students to discuss these criteria with a view to drawing up a composite list. Criteria to be prioritized and recorded
• Lecturer to write down most important criteria from each group, then next most important criteria and so on. Combine criteria as appropriate
• Agree criteria as a whole group – amending initial list as required
• Number agreed criteria
• Discuss and agree distribution of marks/grades
• Agree format for proforma

Example assessment criteria can be found in the appendices of this document.

Follow up Reading

Boud, D.J. (1995) Enhancing Learning through Self Assessment, London: Kogan Page

Hinett, K. and Thomas, J. (1999) Staff Guide to Self and Peer Assessment, Oxford: The Oxford Centre for Staff and Learning Development, Oxford Brookes University

Hounsell, D., McCulloch, M. and Scott, M. (eds) (1996) The ASSHE Inventory: Changing Assessment Practices in Scottish Higher Education, Sheffield: The Centre for Teaching, Learning and Assessment University of Edinburgh and Napier University, in association with The Universities’ and Colleges Staff Development Agency

Jarvis, P., Holford, J. and Griffin, C. (1998) The Theory and Practice of Learning, London: Kogan Page

Knight, P, (ed) (1995) Assessment for Learning in Higher Education, London: Kogan Page

LTSN Engineering el al. (2002) LTSN Engineering Working Group Report: Assessment of individuals in Teams, LTSN Engineering

Race, P., (2001) The Learning and Teaching Support Network Generic Centre Assessment Series No9: A briefing on self, peer and group assessment, York: LTSN

Russell, M. Haritos, G. Combes, A., (2006) Individual student scores using blind and holistic peer assessment, Engineering Education vol.1 issue.1

Toohey, S. (1999) Designing Courses for HE, Buckingham: The Society for Research into Higher Education and Open University Press

QAA (2000) Code of practice for the assurance of academic quality and standards in higher education. Section 6: Assessment of students
https://dera.ioe.ac.uk/id/eprint/9713/2/COP_AOS.pdf

Twelve Tips on Self and Peer Assessment

Adapted from the UCoSDA Project (1997)

• Start small: Don’t be too ambitious at first
• Be relevant: Make sure what you are doing supports the learning outcomes
• Convince: Make sure the students know what they are doing is useful
• Clarify: Let the students know what is expected of them
• Criteria: Be sure the students understand the assessment criteria
• Reflect: Think the process through careful
• Moderation: Address issues of reliability
• Rehearse: Provide opportunities for the students to try things out
• Liaise: Don’t try and reinvent the wheel, access good practice
• Be flexible: There are no hard and fast rules, adapt as you go along
• Innovate: Each event as new but recycle the useful elements
• Persist: Don’t let problems discourage you

Appendix 1 – Example Assessment Criteria

Based on questions developed at Loughborough University

Assessment Title: Personal Effectiveness

Question Co-operation
Description This covers attendance at meetings, contribution to meetings, carrying out of designated tasks, dealing with problems
Range 0-5
Score 0 no help at all
1 quite poor
2 not as good as most of the group
3 about average for the group
4 better than most of the group
5 really excellent

Question Communication
Description This covers effectiveness in meetings, clarity of work submitted to the group, negotiation with the group, communication between meetings and providing feedback.
Range 0-5
Score 0 no help at all
1 quite poor
2 not as good as most of the group
3 about average for the group
4 better than most of the group
5 really excellent

Question Enthusiasm
Description This covers motivation, creativity and initiative during the project
Range 0-5
Score 0 no help at all
1 quite poor
2 not as good as most of the group
3 about average for the group
4 better than most of the group
5 really excellent

Question Organisation
Description This covers skills in self-organisation and the ability to organise others. It also covers planning, setting targets, establishing ground rules and keeping to deadlines
Range 0-5
Score 0 no help at all
1 quite poor
2 not as good as most of the group
3 about average for the group
4 better than most of the group
5 really excellent

Question Contribution
Description This covers the overall effort put in by an individual during the Semester.
Range 0-5
Score 0 no help at all
1 quite poor
2 not as good as most of the group
3 about average for the group
4 better than most of the group
5 really excellent

Appendix 2 – Student Briefing Document

Based on questions developed at Loughborough University

Assessment Title: Personal Effectiveness

(Student Briefing Document example provided by J. Glass, Loughborough University.
NOTE: Used with WebPA but not in accordance with the default algorithm for scoring.)

Building Design Project

Peer assessment component (15%): instructions
In line with Department of Civil and Building Engineering guidelines on peer assessment for group coursework, this module will use the Web-PA system. This is an online tool which allows you to ‘score’ your fellow group members on their performance. To access the website and take a tour, please visit:

[WebPA URL]

You have been asked to give each of your fellow group members a mark from 0 to 5 for each of five categories (see overleaf); there will be short explanations to help you do this. Although you should consider your assessments carefully beforehand, the whole process should take you only a few minutes once on line. Any problems, please email me.

How to use Web-PA

Step 1: Between [time] today (dd/mm/yy) and midnight Friday (dd/mm/yy), make time to use the internet to go to the following link: [WebPA URL]

Step 2:
Select ‘student login’ and enter your ‘normal’ University log-in and password.

If you have not used Web-PA before, then it’s probably a good idea to click on ‘take a tour’, which explains all the steps very clearly.

Step 3:
Select [assessment name] from the ‘Open assessments’ menu and then select your group number (you should see your name/’yourself’ and your colleagues).

Step 4:
Once you are familiar with the system, click on ‘Take assessment’ and begin entering scores for yourself and each of your group members.

For each category, you can enter scores from 0-5, for example:

0 ‘Never took the lead’
.
.
.
5 ‘Often lead and did so effectively’

When you have entered all the scores, click on ‘Save marks’ and you are all done.
A few words on the scoring process

• Giving someone a ZERO score for a particular category affects only that category and it will NOT be multiplied through (see equation below). Thus it will not wipe out all of someone’s score!
• Giving someone a score of 5 means that you cannot think of anything whatsoever that they could have done in addition; i.e. they made a full and meaningful contribution in that particular category.
• Remember to give yourself a score as well as your colleagues!

Do not collude or confer when you are doing the scoring; it undermines the process. If there appear to be discrepancies, I will meet with the group and/or individual concerned BEFORE ANY GRADES ARE ISSUED TO THAT GROUP.

How the results are analysed
For information, each person receives a score from 0-3% for each of the five categories below. This percentage is based on the average taken from everyone’s scoring. So for example, if you give yourself a 4 out of 5 and your other three colleagues give you the following marks (2, 3 and 0), then your final score would be:

Sum of the scores (4+2+3+0)/ Number of people (4) = 2.25
This is divided by the maximum possible score (5) x 3% = 1.35%

Reminder: mark scheme
The peer scoring in this module affects coursework items, to a value of 15%.

Peer assessment

Marks will be given for:
• Timekeeping – prompt to arrive and communicate (3)
• Brainstorming – active in providing ideas and suggestions (3)
• Taking the lead – putting yourself forward and leading (3)
• Supporting – helping other group members (3)
• Being positive – being flexible, open to ideas etc (3)

This component is worth 15%, applied to course work.

Feedback of results
Each person’s final grade for the whole module will be issued during Week 12, when all the peer scores have been completed.

Note that you will be given grades NOT percentages; final marks will not be issued until the end of the year when the Programme Boards have taken place.

Individual/group support
If you have any queries at all, please email [tutor name] on [tutor email] , call [tel] or drop into [room no] (during office hours).