Supplementary Information About Self and Peer Assessment


Supplementary Information About Self and Peer Assessment


Introduction

This document has been developed originally by Jan Tennant (Professional Development, Loughborough University) and subsequently Adam Crawford (engCETL, Loughborough) as supplementary information to the professional development ‘self and peer assessment’ course. The information has been adapted and supplemented to support the web-pa peer assessment system.


The rationale for using self and peer assessment methods within individual modules, and as part of the overall assessment strategy for programs of study, have been well documented. It is possible to find a vast amount of information about why both self and peer assessment used, some of these sources of information can be found in the appendices.


Appendices are attached to provide exemplars of Assessment Criteria, and briefings provided to students.



Measure Group Work

Group work is often assessed on more than one level. Some of the most common are as follows:



Why Self and Peer Assessment?

In general the reasons for why self and peer assessment are used is related to the meanings of the terms. Generally the assessment forms are under stood to be;


Self assessment being about … making judgments about the standard of the students own work.


Peer assessment being about … making judgments about the standard of other students work.



Benefits for Students

The direct benefits to student are often over looked when considering the use of peer and self assessment. These benefits are normally grouped into four categories and seldom expanded. To try and assist in understanding these benefits the main ones are listed below, these include:



Benefits for Staff

There are also benefits for staff in incorporating an increased variety of assessment methods into their assessment practice. The benefits noted below should be read in relation to self/peer assessment though are not exclusive to this assessment method.


Benefits include:



Some Risks…

As with any form of assessment that is used there are associated risks. The main areas of risks identified in research are covered in the following sections below.



Reactions of Colleagues and External Examiners

Self and peer assessment challenges well-established beliefs about who should rightfully assess students work and the respective roles/responsibilities of lecturers and students. The methods can be contentious and invite rigorous debate about the maintenance of standards.


As noted by Jarvis et al (1998), traditional views presume that the student (as the recipient of knowledge imparted by the lecturer) cannot be the right person to judge whether knowledge has been correctly learnt since they are not “experts” and therefore lack mastery of the content. If, by contrast, the student is viewed as an active participant in a learning process where learning is constructed by the learner rather than simply received, with students playing a role in both what they learn and deciding whether they have learnt it, then self and peer assessment becomes a legitimate activity for students.


Prior to implementation, it is advisable to involve colleagues and external examiners in discussions about the introduction of self/peer assessment and to explore issues related to design, implementation and review.



Demands on Time

Time is needed in setting up the self/ peer assessment processes. Areas to be addressed include:



Robustness of Assessment

Like all assessment methods, if self and peer assessment methods are to have credibility then it is crucial that issues of validity, reliability and explicitness are addressed (see Section of the Code of Practice on Assessment – QAA).
http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/section6/



Reliability

The degree to which the assessment procedure produces consistent results. Reliability is about striving for inter- and intra- assessment reliability ie the assessment should be both independent of the assessor involved and of when/where the assessment is carried out. Well-articulated assessment criteria are most important in promoting reliability. Having analysed numerous studies of self assessment, Boud and Falchikov (1995) conclude that there is no general tendency for students to either over-estimate or under-estimate their own performance, although both have occurred in different studies.



Validity

The degree to which the assessment procedure actually measures what it is supposed to measure. Important to determine whether self/peer assessment supports the evidencing of achievement of the stated module/program learning outcomes.



Explicitness

What is required should be clearly defined and transparent to all parties. Assessment criteria must be explicit – this is particularly important in self/peer assessment.



Student Response

Students may feel anxious when embarking on self and/or peer assessment. These concerns may reflect such things as:


Such concerns should not be minimised and opportunities must be made for these anxieties to be aired and considered.



Assessment Criteria

The need for, and importance of, clearly defined assessment criteria when using self/peer assessment cannot be overstated.


The more students are involved in generating the criteria the greater their understanding of the meaning of each criterion. This is likely to assist in increasing both the validity of the assessment and its overall reliability.


Opportunities for students to develop and practice their assessment skills proves an effective means of improving reliability, as does student involvement in the setting of criteria (Toohey 1999).


An example of an approach whereby students are actively involved in the generation of assessment criteria is outlined below:


Example assessment criteria can be found in the appendices of this document.



Follow up Reading

Boud, D.J. (1995) Enhancing Learning through Self Assessment, London: Kogan Page


Hinett, K. and Thomas, J. (1999) Staff Guide to Self and Peer Assessment, Oxford: The Oxford Centre for Staff and Learning Development, Oxford Brookes University


Hounsell, D., McCulloch, M. and Scott, M. (eds) (1996) The ASSHE Inventory: Changing Assessment Practices in Scottish Higher Education, Sheffield: The Centre for Teaching, Learning and Assessment University of Edinburgh and Napier University, in association with The Universities’ and Colleges Staff Development Agency


Jarvis, P., Holford, J. and Griffin, C. (1998) The Theory and Practice of Learning, London: Kogan Page


Knight, P, (ed) (1995) Assessment for Learning in Higher Education, London: Kogan Page


LTSN Engineering el al. (2002) LTSN Engineering Working Group Report: Assessment of individuals in Teams, LTSN Engineering


Race, P., (2001) The Learning and Teaching Support Network Generic Centre Assessment Series No9: A briefing on self, peer and group assessment, York: LTSN


Russell, M. Haritos, G. Combes, A., (2006) Individual student scores using blind and holistic peer assessment, Engineering Education vol.1 issue.1


Toohey, S. (1999) Designing Courses for HE, Buckingham: The Society for Research into Higher Education and Open University Press


QAA (2000) Code of practice for the assurance of academic quality and standards in higher education. Section 6: Assessment of students
http://www.qaa.ac.uk/academicinfrastructure/codeOfPractice/section6/



Twelve Tips on Self and Peer Assessment

Adapted from the UCoSDA Project (1997)



Appendix 1 – Example Assessment Criteria

Based on questions developed at Loughborough University


Assessment Title: Personal Effectiveness


Question Co-operation

Description This covers attendance at meetings, contribution to meetings, carrying out of designated tasks, dealing with problems

Range 0-5

Score 0 no help at all

1 quite poor

2 not as good as most of the group

3 about average for the group

4 better than most of the group

5 really excellent

Question Communication

Description This covers effectiveness in meetings, clarity of work submitted to the group, negotiation with the group, communication between meetings and providing feedback.

Range 0-5

Score 0 no help at all

1 quite poor

2 not as good as most of the group

3 about average for the group

4 better than most of the group

5 really excellent

Question Enthusiasm

Description This covers motivation, creativity and initiative during the project

Range 0-5

Score 0 no help at all

1 quite poor

2 not as good as most of the group

3 about average for the group

4 better than most of the group

5 really excellent

Question Organisation

Description This covers skills in self-organisation and the ability to organise others. It also covers planning, setting targets, establishing ground rules and keeping to deadlines

Range 0-5

Score 0 no help at all

1 quite poor

2 not as good as most of the group

3 about average for the group

4 better than most of the group

5 really excellent

Question Contribution

Description This covers the overall effort put in by an individual during the Semester.

Range 0-5

Score 0 no help at all

1 quite poor

2 not as good as most of the group

3 about average for the group

4 better than most of the group

5 really excellent



Appendix 2 – Student Briefing Document

Based on questions developed at Loughborough University


Assessment Title: Personal Effectiveness


(Student Briefing Document example provided by J. Glass, Loughborough University.

NOTE: Used with WebPA but not in accordance with the default algorithm for scoring.)

Building Design Project

Peer assessment component (15%): instructions

In line with Department of Civil and Building Engineering guidelines on peer assessment for group coursework, this module will use the Web-PA system. This is an online tool which allows you to ‘score’ your fellow group members on their performance. To access the website and take a tour, please visit:

[WebPA URL]

You have been asked to give each of your fellow group members a mark from 0 to 5 for each of five categories (see overleaf); there will be short explanations to help you do this. Although you should consider your assessments carefully beforehand, the whole process should take you only a few minutes once on line. Any problems, please email me.

How to use Web-PA

Step 1: Between [time] today (dd/mm/yy) and midnight Friday (dd/mm/yy), make time to use the internet to go to the following link: [WebPA URL]

Step 2:

Select ‘student login’ and enter your ‘normal’ University log-in and password.

If you have not used Web-PA before, then it’s probably a good idea to click on ‘take a tour’, which explains all the steps very clearly.

Step 3:

Select [assessment name] from the ‘Open assessments’ menu and then select your group number (you should see your name/’yourself’ and your colleagues).

Step 4:

Once you are familiar with the system, click on ‘Take assessment’ and begin entering scores for yourself and each of your group members.

For each category, you can enter scores from 0-5, for example:

0 ‘Never took the lead’

.

.

.

5 ‘Often lead and did so effectively’

When you have entered all the scores, click on ‘Save marks’ and you are all done.

A few words on the scoring process

• Giving someone a ZERO score for a particular category affects only that category and it will NOT be multiplied through (see equation below). Thus it will not wipe out all of someone’s score!

• Giving someone a score of 5 means that you cannot think of anything whatsoever that they could have done in addition; i.e. they made a full and meaningful contribution in that particular category.

• Remember to give yourself a score as well as your colleagues!

Do not collude or confer when you are doing the scoring; it undermines the process. If there appear to be discrepancies, I will meet with the group and/or individual concerned BEFORE ANY GRADES ARE ISSUED TO THAT GROUP.

How the results are analysed

For information, each person receives a score from 0-3% for each of the five categories below. This percentage is based on the average taken from everyone’s scoring. So for example, if you give yourself a 4 out of 5 and your other three colleagues give you the following marks (2, 3 and 0), then your final score would be:

Sum of the scores (4+2+3+0)/ Number of people (4) = 2.25

This is divided by the maximum possible score (5) x 3% = 1.35%

Reminder: mark scheme

The peer scoring in this module affects coursework items, to a value of 15%.

Peer assessment

Marks will be given for:

• Timekeeping – prompt to arrive and communicate (3)

• Brainstorming – active in providing ideas and suggestions (3)

• Taking the lead – putting yourself forward and leading (3)

• Supporting – helping other group members (3)

• Being positive – being flexible, open to ideas etc (3)

This component is worth 15%, applied to course work.

Feedback of results

Each person’s final grade for the whole module will be issued during Week 12, when all the peer scores have been completed.

Note that you will be given grades NOT percentages; final marks will not be issued until the end of the year when the Programme Boards have taken place.

Individual/group support

If you have any queries at all, please email [tutor name] on [tutor email] , call [tel] or drop into [room no] (during office hours).