A Worked Example of the Sorting Algorithm

• Introduction
• A Worked Example
  • Normalise the Scores Awarded by Each Student
  • Calculate the WebPA Score Awarded to Each Student
  • Calculate the Student’s Final Grade
  • PA Weighting
  • Weightings and Penalties
  • Non-Completion Penalty

Introduction

This document demonstrates a worked example how the WebPA scoring algorithm is used to obtain marks for a student assessment.

This is a very simple example assessment, focussing on a single group, marking each group member against a single question.

For readability, most of the numbers used later, in this example, are rounded to 2 decimal places. The actual WebPA algorithm is more precise and provides a greater accuracy.

A Worked Example

Alice	Bob	Claire	David	Elaine	Total awarded
Alice	4	4	3	2	1	14
Bob	3	5	3	2	0	13
Claire	4	4	4	4	4	20
David	3	5	4	3	1	16
Elaine	–	–	–	–	–	0
Total Received	14	18	14	11	6

The rows across are the marks awarded by each student, whereas the columns down shows the marks each student received. The marks received are totalled at the bottom.

A quick review of the scores shows that Bob performed the best, while Elaine has performed the worst. Also, Elaine didn’t submit any marks for the peer assessment at all, as denoted by the dashes in her row.

Normalise the Scores Awarded by each Student

All the marks each student awarded are added up (total up each row).

Alice awarded: 4 + 4 + 3 + 2 + 1 = 14 marks Bob awarded: 3 + 5 + 3 + 2 + 0 = 13 marks Claire awarded: 4 + 4 + 4 + 4 + 4 = 20 marks David awarded: 3 + 5 + 4 + 3 + 1 = 16 marks Elaine awarded: 0 + 0 + 0 + 0 + 0 = 0 marks

Figure 2 – The marks awarded are added

Once it is known how many marks each student has awarded, the scores that they gave can be received and the fractional score for each student can be calculated. In other words, every student in the group has an identical ‘cake’, but they can give different sized slices to each other (and themselves!).

Alice awarded a total of 14 marks (so her cake was split into 14 slices).

Alice gave herself 4 marks = 4 / 14 = 0.29 Alice gave Bob 4 marks = 4 / 14 = 0.29 Alice gave Claire 3 marks = 3 / 14 = 0.21 Alice gave David 2 marks = 2 / 14 = 0.14 Alice gave Elaine 1 marks = 1 / 14 = 0.07

Figure 3 – How the fractional scores are calculated

These are the fractions awarded by Alice to each student. As a test, all the fractions can be added up and they should equal 1.

This process is repeated for every student for every question. In this example there is only one question so we only need to add up this one question.

Calculate the WebPA Score awarded to each Student

Before each students WebPA score is calculated, there’s one more bit of information that is needed; how many students were in the group, and how many of them submitted marks.

In this case the group had 5 members, which means 5 cakes should be split between 5 members. Unfortunately, Elaine didn’t submit any marks, so only 4 cakes are available!

To compensate, WebPA calculates a multiplication factor to bring the total number of cakes back up to 5. This value is identified as the “fudge factor”.

5 (students in group) / 4 (students submitting) = 1.25 (our fudge factor)

Figure 4 – Calculation of the multiplications factor

The WebPA scores can now be calculated. Taking Alice as an example, we add up all the fractional scores she received. We calculated Alice’s own fractions above, but the other students’ fractions were produced in exactly the same way.

Alice awarded herself 0.29 (as we saw above). Bob awarded Alice 0.23 (= 3 / 13). Claire awarded Alice 0.20 (= 4 / 20). David awarded Alice 0.19 (= 3 / 16). Elaine didn't submit, so she effectively awarded 0.00.

Figure 5 – Fractional scores awarded to the members of the group

If all the individual factors are added up, the WebPA score is calculated. For Alice:

Alice's WebPA score = 0.29 + 0.23 + 0.20 + 0.19 + 0.00 = 0.91

Figure 6 – Alice’s WebPA score calculation

If everyone had submitted the assessments, that would be fine, but remember we had one non-submission, so we have to bring in our fudge factor from earlier.

Alice's actual WebPA score = 0.91 x 1.25 (the fudge factor) = 1.14

Figure 7 – Alice’s actual WebPA score

If the same calculations are done for each of the other students, we find:

Bob's actual WebPA score = 1.18 x 1.25 = 1.47 Claire's actual WebPA score = 0.89 x 1.25 = 1.11 David's actual WebPA score = 0.68 x 1.25 = 0.85 Elaine's actual WebPA score = 0.33 x 1.25 = 0.41

Figure 8 – Each of the groups WebPA score

As a quick check, adding up all the WebPA scores for every student, it should equal the number of students in the group. Adding up the scores gives 4.98, which allowing for rounding errors (we’re using 2 decimal places in this example) is 5, the total number of students assigned to the group.

Calculate the Student’s Final Grade

Now the WebPA score have been worked out, it is easily to calculate a student’s final grade using the overall group mark.

Let’s assume we’ve awarded this group’s work 80%. Ordinarily, this may have given everyone in the group 80%, but now using the WebPA scores to give either proportionally higher or lower marks according to the students actual performance. This is one of the clear benefits of using WebPA.

Alice's Grade = 1.14 x 80 = 91.2% Bob's Grade = 1.47 x 80 = 117.6% = 100% (we don't give grades above 100%) Claire's Grade = 1.11 x 80 = 88.8% David's Grade = 0.85 x 80 = 68.8% Elaine's Grade = 0.41 x 80 = 32.8%

Figure 9 – Calculating the students final grades

It is clear that three of the students did better than the group mark would have suggested, but two students were worse off. According to the peer assessment, that’s a fair result.

PA Weighting

The first parameter that can be changed is the PA Weighting. This defines how much of the overall group-mark should be fixed (given to all the students automatically), and how much should be peer-assessed (calculated using the algorithm above).

If instead of peer-assessing the entire group mark it is possible to use 50% instead, then all 5 students would automatically receive the other 50% of the group mark. In this case, with an overall group mark of 80%, that’s an automatic 40% fixed, and the remaining 40% will be adjusted by the peer assessment. The students final grades would then become:

Alice's Grade = 40 + (1.14 x 40) = 85.6% Bob's Grade = 40 + (1.47 x 40) = 98.8% Claire's Grade = 40 + (1.11 x 40) = 84.4% David's Grade = 40 + (0.85 x 40) = 74% Elaine's Grade = 40 + (0.41 x 40) = 56.4%

Figure 10 – WebPA weighted marks

The spread of grades has been reduced substantially in Figure 12. Bob’s previous grade was 117.6% but this has been brought down to 98.8%. Similarly, Elaine’s low score of 32.8% has been brought up to 56.8%.

Weightings and Penalties

There are other factors that can come into play which affect the overall scores received by each student. The calculation above assumes that the entire group mark should be adjusted using the peer assessment. This can lead to large spread of marks. Fortunately, WebPA allows the algorithm to be adjusted, using a couple of extra parameters, controlling the spread of marks.

Non-Completion Penalty

The second parameter is the non-completion penalty. For students who didn’t take the peer assessment, a penalty of up 100% can be awarded to the final grade.

As an example, awarding a non-completion penalty of -10%, Elaine, who failed to take the assessment, would receive this penalty:

Elaine's Grade = weighted WebPA mark – non-completion penalty

Elaine's Grade = 56.4 - 10% = 50.76%

Figure 11 – Non-completion penalty calculation