FAQ

Here you can view the frequently asked questions on the WebPA project, if you do not find an answer, please check the Community Mailing List.

Use the below contents list to navigate to a particular section, and click on the relevant question to you.

• Assessment
  • Forms and Criteria
  • Groups
  • Hardware
  • Loading data on staff & students
  • Marking and Reporting
• Software

Assessment

• Can I run more than one assessment for the same assignment?

Yes, you may run as many assessments as you want, as often as you like – even concurrently.

• Can students mark each other as they go along?

No. The students can only take an assessment once. You should schedule your assessment to run when your group activity has finished. students can then submit marks for all the sections/criteria.

• How do I get the student’s scores?

Once an assessment has been completed (i.e. The deadline for submissions has passed) it will appear in the “closed” tab. From there, you can create a mark sheet (It’s the last of the small buttons next to the assessment’s name).
The mark sheet sets the parameters which tweak the scoring algorithm (penalties for non-submission, etc). You can create as many different mark-sheets as you like, but you only need one to access the reports.
Once you’ve added a mark sheet to an assessment you’ll see a “view reports” button next to it in “My Assessments” (just look under the ‘Marked’ tab).
There are a few different reports in there, and you can download them into Excel as well as viewing/printing them. If you haven’t provided the overall group mark, then only some of the reports will be available because the algorithm needs the overall marks to do it’s work properly.

• What should I do if I have identified that one or more students in the group are not contributing to the group effort?

By free-riders, we mean those students who occasionally attend meetings, and say they will do things, but actually do very little. They bluff the supervisor and expect to be included in the marks whilst simply going through the motions. If the student is a good communicator this can be very hard to detect.
The Web-PA system is designed to identify and adjust the marks for such students accordingly. You should, however, remind the whole group that all members have a responsibility for the effective functioning of teamwork and you can moderate the extreme effects of peer marking by altering the proportion of the team mark that is affected by the peer assessment (PA Weighting,). This factor is entered when you create a mark sheet at the end of the assessment.

• What should I do if one or more students have not completed the WebPA form during the assessment period?

Failure to complete causes considerable difficulties and it is worth putting in a little effort to maximise the response rate. We suggest that you send a reminder email to those who have failed to complete a form two days before the closing date: use the in-built email system and select ‘students who have not submitted’. Soon after the assessment is closed, check to see how many students have failed to submit. If there are more than three or four, consider re-opening the assessment for, say 24 hours and sending a second ‘final’ email reminding students that their mark may be adversely affected unless they complete a form.

• What should I do if one or more students in a group has effectively withdrawn from the course yet is still registered on the mod?

This is where a student has failed to attend group meetings, and has taken no part in the group activity at all. Contrast this with free-riders in a later FAQ.
You need to be decisive here. First gather the facts together with any available evidence that the student has failed to participate. If you find out early enough in the assignment for the situation to be remedied then invite the withdrawn student to a face to face interview to discuss. If the assignment has ended, inform the student in writing that he/she will be assessed separately from the team and make a tutor’s judgement on the mark based on actual proven contribution (this may be zero). Remove the student from the Web-PA team and allow the assessment to run.

• Why don’t changes in ‘My Forms’ or ‘My Groups’ affect ‘My Assessments’?

Forms, Groups and Assessments exist as separate entities within the system. When you create an assessment the form you selected, and the collection of groups you chose, are copied into the new assessment. This means any subsequent changes you make to either the forms or the groups will not affect the assessment.
This is entirely by design. Any changes to the criteria being used, or the group composition, during an assessment could throw off the Web-PA algorithm and generate bogus results. To minimise the risk of you accidently changing the form or groups during an assessment, Web-PA simply doesn’t allow it.
If you need to change the criteria being used, or the groups being assessed, you can make the relevant changes in My Forms or My Groups, then edit your assessment to select the new form/collection of groups. You can do this up until the first student submits marks – at that point, the assessment is locked and the choices you have made are locked in.

Forms and Criteria

• Can you give me some examples of appropriate criteria for Peer Review?

The criteria you choose will clearly depend on the nature of the group assignment. It is a good idea to link your criteria in some way with the Intended Learning Outcomes of the module.

For examples of criteria used by other practictioners view example forms. Yes, you may run as many assessments as you want, as often as you like – even concurrently.

• Do I need to complete all the descriptors for the meaning of each numbered response?

No, you can complete as many as you wish, but complete at least the minimum and maximum descriptors. Only completed descriptors will appear on the form.

Groups

• Are there any restrictions on group/team size?

The Web-PA system does not restrict group or team size. However, we do not recommend running assessments for groups with only 2 members. The algorithm appears to provide best results for teams of 4-6, but has been successfully used with teams from 3 to 8.

• How can I create individual group names?

The WebPA team were asked how an academic tutor can give the groups in the collection individual names.

Method 1
There are two ways in which this can be achieved. The first is to create the group collection through the normal process up to step 3 of 5. At this point you should not enter any information, but just proceed to the next step by clicking the next button. Complete the rest of the wizard as normal.

Once the wizard is complete and you have clicked the ‘finish’ button the next screen offers you a number of choices. You need to select the ‘group editor’ option. This will take you to a new screen where you are able to add the groups one by one.

To add the groups one by one, you will need to click on the ‘add groups’ in the ‘Groups’ area of the page. This will take you through to another new screen. Here you will need to type the name of the group in the text box in the New Group area and click the add button. You will then see the group names listed in the area below ‘Available Groups’. To then add the students to the group you will need to return to the group editing page.

If you have not added all the groups you need to the collection at this time, you can always add more in the future via the ‘add/remove groups’ button in the ‘Groups’ area of the edit screen.

Method 2
The second option for providing individual names for the groups is by creating the groups as normal through the wizard. Once you return to the edit groups screen you can then click on each group in turn, or click the pencil button to the left of the group name. Once you have done this, you will be taken to a new screen, which allows you to edit the detail about the group.

At the top of the screen in the ‘Group Name’ area you can edit the name given to the group. Once you have changed the name of the group you will need to click the ‘save changes’ button. You can repeat this for each group that you would like to re-name.

Method 3
The third option is to have the administrator add the group name to the uploading of the student within the admin area of WebPA. You will also need to supply them with a name for the collection of the groups. Once you have been notified that the students have been uploaded with their groups it is worth you check the groups are correct via the my groups area.

• What strategies are available for selecting groups?

  • Randomly selected, e.g. alphabetically by surname
  • Self selected by students. This often result in some students being grouped by gender, race, ethnicity and geographic location (e.g. same house) and ALSO some students being isolated by the same factors.
  • Seeded groups. One student who is a high academic achiever, or has in some way previously demonstrated better than average ability in the type of activity being assessed, is allocated to each group each. The, remaining students are randomly or self selected to fill these remaining group places.
  • Ranked groups. Students are allocated to groups based on academic performers, e.g. all top students go in one group all bottom students in another. This may result in some dysfunctional groups but will provide a wide variance in performance.
  • Tutor selected. Tutor defines parameters and allocates groups based on these parameters (e.g. to get a mix of Belbin roles in every group).

Hardware

• What would you suggest as a baseline hardware configuration?

I will need to get back to you on the requirements for a baseline, I should be able to get this from the Guys at Hull. Here at Loughborough the system runs on a server which is managed for us.

• If a large number of users start to use WebPA here, we would look at having failover capabilities in place just in case something serious happened to the hardware. Do you know of any other WebPA users who use failover or are users sticking to just one server & accepting that the WebPA system will be down should that server fail?

We have a fall over, which is to a back up server which has taken an image over night. For us this is quite important as we have over 6,000 students taking an assessment in one year. Having talked to academic tutors about contingencies when there is a fall over/ failure they are happy to revert back to group marks and explain the situation to the students. Although this may not be ideal in many cases academic tutors are told to have back up plans for such events.

Loading Data on Staff & Students

• How do you recommend this is approached? We could extract data from our SunGard Banner system but what format would you expect it to be in for loading into WebPA?

For the pilot of WebPA we have a system where you can batch load with a csv file through the admin interface. We specify the layout of the data, but we only take the minimal information about users. In the longer term we have left the choice of long term integration to each institution. Following the user group we will be looking at the possibilities of APIs again. Here at Loughborough we hold the user details in a separate database as the information is maintained and held for a number of different applications.

The information required is;

for staff and students

institutional_reference, forename, lastname, email, username,module_code, department_id, course_id, password

and for the module information

module_code, module_title

You may also be interested to know that one of the other institutions using WebPA are looking at developing a power link for Black board/webct vista for the transfer of student information. A basic power link has now been created and is available from the project source forge area.

Marking and Reporting

• How can I avoid Web-PA producing an inappropriately high number of First Class marks for a module?

Web-PA acts on the team marks that you supply balancing the winners with the losers. If you award very high team marks, the end result will be high individual marks. The overall module average remains unaffected by Web-PA and hence in your control, unless you apply a non-completion penalty which will bring the average down.

• How should I apply the mark penalty for failure to complete a form?

Add a mark penalty for non-completion with great care! It can have a very damaging effect on an individual’s mark. And students who fail to complete a form have often already been marked down by their peers.

• How should I apply the peer weighting factor (PA Weighting)?

This depends on the nature of the assignment and its intended learning outcomes. An assignment that is designed to develop teamwork should logically carry marks that accrue from the ‘team’ effort as opposed to that of the individual. A good starting point is 50% team and 50% peer assessed. You should consider increasing this a little where the technical contributions of the individual dominate. A larger PA weighting will give a larger standard deviation in the group marks. It is good practice to advise your students in advance that a proportion of the team marks will be peer assessed.

Software

• Having installed the original software, do you provide patches & new releases? If so, how often are patches & upgrades released? (link to patch documentation)

We provide patches as and when they are identified/provided and fixes – this tends to be immediately after an installation and then when any are found after that. We are trying to provide patch notification thought the JISCmail list and the project website

Upgrades are made when major improvements are made or there have been sufficient patches to warrant a new release. Through the life of the JISC funding we have tried to make a release every six months. However, out of the funding the releases may be more spread out. I have tried to make sure that we have only changed the code and kept things like database changes to the minimum.

• Looking at the item on the WebPA website relating to the another institutions implementation of WebPA, they mention using Apache web server & MySQL for the database. Checking with the technical guys here, they would use IIS web server & SQL Server for the database. Would that cause any issues?

Here at Loughborough we run a flavour of Linux with PHP and MySQL, at Hull they are running Windows with PHP and MySQL. There are some tweaks that need to be made for the email functionality to work, but I think they are covered in the installation guide (currently being up dated). As for SQL Sever, we have never run it on this. There will be some changes needed to work for the SQL queries but we can discuss doing this. Our problem would be that we do not have access to SQL Server to do the development of this, but we are happy to work closely with anyone to get this done, if that is your requirement.