1
Creating Online Evaluations
Follow these steps to set up your module evaluations once you have generated an .xml file using the MEG (see EvaSys User Guide 1: Using the Module Evaluation data Generator for instructions).
Web addresses to save:
EvaSys: https://ncl.evasys.co.uk/evasys The Module Evaluation webpage:
http://www.ncl.ac.uk/quilt/resources/monitoring/module.htm. This contains links to the Policy on Surveying and Responding to Student Opinion, the Module Evaluation question structures and sets, the full set of EvaSys User Guides, and hints and examples of good practice to help ensure smooth running of the evaluation process and promote good response rates.
To log in to EvaSys for the first time:
1. Go to https://ncl.evasys.co.uk/evasys, and log in using your standard user id and 123456 as your password.
Important note: If your machine has popup blockers set up you may see an error message the first time that you try to log in to EvaSys. If this happens, click on the bar at the top of the window and select the option to ‘always allow popups from this site’. Make sure that you do this before you attempt to log in again – after 3 failed attempts to log in your account will be temporarily suspended.
2. To change your password select ‘System Settings’ from the top menu, and then ‘My Profile’ from the left hand menu. Please change your password to something different from your standard university one.
Import the data in to EvaSys:
1. Select the ‘Subunits’ heading from the top menu.
2. From ‘Central Evaluation’ menu on the left of the screen, select ‘Data import’. 3. In ‘load XML file’, click browse to find the file you created in the MEG.
4. Click ‘import’ and a large table of data will be generated.
5. Select the correct period from the dropdown box at the top, e.g. Semester 1 14/15 (it doesn’t matter whether you’re doing mid or end of module evaluations - just select the right semester). It is essential to select a semester: if you leave the survey period as the default ‘Repetitive’ your surveys will be regenerated each semester using the same data.
2
Note: The table will show a series of question sets for each module. The codes relate to the following question sets:
Compulsory questions for standard modules:
MODCORE1 Core questions
MOD-TEACH Staff specific questions (one set for each member of staff teaching on the module)
MOD-OPEN Summary questions
Alternative question structures:
STUDY-CORE Independent study questions FIELD-CORE Fieldtrip questions
CLINIC-CORE Clinical case based problem solving questions DIST-CORE Distance learning questions
DESIGN-CORE Design, Studio and Performance module questions
Optional questions:these will only appear if you’ve selected ‘yes’ in the data generator
MOD-SEM Seminar questions
MOD-PRAC Practical questions
MOD-GROUP Group/Team work questions
SUB-xxxx School/Subject Questions (where xxxx is your school’s abbreviation)
MOD-SUPER Supervision focussed questions 6. Click on ‘import’.
Amending/adding to evaluation data in EvaSys
Before moving on to the ‘Generate Evaluations’ stage below, check whether you need to amend or add to the evaluation data you have imported into EvaSys. You will need to do this if any of the following apply to the evaluations you are setting up. The relevant user guides are all available to download from
http://www.ncl.ac.uk/quilt/resources/monitoring/module.htm. If you need to do two or more of these steps you are advised to complete them in the order shown here.
A. You are setting up an evaluation for a module with a long title (more than 75 characters)
If you are setting up an evaluation for a module with a title that is more than 75 characters long you must manually amend the title of the evaluation in EvaSys. If you fail to do this there will be technical problems with subsequent years’ surveys, which will overwrite
existing data and corrupt the module evaluation results that are stored in the system. To do this follow the instructions in User Guide 17: Amending the Title of an Evaluation.
3 B. You are using module specific questions
If you wish to add module specific questions to an evaluation you need to do that at this stage. To do this follow the instructions User Guide 11: Using the Module Specific Questions Tool.
C. You need to add an external staff member
If your school uses external staff members who are not registered on the SAP system to teach modules their details will not automatically pull through when you produce an .xml file in the MEG. If you wish to include Staff Specific questions about members of external staff you can manually add their details to the relevant evaluation in EvaSys, by following the instructions in User Guide 4: Manually Adding an External Staff Member.
D. You need to manually add participant emails
If your school runs CPD modules attended by professionals you can choose to send the emails from the system to their professional email address rather than their Newcastle University address. To do this follow the instructions in User Guide 5: Manually Adding Participant Emails.
Generate Evaluations
1. In the Subunits area, select ‘Generate Survey’ from the Central Evaluation menu on the left hand side of the screen.
2. The ‘Generate Surveys’ screen will open. Tick the ‘Switch to Module Surveys’ box at the top of the screen.
3. Box 1 (Subunit) should show your School. Box 2 (Survey Period) should show the semester and year – if more than one shows, select the appropriate one.
4. Box 3 (Module) shows every module for your school and selected survey period where data has been uploaded into the system, but the survey hasn’t yet been created for it. You can create a single evaluation, or to save time if you will be running several
evaluations with the same dates you can create them as a batch. Select the module(s) you require by highlighting them in the Module box (you can hold down CTRL and click with the mouse to select individual modules, or drag your mouse down the list to select a block of modules).
5. In the ‘Online Evaluations’ section at the bottom of the screen select ‘PSWD based’, and tick ‘Use Time control’.
6. Click on ‘Generate Surveys’.
Note: If you opted to include any of the school owned questions (School Specific questions, Subject Specific questions and School Staff Specific questions) when you generated the xml file from the MEG, but have not set them up in EvaSys, you will not be able to generate the evaluations. To set these question types up so that you can add them to an evaluation follow the instructions in User Guide 3: Creating School, Subject or Staff Specific Questions, available to download from http://www.ncl.ac.uk/quilt/resources/monitoring/module.htm. Alternatively if you have opted to include school owned questions in error you will need to
4
delete the data you have uploaded, then create a new xml file and upload it again. See the Avoiding and correcting common errors box at the end of this document for instructions on deleting the data.
7. A confirmation message will appear at the top of the screen confirming the number of surveys that have been added. Below that you will see the Scheduled tasks, which determine the date and time that each stage in the module evaluation process takes place.
8. The first scheduled tasks is the ‘PSWD to Respondents’ task: this sends the email out with the link to the evaluation to all the students on the module.
a. Press the ‘Activate’ button to make the task available for editing, then set the date and time when you want the task to take place by clicking on the calendar icon. b. The text for this email has been written centrally and approved by the EvaSys
Implementation Steering Group and the Students Union. Please don’t delete any of the text, although you can add information to it if required.
c. After you have completed each task you can either press ‘Save’ to apply the settings to the evaluation you are currently working on, or if you are setting up a batch of evaluations you can press ‘Apply to all’ to add these settings to all of the evaluations in the batch. A warning message will appear asking you to confirm that you wish to apply the settings to all of the surveys in the batch; press OK to confirm.
9. To move on to the next task press the ‘Edit’ button in the second tab, ‘Online Survey Reminder’. This task sends reminder emails to the students who haven’t yet completed the evaluation.
a. Press ‘Activate’, and set the date and time when the first reminder will be sent using the calendar function as above.
b. You can choose to send multiple reminders by clicking on the radio button and selecting the reminder frequency.
c. Edit the text of the email to include a signature (practice varies between different Schools; for example, some use the DELT, DPD, or School Manager). Note that changes you make to the email text will only apply to the individual evaluation you have open, even if you have selected the option to set the tasks for all of the surveys in a batch. See below for instructions on editing the reminder emails for other surveys in the batch.
d. Press ‘Save’ or ‘Apply to All’, depending whether you are setting the reminder frequency for one survey or a batch of surveys.
10.The next task is the Response rate notification: this is an optional feature, which sends out an automated email to the module leader if responses haven’t reached a certain level by a certain date. If you decide to use this feature press the ‘Edit’ button in the third tab.
a. Press ‘Activate’, and set the date and time when the Response Rate Notification email will be sent out by using the calendar function as above.
5
b. In the ‘Notification at a response rate below’ box set the rate below which you want the email to be sent. The default is 50%, and you can change this by typing in the text box. The email will only be sent if the response rate is below the level you set by the date and time you have selected.
c. Add recipients from the ‘List of Recipients’ section. The default ‘Instructor of the course’ will go to the module leader and other teaching contributors for each module; if you wish you can add other recipients, e.g. the DPD or a member of admin staff – use the drop down box to find them from the list.
d. Once you’ve selected the recipient you want to use press the ‘Add’ button – once they appear in the box below the dropdown they have been added.
e. Press ‘Save’ or ‘Apply to All’, depending whether you are setting the reminder frequency for one survey or a batch of surveys.
11.The final task is the Finish Survey: this will set the date and time when the survey will close; after this the students will not be able to access the evaluation. Press the ‘Edit’ button in the forth tab.
a. Press ‘Active’, and set the date and time when the survey will close using the calendar function as above.
b. You can choose whether to use the ‘Automatic report dispatch’ function, which will email the results reports automatically to the module leader and other teaching contributors as soon as the survey closes. If you wish to use this option tick the ‘Automatic report dispatch’ box.
c. Press ‘Save’ or ‘Apply to All’, depending whether you are setting the reminder frequency for one survey or a batch of surveys.
Sending out results reports
The Module Leader will be sent a single report with all answers and information generated by module evaluation.
Other teaching contributors will be sent separate reports with information generated from core questions, school/subject specific questions if used, and staff specific questions about themselves.
Results will not be sent automatically to admin staff, DPDs or Heads of Academic unit. As an EvaSys admin user you are able to download results reports from EvaSys for distribution to these members of staff if required. There are instructions for doing this in User Guide 8: Viewing and Downloading Evaluation Results, available from
http://www.ncl.ac.uk/quilt/resources/monitoring/module.htm.
Finally, amend email text for automated reminder emails:
If you have set up the scheduled tasks for a batch of evaluations together, you will need to amend the email signature for the Online Survey Reminders for each module evaluation you have created. To do this:
1. Still in the ‘Scheduled Tasks’ screen, click on the ‘Next survey’ button at the top right. This will scroll to the next survey in the batch.
6
3. The Online survey reminder window will open, with the email text showing at the bottom of the screen. At the bottom of the email text insert the correct person’s name and job title as the email signature. Click Save.
4. To save time, before you move to the next survey you can copy the signature of the email, ready to paste into the other reminders that you need to amend.
5. Repeat steps 1-3 for each evaluation in the batch.
Avoiding and correcting common errors
Provided that the data in MOFS is correct, and you selected the correct question sets in the ‘Evaluation Parameters’ section in the MEG, your surveys should not contain any errors. However, if you do discover any errors in the evaluation data in EvaSys it is essential that you delete out the information you have uploaded, before correcting the error and generating a new xml file to upload in its place. To do this:
1. Click on your subunit, and find the module that you need to delete the data for. If the module is listed more than once take care to select the one for the correct evaluation period – this is given at the end of the evaluation name, e.g. (End Sem 1 13/14). The date in the ‘Created’ column shows when the data was imported into EvaSys, so this can also help you identify the correct evaluation to be deleted. Note that it is NOT possible to recover any evaluation data that is deleted in error.
2. Click on the red cross in the ‘Actions’ column at the end of the row of information for the relevant module.
3. A warning message will appear that says ‘User to delete:’ followed by the name of the module leader. Provided that you have clicked on the cross in the correct line of information this will only delete the data for the evaluation, and not the user, so if you are confident of this go ahead and click ‘ok’.
