NSS All questions 2015
5.3 Content Analysis of Verbatim Comments
5.3.2 Assessment and Feedback.
Word Length Count Weighted
Percentage (%)
Feedback 8 118 3.05
Course 6 88 2.27 Coursework 10 76 1.96 Staff 5 50 1.29 Tutors 6 49 1.27 Interesting 11 40 1.03 Assignments 11 34 0.88 Modules 7 34 0.88 Teaching 8 34 0.88 Skills 6 33 0.85 Helpful 7 32 0.83 Experience 10 29 0.75 Marking 7 29 0.75 Industry 8 28 0.72
Table 15 - Word Frequency Assessment and Feedback and Positive Comments.
From the data the following information can be extracted around the key areas
Word Count Weighted Percentage %
Coursework, assignments, 110 2.84
Feedback, marking 147 3.80
Lecturers, tutors, staff, 201 5.19
Course, modules, 122 3.15
Teaching, skills, interesting, experience, industry, helpful
225 5.06
Table 16 - Word Frequency Results: Assessment and Feedback and Positive Comments.
The data is suggesting that the students on the four programmes are reporting very different experiences in relation to ‘Assessment and Feedback’ despite a clearly defined policy and procedure within the School regarding the management of assessment and feedback. The quantitative data indicates that the ‘Assessment and Feedback’ category
has lower levels of satisfaction than other categories across all the courses within this case study.
As highlighted in figure 34, particular issues are evident for the Building Surveying and Quantity Surveying students around ‘Assessment arrangements and marking have been fair’ and ‘Feedback on my work has been prompt’. Additionally, students are reporting low levels of satisfaction relating to ‘Feedback on my work has helped me clarify things I did not understand’. The data indicates that the lecturers themselves are the most important aspect in delivering a positive perception of the student experience relating to assessment, closely followed by the quality of the feedback and the quality of the marking. The actual assessment tasks as represented by the coursework brief are shown to be important as are the clarity of the marking criteria. The quality of the feedback in terms of a clarifying ‘things the student did not understand’ appears to be an important factor in how they perceive the whole assessment experience. The course and/or module content also plays a significant role with the skills of the tutors in making the subject area interesting and relating the assessment to industry having an influence on the perceptions of quality and the student experience. This is an issue identified via the verbatim comments rather than the survey statements as the statements do not directly refer to the influence of the staff on the process, how the assessment aligns with the module content and how it aligns with industry practice.
Word Length Count Weighted Percentage (%)
Feedback 8 185 4.14 Coursework 10 119 2.66 Lecturers 9 82 1.83 Marking 7 56 1.25 Modules 7 48 1.07 Assignments 11 45 1.01 Lectures 8 42 0.94 Course 6 41 0.92 Library 7 40 0.89
Group 5 38 0.85 Students 8 37 0.83 Tutors 6 37 0.83 Enough 6 35 0.78 Sometimes 9 30 0.67 Staff 5 29 0.65
Table 17 - Word Frequency Assessment and Feedback and Negative Comments
From the data the following information can be extracted around the key areas
Word Count Weighted Percentage %
Coursework, assignments, 164 3.67
Feedback, marking 241 5.39
Lecturers, tutors, staff, 148 3.31 Course, modules, lectures,
library
171 3.82
Group, Students 75 1.68
Enough, sometimes 65 1.45
Table 18 - Word Frequency Results Assessment and Feedback and Negative Comments.
Again, from the initial review of the data it appears that a larger range of factors impact on the student reporting a negative student experience than on a positive experience. The data indicates that the feedback and marking are the most significant factors in delivering a negative perception of the student experience relating to the assessment, closely followed by a range of additional factors including the coursework/assignments themselves, the actual structure and content and the lecturing staff. The actual assessment tasks as represented by the coursework brief are shown to be important in terms of clarity for the students and the support they receive. The course and/or module content also plays a significant role with the skills of the tutors in making the subject area interesting and relating the assessment to industry having an influence on the perceptions of quality and the student experience. The planning and scheduling of the assessment impacts on the student view in terms of a positive or negative perception of
the actual assessment and feedback. The verbatim comments also indicate that the volume of assessment impacts on the student experience in a positive or negative way. Interestingly, the students link issues such as ‘Academic Support’ with other specific areas such as ‘Assessment and Feedback’ and ‘Teaching on my Course’ within the verbatim comments in a manner that widens how they process their experience and indicates students interpret their experience outside the narrow confirms of the survey.
Further analysis of the comments has been undertaken to establish the context of the results of the Word Frequency analysis. The main issues highlighted by the analysis are identified below;
Word Positive Influencer Negative Influencer
Good • Good Feedback
• Good coursework
• Feedback not good
Coursework, assignments, work • Good coursework, interesting
• Too much coursework • Poor timing
• Very poor coursework • Unequally planned • Poorly managed • Poor criteria • Unclear Feedback • Timely comments
• Returned on time • Relevant
• Helpful, very good
• Unfair marking • Very poor quality • Slow/Late feedback • Generally useless • Unclear
• Vague and inconsistent • Lacking in detail Lecturers, tutors, staff, • Certain tutors are excellent
• Good standard of knowledge
• Lecturers do not give feedback
• Lecturers not accessible • Poor communication skills Course, modules, • Demanding but enjoyable
• Course runs smoothly
• Modules not specific • Irrelevant subject material
Teaching, skills, interesting • Some interesting coursework
• Relevance to industry and career
• Many cancelled classes • Tutors poor communication
skills
• Tutors cannot communicate properly
• Lack of industry relevance Table 19 - Word Frequency Assessment and Feedback Contextualised Results.
Examples of the results of the text search of the most frequently identified words are shown below. The results of the text search are illustrated using a word tree. Further search of the raw data was undertaken in order to contextualise the comments from the students.
5.3.3 Learning Resources
Learning resources within the context of the NSS applies to the provision of such support facilities such as the library, the use of the university Virtual Learning Environment (VLE), the general estate facilities such as the quality of the classrooms, study space and computing facilities. The provision of these facilities is generally controlled and maintained centrally by the university although students do not typically distinguish between those facilities controlled on a School basis and those provided by the university centrally.
Word Length Count Weighted Percentage (%)
Library 7 51 10.92 Resources 9 23 4.93 Books 5 21 4.50 Facilities 10 18 3.85 Available 9 16 3.43 Blackboard 10 14 3.00 Enough 6 12 2.57
Lecture 7 10 2.14 Difficult 9 5 1.07 Computer 8 4 0.86 Great 5 4 0.86 Rooms 5 4 0.86 Theatres 8 4 0.86 Construction 12 3 0.64 Course 6 3 0.64
Table 20 - Word Frequency: Learning Resources.
From the data the following information can be extracted around the key areas
Table 21 - Word Frequency Results: Learning Resources.
The data indicates that the library is the most important aspect in the perception of the student experience relating to the learning resources. The interesting evidence from the initial data is the response referring to the computing facilities although the use of Blackboard VLE is important to the students. The general facilities including the university estate in terms of lecture rooms are a consideration but more analysis is required to establish if it is the facilities or how they are used.
Further analysis of the comments has been undertaken to establish the context of the results of the Word Frequency analysis. The main issues highlighted by the analysis are identified below;
Word Count Weighted Percentage %
Library, Books, available, enough 100 21.42
Computer 4 0.86
Resources, facilities 41 8.78 Theatres, rooms, lecture 18 3.86
Blackboard 14 3.00
Word Positive Influencer Negative Influencer Library, Books, available,
enough
• Open 24 hours • Lack of books • Out of date material • Poor range of books
• Available books rarely available
• ‘Awful’
Computer • Good availability of computers
• Good facilities
• Availability of computers with specialist software
Resources, facilities • Overall resources very good • Good facilities
• Poor resources
Theatres, rooms, lecture • Crowded classrooms
Blackboard • Excellent when staff use it • Notes and lecture slides
available
• Some information not available or after the lecture
Course, construction, great, difficult
• • Lack of construction
materials
• Difficult to find books • Lack of books specified on
the course
Table 22 - Word Frequency Learning Resources: Contextualised Results.
5.3.4 Academic Support
The quantitative data for the ‘Academic Support’ category is again highlighting a very mixed experience as reported by the students within the school. As highlighted in Table 23, the data shows students on the Architectural Design Technology course are generally satisfied with the academic support they receive while the Building Surveying and Quantity Surveying students are less satisfied with the perceived quality of advice available to them related to their studies and the availability of staff.
The data is suggesting that the students on the four programmes are reporting very different experiences in relation to ‘Academic Support’ despite a clearly defined structure within the School for providing support to students at module level, at each level of study and via the Programme Director. The data is useful in highlighting the areas that the students are less positive about within the category of ‘Academic Support’ and raises some questions regarding the expectations of the students on each programme.
Word Length Count Weighted Percentage (%)
Lecturers 9 49 5.84 Staff 5 35 4.17 Tutors 6 35 4.17 Helpful 7 24 2.86 Support 7 20 2.38 Students 8 14 1.67 Always 6 12 1.43 Available 9 11 1.31 Communication 13 11 1.31 Course 6 11 1.31 Advice 6 9 1.07 Emails 6 8 0.95 Excellent 9 8 0.95 Friendly 8 8 0.95 Teachers 8 8 0.95
Table 23 - Word Frequency Academic Support.
From the data the following information can be extracted around the key areas;
Word Count Weighted Percentage %
Lecturers, staff, tutors, teachers 127 15.13 Helpful, support, always, available,
friendly, Excellent
83 9.05
Communication, advice, emails 28 3.33
Course 11 1.31
Students 14 1.67
The data indicates that the lecturers themselves are the most significant factor relating to the student experience relating to academic support as may be expected. All the comments indicating a positive experience relate to staff that appear to be providing excellent support in a friendly, enthusiastic manner and are accessible to those students seeks support. The negative influencers within the verbatim comments are centred on academic staff that are not accessible, have poor language skills or do not provide the support the students expect. The availability of staff via face-to face contact and by email plus the quality of that interaction with staff is shown to be important to the students and is a key influencer in a positive or negative manner.
Further analysis of the comments has been undertaken to establish the context of the results of the Word Frequency analysis. The main issues highlighted by the analysis are identified below;
Word Positive Influencer Negative Influencer Lecturers, staff, tutors, teachers • Knowledgeable, Friendly
staff.
• Provide support. • Accessible staff • Enthusiastic
• Poor English Language and communication skills • Difficult to contact • Provide little support • Research students
Helpful, support, always, available, friendly, Excellent
• Some helpful lecturers • Friendly staff
• Some staff excellent • Staff are willing to support
• Some staff not very helpful • Staff not available
• Not supportive • No support
Communication, advice, emails • Usually respond to emails • Usually quick to respond to
• Provide useful advice
• No support – no/slow response to emails
• Staff reluctant to respond to emails
Course • Course leaders listens to students
• Lots of advice available regarding the course
• Course leaders never available
Students • Good support for students • Excellent support for
students
• No real help for students • Lecturers do not check
which students are on support plans
Table 25 - Word Frequency Academic Support: Contextualised Results.
5.3.5 Organisation and Management
The ‘Organisation and Management’ category is again highlighting a very mixed experience as reported by the students in both the quantitative data and within the verbatim comments. Architectural Design Technology students are reporting higher levels of satisfaction when compared with the three other degree programmes. As highlighted in Table 26, the data shows students on the Building Surveying and Quantity Surveying programmes are less satisfied with the perceived quality of timetabling and how any changes are communicated. The area which is resulting in the least satisfaction is related to the statement ‘The course is well organised and is running smoothly” with the Construction Project Management students indicating only 47% of the students undertaking the survey agreeing with this statement. Again it is apparent from the analysis of the verbatim comments that the students are concerned with the quality of the teaching rooms, the proximity to each other of the space they occupy for lectures and the size of the class in terms of the number of students within the room. There is also some evidence from the analysis that students link the timetable, the class size and environment with the amount of support they receive with assessments.
Word Length Count Weighted Percentage (%)
Timetable 9 26 3.95 Course 6 20 3.03 Rooms 5 14 2.12 Lectures 8 13 1.97 Timetables 10 12 1.82 Lecture 7 11 1.67
Changes 7 9 1.37 Class 5 9 1.37 Organised 9 9 1.37 Students 8 9 1.37 Organisation 12 8 1.21 Module 6 7 1.06 Sizes 5 7 1.06 Timetabling 11 7 1.06 Coursework 10 6 0.91
Table 26 - Word Frequency Organisation and Management,
From the data the following information can be extracted around the key areas
Word Count Weighted Percentage %
Timetable, timetabling, timetables, changes
54 8.2
Rooms, sizes 21 3.18
Lectures, lecture, class 33 5.01 Course, module, coursework 33 5.0 Organised, Organisation 17 2.58
Students 9 1.37
Table 27 - Word Frequency Results: Organisation and Management.
The analysis of the verbatim comments indicates that the timetables are the most important aspect in the students’ perception of how their programme of study is organised and managed. Student comments indicate that some students consider their timetable to be well organised and that the school has responded to there concerns while others are very dissatisfied complaining of ‘Too many gaps in the timetable’, ‘Rooms all over the campus’ and ‘Rooms unsuitable for the class’.
The lectures and where the lectures take place are shown to be significant for many students and are clearly linked in the mind of the students with the structure of their degree programme, the modules and the procedure relating to the coursework.
Further analysis of the comments has been undertaken to establish the context of the results of the Word Frequency analysis. The main issues highlighted by the analysis are identified below;
Word Positive Influencer Negative Influencer Timetable, timetabling,
timetables, changes
• Easy timetable
• Staff listened to student concerns regarding the timetable
• Timetable arrangements have been good
• Layout has been good
• Too many gaps in the timetable
• Poor timetable
• Rooms all over the campus • Timetable is ‘a bit stuffed’. • Not well structured • Disjointed with large gaps • Long day – 9 – 5.
Rooms, sizes, lecture, class • Allocation of rooms disorganised
• Rooms unsuitable for the class
• Rooms too small • All over the campus
• Changed without notification
• People sitting on the floor.
Course, module, coursework • Good support with coursework
• Good module tutors
• Good support from module tutors
• Course poorly organised • No support with coursework
in some modules
Organised, Organisation • Poorly organised
Students • Staff always willing to help students
Table 28 - Word Frequency Organisation and Management: Contextualised Results.
5.3.6 Personal Development
The results of the analysis of the ‘Personal Development’ category of questions has produced a range of results for the four programmes in the School and a significant range of responses for the questions within the category (Table 29). The results show that for Q19. – ‘The course has helped me to present myself with confidence’ all four courses are reporting satisfaction levels in a range between 88% and 66% compared with the University of Salford average of 83%. However, the data shows a clear distinction between the four programmes with the Architectural Design Technology course reporting generally high levels of satisfaction across all statements in the category with the other programme reporting significantly lower levels of satisfaction.
Word Length Count Weighted Percentage (%)
Skills 6 29 4.11 Course 6 24 3.40 Industry 8 14 1.98 Confidence 10 10 1.42 Improved 8 10 1.42 Placement 9 9 1.27 Construction 12 8 1.13 Helped 6 8 1.13 Placements 10 8 1.13 Career 6 7 0.99 Communication 13 7 0.99 Experience 10 7 0.99 Working 7 7 0.99 Environment 11 6 0.85 Group 5 6 0.85
Table 29 - Word Frequency: Personal Development
Word Count Weighted Percentage % Skills, communication, confidence, 46 7.52
Industry, Placement, Placements, experience, career, working, construction,
60 8.48
Course 24 3.40
Improved, helped, environment, group 30 4.25
Table 30 - Word Frequency Results: Personal Development.
The analysis of the verbatim comments shows that students who provided comments state that the course has provided them with opportunities to develop their presentation skills and skills related to their chosen profession. The largest volume of comments relate to a positive experience for the students in developing communication skills, the ability to develop team working skills and the confidence to use these skills to improve their career prospects. The data indicates that the link to the construction industry in the form of placements, employability and general career awareness is high on the agenda for students in terms of skills development. However, for some students there is a perceived lack of this link with industry and this is shown to be a significant negative influencer for the students particularly relating to the construction-based professions. The programme of study and the perception of the module content and how this relates to the students expectations of their career development once working in the construction industry is shown to be a significant factor. The development of soft/transferrable skills is also important to the students. There is some concern evident from the verbatim comments that the students do not feel prepared to undertake a professional role within the construction industry immediately following completion of their studies.
Further analysis of the comments has been undertaken to establish the context of the results of the Word Frequency analysis. The main issues highlighted by the analysis are identified below;
Word Positive Influencer Negative Influencer Skills, communication,
confidence,
• Good skills development in group situations
• Improved presentation skills
• Gradual improvement in communication and analytical skills • Meeting and presentation skills • Team building and team working
skills
• Improved my confidence
• Content is geared towards developing skills
• Development of professional skills • Gained new skills
• Gained transferrable skills Industry, Placement,
Placements,
experience, career, working, construction,
• Placements allow invaluable exposure to industry
• Placement Fairs
• Enhances employment opportunities
• Placement opportunities available • Presentations provide insight into
Construction Industry
• Good opportunity for work placement
• More information regarding work placement
• Placements should be part of the course.
Course • The course has improved my skills • Course has stretched me
intellectually
• Taught me new skills
• Gives you good employability • Represents industry standards and
develops skills
• Gained a lot of confidence since starting the course.
• More industry involvement in course
• Course not directly linked to industry
• Feel totally unprepared for career
5.3.7 Overall Satisfaction
The data indicates that the programme of study and its perceived relevance to the student’s future career is the most significant factor for the student when reporting their perception of the quality of experience within the overall satisfaction section of the NSS survey. The lecturers themselves are also an important aspect in delivering a positive perception of the student experience.
Word Length Count Weighted Percentage (%)
Course 6 55 7.20 Knowledge 9 13 1.70 Interesting 11 12 1.57 University 10 12 1.57 Industry 8 11 1.44 Lecturers 9 10 1.31 Construction 12 9 1.18 Experience 10 9 1.18 Modules 7 9 1.18 Content 7 8 1.05 Overall 7 7 0.92 Relevant 8 7 0.92 Salford 7 7 0.92 Students 8 7 0.92 Working 7 6 0.79
Table 32 - Word Frequency Overall Satisfaction.
From the data the following information can be extracted around the key areas
Word Count Weighted Percentage %
Course, modules, knowledge, content, interesting, relevant
114 13.62
Lecturers, students 17 2.23 University, Salford 19 2.49 Industry, construction, working 26 3.41 Experience, overall 16 2.1