Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 1 von 8
Competency Based eAssessment
This paper is an integrative part of the WP 7.1 of the LearnInfra-project
(http://www.switch.ch/uni/projects/learn_infra/index.html) which is dedicated to competency-oriented assessment at Swiss Higher Education Institutions (HEIs). The eAssessment field is impacted more and more by a competence orientation. Therefore, this work package follows this current trend of focusing on competence orientation especially in connection with educational objectives.
This present paper presents a research perspective (D 7.1.2) and first defines and categorizes the most important concepts and approaches and the reasons for a huger inclusion of the competence orientation. After, the players and some implementation scenarios are described what leads to the conclusion which focuses on the most important trends.
1
Definition of Competency Based eAssessment
1.1 Definitions
First, the most important concepts of Competency Based eAssessment shall be introduced:
Competencies: A cluster of related abilities, commitments, knowledge, and skills that enable a person (or an organization) to act effectively in a job or situation.1 So, competencies refer to knowledge, experience, skills and motivational skills. Thus it can be considered as concrete behaviour, but also as potentials and resources.
There are authors who differentiate between professional competencies (Fachkompetenzen), methodological expertise (Methodenkompetenzen) und social competencies (Sozialkompetenzen).2 Generally, competencies depend from the context and the person and are therefore individual.
eAssessment: The end-to-end electronic assessment processes where ICT is used for the presentation of assessment activity and the recording of responses. This includes the end-to-end assessment process from the perspective of learners, tutors, learning establishments, awarding bodies and regulators, and the general public. Most eAssessment have got fixed structures and point out the reproduction of knowledge. There are no dialogues and other.3
Competency Based Assessment
An assessment process is based on the collection of evidence on which judgements are made concerning progress towards satisfaction of fixed performance criteria.
The competency based assessment of an individual doesn’t take into account the performance of others in the wider group being assessed (as it is the case in norm-referenced assessment), and can be limited to a pass/fail grading (also called mastery/non-mastery).4
Competency Based eAssessment
If we make use of (?) the “e” in order to formulate a definition for competence oriented eAssessment: Competence orientation asks the operationalization of the learning outputs which shall through the eAssessment supported by ICT-working tools.
1 Read more: http://www.businessdictionary.com/definition/competence.html#ixzz2gf1OiYxI, 3.10.13. 2 North, Wissensorientierte Unternehmensführung, p. 152.
3 JISC, e-Assessment Glossary, p. 43. 4 JISC, e-Assessment Glossary, p. 31.
Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 2 von 8 1.2 Categorization
Most authors differentiate between four types of eAssessment: E-testing5, Computer-based assessment (CBA)6, GCSE computer-based tests (CBTs)7 and E-portfolio8. According to the definition, Competency oriented eAssessment are part of the Computer based assessment or can be illustrated in an ePortfolio. The Competency based eAssessment is part of the formative assessment, because it focuses more on the process than on the output. Competence based assessments don’t focus on the simple summative
assessment. The examinee is asked / motivated to show its knowledge in a concrete situation.9
Research work has showed that the biggest challenge for a Competency oriented eAssessment consists of the fact that eAssessment generally focuses on summative outputs, but Competency oriented eAssessment puts a strong focus on the process itself. Finding a link between these two components therefore is the big challenge of eAssessment.
2
Context
Due to the following reasons, Competence oriented eAssessment ha become important:
Throughout the Bologna reform, (summative) assessments have got a more relevant significance and teachers’ correction efforts have augmented. So, eAssessment hasbecome more common at HEI. At the same time, the Bologna reform has brought a deeper focus on competence based efforts, guiding in its consequence to the Swiss formulation of the NQF-competencies. In the European Qualification Framework for Lifelong Learning (EQF-LLL), the descriptors are subdivided into knowledge, hard skills and
competences. In the qualification framework for the Swiss universities (nqf.ch-HS), the descriptors are broken down into knowledge and understanding, the application of knowledge and understanding, judgments, communicative skills and self-learning capabilities.10
In the last years, HEIs have been asked to prepare their students for the labour market, the so called “employability”. In order to prove this employability it has become important to test students’ behaviour / action knowledge.
Also, during job interviews candidates are more and more asked to prove their ability in a concrete professional scenario in order to prove their experience and their knowledge. It can be very helpful for students to share also experiences done for their studies, apart from their professional experiences.
3
Players
Study programmes: Studies show that competence based Assessment is especially important in the vocational education. Here, a lot of research has been done with the focus on learning objectives,
5 E-testing is like traditional testing but is carried out using a computer rather than pen and paper. Cf.
http://www.ocr.org.uk/i-want-to/do/e-assessment/computer-based-assessment/, 7.10.13.
6 Computer-based assessment (CBA) offers an alternative to local assessment and external moderation. Learners work on
assignments as usual, but their files are assessed via computer rather than a local assessor marking print outs and sending them to an OCR Examiner Moderator. Cf. http://www.ocr.org.uk/i-want-to/do/e-assessment/computer-based-assessment/, 7.10.13.
7 Computer-based tests (CBT) are an alternative option to traditional paper-based assessment which allows candidates to complete
electronic tests at a PC, Cf. http://www.ocr.org.uk/i-want-to/do/e-assessment/computer-based-assessment/, 7.10.13.
8 An electronic portfolio, (e-portfolio) is a storage area, where learners upload and submit their work and tutors and assessment
personnel view and mark work. Unlike traditional paper based methods, e-portfolios provide much richer and varied ways of recording and presenting evidence. Learners can submit a range of file formats including word processed documents, spreadsheets, images, video and sound files, http://www.ocr.org.uk/i-want-to/do/e-assessment/e-portfolio/, 7.10.13.
9 Bettina Zurstrassen: Kompetenzorientierte Lehrerbildung in den sozialwissen-schaftlichen Unterrichtsfächern: Blühende
Landschaften in der sozialwissenschaftlichen Lehrerbildung von morgen? In Journal of Educational Sciences, Vol. 8, Nr.2, p.34.
Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 3 von 8 in their curriculum.
But also for HEI players competence orientation has become more important. But nevertheless, it doesn’t make sense for all study programmes. A competence orientation is especially applicable for study
programmes where the student is expected to apply the acquired knowledge to concrete professional scenarios such as Engineering, Medicine, and Educational Sciences etc.12
University: Universities of Applied Sciences which promote their experience based study programmes have a huge interest to focus on competency based assessment. Nevertheless, as employability has become one of the main requirements for a university, all types of HEIs will put their focus on applied sciences that means on concrete professional scenarios in the near future.
4
Implementation
First, teachers should start to formulate learning objectives into/for (?) concrete into competencies. To assess these competencies, cognition, motivation, social skills should be an integrative part of the concrete actions which are going to be assessed by teachers.
In the attachment of this document, you find different scenarios; formulated at Swiss Distance University of Applied Sciences (FFHS) (cf. Annex 1). Here, some of the scenarios show how competencies are valorised by different types. These scenarios are currently adopted according to new technical, organisational and judicial trends will lead to a clarification of eAssessment at FFHS.
Problem based learning can be part of a learning process which allows an active, self-regulated and constructive approach to learning.
5
Outlook / Challenges
The focus remains on formal education (in the sense of hard skills and practical know-how) and
neglectssoft skills. Thus, the challenge in the future will be to establish concrete settings for testing these relevant soft skills.
• The human being becomes more and more “Homo competencies” who is rated according to his
formal knowledge (which diploma does the person have), his practical know-how such (“which project, manual activities does the person have?”) and his shared knowledge (“has the person thaught his knowledge ?)
• The integration of ePortfolisoos as a self-assessment tool support the self reflection and therefore
the competence development of self competences. Teachers and students act as peers and discuss the development of their competences.
• Social Media business platforms such as Linkedin, Xing, academia etc. supports the reflection and
the festhalten, recommendations and should be integrated in the competence evaluation process.
• Formal and informal knowledge prove the competencies of a human being.
11 Barbara Lorig: Kompetenzbasierte Prüfungen im dualen System – Bestandsaufnahme und Gestaltungsperspektiven.
Zwischenbericht. 2012 Bonn.
12 Thomas Prescher, Federick Schulz: „Prozessorientiertes Kompetenz-Entwicklungs-Assessment zur Förderung der
Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 4 von 8
6
Literature
JISC, e-Assessment Glossary, p. 43.
Barbara Lorig: Kompetenzbasierte Prüfungen im dualen System – Bestandsaufnahme und Gestaltungsperspektiven. Zwischenbericht. 2012 Bonn.
Klaus North, Wissensorientierte Unternehmensführung. 2011 Wiesbaden.
Thomas Prescher, Federick Schulz: „Prozessorientiertes Kompetenz-Entwicklungs-Assessment zur
Förderung der Self-Monitoring-Kompetenz im Lehramtsstudium“ in: Hamburger eLMagazin, 2011 Hamburg. Bettina Zurstrassen: Kompetenzorientierte Lehrerbildung in den sozialwissen-schaftlichen
Unterrichtsfächern: Blühende Landschaften in der sozialwissenschaftlichen Lehrerbildung von morgen? In Journal of Educational Sciences, Vol. 8, Nr.2.
Links:
http://www.ocr.org.uk/i-want-to/do/e-assessment/, 7.10.13.
http://www.businessdictionary.com/definition/competence.html#ixzz2gf1OiYxI, 4.10.13.
http://www.ocr.org.uk/i-want-to/do/e-assessment/e-portfolio/, 7.10.13.
Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 5 von 8 Scenario Competency based learning: Case studies
Type Summative
Categorization LMS, specialized assessment, students’ and teachers’ hardware Short description Students work in groups on a case study. The result of their input is a
semester work and a presentation to the teacher and the other students. Through an assessment form, the teacher evaluates the semester work and the presentation and gives the module mark. Detailed description In the module Web Science and Social Media (WSSM), students are
asked to investigate on the social media presence of given company (big national companies such as Post, Swiss etc.). Departing from the results of their investigation, students shall design a social media concept containing mission, vision, objectives and concrete projects. They do this task in groups of 3-4 people. Five days before the f2f-teaching session, they sumbit the concept (in a 20 page doc-format) in an input-forum. During the f2f- teaching session they present their results to their colleagues. Based on an assessment-form, the teacher reviews the concept and the presentation.
PDFs: Modulplan WSSM
Auftrag Studierende für Gruppenarbeit eAssessment-form
Actors Students and teacher (teachers/students) and relation between them Tools Moodle (LMS) : Students’ and teachers’ hardware
Security These types of exercises / assessments allow a huge variety of possible solutions, so that a copy of existing concepts will suddenly be discovered.
As every student has to submit his individual solution, there is no risk of cheating.
Institution Swiss Distance University of Applied Sciences (FFHS) Setting Bachelor of Business Informatics
Analysis Advantages: Students are highly motivated to elaborate a good exercise.
Students apply the learned theory Creativity is demanded
Limitations: Some strategies are online available, so that the teacher
must ensure that they don’t copy the existing strategy. Implementation 2 times (first time in 2012)
Use Scenario sporadically used Group size small
Scenario Working task as admittance for the exam
Type Formative
Categorization LMS, specialized assessment, students’ and teachers’ hardware Short description After every f2f teaching session (5 in total), students get a working task
(Arbeitsauftrag) which they have to submit in an input-forum. Only if they have fulfilled this task, they are allowed to take part in the written exam.
Detailed description In the marketing module, the students have to submit after every f2f-session a short paper according to their working task (Arbeitsauftrag) where they apply the theory of the 3 marketing Ps on a special case. They enter their paper in the entry forum on their LMS moodle and get a feedback from their teacher during the next f2f-session.
Only students who have submitted their tasks can take the module exam.
Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 6 von 8 PDFs: Modulplan Marketing
Actors Teachers as initiator and evaluator of the task / students as executer
Tools Moodle (LMS)
Security As every student has to submit his individual solution, there is no risk of cheating.
Institution Swiss Distance University of Applied Sciences (FFHS) Setting Bachelor of Business Administration
Analysis Advantages: Students repeat the theory after every teaching session. Students get a feedback after every teaching session.
Students apply the theory on a particular case.
Limitations: The admission to the exam depends on the quantity of the
submitted working tasks and not on the quality. So, some students are not motivated to submit a high quality short paper.
Implementation 1
Use Scenario widely used (because there are different marketing-classes) Group size medium
Scenario Screening test
Type formative
Categorization Self-assessment, student, specialized assessment etc.
Short description On the FFHS website, persons who are interested in a study program at a distance Higher Education Institution can fill out an
online-questionnaire in order to find out if they are able to do such a program. Detailed description As the discontinuation rate at a distance Higher Education Institution is high (?), an online self-assessment tool was elaborated and placed on the entry page of the FFHS-website. The potential student has to answer around 51 questions (every of the 17 thematic section contains 3-7 items) about their motivation, personal and working environment, expectations, soft and formal skills etc. After every thematic session the user gets a detailed feedback about his answers, his adaption and important points to consider. At the same time, he gets information (links) about
Link to the self-assessment:
http://www.fernfachhochschule.ch/ffhs/studieren/online-test/online-test
Actors Potential students
Tools Online-questionnaire (on the FFHS-website) Security No cheating (as it is a self-assessment-tool)
Institution Swiss Distance University of Applied Sciences (FFHS)
Setting Pre-study
Analysis Advantages: Students get a feedback about their adaptability for an online-program. So they get aware about the challenges / crucial factors of an online study program and they get an orientation for their final decision for or against an online study program.
Limitation: The tool gives an orientation for the decision making
process. Students shouldn’t take their decision only based on the output of this test; it is only a helping tool.
Implementation Various times
Use Scenario used widely used (the click-rate of this questionnaire is very high)
Group size large
Scenario Scored Online Activities Type formative as well as summative Categorization Student Hardware and LMS usage
Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 7 von 8 collaborate through an LMS (Moodle), where certain elements are
used like forum, wiki, document upload and multiple-choice questions. At the end of the semester, these activities are scored by the teacher where (whether?) this score counts 40% of the overall score.
Detailed description For a better understanding of this scenario, a real example is described here in a bachelor course which contains 24 students and has duration of 1 semester (6 month). The course contains 300 hours of learning, where 10% of it is in f2f presence mode, the rest is eLearning with online-coaching, self-regulated learning and collaborative learning through group-work. There are 20 online-tasks spread over the semester, which have to be done by everybody with a response in the LMS Moodle. The tasks are: providing documented solutions,
commenting on other solutions, start and comment discussions, performing multiple choices and more.
The teacher can locate and follow all individual interactions of the students, he is able to see who takes the lead, who can follow and start discussions, he can follow individual knowledge and problems within the theme itself and has (?)active access to learning materials. The teacher gets an individual picture of every student about learning behaviour, individual interest, thematic problems etc. As f2f meetings are held from time to time (6 times, a 5 lessons), the teacher can prove and reflect this picture for every student within the meetings. After finishing the semester, the teacher will score all necessary activities which influence the total score by a weighting of 40%.
Actors Only teacher and students are involved
Tools Moodle (LMS)
Security Individual teacher/students feedback in f2f inspection-meetings. These feedbacks prove the online activities.
Institution Swiss Distance University of Applied Sciences (FFHS)
Setting Bachelor
Analysis Advantages: It supports the motivation of LMS-online activities of the students, caused by the score relevance. The teacher can follow, steer and comment the educational process of the students during the semester. Also non-active students can be discovered and thus, specific actions can be initiated.
Drawbacks: the online activities have to be proved by f2f meetings, if there are suspicious differences between the provided knowledge of f2f and online activities; an oral test has to prove the actual knowledge of the concerned student.
Even, if a student would be able to cheat all the proving for online activities, he still has to pass a conventional test in order to get a sufficient score. But the advantage of this online activity scoring system is, that the students can bring in their “knowledge over time” and reduce the “only one time proved by conventional test” effect.
Limitations: The amount of students must be “small”, which means less than 25 persons, regarding the limited teachers work time.
Implementation This scenario runs since more than 6 years, so it’s proved more than 12 times. Likewise scenarios (with another scoring of online-activities) are guessed by a number of 20-30 times over the last 6 years. An Example of such an implementation is the bachelor course in “complexity-management with simulation”
Use Scenario regularly used
Group size Group size of assessed students = small (30 students or less)
Overview_CompencyBasedAssessment_v2.doc 25.10.2013 Seite 8 von 8 Scenario Project based online presentation
Type Summative
Categorization Student Hardware, LMS usage
Short description Students work in groups on a self-defined project (project-based learning). An LMS is used for interactions and document-sharing. The teacher can see the progress of the group-works and can interact at any time for help or suggestions. At the end of the semester, the groups have to make a 10- Minute presentation of the work, and after the presentation, they have to answer questions in a discussion with the teacher. The presentation can be made with a video, which they have to upload to the LMS, also the discussion can be made within a forum of the LMS.
Detailed description The self-defined project of the group work has to fulfil certain requirements, given by the teacher. Usually, the students work in a company as their study goes in parallel with their work. So, the students can bring in and work on a real problem from their company. The project-work will be supervised by the teacher andall activities are documented through the LMS. The students can produce and show their final presentation via video-podcast, and they have to upload this onto the LMS. The teacher observes the podcast and then sets up a discussion/questions on the LMA. The students have to enter/answer the discussion. The teacher scores the whole work based on the content, the presentation and the discussion. The scores of these works counts to 60% for the overall score. From time to time, f2f meetings are held (5 times during a semester) in order to keep in touch with the students themselves.
Actors Teacher and students
Tools LMS Moodle, personal audio/video tools from cameras, Webcams or iPad/Tablets.
Security The teacher is involved in the process of the group-work during the whole time. Also, the students have to declare, on which parts they worked. The final discussion with the teacher is also an instrument to eliminate any doubts on the work and/or the students’ participation. Institution Swiss Distance University of Applied Sciences (FFHS)
Setting Bachelor
Analysis Advantage: The students can work on an own problem, which is good for their motivation. They will be scored on this project, so no additional exam has to be made. Also, the students can learn from other groups and their presentations.
Disadvantage: Usually, the scores for the whole group are the same, if
they have similar participation. This participation is not easy to analyse. So, the teacher has to believe, what students declare (but nevertheless the teacher has an instrument to test it: f2f meetings and discussion).
Limitations/Extensions: The group size can also be one person.
Implementation Number of times scenario has been implemented
Use The scenario of scored group work is widely used, but the possibility of doing all things online is rare. One example is the course “Complexity management with simulation”
Group size Group size of assessed students = small (30 students or less)
