VALIDITY &
VALIDITY &
RELIABILITY
RELIABILITY
QUALITIES OF MEASUREMENT
QUALITIES OF MEASUREMENT
DEVICES
DEVICES
ValidityValidityDoes it measure what it is supposed to measure?
Does it measure what it is supposed to measure?
ReliabilityReliability
How representative is the measurement?
How representative is the measurement?
PracticalityPracticality
Is it easy to construct, administer, score and
Is it easy to construct, administer, score and
interpret?
interpret?
BackwashBackwash
What is the impact of the test on the
What is the impact of the test on the
teaching/learning process?
QUALITIES OF MEASUREMENT
DEVICES
In Psychology, we judge the quality and goodness of measuring devices by two
psychometric criteria: reliability & validity
If a predictor is not both reliable and
VALIDITY
The term validity refers to a standard for evaluating tests that refers to the accuracy or appropriateness of drawing inferences from test scores.
It refers to whether or not a test measures what it intends to measure.
On a test with high validity the items will be closely linked to the test’s intended focus.
If a test has poor validity then it does not measure the job-related content and competencies it ought to.
VALIDITY
There are several ways to estimate the validity of a test, including content validity, construct validity,
criterion-related validity (concurrent & predictive) and face validity.
VALIDITY
Content”: related to objectives and their sampling. “Construct”: referring to the theory underlying the
target.
“Criterion”: related to concrete criteria in the real world. It can be concurrent or predictive.
“Concurrent”: correlating high with another measure already validated.
“Predictive”: Capable of anticipating some later measure.
1. CONTENT VALIDITY
Content validity refers to the degree to which subject matter experts agree that the items in a test are a representative sample of the domain of knowledge the test purports to measure.
The test should evaluate only the content related to the field of study in a manner sufficiently representative, relevant, and comprehensible .
2. CONSTRUCT VALIDITY
It refers to the degree to which a test is an accurate and faithful measure of the construct it purports to measure.(concepts, ideas, notions).
Construct validity seeks agreement between a theoretical concept and a specific measuring device or procedure.
For example, a test of intelligence nowadays must include measures of multiple intelligences, rather than just logical-mathematical and linguistic ability measures.
3. CRITERION-RELATED
VALIDITY
The degree to which a test forecasts or is statistically related to a criterion.
Also referred to as
instrumental validity
, it
states that the criteria should be clearly
defined.
There are two major kinds of
CRITERION-RELATED VALIDITY- Concurent & prdictive.4. CONCURRENT VALIDITY
Concurrent validity is used to diagnose the existing status of some criterion.
It is a statistical method using correlation, rather than a logical method.
In measuring concurrent criterion-related validity, we are concerned with how well a predictor can predict a criterion at the same time, or concurrently..
5. PREDICTIVE VALIDITY
This is another statistical approach to validity that estimates the relationship of test scores to an examinee's future performance.
Predictive validity considers the question, "How well does the test predict examinees' future status as masters or non-masters?
5. PREDICTIVE VALIDITY
For this type of validity, the correlation that is computed is based on the test results and the
examinee’s later performance.
This type of validity is especially useful for test purposes such as selection or admissions
.
6. FACE VALIDITY
Face validity is determined by the appearance that items in the test are appropriate for the intended use of the test by the individual who take the test.
Unlike content validity, face validity is not investigated through formal procedures. Instead, anyone who looks over the test, including examinees, may develop an informal opinion as to whether or not the test is measuring what it is supposed to measure.
6. FACE VALIDITY
Face validity alone is insufficient for establishing that the test is measuring what it claims to measure.
RELIABILITY
Reliability is the standard for evaluating tests that refers to the consistency, stability or equivalence of test scores.
It
is the extent to which an experiment, test, or any
measuring procedure shows the same result on
repeated trials.
“Equivalency”: related to the co-occurrence of two items
1. EQUIVALENT-FORM
RELIABILITY
A type of reliability that reveals the equivalence of test scores between two versions or forms of the tests.
Equivalency reliability is the extent to which two items measure identical concepts at an identical level of difficulty.
Equivalency reliability is determined by relating two sets of test scores to one another to highlight the degree of relationship or association..
1. EQUIVALENT-FORM
RELIABILITY
For example, a researcher studying university English students happened to notice that when some students were studying for finals, they got sick. Intrigued by this, the researcher attempted to observe how often, or to what degree, these two behaviors co-occurred throughout the academic year. The researcher used the results of the observations to assess the correlation between “studying throughout the academic year” and “getting sick”. The researcher concluded there was poor equivalency reliability between the two actions. In other words, studying was not a reliable predictor of getting sick.
2. STABILITY RELIABILITY
Stability reliability (sometimes called test, re-test reliability) reveals the stability of test scores upon repeated applications of the test.
To determine stability, a measure or test is repeated on the same subjects at a future date. Results are compared and correlated with the initial test to give a measure of stability.
2. STABILITY RELIABILITY
This method of evaluating reliability is appropriate only if the phenomenon that the test measures is known to be stable over the interval between assessments.
3. INTERNAL CONSISTENCY
Internal consistency is a type of reliability that reveals the homogeneity of the items comprising a test.
It is the extent to which tests or procedures assess the same characteristic, skill or quality.
This type of reliability often helps researchers interpret data and predict the value of scores and the limits of the relationship among variables
.
3. INTERNAL CONSISTENCY
For example, analyzing the internal reliability of the items on a vocabulary quiz will reveal the extent to which the quiz focuses on the examinee’s knowledge of words.
4. INTER-RATER RELIABILITY
Inter-rater reliability reveals the degree of agreement among the assessments of two or more raters.
Inter-rater reliability assesses the consistency of how a measuring system is implemented.
It is dependent upon the ability of two or more individuals to be consistent.
Training, education and monitoring skills can enhance inter-rater reliability.
4. INTER-RATER RELIABILITY
For example, when two or more teachers use a rating scale with which they are rating the students’ oral responses in an interview (1 being most negative, 5 being most positive). If one researcher gives a "1" to a student response, while another researcher gives a "5," obviously the inter-rater reliability would be inconsistent.
4. INTRA-RATER RELIABILITY
Intra-rater reliability is a type of reliability assessment in which the same assessment is completed by the same rater on two or more occasions.
These different ratings are then compared, generally by means of correlation.
Since the same individual is completing both assessments, the rater's subsequent ratings are contaminated by knowledge of earlier ratings.
SOURCES OF ERROR
Examinee (is a human being)
Examiner (is a human being)
