What Is Reliability And Its Correlation Coefficient (r) In Psychometry?

What do you understand by the word ‘ Reliable‘? the dictionary meaning of it is Trustworthy, Dependable, stable , etc. If you have any meaning of it, you are right! Congratulations!! In Psychometry or Psychological testing, Ratability as noun has a little different meaning, lets understand that.

In Psychometry, Reliability means the Consistency of scores of the same person on same test on different occasions. Reliability refers to the accuracy, dependability, consistency, or repeatability of test results.

It  refers to the degree to which test scores are free of measurement errors. Psychological Tests that are relatively free of measurement error are considered to be reliable, and tests that contain relatively large measurement error are considered unreliable.

In psychological testing, the word ‘error‘ does not imply a mistake. Though it has a negative connotation, error implies that there will always be some inaccuracy in our measurements. Our task is to find the magnitude of such errors and to develop ways to minimize them.

History of Reliability in Psychometry

Charles Spearman was the early psychometrician who gave contribution to the development of reliability assessment. He gave ‘Classical test theory‘. Spearman worked out most of the basics of contemporary reliability theory and published a article entitled “The Proof and Measurement of Association between Two Things.’’

In 1733, Abraham De Moivre introduced the basic notion of ‘sampling error ‘.

Karl Pearson (1896) developed the ‘Product Moment Correlation.’

What is Test reliability?

Test reliability Indicate; How much individual differences in test score causable to ‘True’ differences  on the characteristics And how much they are causable to change errors.

What is Random Error?

▫Obtained score(Xi) = true score(T i) + random error(E i).

▫Random error of 2 individual can’t be same.

What is Error Variance?

•Total variance= true variance  + error variance

Error variance –Any condition that is irrelevant to the purpose/

•Error variance for one purpose would be classified under true variance for another.

For Illustration –

• Purpose 1 –measuring fluctuation of mood . ▫True variance – day to day mood changes in score on a test of cheerfulness- depression.
  • Purpose 2 – measuring permanent personality characteristics . ▫Error variance –day to day mood fluctuations.

So, To reduce error variance and to bring more reliability in test  scores . The examiner try to maintain uniform condition –testing environment, instructions, time limit, rapport, etc.

What is Correlation Coefficant (r)?

Reliability  i.e. consistency of score or agreement between two scores expressed in term of correlation coefficient(r). Correlation Coefficient (r) is mathematical index that describes the direction & magnitude of a relationship. It express the degree of correspondence or relationship, between two sets of scores.

Correlation Coefficient range form -1 to 0 to +1.

• Positive Correlation (0.00 to +1.00) •means that high scores on Y are associated with high scores on X, and low scores on Y correspond to low scores on X.
• Negative correlation (-1.00 to 0.00) -means higher scores on Y are associated with lower scores on X, and lower scores on Y are associated with higher scores on X. •Ex – the higher the drug dose, the less active the patients are.
• No correlation (0.00) •The  variables are not related. •Here, scores on X do not give us information about scores on Y. •Ex – correlation between shoe size and IQ.

How to Calculate Correlation Coefficient (r)?

There are two ways to compute/ Calculate correlation coefficient (r).

1. Pearson product moment correlation -To find out followings
   1. Persons position in group .
   2. Amount of his/her deviation above(+) or below(-) the group mean like SS
   3. Correlation between two continuous variables such as height, weight, and intelligence.
2. Spearman’s rho / The rho coefficient (r) ▫for finding
   1. the association between two sets of ranks.
   2. used when the individuals in a sample can be ranked on two variables but their actual scores are not known or have a normal distribution.

Testing the Statistical Significance of a Correlation Coefficient

• If we have r=0.40 means moderate positive degree between X n Y for that sample.
• But we want to apply this (r=0.4) to population .
• for that we need significance level.
• We check its Significance level from tables (significance of correlation in any stat book) depending up on sample size .
• Significant at the 1%(0.01) level= means chances are no grater than 1 out of 100 that the population correlation is zero. Hence two variables are truly correlated .
• Significant at the 5%(0.05) level= means chances are no grater than 5 out of 100 that the population correlation is zero. Hence two variables are truly correlated.
• Significance level- risk of error we are willing to take in drawing conclusions from out data.

What is Reliability coefficient?

• it is a application of correlation coefficient(r).
• Reliability coefficient provide test users with some information concerning the magnitude of the error that is likely to enter into scores from various resources.
• It the correlation of the test with itself.
• It is based on data such as two administrations of the same test, two versions of the same test, inter item correlations and so on, that ought to be highly consistent.
• In most of the cases it is considered as higher the coefficient, better the reliability.
• Approximate minimal satisfactory coefficient is considered to be .70 and/ or higher that that by many researchers.
• The reliability coefficient is the ratio of the variance of the true scores on a test to the variance of the observed scores.

Factors Influencing the Reliability of Test Scores 

Both extrinsic (outside the test) and intrinsic (within the test) factors affect the reliability of test scores.

1.    Extrinsic Factors – These are factors external to the test itself.

1.  a) Group Variability – When a group of test-takers is homogeneous (very similar in ability), the reliability of the test scores is likely to be lowered, and vice-versa.
2.  b) Guessing and Chance Errors – Guessing on a test increases error variance and reduces reliability. For example, in a True/False test, there is a 50% chance of answering an item correctly by guessing.
3.  c) Environmental Conditions –  The testing environment should be as uniform as possible. Factors like lighting, sound, and comfort should be equal for all test-takers; otherwise, reliability is affected.
4. d) Momentary Fluctuations –  Temporary incidents can raise or lower an individual’s score. Examples include a broken pencil, a sudden distracting noise, test anxiety, or making a careless mistake with no way to correct it.

2. Intrinsic Factors –These are factors inherent to the test itself.

1.   a) Length of the Test  –  Reliability has a definite relation with the length of the test. The more items the test contains, the greater its reliability will be, and vice-versa.
2.   b) Homogeneity of Items – This refers to how consistently items measure the same trait. If items measure different functions and the correlations between them are zero or very low, then the reliability will also be zero or very low, and vice-versa.
3.  c) Difficulty Value of Items –  Items with a moderate difficulty level (around 0.5) generally yield higher reliability than items that are either very easy or very difficult.
4.   d) Discriminative Value – When items can effectively discriminate between high and low performers (shown by high item-total correlation), the reliability of the test is also likely to be high, and vice-versa.
5.   e) Scorer Reliability – Also known as reader reliability, this refers to how closely two or more scorers agree when scoring the same set of responses. Reliability is lowered if scorers do not agree.