Item Analysis
You are a member of a 3-person educational team that has developed a competency test that uses 50 multiple choice test questions to determine the knowledge and skill level of learners. After 30 learners have completed the competency test, the team conducted an item analysis.
From the results, there are 10 questions that are identified as being problematic. One of the educators wants to consider each of these questions individually to see if one or more of the questions should be removed from the test.
However, a second educator is strongly against this by stating, “This is a competency test; it has validity because we wrote it—the test questions should not be altered. The grade on the examination should stand!” The third educator wants to remove the ten questionable items from the test and raise all learners’ scores.
For the educator in professional nursing development, this will result in staff nurses who have failed the exam being placed on performance improvement plans and possible dismissal from the agency. For the educator in the academic setting, the learner will not be allowed to progress to the next course, must repeat the current course, and may be dismissed from the academic program.
What quantitative and qualitative information would you provide in order to assist these educators to consider the quality of the test questions?