Week Seven Assignment: Item Analysis Assignment Guidelines & Rubric
Purpose
The purpose of this assignment is to help students apply information regarding item analysis. Students are provided an item analysis as well as test questions. Upon considering the item analysis information, each students makes a decision to either retain the item or revise it. Students revise the item while demonstrating best practices regarding test item construction.
Due Date: Sunday 11:59 p.m. MT at the end of Week 7
Total Points Possible: 175
Requirements:
Description of the assignment:
A printout of an item analysis for a 45-item test on HIV/AIDs, along with the corresponding test questions is found in Course Resources. This assignment has two elements. First the overall reliability of the examination is considered and then the item analysis of specific questions. Within the item analysis document, there are nine highlighted test questions (Questions 1 – 8 and 45). These highlighted test questions are the items to be individually analyze. The analysis of selected test questions consists of two major elements:
Using item analysis numerical results
Appling best practices regarding item construction
Using the Item Analysis Template (found in Course Resources), the student, first, examines the overall reliability of the test and make suggestions for improvement. Following the required review of overall test reliability, the student will examine the required test items (Questions 1 – 8 and 45); individually. The analysis includes consideration of numerical item analysis results AND if the selected test question demonstrated best practices regarding item construction. The student will then provide a decision whether he/she would retain or revise the item. If the student’s decision is that the item should be revised, the student identifies suggestion (s) to improve the item as well as re-writing the test question to be consistent with base practices of item construction. For this assignment the desired difficulty index is .70 – .80 for most questions. The desired discrimination index value is .30 or greater. There are no mastery items on this examination. A cover page and running head are required. If a reference is used, you must provide a reference list.
Criteria for content:
Review of the reliability of the entire examination: In this section the MSN student reviews the reliability results of the entire examination. It includes the following required elements:
Explain what is meant by test reliability
Identify the name of the statistical test used in the item analysis to determine overall test reliability
Identify the overall test reliability
Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
Identify 2 (two) ways/methods to improve the overall test reliability
Analysis for Individual Test Questions: For each of the nine identified test questions (items 1 – 8 and 45), the student will identify and discuss:
If the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
If the question has a poor difficulty index
If the question has a poor discrimination index
If the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Preparing the paper
Criteria for Format and Special Instructions:
The use of the Item Analysis Template is required. It is found within Course Resources.
A reference page is required if a reference is cited. APA (6th ed.) is required.
Category
Possible
Points
%
Remarks
Review of the reliability of the entire examination
30
17
Required content for this section includes:
Explain what is meant by test reliability
Identify the name of the statistical test used in the item analysis to determine overall test reliability
Identify the overall test reliability
Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
Identify 2 (two) ways/methods to improve the overall test reliability
Question One
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Two
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Three
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Four
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Five
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Six
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Seven
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Eight
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Nine
15
9
Required content for this section includes:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
APA Format (6th edition)
5
3
Title page, running head, body of paper, and reference page must follow APA guidelines as found in the 6th edition of the manual. This includes the use of headings for each section of the paper except for the introduction where no heading is used.
One deduction for each type of APA style error
Writing Mechanics
5
3
Rules of grammar, spelling, word usage, and punctuation are followed and consistent with formal written work as found in the 6th edition of the APA manual.
Total
175
100%
A quality assignment will meet or exceed all of the above requirements.
Grading Rubric
Assignment Criteria
Exceptional
(100%)
Outstanding or highest level of performance
Exceeds
(88%)
Very good or high level of performance
Meets
(80%)
Competent or satisfactory level of performance
Needs Improvement
(38%)
Poor or failing level of performance
Developing
(0)
Unsatisfactory level of performance
Content
Possible Points = 165 Points
Review of the reliability of the entire examination
30 Points
26 Points
24 Points
11 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Explain what is meant by test reliability
Identify the name of the statistical test used in the item analysis to determine overall test reliability
Identify the overall test reliability
Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
Identify 2 (two) ways/methods to improve the overall test reliability
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Explain what is meant by test reliability
Identify the name of the statistical test used in the item analysis to determine overall test reliability
Identify the overall test reliability
Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
Identify 2 (two) ways/methods to improve the overall test reliability
Presentation of information was minimally demonstrated in the all of the following elements:
Explain what is meant by test reliability
Identify the name of the statistical test used in the item analysis to determine overall test reliability
Identify the overall test reliability
Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
Identify 2 (two) ways/methods to improve the overall test reliability
Presentation of information in one or two of the following elements fails to meet expectations:
Explain what is meant by test reliability
Identify the name of the statistical test used in the item analysis to determine overall test reliability
Identify the overall test reliability
Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
Identify 2 (two) ways/methods to improve the overall test reliability
Presentation of information is unsatisfactory in three or more of the following elements:
Explain what is meant by test reliability
Identify the name of the statistical test used in the item analysis to determine overall test reliability
Identify the overall test reliability
Provide a discussion if the overall test reliability is sufficient or if improvement is necessary
Identify 2 (two) ways/methods to improve the overall test reliability
Analysis for Individual Test Questions:
Question One:
15 Points
13 Points
12 Points
6 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was minimally demonstrated in the all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information in one, two, or three of the following elements fails to meet expectations:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information is unsatisfactory in four or more of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Two:
15 Points
13 Points
12 Points
6 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was minimally demonstrated in the all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information in one, two, or three of the following elements fails to meet expectations:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information is unsatisfactory in four or more of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Three:
15 Points
13 Points
12 Points
6 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was minimally demonstrated in the all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information in one, two, or three of the following elements fails to meet expectations:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information is unsatisfactory in four or more of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Four:
15 Points
13 Points
12 Points
6 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was minimally demonstrated in the all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information in one, two, or three of the following elements fails to meet expectations:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information is unsatisfactory in four or more of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Five:
15 Points
13 Points
12 Points
6 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was minimally demonstrated in the all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information in one, two, or three of the following elements fails to meet expectations:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information is unsatisfactory in four or more of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Six:
15 Points
13 Points
12 Points
6 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was minimally demonstrated in the all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information in one, two, or three of the following elements fails to meet expectations:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information is unsatisfactory in four or more of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Question Seven:
15 Points
13 Points
12 Points
6 Points
0 Points
Presentation of information was exceptional and included all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was good, but was only partially demonstrated in places; all elements were present:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information was minimally demonstrated in the all of the following elements:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor discrimination index
Identifies if the question should be retained as written or revised
If you selected “revision”, describe ALL of the revisions that are needed
If you selected “remain”, state why the question is good to use
Presentation of information in one, two, or three of the following elements fails to meet expectations:
Identifies if the question violates any principle of item writing (i.e. does it ask a question, are there any tips to the correct answer, etc.)
Identifies if the question has a poor difficulty index
Identifies if the question has a poor dis
Recent Comments