QuestionTake the role of a
psychological examiner administering a standardized test of intelligence to
Johnny, an 8-year-old boy, as a part of a research project. Carefully review
theABS300: Week Two Assessment Scenario. In your initial post, begin by describing your
current understanding of the concept of intelligence. Based on your description
of intelligence, evaluate Johnny’s response in comparison to the responses
identified as correct in the standardization manual. Decide whether or not to
score Johnny’s answer as correct or incorrect. Explain your rationale. Specify
how scoring Johnny’s responses exactly by the manual might impact his true
score on the intelligence measure. Analyze the “four seasons of the
year” test item in terms of each of the following psychological measurement
concepts: reliability, predictive validity, content validity, and cultural
fairness. Evaluate the standards in the APA’s Ethical Principles of
Psychologists and Code of Conduct that address testing bias and the strategies
for multicultural assessments discussed in the article by Horin, Hernandez
& Donoso (2012) in light of the current scenario. Conclude your post by
explaining how you would interpret the assessment results to Johnny and his
parents. 400 words needed. Reference material PSYCHOLOGICAL ASSESSMENT Have you ever wondered what your true I.Q. score is? What if I told you I have a reliable way to measure your intelligence using a common household item? Now, granted, this measurement device is not 100% reliable over a lifetime, but over the course of a few days, it is highly reliable and over the course of a few weeks, still largely reliable. The item? Your bathroom scale! Yes, really, just hop on the scale and weigh yourself and subtract 10 from the number if you weigh between 110-150 pounds; subtract 50 if you weigh between 151-200 pounds; and divide by 2 if you weigh over 200 pounds, and you will have a largely RELIABLE measure of your intelligence.What, you don’t believe me? Define reliability.According to the text authors, reliability simply refers to the consistency of measurements (Cohen, Swerdlik, & Sturman, 2013). If you step on a scale, you should get highly consistent measurements over a few days and still pretty consistent measurements over a few weeks unless you make some drastic changes. Right?So, what’s the problem?The problem is that the measure lacks validity. Yes of course, test results need to be dependable; however, they also need to be valid (meaningful). So, if you think the recommendation to use a bathroom scale to test intelligence lacks validity, you are correct. Validity refers to the extent to which a test measures what it is supposed to measure (Cohen, Swerdlik, & Sturman, 2013).So, why would you want a test that is reliable but not valid?You wouldn’t. You also do not want a test that is not reliable. Let’s say that scales are generally considered to be valid measures of weight—of course, some are better than others. If you step on the scale at 7:00am and it measures your weight to be 125lbs, at 7:01am: 190lbs, at 7:05am: 95lbs, and at 7:10am: 160lbs—all in the same day, the scale is not reliable—the measurements are simply not consistent over a time span when they should be highly consistent.By the use of statistical procedures we can compute a statistic, called the reliability coefficient, for psychological tests. Another useful and related statistic is the standard error of measurement—which is derived from the reliability coefficient. DO NOT TRUST a test if its reliability coefficient is low. For most tests of cognitive abilities, a reliability coefficient of .80 or higher is generally considered to be acceptable. As for validity—whether or not the test measures what the authors claim it measures—unless the test is valid for the particular purpose for which it is being used, the results cannot be used with any degree of confidence. Validity is not one-size-fits-all. Because tests are used for many different purposes, there is not a single type of validity that fits for all testing purposes. Examiners are responsible for the valid use of test results. Validity is highly contextual. In Week 1, I mentioned the TEMAS test, which was developed because tests like the Thematic Apperception Test and the Children’s Apperception Test were not deemed to have high validity for Hispanic and African American children. The WAIS-IV (Wechsler Adolescent and Adult Intelligence Scales-IV) is considered a reliable and reasonably valid measure of intelligence for adults; however, if used with a 3-year-old or someone who does not speak English, the test might continue to be reliable, but would definitely not be valid.You will read more about different types of reliability and validity in the textbook. Chapters 5 & 6 assume the reader has a good understanding of basic statistical concepts. If you managed to enroll in this class out of sequence and have not yet taken the required statistics and research methods classes, or for some other reason, you need a refresher, it is recommended that you read Chapter 3 in the text and review the videos in Films on Demand that are listed toward the end of this week’s instructor guidance.One especially important concept to understand in addition to reliability and validity is the confidence interval. It is common in psychological assessment reports to report a score in the manner below. Jacob obtained a Full Scale IQ score of 121 + 8 on the WAIS-IV.  The chances that his true score is between 113 and 129 are about 95 out of 100. In the example above, the + 8 (read as, “plus or minus eight”) is the 95% confidence interval for Jacob’s score on that test. The process of computing the confidence intervals is somewhat complex; however, test manuals provide confidence intervals in a table in the scoring manual, and computer scoring programs provide the confidence intervals as a routine part of the scoring process.“Why is so much space devoted to the concepts of reliability and validity?”Making sure that tests and assessment procedures are reliable and valid is not just a numbers game. The use of tests and the communication of assessment results impacts the lives of real people every day.Dr. Diaz (name changed) is a highly respected—and very wealthy–ophthalmologist. He started his practice with one clinic and now has a chain of twelve eye (ophthalmology) clinics within a 200-mile radius. He is one of the most sought after eye surgeons in a four-state area.Dr. Diaz was born in New York City. When he was in the 3rd grade, Tony (short for Antonio) was at the top of his class and fluent in English and Spanish. Due to his father’s job opportunities being limited in NY, the family relocated to California (in the 1970s). Despite the fact that Tony’s mother was proficient in English, when she went to enroll Tony in the local school, the counselor noted Ms. Diaz to speak with an “accent” and made incorrect assumptions about Ms. Diaz and Tony. Based on the interview with Tony (who was mostly quiet—a bit nervous about starting a new school—and without formally testing Tony, the school counselor assessed Tony to require remediation and assigned him to the special education class. What was the data used to evaluate Tony’s academic level? The way his mother pronounced words. Was the assessment reliable? Yes—Ms. Diaz had pronounced, and would continue to pronounce, words that way for years. Was the assessment valid? No. Unlike many similar stories, rather than becoming a self-fulfilling prophesy, Tony was eventually moved from special education classes to regular classes by the 5th grade. How was his intelligence discovered? Because Tony was a cooperative child, he did the work assigned in his special education class despite the fact that he thought it was really easy. However, he loved to read and would checkout advanced books from the library. One day, Joey, a student in 5th grade, was having difficulty with some math problems and Tony showed him how to do his homework. The change in Joey’s homework quality was so drastic that his teacher asked him who helped him. Joey told her that Tony showed him how to do the work. The school was in a small town in California, so getting to Tony was a matter of literally walking across the hall. Long story, short, Tony was moved out of special education, he graduated high school with honors, and he earned his bachelor’s and master’s degrees in education (interestingly) from one of the top, private universities in California. He practiced as a high school math teacher for over a decade and developed innovative curricula to teach math and science. He loved to work with students who had been evaluated to be below average in intelligence, mathematics, and/or science. The results of the standardized testing were assumed to be valid indicators of the students’ level of functioning and achievement—and had the student’s not run into Mr. Diaz (Tony) as a teacher, the results might have proven valid over the long-term. However, Tony’s interventions often led to the students achieving at levels much higher than the test scores predicted (keep this in mind when you read about predictive validity). Then, when ready for a career change, Tony went back to college, graduated from medical school and became an ophthalmologist. Dr. Diaz has practiced as a physician specializing in eye health for over a decade now and he still volunteers tutoring youth in the subjects of math and science.Just for Fun: Are there reliable and valid measures of your personality on your Facebook page?Let’s first consider the endless number of quizzes shared by social media users which purport to define your personality from What type of girl are you?to What type of dragon are you? or What kind of candy are you? If you take one of these tests and retake it and the results come out the same—that’sreliability. How do you determine whether or not the tests are valid? Based on the information in your textbook, which types of validity should you be concerned about for these types of tests? Or are these just fun ways to pass the time without any expectations of reliability or validity?On the other hand, is it possible to obtain valid and reliable measures of personality based on your social media data? Park and his colleagues believe it is. Could this be one reason selling social media data is such big business? The recommended article by Park et al. (2014) shows an application of the concepts of reliability and validity to a most interesting assessment context: using vocabulary analysis techniques to assess user personality traits (openness, conscientiousness, extraversion, agreeableness, and neuroticism) based on language posted in social media (Park et al., 2014). It makes you wonder, “What else can be determined by the information people post online on social media pages?”  What do you think?Big Five personality ModelAdditional Resources (Overviews & Refreshers) on Statistical ConceptsMultimedia (Instructions on how to access the Films on Demand database follow the list below.)Taylor, S., & Howes, S. (Producers). (2008). Descriptive statistics [Video file]. In Psychology research in context. Retrieved from https://fod.infobase.com/OnDemandEmbed.aspx?Token=40117&aid=18596&Plt=FOD&loid=73353&w=640&h=480&refThis program provides an easy-to-understand review of several topics from research methods and psychological statistics using applied examples.Taylor, S., & Howes, S. (Producers). (2008). How science works [Video file]. In Psychology research in context. Retrieved from https://fod.infobase.com/OnDemandEmbed.aspx?Token=73350&aid=18596&Plt=FOD&loid=73350&w=640&h=480&refThis program provides an easy-to-understand review of several topics from research methods and psychological statistics using applied examples.Taylor, S., & Howes, S. (Producers). (2008). Measuring validity [Video file]. In Psychology research in context. Retrieved from https://fod.infobase.com/OnDemandEmbed.aspx?Token=40117&aid=18596&Plt=FOD&loid=73361&w=640&h=480&refThis program provides an easy-to-understand review of several topics from research methods and psychological statistics using applied examples.Taylor, S., & Howes, S. (Producers). (2008). Normal distribution [Video file]. In Psychology research in context. Retrieved fromhttps://fod.infobase.com/OnDemandEmbed.aspx?Token=40117&aid=18596&Plt=FOD&loid=73354&w=640&h=480&refThis program provides an easy-to-understand review of several topics from research methods and psychological statistics using applied examples.Taylor, S., & Howes, S. (Producers). (2008). Statistical analysis [Video file]. In Psychology research in context. Retrieved from the Films On Demand database. https://fod.infobase.com/OnDemandEmbed.aspx?Token=40117&aid=18596&Plt=FOD&loid=73357&w=640&h=480&refThis program provides an easy-to-understand review of several topics from research methods and psychological statistics using applied examples.