College Board's New Stance on Test Prep
For decades the College Board has maintained that its flagship test, the SAT, was unteachable. The name itself was once an acronym for "Scholastic Aptitude Test," which represented the test designers’ belief that the test measured inherent and unchangeable ability. However, on May 8th the College Board released a statement on its website that seemed to reverse its position. The article leads with its central point:
"New data show studying for the SAT® for 20 hours on free Official SAT Practice on Khan Academy is associated with an average score gain of 115 points, nearly double the average score gain compared to students who don’t use Khan Academy."
First, I should say that I am pleased that College Board has finally admitted what everyone else in the industry has known for years: preparing for standardized tests in advance improves your performance on those tests. It seems self-evident. But why the sudden change?
Our first clues comes from College Board itself. In a response to an inquiry made by the Washington Post in preparation for its article about the study, College Board’s Senior Director of Media Relations Zachary Goldberg said:
"Too much of commercial test prep teaches to the test — looking for shortcuts and tricks to “beat” the test. The SAT in its old format lent itself to this approach. The College Board and Khan Academy firmly believe in practice, and particularly practice that is personalized to pinpoint areas where learners need additional help. Preparing for the new SAT is the same as preparing for college."
Let me unpack the inconsistencies of this statement.
Prior to 2017, College Board claimed that the SAT was not a teachable test. Now, Mr. Goldberg admits that the old SAT format “lent itself” to the approach of “tricks and shortcuts.” So does that mean that it was teachable all along, and that commercial test prep had found effective ways to help students improve their performance?
The new format, according to him, lends itself to personalized practice. Many commercial test prep companies that provide one-on-one instruction personalize their approach to the students with whom they work. Shouldn't that mean they too can provide real help for SAT takers?
The value in private tutoring is in establishing a relationship between tutor and student that fosters trust and motivational growth. A student must be sufficiently motivated, adequately exposed to the right material at the right time, and provided with the relevant and academically necessary information customized to their needs. With the right tutor there is potential for immense change - not just improvement in the eventual test score but also in the way a student will test in the future and the way that student will approach other academic tasks of this magnitude.
Yes, of course, there are techniques that exist outside of the academic content of the test. That does not mean that those techniques are rendered ineffective by the new format. In fact, simply because the test is still predominantly multiple choice is reason enough to doubt these claims. Answering multiple choice questions is a skill that can be taught and improved, which incidentally is why many assessment specialists, including me, advocate other question formats in favor of multiple choice when designing standardized tests.
Disappointingly, the study examines only the effect of using the material released by Khan Academy. College Board partnered directly with Khan Academy to develop these materials. So, to maintain the claim that test preparation is ineffectual no longer makes business sense, as doing so would lump their own materials in with those they had been dismissing for sixty years. But if College Board does not include other methods of test prep or other content or service providers in their research, championing Khan Academy as the pinnacle of test prep is horribly misleading.
It’s all pretty confusing, isn’t it? So, here is the simple truth: making broad claims about the efficacy of all programs and companies other than Khan Academy is impossible. What large corporations do in classrooms of 100 kids is very different from what small, specialized firms can accomplish in one-on-one or small group sessions, which in turn is very different from what Khan Academy provides with its semi-adaptable software. All of these models have great value and can provide the right service for the right student. Mr. Goldberg, in attempting to clarify, has broadcast College Board’s motives and undermined the conclusions of the study.
Later in the WashPo article, Bob Schaeffer, Education Director of FairTest, rightly calls out College Board for this conflict of interest:
"The College Board’s admission that SAT coaching can boost scores significantly once again demonstrates the hypocrisy of the testing industry. After six decades of aggressively claiming that SAT prep courses do not have a major impact, the College Board has suddenly reversed its position. Of course, the program they now assert can make a big difference is the only one the College Board partners with. Not surprisingly, they did not study the offerings from any test-prep firm, many of which advertise even larger score gains."
It is difficult enough to know how best to serve your children and students, but things get far cloudier when the administrators of a test begin to make claims not only about their test but also about the exclusive partnerships they make. Now, don't misunderstand me. Khan Academy is an excellent tool. It provides instruction and practice for numerous content areas and can be leveraged to great effect by a motivated and insightful student. It is also provided free of charge, making it available to kids and families of all socioeconomic backgrounds and hopefully cutting into the college access gap.
However, Khan Academy does not have a monopoly on effective SAT prep content. Its content does not replace the customizable approach a private testing expert can provide or the regular and familiar structure of a classroom setting. To say otherwise is naive at best; when asserted by the test designer, doing so borders on disingenuous and self-serving. The real takeaway from the study is this: the SAT has content that can be learned, and the changes to the SAT format have only affected what that content is.