The Lone Star Foundation Conference
Public Education Reform in Texas: Comprehensive Progress Report
Austin, Texas - December 7-8, 2000

Austin, TexasIntroduction

This report addresses the assessment of mathematics achievement in Texas as it relates to the goal of increasing student learning.

The report begins with some background information to set Texas in the larger context of mathematics achievement nationally and the ongoing debate over mathematics education in this country. Some considerations about other well-known tests are also given as background.

Following this is a summary of the varied reports and indicators of the success (or lack thereof) in Texas with respect to stimulating achievement gains.

The bulk of the report addresses the details of the statewide assessments in Texas - the Texas Assessment of Academic Skills (TAAS) as it was from 1995 to 1998, the transition to the more recent version tied to the Texas Essential Knowledge and Skills (the TEKS-based TAAS), and finally the future version (TAAS II). These findings generally point to weaknesses in the assessment system that may or may not be resolved with the transition to TAAS II.

Conclusions are generally less than optimistic, although a few rays of hope are offered.

Achievement Levels

Texas, though large, must be considered within the larger national context when it comes to mathematics achievement. There is ample reason to believe that the United States is not doing well in this regard. Some of the most compelling evidence comes from the Third International Mathematics and Science Study (TIMSS) results. The figure illustrates U.S. achievement relative to the other countries evaluated at given grade levels. The findings were less than stellar - the United States is clearly not on top of the world when it comes to mathematics achievement. In fourth grade, our achievement was moderate, and the scores went down from there.

A regression line fitted to the rankings across the grade levels tested shows a remarkably linear outcome. It suggests that, relative to our international competition, our students are losing ground year after year, culminating in a dismal result by the end of high school. This national trend is significant for Texas. It will be difficult for Texas to buck the trend in this national context. Perhaps more significantly, it suggests that keeping up with national achievement levels will still mean failure. The national context does not bode well for Texas.

There is also every reason to believe that making progress relative to national norms becomes more and more difficult as students get older. This effect is evident, for example, in the test score gains in California over the past two years. Such a result is to be expected if education is cumulative, which is safe to assume, if not for a variety of other plausible reasons. Whatever reasons underlie this phenomenon, the effect size is sufficiently large to make it very difficult to correct our national decline across grade levels. It is even large enough to jeopardize gains made in the early grades - they may not be retained as students mature. Such a finding is not uncommon in longitudinal studies. Together, these influences support the notion that we may need to move mathematics expectations to lower grade levels. There are many who advocate setting algebra 1 as the state eighth-grade target for these reasons.

The National "Reform" Movement

The influence of the so-called reform movement in mathematics education is inescapable in Texas. As promoted by the National Council of Teachers of Mathematics (NCTM), these ideas have influenced standards and textbooks throughout the country. Issues with the NCTM, and the reform movement in general, are well-documented on the Mathematically Correct web site (http://www.mathematicallycorrect.com) and need not be detailed here. Suffice it to say that there are many who view the "higher-order" thinking of the reformers' designs as "lower-order" in practice.

The positions advocated by the NCTM and others are finding other ways to influence the individual states. The U.S. Department of Education has become involved, despite the following prohibition to the contrary:

Curriculum, administration, and personnel; library resources

No provision of a program administered by the Secretary or by any other officer of the Department shall be construed to authorize the Secretary or any such officer to exercise any direction, supervision, or control over the curriculum, program of instruction, administration, or personnel of any educational institution

United States Code, Title 20, Chapter 48, Subchapter I, Section 3403(b)

This influence has been effected through cooperation with the National Science Foundation (NSF) among other groups. NSF has funded the development of reform mathematics curriculum materials. Many of these programs were then evaluated as exemplary or promising by a committee of the Department. The programs are now advertised by the Eisenhower National Clearinghouse web site (http://www.enc.org) which is funded by the Department of Education.

As a result, a letter of protest was drafted and has now been endorsed by over 200 individuals, mostly professional mathematicians but including other noteworthy people in education and scientific fields. The letter was published in the Washington Post and Education Week, and makes the objections to the actions of the Department of Education clear. (see http://mathematicallycorrect.com/nation.htm#doesham).

Meanwhile, NSF has funded Systemic Initiatives which effectively promote these reform programs and their related pedagogical approaches in states and districts around the country, including Texas (see "The National Science Foundation Systemic Initiatives," Michael McKeown, David Klein, and Chris Patterson in Sandra Stotsky's What's at Stake in the K-12 Standards Wars, Peter Lang, New York, 2000).

Federal Government Involvement

The reform movement is inextricably bound up with the federal government, far beyond just the Department of Education and the NSD. Current thinking is that the federal government only contributes a small portion of the cost of public education, say six or seven percent, with the implication that the federal influence is not great. This implication is an underestimation of the total impact of the federal government on education.

As just one example, the National Assessment of Educational Progress (NAEP) is a project of the federal government that doesn't show up in the cost of running local schools. The federally funded (but quasi-independent) National Assessment Governing Board oversees the tools used to measure achievement at the national and state level.

In fact, considerable national influence in education flies below the radar of the six or seven percent figure. Furthermore, there are so many federal departments and agencies involved that the federal impact on education becomes almost impossible to quantify. As but one example, a recent survey has identified the federal contributors to the efforts to wire schools for the Internet:

U.S. Department of Education

U.S. Department of Commerce

U.S. Department of Defense

National Science Foundation

National Aeronautic and Space Administration

Federal Communications Commission

Office of Science and Technology Policy

(http://www.media.mit.edu/people/vanessa/nii/toc.html)

Even within the six or seven percent figure, the influence of the federal government may exceed the proportion of the funding it provides. In the Systemic Initiatives, this is accomplished in much the same way as some investors leverage their funds through the use of futures or options. It even does this by other "strings" that come with funding:

The Federal Government closely controls how the funds it gives are spent, which hampers local innovation. Also, the … funding that it gives is responsible for more than 50 percent of the administrative work in many school districts, due to the extensive paperwork requirements that come with Federal assistance.

106th Congress , 1st Session, March 11, 1999, Page S- 2566

(http://www.senate.gov/~rpc/rva/1061/106145.htm)

In summary, attempts to improve mathematics achievement in Texas necessarily must occur within the confines of the national context. These include poor achievement levels, the promotion of various programs and methods that may be counter-productive, and the confining consequences of federal government involvement.

Background II: No Test is Perfect

Much of the material herein will focus on the Texas Assessment of Academic Skills (TAAS) as a measure of mathematics achievement. In this regard, several other measures will be referred to as well. Lest the reader consider some other test as the "gold standard" for measuring mathematics achievement, some comments are offered here about two of the most highly-regarded assessments.

The National Assessment of Educational Progress (NAEP)

Results from the NAEP are frequently used to make longitudinal comparisons or comparisons among states. While we are fortunate to have this measure of achievement in mathematics, it is less than ideal for many research applications.

It is not commonly known that there are actually two NAEP tests. The one that most are aware of is the state assessment, used periodically to make comparisons among states and given wide attention in the press. The other is the little-known national trend test. This is used to measure progress in the country over longer time periods and for equating different versions of the state assessment. This means that, unfortunately, the commonly used data reflect achievement goals that are a moving target.

The NAEP is designed to survey achievement across states. As such, NAEP objectives are relatively broad. They may fit better with the curriculum in one state than another, although this fit may change as the state assessment is revised.

NAEP data are collected from a sample of schools in each state and participation is voluntary. Although statistical methods are employed in the attempt to adequately represent each state by the schools that are sampled, this process can only be approximate. Likewise, a "matrix" design is used for item administration, so that different students take different samples of items. NAEP cannot provide data on individual students or track progress in the same sample of students. Furthermore, only a subset of grade levels are tested and the tests are not given each year.

Thus, the NAEP is seriously limited with respect to research applications. There are also concerns about the content domain sampled by the NAEP. Some interesting comments about NAEP and the National Assessment Governing Board (NAGB) were provided by one of its former members, Chester E. Finn (Statement by Chester E. Finn, Jr., Prepared for delivery to the National Assessment Governing Board, Washington, D.C. January 21, 1998, http://www.mathematicallycorrect.com/onnvtm.htm#finn). Selected sections of his address are reproduced below:

In 1990, we began to steer NAEP toward the recommended standards of the National Council of Teachers of Mathematics (NCTM), which had been published in 1989. In revising the NAEP math framework for the 1992 assessment cycle, and revising it yet again for the 1996 cycle, the Board steered ever closer to NCTM's view of what constitutes desirable standards, curricula, instructional methods and assessment strategies. Steadily more emphasis was placed, for example, on "open response" test items, on calculator use, and on conceptual understanding. That evolution is plain in numerous NAEP documents, including the 1996 mathematics Report Card. The three "math abilities" and three forms of "mathematical power" depicted in that report, for example, are pure NCTM doctrine.

Indeed, I think it's accurate to observe that the Board's math frameworks for NAEP would likely have embraced the NCTM approach even more fully had it not been for concerns about trend lines and about the assessment getting out too far in front of school practice. But the frameworks and assessments went a great distance toward NCTM, ever closer as the 1990's progressed. I would estimate that by 1996 the NAEP math framework had become at least 75% reflective of NCTM's recommendations.

That's a mighty quick and abrupt shift for the "Nation's Report Card", a hint, it's now clear, of the danger inherent in allowing so sensitive and consequential a measuring stick to be bent to the contours of curricular fads and educational "innovations" that carry plenty of big names and paper credentials but that have not, in fact, proven themselves in the real world.

There's just one hitch. The math world is again changing. Since those NAEP frameworks were developed (with, as I said, perhaps 75% NCTM influence), earthquakes have begun to rattle the once-solid structure of NCTM math. It's too early to be sure, but there is reason to suppose that American education has started moving away from some key NCTM assumptions.

What does this mean for NAEP and NAGB? To me, it means that if a nationwide math consensus project were launched today and conducted honestly, i.e. with a full measure of consumers and real scholars and not again dominated again by CCSSO and NCTM, the result would almost certainly be less reflective of NCTM-style math than was the 1996 framework.

In fact, it appears to me, in today's context, that the NAEP framework is itself worrisomely derriere garde, reflecting the math assumptions of 1989, assumptions now no longer universally shared and no longer viewed as the cutting edge but as the receding edge of a failed revolution.

Secretary Riley recently called for an end to the "math wars". That's a splendid sentiment, even if many in his own Department believe it must mean the surrender of long division to the "problem solving" forces of the NCTM.

Maybe NAGB can become the peacemaker. But for that to happen, I respectfully suggest, the Board must itself go back to ground zero, jettisoning the specifications that it inherited, and starting afresh. That means enlisting new people, new assumptions, new item specs and, in time - let's be honest - developing a new NAEP math framework, too.

For all its faults, in matters of curriculum the Congress is not a bad proxy for the consuming public. I don't believe most of the public wants its children to learn fuzzy math. I don't believe the Congress does, either.

In summary, NAEP scores are very useful but there are many reasons to argue that it should not be considered as a "gold standard" and there are many limitations to the utility of NAEP scores for evaluating progress in research applications.

The College Entrance SAT Exam

There are individuals and groups that are critical of college-entrance SAT exams provided by the College Board (http://www.collegeboard.org). Some complain about these scores being used for entrance requirements and argue that this process is exclusionary or culturally biased. Despite ample evidence of predictive validity, others argue that the SAT (and standardized tests in general) only measure test-taking ability. Still others feel that the math score is not a good indicator of mathematics achievement and many universities require one of the SAT II Subject Tests in mathematics as well.

In the midst of these issues, the SAT itself is changing. These changes may add to the confusion about SAT scores. Many are aware of the scoring change instituted in 1995 known as "recentering." This change adjusted the interpretation of score values, and it is not known whether or not all score users correctly adjust for this change when making comparisons. But, these are not the only changes to the SAT. Qualitative changes in the tests have taken place and show up in the data for the 1996 test year as illustrated in Appendix A.

The College Board itself cautions on the use of aggregate SAT data for comparisons because not all students take the exams, among other reasons (http://www.collegeboard.org/research/html/rn01.pdf). Not only has the participation rate changed over time, but the number of times each student takes the exam has changed as well. Finally, the impact of differential participation in SAT preparation programs can bias conclusions. Thus, there are many risks to using aggregate SAT data to draw inferences.

In summary, even the most respected of assessments are not without limitations. Assumptions and implications go along with the use of any given measurement tool.

Mathematics Achievement Gains in Texas: The Texas Miracle

"The Texas Miracle" is an expression that has been used to refer to significant score gains demonstrated on assessments in Texas, including measures of mathematics achievement. These gains are most notable using the Texas Assessment of Academic Skills (TAAS). The results show consistent and dramatic gains in the statewide data. The most commonly-referenced metric is the percentage of students meeting the minimum expectations at a given grade level. The figure shows this growth over several years for three illustrative grade-levels. Graduation from high school is contingent on passing the tenth-grade exam, so this increase reflects a high-stakes context. In addition to over-all improvements, TAAS scores also reflect a decreasing "achievement gap" between whites and minorities. These findings have gained Texas considerable recognition.

Of course, the TAAS tests are keyed to the learning expectations in Texas. But, even by the more general National Assessment of Educational Progress (NAEP), Texas students have made substantial gains relative to the country as a whole, especially in fourth grade. The eighth-grade results are less dramatic, particularly from 1992 to 1996. The results of the 2000 administration of the NAEP will not be available until 2001.

When the confounding variables of economic status and other demographic and school variables are controlled for, the improvements seen in Texas are still outstanding according to a substantial report from the Rand Corporation (David Grissmer, Ann Flanagan, Jennifer Kawata, and Stephanie Williamson, "Improving Student Achievement: What NAEP State Test Scores Tell Us," Rand, MR-924-EDU, 2000).

Texas Gains as Somewhat Less than a Miracle

Several other indicators are less flattering regarding achievement trends in Texas over time. The first of these is the Texas end-of-course test for algebra 1. Although there have been gains in these scores over the years, most students are still failing this exam. This fact is especially frustrating given the gains that have been seen on the grade-level TAAS exams and the importance of algebra to future academic success.

Even less encouraging are the results for the college-entrance SAT and ACT exams illustrated below. Texas is not making any real improvement on these tests relative to the rest of the country. This result may be mitigated somewhat by the fact that the participation rate has increased in Texas, but these findings are still not consistent with TAAS results and suggest that any gains are not being carried through to college entrance.

Finally, Advanced Placement (AP) Calculus AB exam scores for 2000 show that the failure rate for Texas students continues to exceed that of the nation as a whole. Again, participation rates are increasing. Nonetheless, the top students from Texas are still lagging behind the country as a whole.

Questioning The Texas Miracle

Shortly before the November election, another report was released by the Rand Corporation with a decidedly different interpretation of test results in Texas (Stephen Klein, Laura Hamilton, Daniel McCaffrey, and Brian Stecher, "What Do Test Scores in Texas Tell Us?" (2000), Santa Monica, CA: Rand IP202). In part, this issue paper compared NAEP results to TAAS results to conclude that the gains in achievement in Texas had been overstated. Only the mathematics results will be addressed here.

As part of their evidence, these authors point to score gains in Texas over a 4-year period as measured by TAAS (1994 to 1998) and for both Texas and the nation as a whole based on NAEP data (1992 to 1996). Gains were expressed in standard-deviation units based on mean scores. Fourth-grade gains in Texas as measured by TAAS were much larger than those measured by NAEP, but the Texas gains on NAEP were much greater for Texas than for the nation as a whole. By contrast, eighth-grade NAEP gains were not much different for Texas than they were for the nation as a whole, although gains measured by TAAS were still very large. The implication is that both 4th and 8th grade results are overstated by TAAS, and that in 8th grade the gains in Texas are on a par with the nation as a whole.

These findings should not be surprising for the following reasons:

The tests don't measure the same thing, The TAAS is curriculum specific, while the NAEP is not.

The motivation to study for and achieve on the NAEP is different than it is for the TAAS.

Gains in eighth grade are harder to produce than gains in fourth grade.

The NAEP mathematics tests were revised in 1996, so it is a moving target. The changes may have mitigated against Texas gains.

The scores addressed by the authors are means, not the percentage of students meeting a criterion, which could influence results.

A second method the authors use to measure gain using NAEP mean scores is to compare 4th graders in 1992 to (different) 8th graders in 1996. NAEP scores are designed so that the common scale can be used to measure growth across grade levels in this manner. The Texas mean increased from 229 to 285, a 56 point gain, while the national mean increased from 227 to 281, a 54 point gain. The authors conclude that the gains in Texas were almost identical to the national gains.

This comparison is subject to many of the same influences noted for the first method. Also, this technique obscures any gains that are made by 4th grade and simply retained over the ensuing four years.

In short, the findings in their report indicate that fourth-grade gains are larger than eighth-grade gains, and that TAAS-measured gains are larger than NAEP-measured gains. Neither of these findings is surprising.

Although not the focus of the current review, the recent Rand paper also addresses equity issues that should be mentioned here. The Rand authors report the white-black mean difference and the white-Hispanic mean difference as indicators of achievement gaps across these groups. These gaps were seen to decrease for TAAS means from 1992 to 1996 but to increase for NAEP means from 1994 to 1998.

The Rand authors correctly note that ceiling effects on the TAAS could have the net effect of making the gaps appear to decrease. Using mean scores for analysis while Texas (and even NAEP reporting) focuses on the percentage of students who meet a minimum standard could also be related to this finding. Lack of comparability of the tests, differing motivation for success, and the changing nature of the NEAP could also have contributed to these results.

The Rand authors clearly consider the NAEP to be the gold standard. In fact, they state that the "large discrepancies between TAAS and NAEP results raise serious questions about the validity of the TAAS scores." The perception could be easily turned around, arguing that the broader focus of NAEP, or the nature of the NAEP items, make it the less useful measure.

Clearly the truth is that the Texas findings lie somewhere between myth and miracle.

Grade Inflation and Equity

One of the values of external testing is that it provides an independent check on course grades. The value of such an external verification has increased as concerns over grade inflation and social promotion have grown. The simple check for grade inflation is to compare course grades to test performance. The TEA has provided the evidence for algebra 1 achievement as summarized below (Student Performance Results, 1998-1999, Statewide and Regional Results, Texas Education Agency, Student Assessment Division):

Performance on Algebra I EOC Test Compared to

Performance in Algebra I Course

Thus, there was agreement between the course grade and the test score for 62% of the students - 43% who passed both and 19% who failed both. Of concern are the 38% where there is a mismatch. The great majority of these are students who pass the course but fail the exam. This is an indication of grade-inflation. Indeed, most of the students who fail the exam still pass the course. Effectively, the expectations that must be met to pass the course appear lower than the expectations that must be met to pass the exam.

The same document also breaks this information down into three ethnic groups and for the economically disadvantaged versus those who are not economically disadvantaged. The 2% course failure rate for those who pass the exam remains constant across these tables. However, the percentage who fail the exam yet pass the course varies across these breakdowns. By ethnicity, these values are 48% for African American students, 41% for Hispanic students, and 30% for White students. The value is 42% for economically disadvantaged versus 32% for students not in this category.

This relative difference between the course and exam expectations suggests that the grade inflation is greater for minority groups and the economically disadvantaged than for others. This sort of finding is an example of "the soft bigotry of lower expectations" that George W. Bush referred to so frequently as during his presidential campaign.

Minimum Competence, Low-Difficulty Exams

The TAAS mathematics examinations are essentially minimum-competence tests. That is, they are used to assess whether or not students meet minimum expectations. While other score metrics are provided for TAAS tests, the minimum-competence scores receive the most attention. The TAAS mathematics examinations are also arguably too easy. Evidence of this will be provided in subsequent sections. The emphasis here is on the nature and impact of scores for minimum-competence, low-difficulty exams.

This first figure illustrates a hypothetical achievement distribution before testing is introduced and a second distribution showing an achievement gain sometime later. A theoretical cut-point representing the measurement of minimum-competence is also shown. As achievement improves, the number of students below the cut point decreases.

The second figure illustrates what can happen if instruction is focused too tightly on the minimum-competence level. Although the number of students below the cut point decreases, overall achievement is not improved. In fact, it can actually get worse. This may or may not actually happen. Som of the top students may have sufficient resources to achieve beyond the level of classroom instruction. This may depend on the socio-economic status of the students. For many, this situation may actually close the door to high achievement.

This figure illustrates what can happen when the difficulty of the test is too low. Students stack up near the top of the resulting skewed distribution. When this happens, the test is not sensitive to high achievement levels. The test is then not useful for differentiating between moderately high and very high achievement. If the difficulty level of the test is low enough, the test will not be useful for measuring progress toward achievement goals. Students at the lowest achievement levels can still demonstrate achievement gains, but progress will be difficult to demonstrate for a large proportion of the distribution.

This ceiling effect can lead to the false appearance of decreasing achievement gaps. Student groups that start out higher on the distribution will be likely to demonstrate smaller achievement gains than student groups that start out lower on the distribution.

The actual distributions of TAAS exit exam scores demonstrate what can happen with the combination of minimum-competence tests and low difficulty levels. Note that the number of students scoring below the cut point (which is about 40 out of 60 items correct) has decreased substantially over the years in the figure. However, the test results are not able to give us a good picture of what has happened at upper achievement levels. The actual achievement of the top students could be stagnated - or even worse.

These testing phenomena may help explain why TAAS scores demonstrate striking gains while algebra 1 scores are still disappointing and college-entrance scores show no real progress.

TAAS Items from 1995 to 1998

Previously, we examined individual TAAS items for the tests given from 1995 to 1998. We have reported the results in elsewhere in detail (Paul Clopton, Wayne Bishop, and David Klein, "Statewide Mathematics Assessment in Texas", http://mathematicallycorrect.com/lonestar.htm). The key findings will be reviewed here.

The TAAS items were based on the expectations given in the Essential Elements. However, item specifications were written to match Essential Elements covering a three-year span including the test target year and the two prior years. For example, fifth-grade items were written to Essential Elements for grades 3, 4, and 5. The consequence of this is that the items averaged about one year below the target grade level given in the Essential Elements, and this is without commenting on whether or not the Essential Elements represented sufficient academic progress. Worse yet, the item specifications for the exit exam mirrored those for the eighth grade test, making high school graduation essentially dependent on achievement at roughly the seventh-grade level according to the Essential Elements.

We also rated the grade level equivalence of each exit exam test item against the California Mathematics Standards. The new California standards provide a good benchmark because they are quite specific, highly rated among American standards documents, and closely tied to progress in the most successful countries. They were also designed explicitly to support the readiness to study algebra 1 in the eighth grade as a state goal. The results suggest that achievement at the sixth grade level by California standards was sufficient to pass the TAAS exit exam. This supports the notion that the TAAS exams address a low-difficulty level as discussed in the prior section.

We also examined individual items on the algebra 1 end-of-course exams for the same four-year sample of tests. Items were rated on a 5-point scale running from prior to pre-algebra through high-difficulty algebra. We found that most items fell into either the pre-algebra category or the low-difficulty algebra category. Clearly, no more than very modest algebra achievement is required to pass this test.

Perhaps the most important finding came from comparing the estimated grade levels for the eighth-grade exam and the tenth-grade exit exam to the estimated grade level for the algebra 1 end-of-course exam. Although the algebra 1 exam appeared to be at a notably low difficulty level, the jump in grade level achievement required between the eighth- or tenth-grade exams and the algebra exam was striking, roughly two grade levels by California standards. This suggests that the preparation for algebra, as measured by TAAS exams, is likely to be insufficient, leaving students who pass their TAAS exams at a high risk of failure in algebra. As noted above, this continues to be the case in practice. Performance criteria in the grades leading up to algebra are simply not sufficient to support success. Although the TAAS exams are said to be criterion-referenced tests, the performance criteria in these grades were set using what is essentially a normative process based on data that was collected many years ago.

In short, low-difficulty items and low performance expectations in the TAAS tests may now be contributing to the lack of success in introductory algebra.

Other Indicators of Low Difficulty Levels

There are other indicators to support the notion that the TAAS tests set difficulty levels or performance expectations too low. One such indicator is found in a report from the American Federation of Teachers ("Setting Higher Sights: A Need for More Demanding Assessments for U.S. Eighth Graders," Washington, D.C, July, 1998). Their study looked at eighth-grade exams including the TAAS and several other widely used tests. They found that all of the tests contained mostly easy items and a few items of medium difficulty. The TAAS showed up as the weakest among the tests they looked at, although they found none of them to be satisfactory.

It is also possible to compare TAAS scores to scores on other exams when the tests are given to the same students. This figure shows the median national percentile rank for students whose TAAS scores were just at the passing cut point. This gives an approximation of what passing the TAAS means relative to national norms. The result provides further confirmation that the TAAS performance level requirements do not support a sufficient level of continued academic growth across grade levels.

Recalling the progressions of U.S. students from grade 4 to grade 12 in the TIMSS data, these findings suggest that Texas is far from correcting the problem seen in the nation as a whole. If anything, the TAAS may contribute to making this situation worse.

The TEKS-Based TAAS

The newly developed Texas Essential Knowledge and Skills (TEKS) replaced the Essential Elements as the curriculum requirements for the 1998-1999 school year. This year also began a two-year process of phasing TEKS requirements into the TAAS exams. Understanding this process requires a little historical background. When the TAAS was developed, the contents were broken into Objectives based on the broad specifications found in the Essential Elements. Within each Objective, a set of Instructional Targets was developed from the details of the Essential Elements. Items were then written to match the Instructional Targets.

The TEA has provided a brief description of the phase in process in the form of a memo from the Commissioner of Education (http://www.tea.state.tx.us/taa/tasas991025.html). The following attempts to translate that memo in light of the old test design.

In the first year of the phase-in it appears that TEKS specifications replaced the Instructional Targets while the Objectives statements remained unchanged. When the TEKS content was in agreement with content based on the old Instructional Targets, no further changes were required. Items based on TEKS requirements that did not align with the old Instructional Targets were not included in the first year (1998-1999). At least theoretically, some Instructional Targets could have addressed content that was not present in the TEKS, resulting in some items being deleted.

In the second year of the phase-in (1999-2000), content based on the TEKS was to be included even if it was not aligned with the old Instructional Targets. The statements of the test Objectives still remained unchanged although what they mean in practice has been altered since they are now elaborated by TEKS statements rather than the old Instructional Targets.

The language in the memo from the Commissioner talks about EE items and TEKS items as though they were completely different animals, which is probably misleading. No doubt many EE items could serve well as TEKS items. The actual extent of the changes to the nature of the test may be less than the Commissioner's language might suggest.

One often hears that the TEKS are more rigorous than the Essential Elements were. Hopefully, this might help with the low-difficulty problems of the TAAS as discussed above.

However, the performance criteria for the TEKS-based TAAS has not been changed. Therefore, if the TEKS are more rigorous and, consequently, if the TEKS-based TAAS items are more rigorous, then we should expect lower raw scores for the newer exams.

Because the performance criteria have not changed, the equating process should result in lower cut scores as the tests become more rigorous. This point is clearly addressed in the memo from the Commissioner noted above.

We can now address this point directly by examination of the raw score equivalents for the cut points at each TAAS grade level. The cut points are illustrated across grade levels for the 1998, 1999, and 2000 TAAS administrations.

Indeed, as suggested above, the performance criteria for the 2000 examinations represent lower cut scores than were seen in the 1998 exams. Since the performance criteria remain unchanged, this does suggest that the test content is at a more rigorous level. The magnitude of these differences is greatest in grades 6 through 8, which may be appropriate since the older TAAS seemed weakest in the grades leading up to algebra.

On the other hand, the magnitude of these differences is not great. This suggests that the increase in rigor may be less than some have suggested. While this change does mean that the TAAS ceiling effects should be less severe, it is not clear that this reflects substantial improvement in the sensitivity of the tests to higher achievement levels.

Importantly, since the performance criteria have not changed, the utility of this increase in rigor for stimulating greater achievement is questionable. This information suggests that the shift to a TEKS-based TAAS can provide only a slight improvement at best.

Quadratics in the Algebra I TEKS

Quadratic equations with real solutions are a traditional hallmark of Algebra 1. This content should be treated in depth in algebra 1. Unfortunately, this depth of learning is threatened by the weakness of some so-called reform mathematics programs. Attention to quadratics will therefore be given here to address algebra 1 under TEKS.

The standards for this topic area in the Algebra I section of TEKS are given below:

It is interesting to note that the solving of quadratic equations appears only in standard 2A. Even there, where finding solutions actually appears, four methods of solution are given, only one of which is the algebraic solution. Even if testing excludes solution by concrete models, the isolation of algebraic solutions to a fraction of one standard means that they will not receive much attention in testing. Note as well that there is no stipulation with respect to algebraic methods - factoring, completing the square, and the use of the quadratic formula.

The California algebra 1 standards related to quadratics are given below for comparison. Other standards relate to graphing functions and to work with polynomials. Thus, work with quadratics may be included under other California algebra 1 standards. The standards listed below focus on quadratics and their solutions most explicitly.

Note that the California standards are specific about the algebraic solutions of quadratic equations. Unlike Texas, where the quadratic formula is given in the exam, students in California are expected not only to memorize it, but to be able to follow and understand the proof of this formula as well.

The limitations of the TEKS with respect to quadratics are consistent with those throughout the algebra 1 TEKS. Algebraic methods are de-emphasized, especially when difficult. They are also unnecessarily vague, making students, teachers, and test-designers unsure as to the exact expectations. These characteristics may make higher test scores possible, but they will not promote greater achievement in algebra.

Algebra I TAAS Problems Requiring the Solution of Quadratics

This section presents problems from the TAAS Algebra I end of course exam. The items are from the Spring 2000 administration. Items were selected that involve the solutions of quadratics. The Formula Chart, distributed with the exam, includes the quadratic formula. However, it is not needed to solve the problems.

Possible methods of solution are discussed for each item. Note that all problems can be worked by simply substituting in the response choices given in the problem and evaluating the resulting expression. In many cases, this is the easiest method of working these problems.

These items are Copyright © 2000, Texas Education Agency, and available on their web site (http://www.tea.state.tx.us).

4 Which equation best represents the data in the table?

F y = -x^{2 + 2}

G y = x 2 - 2

H y = - x 2

J y = - x 2 + 4

K y = x 2 -4

Students will most likely solve this problem by substituting the tabled x values into the equations, evaluating, and comparing the result to the tabled y values. The easiest substitution is to start with x = 0. This one substitution gives the answer directly.

 

 

 

5 At which points does the graph of f(x) = 2x2 + 6x + 4 intersect the x-axis?

A (-4, 0) and (-1, 0)

B (-2, 0) and (-1, 0)

C (1, 0) and (2, 0)

D (-2, 0) and (2, 0)

E (2, 0) and (4, 0)

All points in answer choices are on the x-axis as required by the problem.

No doubt a very few students will actually factor the function expression to get (2x + 2)(x + 2), which gives the solution (B) directly.

Most students will solve the problem by substituting the x values in the answer choices. A quick look at the function shows that x cannot be positive. This only leaves answer choices A and B. Thus, only one trial value, say x = -2, must be plugged in to ascertain the correct answer.

 

 

8 If a sports league has t teams and each team plays all the other teams twice, the total number of league games, g, is given by this function.

g = t2 - t

The High Plains Softball League plays a total of 72 games, with all teams playing each other twice. How many teams are in the league?

F 7

G 8

H 9

J 10

K 11

The difficult way to solve the problem is algebraically. Setting g = 72 gives72 = t2 - t so t2 - t - 72 = 0

This can be solved by the quadratic formula (given), or by factoring to (t - 9)(t + 8) and knowing that -8 cannot be the solution.

Most students will solve the problem by substituting in the answers for t and evaluating until the result is 72.

10 The area of an isosceles right triangle is described by the equation

x2 = 576

where x is the height in centimeters of the triangle. What is the height of the triangle?

F 12 cm

G 24 cm

H 96 cm

J 192 cm

K 288 cm

This problem asks for nothing more than the square-root of 576. One could start squaring each answer, stopping at the second choice. However, 576 is between 400 and 900, so the answer must be between 20 and 30 which leaves only one choice.

 

 

13 What are the solutions to this equation?

x2 + 2x + 8 = 4 - 3x

A x = -3 or x = -4

B x = 3 or x = 4

C x = -1 or x = -4

D x = 4 or x = -3

E x = 4 or x = 1

The simplest solution to this problem is to simplify to x2 + 5x + 4 = 0 and then factoring to get (x + 1)(x + 4), which gives the solution (C) directly.

For those who don’t know how to factor, or choose not to, the answers can be tested by substitution, with or without simplifying first. The easiest trial substitutions are 1 (choice E) and -1 (choice C), which will yield the correct answer.

 

 

 

16 The longer leg of a right triangle is 3 inches longer than x, the length of the shorter leg. The hypotenuse is 15 inches long. The following equation shows the relationship between the sides of the triangle.

x2 + (x + 3) 2 = 225

What is the length of the shorter leg?

F 6 in.

G 9 in.

H 12 in.

J 18 in.

K 54 in.

Those who start out by simplifying the left side will find 2x2 + 6x + 9 = 225. Getting this far, it becomes obvious that some work will be required to factor or to use the quadratic formula. Thus, even these students will probably elect to substitute in the answers. After trying the second choice the answer will be known.

A fair number of students will recognize 225 as 15 squared. Having worked Pythagorean Theorem problems, they may quickly recognize 92 + 122 = 152 which is an instance of the 3-4-5 special triangle that appears in many problems. These students will essentially be able to find the correct answer by direct recall.

 

26 What are the solutions to this equation?

2x2 + 7x - 15 = 0

F x = -5 or x = 1.5

G x = -3 or x = 2.5

H x = -2.5 or x = 3

J x = -1.5 or x = 5

K x = -5 or x = 3

The easiest solution to this problem is likely to factor it to (2x - 3)(x + 5) to give the solution.

The quadratic formula can also be applied without much difficulty since the term under the radical is 169, a perfect square.

For those who elect to solve by substitution, the easiest evaluations will focus on the integral values of x. Trying 3 and 5 rules out the last three choices. Then trying either -3 or -5 will give the correct answer without ever squaring 1.5 or 2.5.

 

30 The area of a rectangle is given by the equation

2x2 = 450

where x is the width of the rectangle. What is the width?

F 5

G 9

H 15

J 25

K 30

This problem can easily be solved by substitution. Many will simplify to x2 = 225 and recognize the perfect square immediately.

 

 

 

 

 

The review of test items is informative. There aren't many problems to begin with, so students could miss them all and still pass the exam. Much more troubling is the fact that students do not need to know how to solve quadratic equations at all to find the correct answers to the test items. All they need to know is how to substitute the answer choices into the problem statement and evaluate the result. In fact, given the nature of the test items, this is a reasonably efficient strategy as well.

Students who practice evaluating expressions for given replacement values of the variables could do quite well on these items. Unfortunately, this means that teachers could address only numerical evaluation of expressions and skip quadratics altogether and still have students do fairly well. Of course, most teachers would not use this tactic, but it is frightening nonetheless and plays into the criticism of "teaching to the test."

Evaluating the Grade 8 TAAS Examination

The eighth-grade TEKS standards must serve as the culmination of the preparation for algebra if students are to be ready for algebra by ninth grade. To further understand how ready-for-algebra students are expected to be after eighth grade, the eight-grade TAAS examination from 2000 was evaluated in detail.

The eighth-grade TAAS exam item content is summarized below:

A great majority of the problems are "word problems" while virtually none present equations with variables or even arithmetic expressions.

There are many items requiring the addition or subtraction of money amounts in decimal notation, or their multiplication or division by small integers.

There are many problems requiring interpretation or working with data in graphs and tables.

There are many proportion problems and simple rate problems, but these are only preliminary to linear functions (e.g., slope and intercept are not required)

There are some area and perimeter and simple volume problems.

A few problems involve perspective or two-dimensional drawing interpretation.

A few problems involve mean or median.

There are some problems that involve work with decimals (other than money), fractions, and percents, although these are generally at relatively low difficulty levels.

One problem addresses representing a value in scientific notation.

Two problems address the unknown length of the side of a triangle. These can be solved using the Pythagorean theorem (given on the formula sheet), but it is not needed to solve the problems since the correct answers are also the most reasonable guesses and one is a "special triangle."

Two problems ask the student to identify the correct simple equation or inequality (with variables) to match a given situation.

One problem asks the student to identify the correct numeric expression (no variables) to match a given situation.

 

The following topics were not addressed by the exam:

Identifying irrational numbers or differentiating between rational and irrational numbers.

Differentiating between terminating and repeating decimals

Computing with simple and compound interest.

The use of exponents, powers, and roots, such as:

working with fractions using exponent rules

the meaning and use of negative exponents

multiplying and dividing exponents with a common base

approximating a square root (although some might do so on the triangle problems).

Correct use of order of operations and parentheses (although some students might do so in working some of the word problems).

The meaning and use of absolute value.

The properties of rational numbers (identity, inverse, distributive, associative, commutative).

Early work with variables, including manipulations with monomials

Linear functions.

Basic work with nonlinear functions.

 

In California, the grade 7 standards serve as the culmination point for the preparation for algebra. Some of the California grade 7 standards that do not appear to be required for the eighth-grade TAAS include:

Number Sense

1.4 Differentiate between rational and irrational numbers.

1.5 Know that every fraction is either a terminating or repeating decimal and be able to convert terminating decimals into reduced fractions.

1.7 Solve problems that involve discounts, markups, commissions, and profit and compute simple and compound interest.

2.1 Understand negative whole-number exponents. Multiply and divide expressions involving exponents with a common base.

2.2 Add and subtract fractions by using factoring to find common denominators.

2.3 Multiply, divide, and simplify fractions by using exponent rules.

2.4 Use the inverse relationship between raising to a power and extracting the root of a perfect square integer; for an integer that is not square, determine without a calculator the two integers between which its square root lies and explain why.

2.5 Understand the meaning of the absolute value of a number; interpret the absolute value as the distance of the number from zero on a number line; and determine the absolute value of real numbers.

Algebra and Functions

1.2 Use the correct order of operations to evaluate algebraic expressions such as 3(2x+5)2

1.3 Simplify numerical expressions by applying properties of rational numbers (e.g., identity, inverse, distributive, associative, commutative), and justify the process used.

2.1 Interpret positive whole-number powers as repeated multiplication and negative whole-number powers as repeated division or multiplication by the multiplicative inverse. Simplify and evaluate expressions that include exponents.

2.2 Multiply and divide monomials; extend the process of taking powers and extracting roots to monomials when the latter results in a monomial with an integer exponent.

3.1 Graph functions of the form y = nx2 and y = nx3 and use in solving problems.

3.2 Plot the values from the volumes of three-dimensional shapes for various values of the edge lengths (e.g., cubes with varying edge lengths or a triangle prism with a fixed height and an equilateral triangle base of varying lengths).

3.3 Graph linear functions, noting that the vertical change (change in y-value) per unit of horizontal change (change in x-value) is always the same and know that the ratio ("rise over run") is called the slope of a graph.

3.4 Plot values of the quantities whose ratios are always the same (e.g., cost to the number of an item, feet to inches, circumference to. diameter of a circle). Fit a line to the plot and understand that the slope of the line equals the quantities.

The missing material could largely be characterized as:

More difficult work with fractions and percents.

Understanding and working with powers and roots and representations with exponent.

Understanding and using absolute values.

Manipulation of simple equations and the understanding the properties of numbers.

Understanding and using linear functions, including work with slopes and intercepts.

Introductory work with non-linear functions.

Are the California objectives unreasonably high? Or, do they reflect material that might reasonably be covered as the final preparation for algebra? While these questions have been and will continue to be the subject of debate, there is one simple answer to the question. The California expectations include topics that are covered in a reasonably solid pre-algebra text. Reasonable success in a solid pre-algebra program should reflect learning in these content areas. Furthermore, reasonable success in a solid pre-algebra program is, no doubt, the strongest positive predictor of success in algebra. Thus, the objectives recommended just before algebra in California do appear to be reasonable.

This analysis suggests that the knowledge and skills required by the grade 8 TAAS exam are not consistent with the expectations that would promote success in algebra. In short, the content reflected in the eighth-grade TAAS exam is not sufficient to assure, with any degree of certainty, that students will succeed when they attempt algebra. This finding is entirely consistent with the actual experiences of many Texas students.

Evaluating the K-8 TEKS for Number Sense and Early Algebra

As the forgoing section may already suggest, at least part of the weakness of the newest grade 8 TAAS exam is mirrored in weaknesses in the TEKS. The weakness is evident both in level and in specificity.

In many areas, the TEKS fall further and further behind the California expectations as grade levels increase. It is not uncommon for the TEKS to fall two years behind. Work with fractions is a good example.

In other areas, the lack of explicit statements in the TEKS for certain content areas puts those contents at risk. No where is this more evident than in the treatment of the properties of numbers. While California is quite explicit about what is to be learned and when, the TEKS contains only vague suggestions of this content in standards dealing with patterns.

To trace the development of number sense and some early algebra content across grade levels, the expectations in the TEKS were compared to California, Singapore, Japan, and the local San Diego expectations across several content areas. These comparisons are summarized in Appendix C. The summary will provide further clarification of the differences noted above.

The Move to TAAS II - SB 103

With the passage of Senate Bill 103, Texas is headed for another redesign of the assessment system, now dubbed TAAS II. Introductory documents are provided on the TEA web site (http://www.tea.state.tx.us), although this is clearly a work in progress.

A legislative summary of SB 103 is provided (http://www.tea.state.tx.us/brief/doc2.html), which says, in part:

SB 103 expands the statewide student testing program. The bill maintains the grade levels and subject areas currently tested with the following changes. Exit-level testing required for high school graduation is moved from grade 10 to grade 11 and will ... include at least Algebra I and geometry with the aid of technology. ... The bill further specifies that the grade 11 test measure mastery of minimum skills necessary for high school graduation and readiness to enroll in higher education but does not require students to demonstrate readiness for higher education in order to graduate. ... the bill requires assessment of ... mathematics ... at grade 10; adds assessment of mathematics ... to grade 9;... In mathematics, the bill mandates the use of technology on the mathematics tests at grades 8 through 10 where algebra is assessed. In addition, the bill eliminates end-of-course examinations in Algebra I

The TAAS II will still be TEKS based, but some significant changes are coming as a result of the legislative mandate.

The proposed objectives for TAAS II have been released by the TEA and are currently being reviewed. The plan for TAAS II maps out six objectives through grade 8. Each objective is mapped to TEKS statements for the corresponding grade. These may be casually identified as:

1) number sense

2) patterns and algebraic reasoning

3) geometry

4) measurement

5) probability and statistics

6) processes and tools

The sixth category is really a process category rather than one that addresses actual content. As in most attempts to stipulate process objectives, these show little differentiation from one grade level to the next. The other five objectives show progress from grade to grade.

According to the plan as it is unfolding, these six objectives map to ten objectives in high school. For grade 9, the number sense and patterns and algebraic reasoning strands are replaced with five objectives covering the expectations from the Algebra I TEKS. The same expectations are then repeated in grades 10 and 11.

The grade 8 expectations for objectives 3 and 4 map to objectives 6 to 8 in grades 9 and 10, and the eighth grade criteria are essentially replicated for these two years. In grade 11 these strands are replaced by statements mapped to the TEKS geometry standards.

Objectives 5 and 6 do not have corresponding high school standards so, with one exception, the eighth-grade expectations are simply repeated all the way through the 11th grade (exit) exam.

The transition of the objectives into the high-school exams is summarized in the following table:

Mapping Objectives Across Grade Levels for TAAS II

The TEA is currently surveying educators about the proposal to assess TEKS expectations at each of these grade levels. For each objective and at each grade level, two questions are being asked:

Based on the recommendations you have made, if any, is this objective essential to measure on TAAS II?

Will students have received enough instruction on these TEKS student expectations by the spring of the school year to be adequately prepared to demonstrate mastery of this objective?

The survey raises several questions. What if a large number of educators respond in the negative? Is it possible that entire objectives will be eliminated from the assessment? There is a viable concern that some of the content within a given objective may not be covered by the time of the spring assessment. Eliminating some objectives altogether seems rather extreme.

It is also unclear why the legislative summary refers to algebra assessment in grade 8 but the proposed objectives do not introduce it until grade 9. The use of technology (calculators) is stipulated at grade 8 because of the inclusion of algebra, yet the proposed objectives still reflect only the same grade 8 TEKS that are used in the current TEKS-based TAAS exam.

Perhaps the biggest questions show up in the high school exams. Although the minimum requirements for high school include algebra and geometry, there is no stipulation as to the grade levels in which these courses will be completed. Since some students will not complete algebra 1 in grade 9, there may be questions about opportunity to learn that come into play. On the other hand, it is not clear what specific algebra standards may be deemed testable for each grade from 9 to 11. Is it reasonable to test a little more of algebra 1 each year? If so, what are the curricular implications of such a test design? What happens to students who complete algebra 1 in grade 8?

The grade 11 exam serves as the high school exit exam. The design is likely to require that only a limited portion of algebra will be required. If so, what parts of algebra are not required for graduation? What parts of geometry will not be required?

The questions on the TEA survey are related to opportunity to learn issues. Unfortunately, this approach can enshrine pre-existing deficiencies - material not covered in the past will not be tested, decreasing the probability that it will ever be covered.

At this point, it is difficult to predict what may happen with the objectives and the TEKS standards in TAAS II. It does appear, however, that the tentative plan reflects the highest level of expectations that will be evaluated with TAAS II.

Parting Thoughts

The assessment of mathematics achievement in Texas is at a transition point and at this point there seem to be more questions than answers. The consequent implications for the prospects of improving mathematics achievement statewide are also unclear. The forgoing presentation can provide a few clues.

The current pass rate for the end-of-course algebra exam is too low to be socially acceptable as a requirement for high school graduation. This undoubtedly means that the test items, the performance criteria, or both will be weakened rather than strengthened. This, despite whatever the resulting test scores might be, should be taken as a threat to real algebra success in Texas.

Without changes to item difficulty, content coverage, and performance criteria in the grades leading up to algebra, it is unlikely that TAAS II will be of great help in resolving the lack of readiness for algebra that appears to be a problem in Texas. This should be taken as a threat to continued achievement gains.

There is every indication that an emphasis on minimal-competence will continue. It will take a concerted effort on the part of all involved to incorporate a broader range of achievement into the sensitivity range of the new exams. Without doing so, Texas will continue to be blind to higher achievement levels.

Just as the exit exam is at risk of addressing an even weaker variant of algebra than the current end-of-course exam, there is every reason to believe that the geometry items will also have to be at a low level in order to get the pass rate high enough. This may threaten the content of the geometry courses.

For practical reasons, the exit exam can only address a subset of the expectations for algebra, geometry, and the other TEKS carried over from grade 8. Once the test design specifications (blueprints) become known, there is a risk that large portions of the learning objectives will be considered unimportant.

It is difficult to imagine how the inevitably watered-down assessments of algebra and geometry will function effectively as a college-readiness exam as stipulated by the legislation. Many institutions of higher learning in Texas already require algebra 2 in high school, and more will soon. This content, however, is not in the current plan. The legislation clearly allows college-readiness items on the grade 11 test that are not part of the score used as the exit criteria, but the current plans do not seem to take advantage of this fact.

In short, TAAS II may not help improve actual mathematics achievement in Texas. There is some room for hope in the option for college-readiness items. There is even room for hope since the TEKS are sufficiently vague as to allow richer content to be addressed on the examinations. Yet, despite a few rays of hope, there are good reasons to fear that the problems with TAAS will not disappear with the advent of TAAS II.

Recentering of SAT Scores

In April of 1995 the College Board began using a new scoring scale for the SAT. The adjustment to the scoring procedure has been called "recentering," since part of the intention is to move the means for both the Verbal and Math (Quantitative) portions of the test back to 500. There are practical implications to the recentering of scores. The most obvious is that the new scores are not equivalent to the old ones, and care must be taken in interpretation. To illustrate this, SAT means for California over five years are as follows:

California Mean SAT Scores

The total SAT mean score in California has risen nearly 100 points, which is very similar to the change nationally. Any district, school, or other group that looks at changes in SAT scores between 94-95 and 95-96 without adjusting for the recentering process is either unintentionally fooling themselves or intentionally trying to fool others. Such claims are not unheard of.

Stability of School Mean SAT Scores Across Years

One way to look at the changing behavior of the SAT scores is to examine the consistency of school mean scores across years. This method looks at scores for each school relative to other schools, so that adding a constant value to one year would have no effect. The year-to-year correlations of the school mean SAT scores are presented are given below based on 731 California high schools:

Year to year correlations for verbal (above diagonal) and math (below diagonal) scores

It is evident from this table that the correlations with prior years declined for the 95-96 data. For example, the correlation with the preceding year for verbal scores was .907 in 94-95 but only .776 in 95-96. In math, this correlation dropped from .924 for the 94-95 data to .809 for the 95-96 data. These differences, although statistically significant, may not seem very large. However, it should be remembered that the proportion of common variance, or overlap, is computed by squaring the correlation coefficient. Thus, the overlap with the prior year's verbal scores dropped from 82.3% in 94-95 to 60.2% in 95-96. Likewise, the overlap with the prior year's math scores dropped from 85.4% in 94-95 to 65.4% in 95-96.

It would seem extremely unlikely that this sudden change in the consistency of scores from year to year could be accounted for by any sudden real changes in abilities and achievement in the schools. Nor is there any indication that sudden changes in the proportion of students taking the SAT exams could account for this difference.

Is it possible that recentering could account for this finding? If the recentering process was linear, then there would be no effect at all on year-to-year correlations. It may be the case that the non-linearity in the renorming process contributed to this sudden decrease in consistency from year to year. However, the degree of non-linearity in the recentering process is fairly small and thus the correlation between old and new (recentered) scores for any one test would have to be very high.

Thus, it seems unlikely that recentering can account for much of this sudden shift in the consistency of school mean scores. What then can account for this shift? It appears that the 95-96 exam is qualitatively different in the abilities and achievement being tested.

Correlations Between School Means for Verbal and Math SAT Scores

Another way to examine the behavior of SAT scores is to look at the relationship between the verbal and math scores. High correlations are expected between these two areas. On the other hand, if the correlations are extremely high it would cause us to question whether or not there is any distinction between the content domains measured by the two subtests. The correlations between the school means for verbal and math scores over the last five years are given below:

Correlations between school mean verbal and math scores for 731 high schools

Reading down the diagonal of the table gives the correlation of school mean verbal and math scores in the same year. It can be seen that the value has increased each year. The biggest increase is from the 94-95 correlation of .890 to the .930 correlation for the 95-96 school year. Again, it must be recalled that the proportion of overlap is the square of the coefficient. The percentage of overlap changed from 79.2% in 94-95 to 86.5% in 95-96. Thus, it appears from these values that the scores in the two content areas now have more in common. Whether this degree of overlap is too much and the distinction between verbal and math achievement has become threatened is subject to debate. The correlations for individual students rather than school means should be considered in addressing this question, as will as correlations with other tests.

The off-diagonal sections of the table show the correlations of the school means for math scores in one year with the verbal scores in another year, and vice versa. The cross-year correlations are consistently lower for the 95-96 scores. While the correlation between verbal and math for the same year has increased in the 95-96 data, correlations across both areas and years have gone down. Again, it is difficult to conclude that this finding could be a function of recentering. This leads to the speculation that there have been qualitative differences in the specifics of achievement addressed by both the verbal and math tests for 95-96.

Qualitative Changes to the SAT

The recentering process is purely objective and based on data. But other changes in the examination have occurred as well. The fact that both of these changes are happening at the same time leads to further confusion. In her piece on SAT restructuring (http://www.mdn.org/1996/STORIES/SAT2.HTM), Elizabeth McKinley quotes Brad Quin, associate director of SAT, as saying that those who question the recentering are " political pundits who are lamenting the demise (of the old style test) and can't look beyond their noses." She also notes that Jennifer Marshall of the Family Research Council suggested that the restructuring of the SAT to fit restructured classrooms nullifies the test's objectivity.

In a discussion of changes in the SAT, Kaplan Educational Centers (http://www.kaplan.com/) noted that the newer version stresses critical reading and thinking skills, and now permits the use of calculators. The discussion promoted by the recentering process has complicated any real understanding of the qualitative changes to the SAT.

Elementary Mathematics Comparisons

The following tables illustrate grade level specifications for various early achievements in mathematics, focusing primarily on number sense and early algebra. Related Texas standards are listed following each table.

The sources used are:

Texas: Texas Essential Knowledge and Skills (TEKS), September 1, 1998.

California: Mathematics Content Standards for California Public Schools, adopted December 11, 1997.

Singapore: Objectives of the Primary Mathematics Curriculum, Primary 1 to Primary 6 (Normal Course)

Japan: Mathematics Program in Japan, (Kindergarten to Upper Secondary School), Research Center for Science Education, National Institute for Educational Research, Tokyo, Japan, October, 1989

San Diego: Mathematics Content and Performance Standards, draft dated September, 1997. Subsequent revisions have been made but are not reflected here.

Disclaimer: The grade level specifications identified here were developed on short notice and often required making a educated guess. The material for Japan in particular is very brief and is sometimes unclear in translation. It is likely that some items that are not indicated for Japan are actually covered there. In other cases, the wording in a document may not match closely enough with the items listed, and again some inferences were required. Where matches were particularly difficult, a "?" appears by the grade-level indicator in the tables.

Reading and Writing Whole Numbers

Note: California does not address larger numbers in the same manner as the typical number sense standards. Instead, in Grade 5 students are expected to "estimate, round, determine, and interpret the meaning of very large (e.g., millions) ... numbers."

Texas

read and write numbers to 99 to describe sets of concrete objects. (1.01D)

use concrete models to represent, compare, and order whole numbers (through 999), read the numbers, and record the comparisons using numbers and symbols (>, <, =). (2.01A)

use place value to read, write (in symbols and words), and describe the value of whole numbers through 999,999; (3.01A)

use place value to read, write, compare, and order whole numbers through the millions place (4.01A)

Addition and Subtraction of Whole Numbers

Texas

model and create addition and subtraction problems in real situations with concrete objects. (K.04A)

model and create addition and subtraction problem situations with concrete objects and write corresponding number sentences (1.03A)

learn and apply basic addition facts (sums to 18) using concrete models. (1.03B)

select addition or subtraction and solve problems using two-digit numbers, whether or not regrouping is necessary (2.03B)

solve subtraction problems related to addition facts (fact families) such as 8 + 9 = 17, 9 + 8 = 17, 17 - 8 = 9, and 17 - 9 = 8. (2.05D)

use patterns to develop strategies to remember basic addition facts (2.05C)

recall and apply basic addition facts (sums to 18); (2.03A)

model addition and subtraction using pictures, words, and numbers (3.03A)

select addition or subtraction and use the operation to solve problems involving whole numbers through 999. (3.03B)

use addition and subtraction to solve problems involving whole numbers (4.03A)

Comments

Modeling of addition and subtraction occurs in grades K, 1, and 3.

Addition facts are learned and applied in grade 1, but recalled and applied in grade 2.

Inverse relationship, such as use to check results, is unclear.

Whole Number Multiplication and Division

Texas

model, create, and describe multiplication situations in which equivalent sets of concrete objects are joined (2.04A)

solve and record multiplication problems (one-digit multiplier) (3.04B)

identify patterns in multiplication facts using concrete objects, pictorial models, or technology (3.06B)

identify patterns in related multiplication and division sentences (fact families) such as 2 x 3 = 6, 3 x 2 = 6, 6 ÷ 2 = 3, 6 ÷ 3 = 2. (3.06C)

learn and apply multiplication facts through the tens using concrete models; (3.04A)

uses patterns in multiplication and division. (4.06)

recall and apply multiplication facts through 12 x 12; (4.04C)

represent multiplication and division situations in picture, word, and number form; (4.04B)

use multiplication to solve problems involving two-digit numbers (4.04D)

use patterns to develop strategies to remember basic multiplication facts; (4.06A)

solve division problems related to multiplication facts (fact families) such as 9 x 9 = 81 and 81 ÷ 9 = 9 (4.06B)

use division to solve problems involving whole numbers (no more than two-digit divisors and three-digit dividends without technology); (5.03C)

use multiplication to solve problems involving whole numbers (no more than three digits times two digits without technology); (5.03B)

Comments

Models for multiplication are used in grades 2 to 4, while division is modeled in grade 4.

Multiplication facts are learned and applied in grade 3, but recalled and applied in grade 4 where patterns are used to develop strategies to remember them.

Multiplication by a single-digit is targeted for grade 3, but the size of the other factor is not noted and probably would not be taken to a 3- or 4-digit number.

The use of the inverse relationship between multiplication and division is unclear, showing up in patterns but without mention of applications such as checking results.

The only division that occurs in grade 4 is taken directly from facts, so actual computations do not appear until grade 5.

Operations with Money Units

Note: The Japan Standards do not make reference to money. It may be the case that these sections were removed during translation, as their monetary units are dissimilar to ours.

Texas

use words and numbers to describe the values of individual coins such as penny, nickel, dime, and quarter and their relationships (1.01C)

determine the value of a collection of coins less than one dollar. (2.03C)

determine the value of a collection of coins and bills. (3.01C)

use place value to communicate about increasingly large whole numbers in verbal and written form, including money. (3.01-)

use place value to read, write, compare, and order decimals involving tenths and hundredths, including money, using concrete models. (4.01B)

list all possible outcomes of a probability experiment such as tossing a coin; (4.13A)

Comments

The TEKS do not address operations with monetary units. Comparisons (grade 4) may imply subtraction. California standards for multiplication and division use whole number multipliers and divisors.

Decimal Number Sense: Read, Write, Compare, Order, and Place Value

Texas

use place value to read, write, compare, and order decimals involving tenths and hundredths, including money, using concrete models. (4.01B)

use place value to read, write, compare, and order decimals through the thousandths place. (5.01B)

Addition and Subtraction of Decimals

Texas

add and subtract to solve meaningful problems involving whole numbers and decimals (4.03-)

use addition and subtraction to solve problems involving whole numbers and decimals (5.03A)

use addition and subtraction to solve problems involving fractions and decimals (6.02B)

Comments

The TEKS to not specify the ranges of decimals involved in these operations. However, grade 4 students represent them through hundredths and grade 5 students represent them through thousandths, so these may be the ranges used in these operations as well.

Decimal Multiplication and Division

Texas

represent multiplication and division situations involving fractions and decimals with concrete models, pictures, words, and numbers; (7.02A)

use addition, subtraction, multiplication, and division to solve problems involving fractions and decimals; (7.02B)

Comments

Although the TEKS does not include the money model explicitly, it would appear to fall under the grade 7 standards for modeling multiplication and division with decimals.

Use of Fractions

Texas

use fraction words to name parts of whole objects or sets of objects (2.02)

name fractional parts of a set of objects (not to exceed twelfths) when given a concrete representation. (2.02B)

name fractional parts of a whole object (not to exceed twelfths) when given a concrete representation (2.02A)

use fraction names and symbols to describe fractional parts of whole objects or sets of objects. (3.02)

construct concrete models of fractions; (3.02A)

compare fractional parts of whole objects or sets of objects in a problem situation using concrete models; (3.02B)

use fraction names and symbols to describe fractional parts of whole objects or sets of objects with denominators of 12 or less (3.02C)

construct concrete models of equivalent fractions for fractional parts of whole objects. (3.02D)

locate and name points on a line using whole numbers and fractions such as halves. (3.10A)

describe and compare fractional parts of whole objects or sets of objects. (4.02)

generate equivalent fractions using concrete and pictorial models; (4.02A)

model fraction quantities greater than one using concrete materials and pictures; (4.02B)

compare and order fractions using concrete and pictorial models (4.02C)

relate decimals to fractions that name tenths and hundredths using models. (4.02D)

locate and name points on a number line using whole numbers, fractions such as halves and fourths, and decimals such as tenths. (4.10A)

use fractions in problem-solving situations. (5.02)

generate equivalent fractions; (5.02A)

compare two fractional quantities in problem-solving situations using a