The Lone Star Foundation Conference
Public Education Reform in Texas: Comprehensive Progress Report
Austin, Texas - December 7-8, 2000

A DECADE OF TAAS

WHAT HAVE WE LEARNED

CARL SHAW

PREPARED ESPECIALLY FOR

PUBLIC EDUCATION REFORM IN TEXAS

A COMPREHENSIVE PROGRESS REPORT

Austin, TX 78758-4457

A SYMPOSIUM HELD AUTHOR

December 7 And 8, 2000 Carl Shaw

The Driskill Hotel 501-756-8315

Austin, Texas

ATTACHMENTS*

CRITERION REFERENCED TESTS

CONFLICTS IN EDUCATIONAL POLICY

CARL SHAW

PUBLIC EDUCATION REFORM IN TEXAS

A COMPREHENSIVE PROGRESS REPORT

Austin, TX 78758-4457

FOR POLITICAL GAIN?

CARL SHAW

PUBLIC EDUCATION REFORM IN TEXAS

A COMPREHENSIVE PROGRESS REPORT

Austin, TX 78758-4457

REFERENCED TESTS

CARL SHAW

PUBLIC EDUCATION REFORM IN TEXAS

A COMPREHENSIVE PROGRESS REPORT

Austin, TX 78758-4457

501-756-8315

A DECADE OF TAAS

WHAT HAVE WE LEARNED

"…They probably did it for the same reasons that the engineers let the space shuttle disaster happen - powers above them failed to heed their warnings. They chose to remain publicly silent until the bodies were recovered from the sea…"

The thesis for this Lone Star Foundation presentation finally addresses a question that has puzzled many people for years: why have so many testing experts armed with knowledge of the truth allowed the TAAS testing program to degenerate its current state of academic fraud.

The conclusions drawn are based upon a decade of "inside the tent" awareness of the games self-serving educators have played with the lives of children and the pocketbooks of taxpayers. It is as if the political establishment ordered the education bureaucracy to "make us all look good".

The following is just one example of hundreds that seem to be calculated to distort the true status of academic attainment in Texas.

A magazine cost $3.75. Lenny gave the clerk $20 for the magazine. How much change should Lenny have received from the clerk?

At a restaurant Steve ordered food totaling $6.85. If he paid with a $20 bill, how much change should he receive?

Analogous to the shuttle, we have destroyed the educational lives of children and have left them afloat in the sea of academic mediocrity. We choose to leave the children floundering and instead reward those who have conspired to create the tragedy.

Lone Star Foundation, December 7, 1999^{th grade question 97-98}

A magazine cost $3.75. Lenny gave the clerk $20 for the magazine. How much change should Lenny have received from the clerk?

At a restaurant Steve ordered food totaling $6.85. If he paid with a $20 bill, how much change should he receive?

Analogous to the shuttle, we have destroyed the educational lives of children and have left them afloat in the sea of academic mediocrity. We choose to leave the children floundering and instead reward those who have conspired to create the tragedy.

Lone Star Foundation, December 7, 1999

A DECADE OF TAAS

WHAT HAVE WE LEARNED?

Most of the inside information I gained on the TAAS was gained as Director of Research and Evaluation in Houston and Ft. Worth ISD until I retired in 1997. I have served on too many state and national policy committees on testing to remember. I have made literally hundreds of trips to Austin for committee meetings. Since then I have been an observer of the Texas Legislature, the Texas Education Agency, the Windham education program, the Juvenile Justice programs in Dallas and Harris counties, private vendors serving those counties, and the Texas Juvenile Probation Commission.

Additionally, I observed the aborted attempts to implement basic skills qualifications testing for in-service teachers and to implement a career ladder program. Since this discussion is to focus on the TAAS today’s observations defer any discussion of teacher evaluation until the recommendations section. In the final analysis TAAS has come to be the only variable of importance in the accountability system of Texas.

I divide the history of TAAS into three overlapping eras. First, there was the Inception phase during which the tests were defined, drafts written and trial forms administered. Second, there was the Implementation phase during which the TAAS became a routine in the schools and the rules on TAAS interpretation were widely disseminated. Third, there is the Institutional phase that is still evolving into an even larger scale program.

INCEPTION

The need for a program like TAAS was obvious. Houston and Dallas ISD had data that high school students had extremely low achievement in their content courses. The nationally normed achievement data showed extremely low basic skills levels were widespread, especially in grades 5 through 9. Too many students were not finishing high school. The inception phase was one of great excitement to those involved in defining the TAAS. The legislature set up a framework in which the test was to operate. Students were to be tested in math, reading and writing. Students were required to pass the Exit TAAS to receive a high school diploma. Students who failed were to receive remediation, study guides were to be printed and score reports were to be sent to parents. TEA convened committees who met and defined what may very well be the best set of state test specifications ever written. Each school in the State of Texas became involved with the field test administration. Philosophical perspectives were discussed and there was a general feeling that the education of our youth was getting a great boost by the statewide process taking place. There was an openness and frankness in the meetings that permitted the sharing of thoughts.

It had been generally agreed what each and every student should be expected to learn at each grade level. A test would be would be written to see if the students had attained the grade skill levels. The test would be a criterion-referenced test. In a criterion-referenced test of educational attainment the contents the test is to measure are defined and explicitly defined to teachers and students. Standards are set and students who had received proper instruction and profited from it should pass the test. After all, everybody must have high levels of basic skills. There would be no judging of the relative attainment of students one to another. In other words the bell-shaped distribution and the relative attainment levels were not relevant so norms would not be established.

Large and numerous committees of teachers and TEA staff members met to judge the questions written to the test standards they had produced. TAAS was truly built with large-scale input from thousands of teachers. Teachers had played a very major role in its formulation.

IMPLEMENTATION

The Texas Education Agency contracted to have the tests written. The Texas Education Agency contracted to have the tests printed. The Texas Education Agency contracted to have the tests distributed. The Texas Education Agency contracted to have the tests scored. The Texas Education Agency contracted to have the results calculated. The Texas Education Agency contracted to have the results reported.

The Texas Education Agency brought large committees of teachers in to review the results of pilot tests questions. The Texas Education Agency helped the committees decide which questions could appear on the next test. The Texas Education Agency told the test construction contractor if they had put the right questions on the tests.

The Texas Education Agency decided on the rules for testing. Texas Education Agency TAAS staff trained Education Service TAAS Coordinators on the testing rules. Education Service TAAS Coordinators trained District TAAS Coordinators on testing rules. District TAAS Coordinators trained School TAAS Coordinators on testing rules. School TAAS Coordinators trained individual TAAS Test Administrators (teachers) on testing rules. TAAS Test Administrators (teachers) trained the students on testing rules. All of the above were responsible for fair and accurate test results.

Thousands upon thousands of calls came in to District TAAS Coordinators from the School TAAS Coordinators requesting specific details on who could be eliminated from testing based on the Special Education, the Limited English Proficiency rules, the length of enrollment rules, and the enrollment in special juvenile facility rules. The Texas Education Agency seemed to change each of these rules each year. Often it seemed like the Texas Education Agency changed the rules long after there was any time to implement the rules. More committees met to massage the exemption rules. TEA changed the exemption rules several times in some years. TEA didn’t always make the rules needed in the field. TEA didn’t always supply all of the tests mandated by the legislature. The schools were unable to provide adequate remedial instruction.

The test booklets and all related material arrived in ever increasing numbers as the years progressed. Semi-truck loads of secure test material sometimes showed up at the Houston ISD Test Materials Center. Sometimes a single semi transported the answer documents to Iowa for scoring to help meet deadlines. Once a small airplane delivered the results to Dallas and Houston so the results would meet the Friday deadline (they arrived at about 6:30 on a Friday evening). The District TAAS Coordinator worked all weekend to get results processed, as smaller districts around the county had already released their results. TEA spokespersons told others in the district that the result had all been released. TEA and the vendor certainly went out of their ways to fulfill their obligations to the letter. In fact the national contractor has now opened major facilities in Austin and has transferred dozens of people from Iowa to Austin.

Texas Education Agency TAAS staff trained Education Service TAAS Coordinators on the use of the TAAS Test Specifications. Education Service TAAS Coordinators trained District TAAS Coordinators on the use of the TAAS Test Specifications. District TAAS Coordinators trained School TAAS Coordinators on the use of the TAAS Test Specifications. School TAAS Coordinators trained individual TAAS Test Administrators (teachers) on the use of the TAAS Test Specifications. TAAS Test Administrators (teachers) were expected to improve and modify their instruction so student test scores would improve. District TAAS Coordinators distributed test specifications to the schools over and over and over until school staff accepted that the TAAS was here to stay.

The grades and time of year TAAS was administered was changed to be in all grades three through eight and high school exit in the spring. More tests were added so that Exit tests were administered four times a year and still too many failed to graduate. Training sessions on the written composition scoring standards were conducted by TEA and disseminated as above. The seventy percent correct passing standard was phased in. Performance tests in science and social studies tests were tried once, and were a big failure so they were dropped. Three different attempts were made before a compromised end of course Algebra 1 test was implemented. TEA issued the study guides several years late. TAAS became the number one accountability indicator in the state’s efforts to evaluate schools and districts. After seven years effort the Essential Elements (curriculum standards) on which the TAAS and statewide curriculum were based were change to the Texas Essential Knowledge and Skills. Very few changes were made in the TAAS because review TEA indicated the old questions matched the new standards.

Many people wanted a statewide norm-referenced test and a very ineffective one was administered for two years. It was full of practical and theoretical mistakes, but the results were let stand for fear the state would loose millions in Federal funds. The pressure for a norm-referenced test did not let up.

The first and major technical error with the TAAS came with the implementation of the Texas Learning Index (TLI). Many educators throughout the state had been used to the norm-referenced approach in which students may be compared with a national sample of students on the same test and wanted that same kind of score with TAAS. The notion that the same test could serve as a both a criterion-referenced and a norm-referenced test was advanced. The first idea was that some questions on the TAAS would count toward the passing criterion and other, more difficult, questions would count toward a state norm in a sort of two-tiered approach. That idea might have worked, but was never tried. Instead the mistaken notion that all questions could count for both sets of scores was advanced.

In spite of the advice against it and in spite of strong recommendations by a committee of national experts to the contrary, TEA wrongly adopted the policy of using all questions on the TAAS for both sets of scales: norm-referenced and criterion-referenced. The supposedly norm-referenced scale used by TEA is known as the Texas Learning Index (TLI). The national committee had indicated that the procedure was wrong, but that at least all scaling errors would be of a similar magnitude. This might not have been as serious as it has become had the scores on the TAAS test remained approximately the same as when the TLI was implemented. The higher the scores became on TAAS, the more distorted the TLI was. So for about the last six years the state has wrongly issued a TLI for each student that does not convey any known standard of learning and therefore cannot measure learning gain. The seriousness of the wrongly used procedures is growing in magnitude each year.

The TLIs have essentially destroyed the initial intent of the criterion-referenced design. Computing the TLI in and of itself does not affect the inherent technical characteristics such as validity or reliability that the TAAS may possess. However, the reported TLI has been used in technical studies that produce inaccurate estimates of validity and reliability.

THE TLIs ARE WORSE THAN USELESS, THEY ARE DOWNRIGHT WRONG AND THEY ARE FRAUDULENT USE OF TEST DATA. ANY AND ALL CALCULATIONS USING TLIs ARE BASED ON INAPPROPRIATE DATA GOING INTO THE FORMULAS SO NONE OF THE RESULTS ARE DEPENDABLE.

There is no doubt that the TAAS vendors have individuals on their staffs that know the TLI is inappropriate but produced it anyway. "Why would they do such a thing?" you ask. They probably did it for the same reasons that the engineers let the shuttle disaster happen - powers above them failed to heed the warnings. They chose to remain publicly silent until the bodies were recovered from the sea.

The Texas Education Agency has control of the testing process. The field of student assessment must have independence in the development of tests. A series of checks and balances is necessary and all involved must be focused on the true status of achievement and nothing else. This is because THREE THINGS CONTROL THE NUMBER OF STUDENT WHO WILL PASS A TEST. These are: 1) the level of instruction the students have received, 2) the intrinsic difficulty of the questions, and 3) the rigors of the passing score. Each of these must be managed with honesty and integrity otherwise changes in test scores may be little more than an aberration.

INSTITUTIONALIZATION

The first step toward making the TAAS a true part of the institution of education in Texas was probably during the Governorship of Ann Richards. There were statewide awards for outstanding educational progress and schools were awarded cash incentives. These awards set the tone for what was to follow. Simultaneously with these awards the scores in Houston were being reviewed for blatant cheating. Based on that review, it was obvious that approximately 75% of those awards were based on scores derived from blatant cheating.

It should be understood that cheating has been an integral part of reality in the world of testing. The Texas TAAS program has not been immune to that reality. To deny this is to ignore the obvious. In some, and perhaps in most, districts cheating has become a way of life. There are so many ways that cheating has occurred on TAAS in Texas that there is not time to list them all. But, before the counterattacks begin, let me define cheating.

The underlying purpose of cheating is to produce a test result - be it at the student, the classroom, the grade, the campus, the district, or the state level- that results in the overstatement of the academic skill level of students at any or all of these levels. The kinds of cheating can range from a lack of professional discretion to near criminal activity. These activities serve to inflate and distort scores so the lose any meaning they might have had. The following is a brief list of examples that are so widespread that they are indigenous to the TAAS system:

Taking instructional time from academic contents to devote an inordinate amount of time to TAAS drills.

Spending instructional enrichment funds on TAAS study guides and failing to buy adequate enrichment materials for other needs.

Requiring teachers to make lesson plan modifications to include TAAS specifications where they don’t belong. For example, a TAAS skill might not fit in all science lessons.

Not testing all students who should be tested even if those students are in school on the day of the test. Discouraging attendance on test days for students expected to fail.

Manipulating exemptions to deliberately affect accountability ratings.

Reviewing specific questions with students before, during or after the test and letting that review affect answers.

Post-test changing of answers by school officials when students are not present. This appears to be much more widespread than one could imagine.

Limiting the item pool to non-rigorous questions for the test. There are hundreds of examples of questions experts consider too easy. But there are very few, if any, that experts would consider too hard.

Completely overlooking those students who never progress beyond the ninth grade level in the high school ratings.

Teaching to the test takes on many forms. The contents of the TAAS are so skills based that there is no way the term "academic" belongs in the title. Somehow the myth that teaching to the TAAS is an academic activity must be dispelled.

With affronts such as those to the integrity of the test data any discussions of technical issues such validity and reliability are moot. A valid test must have full security in order to be an honest sample of the effect of legitimate instruction. The test results may be used diagnostically, but even that gradually erodes validity. The TAAS simply is not independent enough from manipulation of scores to be of use.

Many sophisticated techniques were available to the Texas Education Agency that could have been used to work with the school districts to curtail some of the more blatant cheating. For example, tracking of individual student’s scores checking for outlandish variations in scores from one year to the next. Instead the TEA relied on school district personnel to take the initiative in monitoring cheating. This is a real example of how complacency has destroyed TAAS.

As if all of this was not enough school districts and the TEA practice deceptive practices in reporting the test results to the public. The positive spin put on the data has gotten to the point that the public trust has been repeatedly violated by those in position of responsibility to the point that the average person who trusts those in authority believes a falsehood about public schools. For example, there is local data such as the SAT9 in Houston and the ITBS in Dallas that indicates how far below grade level unacceptable numbers of students are functioning, but that sort of data is ignored in the state accountability. There is no doubt that Texas ranks far below the national average in student achievement. Those who would keep such information from the public are intellectually dishonest and are doing a disservice to the people of Texas - especially the students.

Careful research and reporting of data requires that the many data sources be balanced in interpretation as it relates to the topic at hand. Therefore, data such as graduation rates, the data on college readiness, and efforts to assist the lowest of achievers catch up must be brought into the discussion.

The data on graduation rates has not changed in the last decade and is a prime example why it is obvious that data surrounding the TAAS appears to be inflated. Yet the dropout data are manipulated by TEA and unless one knows the difference in the interpretation of dropout and graduation rate one would actually think that nearly all Texas students graduate. However, if the facts are read carefully one will discover that TEA reports that only about 60% of the ninth grade students actually do graduate. A recent report released by the National Center for Educational Statistics reports that Texas has the 3^{rd lowest high school completion rate.}

The data on college readiness is just as blatant. In 1994-95 TEA publications indicated 43% and 53% of the students had TAAS scores that indicated college readiness and in 1999-00 those percents had risen to 77% and 78%, respectively. Yet this change is not reflected in the Scholastic Aptitude Scores or in the Advanced Placement Test scores of students statewide. The truth is that Texas ranks 47th among the states on the SAT tests. Texas ranks 19th out of 23 in states where 50% or more took the test. College entry data shows very low performance no matter it is analyzed.

The following chart shows considerable evidence that education is, at best, stagnated in Texas.

Chart 1: Educational Stagnation In Texas

EVIDENCE CITED HISTORIC DATA RECENT DATA

The "straw man" of TAAS has created a data source independent of other measures of educational attainment such as those above measures presented above. This way their data can possess circularity needed to avoid facing the real issues.

It would be inappropriate for one to say that TAAS shows that education in Texas is in a state of shambles. However, it is appropriate to say that education in Texas has been put in a state of shambles by the way TAAS has been institutionalized. The TAAS data do not provide a clue about the state of education in Texas in the year 2000 nor does it have one bit of usefulness in predicting the future.

The many "TAAS WATCHERS" are in a position of weakness because the data and so many of the reports produced from that data have an agenda. The sad part of this is that so many good researchers and so many technically incompetent are willing to prostitute themselves professionally to be a part of the "good news team." The data released by Texas State Agencies has generally been positive and has been presented with a positive spin. Likewise, some research groups have generally been positive and have presented the data with a positive spin. But, just how believable are these data and reports? The most insidious prostitution comes from within TEA and their contractors because those faults have never seen the light of day.

However, there are "papers with an agenda" produced outside of the agency. The most persistent and blatant of this "good news data" is produced in the Sociology Department at the University of Houston. To the knowledgeable reader it is easy to see the weakness in the TAAS data as the authors point it out to the reader. It is odd that neither the body of their reports nor the substances of their conclusions are particularly flattering of TAAS. It is their disclaimers and contentions that are flattering. Perhaps it is because they are, in part, commingling research and political power in Texas.

The Sociology of Education group at U of H "… elected to return to using TLI scores for the TAAS … because school district personnel are more familiar with them … pg. 14 (1999). This indicates some trepidation on their part on the use of TLI, but because TEA provided the TLI they felt they should use it. But, even more of a problem is the assumptions required in their analysis are not met. They have used techniques that require the tests being compared to at least be on equal interval scales but the TLI is far from an interval scale. For example, they indicate "…there is more variance in TAAS performances in the urban district than in the suburbs due in part to ceiling effects and to generally higher minimum scores in the suburbs …Statistically speaking, correlation coefficients between two tests will often be larger if there is greater variation in the scores on each test than if variation in the scores is more restricted." Pg. 9 (1999)

This same report shows that TAAS is a very easy test. They illustrate the lack of rigor on the TAAS in graphic form and then summarize the charts by saying "…Passage of the TAAS is likely for many students whose NPR scores are below national average…" pg 15 (1999). In plain English this means that a lot of students who are unable to show even minimal attainment on the SAT9 pass the TAAS with flying colors.

The conclusions the study states:

"In short, the TAAS and the widely-used norm-referenced tests that were examined both measure a common domain of achievement and do so rather equally well. True score analysis suggests that when measurement error is controlled in the two tests the TAAS and the Stanford 9, and presumably the TAAS and other norm-referenced tests, can be used interchangeably. Ceiling effects that result in a larger range of TAAS success than high norm-referenced performances seems to be substantially mitigated when the standard of TAAS mastery is applied." Pg. 27 (1999) emphasis is this author’s

NOWHERE DID THE STUDY ADDRESS THE ISSUE OF INTERCHANGEABILITY. A KNOWLEDGEABLE READER WOULD READ JUST THE OPPOSITE CONCLUSION FROM THE DATA THEY PRESENTED. THEIR CONCLUSIONS REALLY SAY THAT IF WE PLAY PRETEND EVERYTHING IS OKAY.

THIS WIDESPREAD DECEPTION OF THE PUBLIC BY TEA AND SOME OF THEIR VENDORS AND RESEARCHERS MUST STOP. FAILING TO REPORT ALL THE RELEVANT DATA OR FAILING TO USE THE MOST RIGOROUS PROFESSIONAL ETHICS MUST STOP.

Some of the studies on the technical characteristics of TAAS should be spread around to some of the excellent psychometric research departments at Texas Tech, Texas A&M, North Texas State just to name a few. Perhaps if that had been done, the checks and balances so badly needed would already be functioning.

CONCLUSIONS

^{The TAAS was planned from a very wide perspective and may very well represent the best set of frameworks for a statewide assessment system. Even from the perspective of an accountability system, Texas has a very good and noble plan. But, the politicization of TAAS has done a lot to destroy the integrity and public confidence in the test. In other words once the blueprints were developed the construction went on without adequate inspection.}

^{There is no way that statewide use of a totally wrong technique can make it right nor that the end justifies the means. If both criterion-referenced and norm-referenced test scores are needed then both kinds of tests must be given. The calculation of the TLI should never have begun and it must cease immediately.}

^{The Institutionalization phase of TAAS brought about putting a spin on the data. Only good news could be released to the public. Some of the following illustrate the point:}

<sup>THE STATE SAYS THE TRUTH ISTHE STATE SAYS THE TRUTH IS

The TAAS is uniformly administered
Cheating on the TAAS is rampant
The dropout rate is about 5%
About 60% of 9th graders graduate
TAAS says the equity gap is closing
National measures say it is not
TEA contends to set a high standard
Carefully done studies disagree
TEA is seeking to raise standards
Teachers vote on test contents
The TAAS is a good test
The TAAS specifications are good
Texas sets a high standard
The standards are for basic skills
Texas is raising its standards
Political pressure lowered standards
Independent research backs TAAS
Much other research criticizes TAAS
TAAS measures academic achievement
The TAAS measures nothing validly</sup>

^{Managers and policy makers must have true and adequate student assessment information in order to do their jobs. Misinformation results in poor decisions about the kind of instruction students need and the impact such instruction may have on them.}

^{The cost of misinformation and misguided instructional planning has been unbelievable. In recent years TEA has spent close to $50,000,000 each year on TAAS. They will be spending even more in the future. In recent years the school districts have spent cumulative millions on substitutes to allow time for teachers to travel to Austin for TAAS and TEKS committee meetings, curriculum revision, materials to raise TAAS scores and on test administration costs. They will be spending even more in the future. There is no way to estimate the price the students of Texas will pay for this colossal hoax called TAAS. No doubt many of them will continue to pay for it the rest of their lives.}

^{But, probably the most devastating circumstance we are in is the notion that those in power can quiet legitimate concerns if it does not serve their immediate concerns. A democracy must be built on the free flow of true and accurate information. The power of one agency cannot be used to continue perpetuating misinformation about the outcomes of the education of our youth that simply cannot stand the test of truth. There are thousands of functionally illiterate students in the high schools of Texas today.}

^{The children of Texas are losing time they don’t have to spare. Due to our failure to confront the embarrassingly low achievement throughout the state a decade was lost. Pretending to have closed the equity gap, while in fact we have not just places more and more of our youth in despair. Just how long will power politics pretend all is well? I do not know, but I do know with all of my heart and technical expertise that the price Texas students will pay will continue to increase if Texas does not pursue a testing program focused on straightforward honesty and integrity. Once those results are ready then serious remediation must begin. We need all of our youth to be ready to take their places in the workforce.}

^{When discussing this situation with an associate, the following analogy was drawn. The situation with TEA and their cohorts and TAAS without checks and balances is like having a baseball pitcher call balls and strikes, having the first baseman call that bag, and having the second and third base umpires sitting in the home team dugout.}

FOR REAL CHANGE

It is clearly past the time that Texans demanded a new accountability system. The original intent and integrity of TAAS has long since been lost. However, it is essential that assessment be expanded in Texas. Although the new system would focus on students, teachers would be brought into the accountability mix. The proposed changes have four components.

^{First, a new testing oversight Agency that is politically isolated and technically specialized must be established. The academic integrity of TAAS essentially collapsed throughout Texas under the burdens of the demands placed on it by a political, educational and vendor-based structure. I propose forming a Texas Educational Quality Commission that would report directly to the legislature. The Texas Educational Quality Commission would:}

^{a. Replace the test writing and accountability roles of the Texas Education Agency whose focus has become too contaminated by political agendas.}

^{b. Distribute the results of any student testing conducted directly to the parents and schools with Commission established information centers accessible to parents, teachers and administrators.}

^{c. The Texas Educational Quality Commission would establish standards and training criteria for measurement specialists to work with the parents and schools.}

^{Second, the Educational Quality Commission should implement a complete student assessment system. All testing conducted by the state must be clearly meet national grade level standards. No if, ands or buts permitted; otherwise, the testing program will once again reward an ambiguous level of mediocrity. They should implement:}

^{a. A nationally normed achievement test immediately. The tests should be given without being permitted to lose security. Such tests provide much broader range of questions than can a restricted criterion referenced test. These tests must be kept secure otherwise they will become a meaningless measure of the results of test preparation exercises. Teachers must not teach to these tests.}

^{b. A two-tiered system of criterion-referenced tests should be implemented. Students would be administered a series of basic tests beginning with reading, math, science and social studies. One test would measure the minimal skills that all students must have as they progress through the grades. Students doing poorly on one of these tests would be placed in intense, redirected teaching environment. Especially successful teachers would conduct those classes. The second set of criterion-referenced tests would be for advanced students. Students who measured of the basic skills test would be administered an advanced test and those with extremely high scores would become "distinguished" students eligible for special advanced programs. Students scoring at or above the 75%ile on HIGH SCHOOL ADVANCED COURSE EXAMS would be awarded an academic scholarship administered by the Texas Educational Quality Commission. Students scoring at or above the 75th percentile on the HIGH SCHOOL TECHNICAL SKILLS TEST would be awarded a post high school technical scholarship.}

^{Third, the Educational Quality Commission must devise a meaningful set of measures to evaluate teacher effectiveness. Those outstanding teachers need recognition for their special contributions.}

^{Fourth, at the heart of the system would be a tracking system that would accurately identify the needs of each student. Additionally, it would serve to assure that those individuals needing remediation did in fact receive effective help. With adequate scientific rigor meaningful studies of educational practices could be completed. All individuals in a state sponsored educational program from kindergarten through workforce training, prison or graduate school could be in one master skills development bank. Each test would serve as conformation of the progress of individuals in the many education programs. All individuals could then be in the accountability system.}

^{It is acknowledged that this proposal needs refinement, but it is a starting point for discussion. Just how much longer can we afford to play "let us pretend"? If the points made during this symposium are internalized by policy makers perhaps in a few short years Texas will have no "academic skeletons" to hide.}

Dr. Carl Shaw - Former graduate professor at the University of Houston (teaching Test Theory, Learning Theory. Education Psychology & Research Design), Executive Director of Student Assessment at Houston ISD, and Director of Research & Evaluation at Forth Worth ISD.

                           *      *     *     *      *                          

Paper presented at the Lone Star Conference on
PUBLIC EDUCATION REFORM IN TEXAS - COMPREHENSIVE PROGRESS REPORT
December 7th and 8th, Austin, Texas. Contact information for the Lone Star Foundation 10711 Burnet, Suite 333, Austin, Texas 78758 (Telephone 512-339-9771).

Published as a public service by
EducationNews.org