Tuesday, August 05, 2008

The Test Scores Are Here! The Test Scores Are Here! 2008 Edition

I come back from out of town and see the "test scores" have arrived! Whoo-hoo! Test scores! Excitement! Drama! Statistics!

Okay, maybe you're not of the uber-nerd variety and would not choose to recover from the muscle soreness and sleep deprivation of five days in the Weminuche Wilderness by going, page by unwieldy page, through the New Mexico Public Education Department's school-by-school report for hours and hours. Yes, I am a dork. (And I really, really, really urge anyone who cares at all about this sort of stuff to raise your dork-o-meter and check the scores out for yourself)

More likely you've run across newspaper and television reports on the new scores. In past years, your humble blogger has written incredibly long-winded, mind-numbingly boring "analysis" about these scores and why the news organizations and, concomitantly, the general public just doesn't get it.

Well, I'm going to spare you the ultra-long-winded recap of this year's scores (okay, this turns out to be a total lie, as you'll see below) because: 1. you deserve to be spared; 2. the news hasn't really changed; 3. my shoulders are really tired from holding up a backpack for days and days.

Instead, I'm just going to make a flimsy point or two now, reserving the opportunity to bore you more when I get the feeling back in my fingers.

APS Test Score Highlights 2008

This is quite definitely the year when the number of "failing" schools got so high, with failing schools found in all sections of the city, that a significant backlash against No Child Left Behind is sure to follow. The old political chestnut "the enemy without unites within" seem to apply here. In the past, some schools failed and others passed. Media, parents, etc. could point to the passing schools and call them "good schools", while bemoaning the horrible "failing" schools as places to avoid.

Well, in 2008....no APS middle school passed. Not even that last domino of "good schoolness" Desert Ridge Middle School was able to "pass". Every middle school is "failing".

Every APS high school is "failing". Yes, even that La Cueva school failed.*(see note way down below)

Now instead of having a socio-political dynamic in which "rich" folks living up in the "passing" Northeast Heights could look down, in more ways than one, at the "failing" lower elevations of town...

We're all in the same boat now, baby. We're all, each and every middle/high school, "failing". Instead of being able to demonize other schools, we are left with no other choice now than to demonize the test itself, and in ways and to an extent never possible when it was only the "poor" schools "failing".

In other words, No Child Left Behind has finally, finally, FINALLY pissed off the wrong people. It will be fun to see how perceptions of the tests change in the next year or so, despite the fact that news organizations are still churning out the horrible drivel linked above (that KOB TV story is particularly unperceptive...heck, let's link to it again and marvel at it's vapidness).

As to why all the middle/high schools failed (I'll get to elementaries in a second), here's a quick, redundant to previous years, explanation:
  • "Students With Disabilities" failed. You basically have two types of public schools: ones really struggling and incapable of having their overall student body make "adequate progress" (by percentage) and another set of schools that pass in ALMOST ALL the many test-taking sub-groups, but "fail" particularly in the areas of Special Education students ("Students With Disabilities") and "English Language Learners" (which, unlike almost all Education jargon, is actually almost a self-explanatory term). "Failure" in these sub-groups meant overall "failure" to a number of schools one might describe as "successful" middle/high schools. La Cueva High School, for instance. Desert Ridge MS. My school, Jefferson MS, once again "failed" with these particular groups.
  • One big reason that middle/high schools all "failed" and only many elementary schools "failed" had nothing to do with the quality of education (like that is really the point, anyway), but instead centered on the simple fact that middle/high schools are bigger than elementaries.
  • Caution: now we're getting to that statistical stuff that you NEVER, EVER see/hear the media get into when discussing this topic. As you can see when you look at the reports (and I really, really want you to do that), schools had to have 25 students in a particular sub-group test (technically as FAY "Full Academic Year" attendees at the school) to have that sub-group scores count. This is done because statistically such a small number of testers isn't seen as "valid". Well, middle/high schools have hundreds, if not thousands of students, and thus almost always have enough students of every sub-group to be statistically valid.
  • Elementary schools are much smaller, and don't always have enough "Students With Disabilities" to count.
  • Here's a funny example: I don't make it to Ventana Ranch much, as it is approximately 275 miles from my house in the South Valley, but evidently they have somehow split Ventana Ranch Elementary into two entities: Ventana Ranch Elementary West and Ventana Ranch Elementary East. Ventana Ranch Elementary West is a "passing" school. East is "failing". East is "failing" because of poor scores by its "Students With Disabilities". It just so happened that exactly 25 "Students With Disabilities" took the test at Ventana East and they bombed the test (more about that in a second, promise). Ventana West had only 18 "Students With Disabilities" take the test..and they bombed, as well. Nevertheless, because West had a statistically irrelevant 18 students take the test they are "passing". Huzzah!!!! Glory Be!!!! Ventana Ranch East is "failing" Burn them! Burn!
There are plenty of other similar "funny" stories embedded in the test score data, but I've already broken my pledge to make this year's "analysis" any less mind-numbing that in previous years. I'll just finish by, once again, asking everyone (and that includes you Albuquerque Journal and KOB-TV) to read through the school-by-school reports. There's a lot there...for instance I haven't even gotten into the whole "Safe Harbor Provision" idea.

But getting back to that finishing thing. The testing of "Students With Disabilities" and "English Language Learners" using the current No Child Left Behind criteria is a travesty. As teachers we want accountability, but part of that accountability is taking the incredible amount of bureaucratic time and energy to follow all the steps necessary to place a child in Special Education or as an "English Language Learner".

Having gone through the myriad referral steps, teachers and others make placements for these students and prepare individualized programs for them. They are in these special programs because it has become clear to everyone that they have deficits which must be addressed. Yet, we expect them, at testing time, to miraculously lose the deficits that led them to be placed, and be "just like everyone else".

What a crock. What an incredible disservice to these children. Do you know how many "Students With Disabilities" and "English Language Learners" get the impression that it is "their fault" their school "failed"? Do you have any idea what that does to a kid who already knows they struggle, and thinks of school as a hellish place that continuously points out their inadequacies?

This disservice has to stop. And, in your humble blogger's opinion, the inclusion now of schools like La Cueva and Desert Ridge as "victims" of this travesty will add necessary fuel to the political fire, and bring about the end of No Child Left Behind as we currently know it.

Accountability will go on, and there will continue to be plenty of problems, but at least one incredibly idiotic provision will go away leaving us at least one step closer to a true standard of "passing" or "failing" schools.

Bring the torches, townspeople, let's storm the castle.

P.S.: *From reading around, I see that Sandia HS supposedly "passed", even though its "Students With Disabilties" passage rate was well into "failure" range. Sandia gets a pass with an asterisk due to the "Safe Harbor Provision", which I was going to try avoid describing. "Safe Harbor" states that schools which haven't made AYP in previous years can claim a "pass" in previously failing sub-groups by reducing the number of those non-proficient in those sub-groups by 10% or more. Not that Sandia HS folks will ever have to explain this little asterisk, because nobody in the media is ever likely to bring it up.

Well, I don't count asterisks, whether its Barry Bonds or Sandia High School. So, I'm sticking with Sandia "failed". Period. No asterisk.

And yes, you may think I'm being harsh about this asterisk business, but I can foresee a time...roughly 11 hours from now (9:27 p.m., 8/5/08) in which the folks at Sandia HS are going to get together for their first day of the new school year and celebrate "passing", as if Sandia is doing something different/better/more wonderful than any other high school in APS, when the fact of the matter is that their scores aren't any different when it comes to Special Education kids. 21% of their Special Education kids passed the Math test. 34% passed the Reading test. Numbers right in line with those at "failing" La Cueva, interestingly enough. Or unbelievably maddening, outrageous and preposterous...depending on your point of view.

If I hear of one, just one Sandia HS person gloat about this little asterisk win as if it is anything but a lucky bit of statistical whimsy........argh.

9 comments:

Anonymous said...

Oh Scot,

Once again you put the whole testing nightmare in perspective. I agree 100% about how heads are going to roll now that the white fright schools are "failing". Karma's a bitch, ain't it!
Anyway I'm sending your perspective to everyone I know. Thanks again for your unique insite.

another mouse

James said...

Mr blogger...

So is it my understanding that Monte Vista Elem would have also failed if 3 more disability students were designated 'FAY'?
link

jscotkey said...

James:

Thanks for taking time to go to the school-by-school listings.

I didn't get into because the post was already way, way too long, but some elementary schools, Monte Vista being one, would have passed even with "Students with Disabilities". You're right that with 22 students tested in this category and and scores a few percentage points below require proficiency in both reading and math it looks like Monte Vista would have failed if only three more "Students With Disabilities" were tested.

But the statistical scoring gurus use something called the "lower bound confidence interval" to determine whether a sub-group really "passes". A statistical range, a margin of error is a term you hear more often, is determined based on the number of sub-group students tested. The smaller the number tested, the bigger the range, both high and low. As more test-takers participate, the statistical range shrinks on both sides of the actual percentage. If you go back and look at scores you'll see that listed, and how it raises the "lower bound confidence interval" in the bigger testing groups to a point much closer to the actual required percentage.

The upshot of this overly-long explanation is that Monte Vista "Student With Disabilities" scores would have certainly been above the "lower bound confidence interval" at the rate the 22 students scored, and even had he other three needed to make the total 25 bombed the test Monte Vista would have "passed".

Taking Reading, for instance, the required proficiency percentage was 44. From looking at a school like Ventana Ranch Elementary East, with exactly 25 test-takers in the group, you find that the "lower bound confidence interval" is 24.1%. Twenty percent lower than the required number! That's a lot.

Monte Vista, if I count correctly, had 9 "Students With Disabilities" "pass" the Reading test (9/22 = 40.9%). If they had tested three more students in the group they still would have had 36% pass (9/25 = 36%) even if those other three bombed the test. Thus, way over the 24.1% we mentioned above.

Again, sorry for the explosion of verbiage here...I envy the math teacher's ability to explain this stuff succinctly..it's a trait I've never been able to master. Obviously.

P.S.: I know it's obvious, but creating a range twenty percent below the supposed required percentage is a huge benefit to small (i.e. elementary) schools. Who out there can find all the cases of elementaries helped by this scoring method? Let us know what you find! Collect them all!

James said...

Dangit... Now you've got me hooked.

:)

jscotkey said...

James:

And thanks for the idea of linking to the actual pages for each school. You know, I should have thought of that a tiny, tiny bit earlier...

P.S.: It is kinda addictive, ain't it?

Unknown said...

As a student in APS as No Child Gets Ahead went into action, I have seen the ratings for the schools I attended plummet, even as teachers struggled to keep up with new paperwork and counter-intuitive processes. I watched as the special education department staff tried to teach for each students needs, but were held up in tangles that guaranteed failure.

Unknown said...

Thanks for the post- I am proud to say I joined your wonkfest over here in the North Valley to sort out what's going to happen to my bright and wonderful daughter this year. Her school, a wonderful place full of fantastic teachers, is in its 4th year as failing & is moving into corrective action (mostly failing bc of Special Ed/ ESL groups).

As far as I can tell, that means more "short term assessments" (tests), so instead of having any science or art or social studies or what have you, these poor kids are going to be drilled and tested, drilled and tested to make sure they meet their standards. What kind of education is that? It sounds HORRIBLE.

We ended last year trying to get my daughter into the gifted program. Under NCLB these days getting into gifted is such a long, tedious process (mostly falling on the teachers), that my DD's school got NO referrals last year. We got her screened & now all she has to do to get 3 hours of pullouts a week is submit to another 8 weeks of evaluation, then take a 6-hour test. This is how they treat the good students I will say for the record that she is also special ed, so her phenomenally high test scores count heavily in that otherwise failing category.

Meanwhile under title I, everyone in the school can sign up for free with some crappy fly-by-night tutoring program (Jim Bobs Real Good School Service! We got computers! And Internets!). When I brought this up to the head of DD's assessment team, her head nearly popped off. She is FURIOUS that NCLB has shifted all the focus off the good kids and have funded remedial education at the cost of cutting all enrichment and gifted programs. She believes (and I agree) it's all part of the program to drive the good kids out of public schools.

You know what? I'm almost there. To be honest, as much as I love this school, it is pathetic that only a third of them can read or do math at grade level. If nothing else, it means there's only a slim chance my kid has someone who is on the same page she is. The gifted resource teacher's reaction? "Have you considered Manzano Day School?"

Listen, this program is fucking the good students and fucking the teachers. It's destroying our public education system. In the interest of my bright daughter getting a good education, should I mortgage the farm and send her to the happy white bubble schools, driving halfway across the city? Or do I stick it out with the neighborhood school, fighting a rigged system the whole way?

ched macquigg said...

somebody needs to point out that it does little good to talk about a problem if you have no seat at the table where the solution will be decided.

Winston Brooks said it straight out; decision making authority is moving back to 6400 Uptown Blvd where it belongs.

You are invited to give them all of the "advice" that you care to.

Anonymous said...

Isn't this happening in other states? Why haven't we been hearing about it? Well, I hear that some states like Florida, have chosen ridiculously easy tests that 90% of their kids can pass.

I want to know what happened in Utah since they ditched NCLB. Why don't we do that? We already ditched some of the "Abstinence Only" sex education money, didn't we?