Thursday, October 15, 2009

NAEP: National Test, State Pissing Contest

For a nationwide test, the National Assessment of Educational Progress (NAEP) sure gets lots of state examination by the media. A search of newspaper stories on yesterday's release of NAEP Math scores underscores this fact.

All the newspapers want to know is how State X did versus the national average. The Albuquerque Journal is no different, using the scores as evidence of New Mexico's "achievement gap" between white and students of color.

Same story throughout the nation: "NJ eight graders show math gains", "New curriculum helping Georgia scores", "Despite improvement, DC students' math scores still lagging", "Mass. leads US on test scores", "California math scores still among the lowest", "Math tests: Wash. eighth graders improve", "Fla. scores on math test hold steady", "Hawaii students' math improves but still below US average", etc., etc. etc.

Everybody wants to be on a team (tribe) and beat the other teams (tribes). And the way NAEP is scored, you can't have school-by-school tribes (or at least I've never seen them) like Adequate Yearly Progress, so states are the lowest tribal denominator. Go tribe! Yeah, rather arbitrary invisible geographical demarcation! Fight, team, Fight!

Which is silly and no big deal, EXCEPT when local reporters myopically focus on the tribe, and exclude meaningful examination of the national results of a national test. Take the Journal story, for instance. In addition to some rather silly comments from NM Education Secretary Veronica Garcia, it wastes tons of valuable print inches comparing New Mexico scores to other states and alluding to the "achievement gap" in New Mexico, while omitting any national data on the "gap" situation.

So I looked at the national data, which is actually quite nicely displayed in self-designable graph/tables at the NAEP site. Guess what:
But you don't see/read stuff like this because local reporters want to tribalize it all into "Us v. Them" statistical football games.

That's a mistake. One that is likely to be compounded when the Journal continues its series on the "achievement gap" in New Mexico, and uses the NAEP data only far enough to promulgate the idea that this "gap" is a New Mexico problem. Which it isn't, anymore than it's a Rhode Island problem, or a Wyoming problem.


P.S.: If one is interested in all this, the NAEP site is definitely worth a visit. The ability to make your own graphs is quite lovely in an edu-nerdcore sort of way.

3 comments:

SydTheSkeptic said...

Well, what do YOU know? You're not a journalist. You're just a silly blogger.

Hehe.

Seriously, it doesn't take an actual investigative reporter (a contradiction in terms in these days of info-tainment) to see why the news industry is flailing while non-journalists (bloggers and such) are increasingly doing the foot work THEY should've been doing.

Anonymous said...

Thanks from the team that built the graphics you get to play with. Have you seen http://nces.ed.gov/nationsreportcard/naepdata ? it is the ultimate in edunerd!

Anonymous said...

I love how Dr. Garcia wants to blame the sample size for the scores. Surely someone with "Dr" in front of her name knows that a very small sample done with rigor gives highly accurate results.