Standardized Testing: Do It Right or Not At All

There's been a lot of wailing and moaning about high-stakes testing in the schools lately. An article by Valerie Strauss in the Washington Post caught my attention because it criticizes a company I used to work for. Strauss has a long litany of delays, errors committed, and fines paid by Pearson for problems with the administration of tests in schools. Another recent article discusses Pearson's role in jacking up the price of the GED. I even know the president of GED Testing Service; I worked with him 20 years ago.

I didn't work with high-stakes testing for kids; I worked for the companies that grew out of Control Data's PLATO division. They provided computerized certification exams for IT professionals who supported software products from companies like Microsoft, Novell, Oracle, etc. They also provided FAA pilot and mechanic exams, stock broker compliance testing, insurance and real estate sales exams, and so on. When I retired a dozen years ago, the business was quickly morphing into computerized delivery of certification testing for medical professionals and was about to enter the SAT/ACT market.

So I have a bit of inside knowledge about the testing business. I was on the software end of things, and wrote the code that delivered and scored computerized tests. I worked with a lot of exam developers and customers to get their exams into our delivery system and the results out of the back end. But the real work in testing is on the front end: the development of the exams themselves.

There were basically two kinds of customers. The first kind was the testing professional, who insisted on doing things the right way. That involves writing a large bank of test questions ("items") and then testing the items' performance in several series of exams to a large number (hundreds, if not thousands) of target candidates who demonstrate the expected range of knowledge of the subject matter. The quality of the items is then statistically determined by how well they predict the ability level of the candidate (which has be assessed separately).

A good item is one that someone with a firm grasp of the subject material gets right and someone who doesn't know the material gets wrong. A bad item has no correlation with subject matter expertise and a terrible one has a negative correlation. Bad items have factual errors, or are poorly written, unclear, misleading or "trick" questions.

Another consideration in writing an exam is the number of "forms" you deliver: in many testing regimes people take the exams on different days, so you have to write many different forms of the exam in order to avoid exposing all the items to the public at once. This is a serious concern because there are quite well-organized cheating efforts that involve people who've just taken an exam doing a memory dump of a few questions they are assigned to remember. With a relatively small crew you can completely reconstruct the exam: within a day your test -- and all the answers -- can be out on the Internet.

When you have multiple forms of an exam, it's critical that the forms be equivalent. That is, each form has to be statistically balanced to have the same degree of difficulty, even though not all items on the form are the same. Otherwise the test wouldn't be fair to all takers.

This means that if you want to give several alternate forms of a 50-question test to millions of kids across a state or a country, you're going to wind up writing thousands of items, many of which will be discarded because they do not accurately predict ability level.

This is extremely expensive and time-consuming. And it's a never-ending process because of the exposure problem and constantly changing curricula. Companies like Pearson manage item banks with millions of items whose statistical performance is monitored and are aged out over time.

That brings us to the second kind of customer: the average guy. The average guy thinks you can just jot down some questions and be done with it. That's probably true for teachers who know the kids in their class, where quizzes plus class participation plus daily homework provide a complete picture for the teacher to assign a grade. But you can't write a standardized test that way.

The problem with developing good exams, in my experience, is that people just don't want to pay for it. Their eyes glaze over as you explain that it'll require subject matter experts writing thousands of items, and months of testing and retesting the items' performance (you can't change a word of an item -- or even its formatting -- without affecting its stats), and analysis of the statistics, and careful construction of equivalent forms.  And then things like content balancing  (making sure subdisciplines of the exam subject matter aren't under- or overrepresented on a particular form) make the exam developer's job that much harder.

Not surprisingly, school districts are particularly concerned about costs and schedules. They never have enough money, and by the time the legislature appropriates it, the company that's supposed to develop and deliver the test may not have the time to do it right.

In my experience, corporate customers constantly changed their minds and added new requirements, but the schedule never changed. With state-wide tests and requirements coming from dozens of school districts, administrators and meddlesome politicians, the software developer in me would imagine the deadline at the end of the school year to be an all-consuming bottomless pit.

Thus, I'm sure that many of the problems Valerie Strauss cited with Pearson's performance are due to changes their customers demanded at the last minute, or customers skipping necessary quality control steps that they didn't want to pay for or have time for due to schedule constraints. From personal experience I'm absolutely certain that many of Pearson's alleged problems are really the fault of politicians, school boards, state education commissions and educators themselves.

I'm equally certain that many of the problems are due to sales guys who promised things Pearson didn't have, management who agreed to schedules their technical people told them outright were impossible, not to mention hardware problems, mistakes in coding and data entry, faulty statistical analysis, mismatching items and their statistics and/or answer keys, and simple cut/paste errors in item text.

Given the constantly shifting educational priorities and curricula, perennially tight school budgets and incessant political bickering I don't see how we'll ever be able to do large-scale standardized testing right, especially not with every state and local jurisdiction trying to reinvent the wheel themselves, and everyone insisting that we do it several times a year.

If we can't spend the time and the money to do standardized testing right, we shouldn't do it at all. With all due respect to my former colleagues, I think we should take the money out of the hands of companies like Pearson and put it back into the schools where it'll do the most good.

Voices In My Head (Blaming The Victim Edition)

I have something I want to say to the victims of Newtown or any other shooting, I don’t care if it’s here in Minneapolis or anyplace else: Just because a bad thing happened to you doesn’t mean that you get to put a king in charge of my life. I’m sorry that you suffered a tragedy, but you know what? Deal with it, and don’t force me to lose my liberty, which is a greater tragedy than your loss. I’m sick and tired of seeing these victims trotted out, given rides on Air Force One, hauled into the Senate well, and everyone is … terrified of these victims. I would stand in front of them and tell them, ‘Go to hell'

This is what happens when you come out of the bubble. You get smacked squarely in the head with your bullshit.

Several things amaze me about this very illustrative incident. We have the usual adolescent temper tantrum that is all too familiar. This stomp down the hallway precedes the equally familiar DARVO, a truly despicable  practice which seems to happen when the mouth foamer knows that he or she is completely wrong.

But the element that really stuns me is just how much of a fucking coward the Bob Davies' are of the world. If he truly has the courage of his convictions, he should go to Newtown and say those things in front of the victim's families. That goes for anyone else out there who hides behind a mic or a blog who thinks that victim's families or frightened children are being used as props or "human shields."

Go say that shit right to their face, fuckos. If you can't muster the sack to do it, then you obviously don't believe what you are saying and just being an immature ass hat.

Heed His Warning

Visit NBCNews.com for breaking news, world news, and news about the economy

Is Joe channeling me?:)

Earth Day +1

Yesterday was Earth Day and Nikto and I were too busy talking about the gun debate and Boston so I thought I would put this wonderful documentary up today. It's an American Experience film, in its entirety, about the history of the modern environmental movement. Interesting how it started with Republicans...

Enjoy!

Well, This is Fantastic!

The Tsarnaevs: Islamic Warriors or Losers?

There's been a lot talk about whether Boston Marathon bomber Dzhokhar Tsarnaev (I'm not going continue the charade of calling him a "suspect") should be read his Miranda rights. It's almost a moot point: he's not going to be doing much talking because he was shot in the throat and may never speak again.

There's been a lot of talk about designating him an enemy combatant. This is crazy on the face of it, since he's an American citizen.

There's even been a suggestion, from one of the right's most brilliant luminaries, Donald Trump, of torturing Tsarnaev.

All of these things presume that Tsarnaev is a terrorist, and it's not at all clear that he is. Philip Mudd, a former CIA Deputy Director, said Sunday on Fox News that Tsarnaev should be charged as a murderer. Mudd thinks, as I wrote last Friday, that the marathon bombing was more like Columbine and not 9/11.

Who is Philip Mudd? Some liberal Obama appointee? Well, Obama did try to appoint him to a high position in the Department in Homeland Security in 2009. But Mudd withdrew after questions about his involvement in waterboarding terrorist suspects:

Mr. Mudd, who joined the C.I.A. in 1985 and served tours in the Near East and South Asia, is considered one of the government’s top experts on Al Qaeda.

“He’s not just an ops guy,” said Frank Cilluffo, director of the Homeland Security Policy Institute at George Washington University. “He’s probing. He asks questions. And he’s open-minded, and I don’t see that every day with this community.”

Several Republican lawmakers expressed anger over Mr. Mudd’s withdrawal. Senator Christopher S. Bond of Missouri said the nomination had become “the latest political casualty of a terror-fighting program no one in Congress objected to until it became politically risky.”

Thus, Mudd is not inclined to be soft on terror.

We don't yet know for sure that aren't any foreign terrorist connections in this case. Further investigation is obviously in order.  But the last thing we want is for our government to automatically treat people like terrorists. Because when we as a nation start treating members of a group as terrorists, other members of that group feel threatened and may become terrorists themselves to protest the injustice of that persecution.

History should be our guide: the invasion of Iraq was a recruiting bonanza for Al Qaeda. Let's not turn what may be a lone act of frustration by two young alienated guys by turning them into martyrs.

Ruslan Tsarni, the Tsarnaevs' uncle, was close enough to know them well, but distant enough not to think they were innocent angels like their parents did. What does he think motivated them?

Being losers, hatred to those who were able to settle themselves; these are the only reasons I can imagine of. Anything else, anything else to do with religion, with Islam – it’s a fraud, it’s a fake.

If we charge the Tsarnaevs as terrorists and enemies of the state we grant them status and renown as soldiers of Islam and martyrs. If we charge them with criminal murder we brand them as murderous losers.

If we want to discourage copycats the choice seems obvious.

Going Forward

It's hard to imagine where the gun safety movement is going to go from here. I've heard many people say to me, "If 20 dead elementary school children doesn't change our nation's gun laws, what will?" I certainly can sympathize with this sentiment.

What has to happen now is out of the box thinking. As I have said previously, bringing a knife to a gun fight never works and the families of the victims of the various shootings that showed up in DC these last two weeks didn't even bring that. They don't understand the nature of what stands in their way. Essentially, it is two distinct groups of people

First you have the gun lobby and the gun manufacturers. They don't give a shit about the 2nd amendment or possible futures in which an American Hitler or Stalin takes over. They care about one thing: money. Their livelihood is being threatened and they are going to do anything to prevent from happening. These people are fucking scumbags who peddle fear and death and they should be exposed as such. Think about how much money they have made since the president took office. Think about how much money has been made since Newtown. So many gun owners rushed to their local gun store because of a fear that ended up being nothing. They have their sheep and they know how to manipulate them.

The second group of people are the paranoid pyschotics that do think Democrats, proggressives and any to the left of the one yard line on the right side of the field is coming to get their guns and send them to re-education camps. These are the same people who sneer at frightened children who write to the president because they are scared, only to be later derisively called props. Or taunt the families of the victims and say things like, "But it must be for the children.." In so many ways, these people are actually worse than the gun lobby and gun manufacturers. The good news is that most of these people are over the age of 40 so time will do its thing in some respects.

Now,there is no earthly way to reach the second group. They are fucking gone into a never, never world of plots and fantasies so profoundly fictitious that it leaves me...even me...completely speechless. So, the attention should be on the first group. And that means one thing. We have to go after the money. The question is...how do we do that?

One idea I had recently was to go in the opposite direction. Rather than try and ban certain types of guns or limit ammo clips, why not simply give away free guns? The federal government could offer free weapons to any citizen who would pass a background check and go through the appropriate training and regular mental health checks. That would also really piss off the anti government spending crowd as they are usually the same demographic as the pro gun crowd. And, to get around the issue of buying the guns from the gun manufacturers, the government could just make their own guns. There are probably plenty of people who could use the work. Heck, that would solve the unemployment problem!

The other idea I had was a PR campaign based on what happened with smoking. If you start to pull together all the various groups that are adversely affected by gun violence, you could really start to change public opinion. These days, smoking is really looked upon like a giant fart that just hangs in the air and won't go away. Lock a bunch of ad guys in a room for a couple of weeks and there is no doubt in my mind that they could eviscerate the gun lobby. You could put together ads with the audio being comprised of the usual rhetoric from the pro gun folks and the video showing shootings, the aftermath, and victims families. The public needs to see this contrast.

Both of these ideas may seem out there but that's what it's going to take. Of course, there is something else that I haven't mentioned because it's not so much of an idea as it is a prediction, which would make both of my ideas moot. If you take a look at how change happens in this country, it only occurs when the right people are adversely affected by whatever issue is on the hot seat. Gay rights, for example, wasn't important until enough parents had children that came out and then that was pretty much fucking that. When it becomes personal, that's when the real change happens.

Right now, there is someone out there..or several someones...that sooner or later is going to be adversely affected by gun violence. I don't who they are or where they live but I do know that they will start a wave that will finally end all this nonsense. This person will likely be someone who arises out of the "pro-gun" crowd and will initially be labeled a "traitor." This same person or people will understand that you don't bring less than a knife or less to a gun fight. They'll know exactly what to do and they will fucking bury the NRA, the rest of the gun lobby, and the pro gun crowd up to their necks.

He or she won't take away their guns nor will they change the second amendment in any way. They will simply expose the anger, hatred, fear, and paranoia for all to see. And then we can finally put a serious dent in the already declining world of violent crime.

They Have More Money

Interesting piece over at the Atlantic about who gives more to charity...the wealthy or the poor. The answer is surprising.

One of the most surprising, and perhaps confounding, facts of charity in America is that the people who can least afford to give are the ones who donate the greatest percentage of their income. In 2011, the wealthiest Americans—those with earnings in the top 20 percent—contributed on average 1.3 percent of their income to charity. By comparison, Americans at the base of the income pyramid—those in the bottom 20 percent—donated 3.2 percent of their income. The relative generosity of lower-income Americans is accentuated by the fact that, unlike middle-class and wealthy donors, most of them cannot take advantage of the charitable tax deduction, because they do not itemize deductions on their income-tax returns.

While it's true and quite obvious that the wealthy give a larger dollar amount, the do not give as much percentage wise, as the less fortunate. Add in the fact that the poorer folks don't get a tax deduction and it seems even more generous. But why?

However, some experts have speculated that the wealthy may be less generous—that the personal drive to accumulate wealth may be inconsistent with the idea of communal support. Last year, Paul Piff, a psychologist at UC Berkeley, published research that correlated wealth with an increase in unethical behavior: “While having money doesn’t necessarily make anybody anything,” Piff later told New York magazine, “the rich are way more likely to prioritize their own self-interests above the interests of other people.” 

They are, he continued, “more likely to exhibit characteristics that we would stereotypically associate with, say, assholes.” Colorful statements aside, Piff’s research on the giving habits of different social classes—while not directly refuting the asshole theory—suggests that other, more complex factors are at work. In a series of controlled experiments, lower-income people and people who identified themselves as being on a relatively low social rung were consistently more generous with limited goods than upper-class participants were. Notably, though, when both groups were exposed to a sympathy-eliciting video on child poverty, the compassion of the wealthier group began to rise, and the groups’ willingness to help others became almost identical.

Hmm...perhaps the wealthy are out of touch?

I think that people that have less money give more because they know what it's like to be poor. Perhaps they didn't have a lot of money in recent memory and can completely relate to the hardship. And the wealthy don't give as much because...well...that's why they are wealthy.

They have more money.

And So It Begins...

Adolphus Busch IV Resigns From NRA

It disturbs me greatly to see this rigid new direction of the NRA. As a starting point, one only has to ask why the NRA reversed its original position on background checks. Was it not the NRA position to support background checks when Mr. LaPierre himself stated in 1999 that NRA saw checks as “reasonable”? Furthermore, I fail to see how the NRA can disregard the overwhelming will of its members who see background checks as reasonable. In fact, according to a Johns Hopkins University study, 74% say they support background checks.

One only has to look at the makeup of the 75-member board of directors, dominated by manufacturing interests, to confirm my point. The NRA appears to have evolved into the lobby for gun and ammunition manufacturers rather than gun owners.

I'm quite proud of the family who has given so much to the state in which I was born. I'm thinking it's just the beginning. The aftermath of the Senate's vote has been quite pointed. As I have said, this does indeed sting, short term but long term? Say goodbye to the gun lobby...

Was Boston Really Terrorism or Just Another Columbine?

With the news that bombings in Boston were committed by guys that had been living in this country for 10 years, people are worried about a wave of "homegrown terrorism." They're wondering what we can do to combat this.

I have a suggestion: stop beating up innocent people.

After numerous false news reports that variously identified "a dark-skinned man," a Saudi, a guy from Nepal, a Moroccan, etc., as suspects in the bombing, there have been several revenge attacks against people who have nothing at all to do with the atrocity: a Bangladeshi network engineer, a woman doctor from Syria, and so on.

This is nothing new. When people are angry they vent their rage against innocent people who vaguely resemble someone they hate. For centuries these sorts of attacks were common against minorities including blacks, Catholics, Irishmen, Hungarians, gays, and especially after 9/11, Muslims.

Tamerlan and Dzhokhar Tsarnaev came to the United States as refugees about that time. How did the intolerance and violence towards innocent Muslims did affect their attitudes about America?

The marathon bombings always seemed like an amateur affair, more like the mass shootings at Newtown, Aurora, and Columbine than a masterminded plot like 9/11. Tamerlan seems to have been alienated from American society. He reportedly said, "I don’t have a single American friend, I don’t understand them." If the Tsarnaevs felt bullied and hated the same way that Dylan Klebold and Adam Lanza felt, Boston may be exactly the same as Columbine and Newtown.

We don't yet know exactly why the Tsarnaevs detonated those bombs. There's simply no excuse for killing innocent people at a marathon. Just like there's no excuse for a man to assault a random Muslim woman on the street.

If you want to stop crimes like the Boston bombing, you have to understand what motivated the perpetrators. Random hatred and mistreatment of Muslims in America may or may not have been the trigger for Tamerlan and Dzhokhar. But anyone who feels oppressed by society could have the same reaction, including gay teenagers, home-schooled Christians, gun owners, you name it.

As we've seen time and again, your ethnicity and political and religious leanings have no real bearing on whether you'll commit atrocities like Boston. Abortion- and lesbian-hater Eric Rudolph was responsible for the bombing at the Atlanta Olympics. Timothy McVeigh's attack in Oklahoma City killed far more people than the Tsarnaevs, and he called himself a true American Patriot.

All that's needed for mass murder is a righteous belief that violence is the appropriate response to a perceived affront. The rest is just details.

Dzhokhar's Joke

People are wondering what motivated Dzhokhar Tsarnaev to commit the atrocity in Boston. His page on the Russian-language social networking site vk.com (short for "v kontakte," or "in contact") is still up as of this writing. It looks fairly normal, with links to a couple of generic pages about Chechnya.

His worldview is listed as "Islam" and his personal priority is listed as "career and money."  Not exactly a terrorist manifesto, is it?

There is one thing that may shed some light on his mindset, a joke he posted on March 19:

В школе задают загадку..Едет
автомобиль. В нем сидят – дагестанец,
чеченец и ингуш.
Вопрос – кто ведет машину ?
Мага отвечает: - Полиция.

Translated into English:

In school they posed a riddle: A car is going along. In it sit a guy from Dagestan, one from Chechnya and one from Ingushetia [regions of Russia that have been torn by insurrection and repression].
Question: who's driving the car?
Maga answers: The police.

Well, I Didn't See That One Coming

It looks like the two bombers of the Boston Marathon were Chechen rebels? Wow, I didn't see that one coming. One of them is dead and the other one is still at large of this post.

His Finest Hour

Everyone keeps talking about how the defeat of the Manchin-Toomey gun bill is the greatest loss the president has experienced and how awful it is. I disagree. In fact, I think it has been his finest hour. Watch his entire speech below from yesterday.



I don't think I have ever been prouder of the man. People are going to remember these words and, when juxtaposed with the 46 Senators that voted against Manchin-Toomey, the American people are going to remember the contrast.

One of the 2.5 Million?

A cop in DeKalb County Atlanta was charged with aggravated assault after pulling a gun on some kids waiting in a drive-thru at McDonalds.

Does the NRA count that as one of the 2.5 million defensive uses of guns they say happen every year?

Three New "Habitable" Planets Found?

Astronomers have found three new planets that are the most similar to earth so far. The scientists used NASA's Kepler spacecraft to observe variations in star brightness to find the planets. No question, this is a great discovery. But reading the popular press, you'd think they'd found little green men peering back at us with big, sad eyes.

The New York Times article is typical, with the headline "2 Good Places to Live, 1,200 Light-Years Away." That's a colossal overstatement, much like their claim that Mars is "habitable."

Mars is in the habitable zone, to be sure, but it's not habitable in any real sense. It is far too cold and the atmosphere too thin and lacking oxygen for a person to survive without wearing a space suit. Crops will not grow except in hermetically sealed green houses. So far there's no trace of any form of life on the surface. Bacteria could likely be persuaded to live there in the soil fairly easily as they do in Antarctica. And, yes, humans could colonize Mars and perhaps thrive there. But it would be little different from living on the moon

The two parent stars, Kepler 62 and Kepler 69, are more than a thousand light years away. Kepler 62-e is on the inner edge of the habitable zone, and is 60 percent larger than earth. Kepler 62-f is only 40 percent larger than earth, and is at the outside edge of the habitable zone. A third planet, 1.7 times the size of earth, was found in the habitable zone of Kepler 69, a star almost identical to the sun. But those are the only things we know about the planets: we don't know if they're made of rock or gas. And we don't know what the atmospheres consist of.

Just being in the habitable zone doesn't make a planet habitable: the diagram on the right shows that Venus and Mars are both well inside the habitable zone of the solar system, but Mars is far too cold and Venus is far too hot for human habitation.

Why? The atmospheres: Mars was too small to hold on to its atmosphere for very long; what oxygen remained combined with carbon or other elements. It's only got a wispy envelope of carbon dioxide.

Venus, a bit smaller than earth, has a very thick atmosphere of carbon dioxide and clouds of sulfuric acid. The temperature on the surface of Venus is almost 900 degrees Fahrenheit. The atmospheric pressure is 92 times greater than earth's at sea level. It's the greenhouse effect gone mad: the same thing would happen to earth if we pumped enough CO2 into the air.

Finally, the only reason that earth is habitable in the sense that people can live here is that it has an oxygen-nitrogen atmosphere. But such an atmosphere is impossible without bacteria and plant life that constantly produce the free oxygen required for animal life. Without those simpler forms of life, the reactive oxygen would oxidize everything around it. The entire earth would rust and perhaps wind up looking like Mars. So these planets we're finding out in the galaxy wouldn't be truly habitable for humans unless something is constantly replenishing atmospheric oxygen -- something we'd probably call life.

There's no question that this is a very cool discovery: it shows that earth-sized planets are common around stars like the sun at the distance necessary for the right temperature for human habitation. But it's way too early to claim they're "habitable" without any knowledge of the planets' compositions and the constituents of their atmospheres.

Beautiful

What Is Wrong with South Carolina Republican Voters?

Yesterday the National Republican Congressional Committee pulled the plug on funding former South Carolina governor Mark Sanford's run for Congress:

Blindsided by news that Sanford’s ex-wife has accused him of trespassing and concluding he has no plausible path to victory, the National Republican Congressional Committee has decided not to spend more money on Sanford’s behalf ahead of the May 7 special election.

Sanford has been a big story since he began his recent comeback. When a seat opened up in the House of Representatives due to the resignation of Jim DeMint, Sanford decided to run, even though everyone thought his political career was over. He beat 16 other Republicans in primaries to win the right to face Democrat Elizabeth Colbert Busch, Stephen Colbert's sister.

Sanford became infamous when he disappeared from South Carolina in 2009, telling aides that he was going to hike the Appalachian Trail. He was actually visiting his mistress in Argentina. A reporter caught him getting off the plane at the Atlanta airport.

To make matters worse, Sanford later gave tearful and sappy interviews about "finding his soul mate." He was accused of using state money to visit his mistress, and he returned funds to the state he'd spent visiting her in 2008. The South Carolina legislature wrestled with impeaching him, but finally decided against it. His divorce from his wife Jenny was finalized in 2010. He left office in 2011.

When the South Carolina House seat opened up, Sanford had the gall to ask his wife to run his campaign for him, saying,“I could pay you this time." His latest problems arose when his ex-wife caught him trespassing in her house. His excuse? He couldn't bear the thought of his son watching the Super Bowl alone.

Sanford has the worst case of serial idiocy I've ever seen. This guy is a liar, a cheat, a thief, a fool and a jerk. And the most incredible thing is that Republican primary voters chose him over 16 other Republicans!

Were those voters totally oblivious to the stench emanating from Sanford for the last four years? Or did they just not care? What kind of people would vote for a man like Sanford? He has violated every institution that Republicans claim to cherish: honesty, faithfulness, holy matrimony, careful stewardship of public funds, and on and on.

Yes, it's true, forgiveness is a Christian virtue. But just because you forgive someone doesn't mean you should put him in Congress.