There are a lot of misconceptions about superdelegates. Every week I hear Mark Shields on the NewsHour say something like "the superdelegates were supposed to make the selection in a situation like this." Well, yes and no. The problem is that superdelegates are from another era (and yes, 1980 is that long ago).
A little history will illuminate my argument. Until 1968, primaries were rare. They were created in the Progressive Era as part of attempts to abolish the 'smoke-filled' room. These turn-of-the-last-century devices (1900-1920s really) included ballot propositions, direct election of senators, nonpartisan offices, the unicameral legislature in Nebraska, and the "plural executive" of states like CA (where the atty general, insurance commissioner, secretary of education, and secretary of state are not appointed, but separately elected). The line-item veto was also part of this, as part of the progressives' weird combination of popular democratic participation and trust in a strong executive. You can see, sort of, how FDR and Huey Long descended from all this.
Well, in 1968, the Democratic party was a fiasco. Hubert Humphrey won because of insiders alone, it was perceived. Protests famously marred the convention. This followed the 1964 Mississippi Freedom Democratic party I earlier blogged about. So, the Democrats sought reform. In 1972, the schedule shifted mostly to primary elections. The purpose was to make the process seem more open. Democrats also insisted on proportional representation of delegate slates. (Previously, you also had many ways of messing with the primary system, even where they existed. For example, there was the "favorite son" situation where, for a prominent example, California's Governor Brown would run as a candidate for president in his own state in the 1960 election, get all others (like JFK) not to put their names on the ballot, then "win" all the delegates of that state in order to be a kingmaker at the convention. Brown delivered for JFK after making a play for the VP slot and losing out).
The unexpected side effect was to change the source of legitimacy in the Democratic party! Once party members began to vote in nationwide, the idea began to grow that only an "elected" candidate should be the nominee. This is a breathtakingly new idea in the history of American political parties.
Republican history was different. First, they were farther along in this process (being more closely linked to the Progressives anyway, since progressives were breakaway Republicans initially). But they also had fewer divisions as a party. They were largely united on the Vietnam war and civil rights in the 1960s and early 1970s. (for both). Nixon's "southern strategy" didn't split the party because the south was not really Republican at that time - all it did was split the Democrats. After Watergate, the party struggled, but not against the perception of inner-party corruption. Democrats had a lock on that perception. Because the Republicans didn't have substantial minority voices to contend with, there was no pressure also to change winner-take-all primaries. Once the Republicans began to contend with a "solid south" for the Republicans, the possibility of party division began to rise. In 2008, we saw party division rampant along sectional and ideological lines, but the WTA system dampened its effects. Republicans are not yet grumbling about this, but if McCain loses, evangelicals may start bitching and demanding proportional representation like the Dems have (if the Dems win, it will show that their party wasn't torn apart by this also, reversing the convetnional wisdom of today)
In 1972, McGovern was perceived by party insiders in the Democratic party as a disaster borne of primaries. In 1976, the primaries produced a 'dark horse' - Jimmy Carter - whose presidency was also considered a disaster. So, in 1980, after the third electoral cycle with primaries, the DNC met and debated how to proceed. It was far from obvious that voters should choose the party nominee. As I mentioned, this was a new idea and none of the party leaders had grown up with it. Nor had most voters. So they adopted superdelegates, hoping there would be a "primary" of sorts also among party leaders.
Well, in the 1980s and 1990s, the superdelegates played no significant role that the public saw. In 1988 and 1992, the primaries produced unexpected nominees and the superdelegates did nothing. By 2000, a generation had grown up believing that the president was to be elected by party members. The advent of 24-hour news cycle and the internet also changed politics from an insiders' game that may occupy a few minutes of a 1/2 hour news broadcast into a widely publicized wash of media attention. Most of the people who designed the superdelegate system 20 years before were no longer in positions of power. So you have a new DNC also that "grew up" believing that primary voters were supposed to make the decisions. Ironically in the context of the current election, it was the Clinton DNC that really made this new view a reality, as baby boomers in their 40s and 50s took over the DNC with their post-Watergate post-1972 political educations.
Then the elections in 2000 and 2004 brought a revival of interest in voting and electoral details, and a tendency to brand the undemocratic features of our political system as illegitimate.
So, whatever the superdelegates were designed for a quarter of a century ago no longer matches the expectations of anyone today. The clear majority of voters, politicians, DNC members, and political operatives assume that party voters get to choose the presidential candidate - indeed, all levels of candidates. This is really a revolution in American democracy that most people don't understand because they think it was always like this. Superdelegates are accordingly nervous. There's a reason that Nancy Pelosi is waving a red flag saying that superdelegates won't go against the voters. They know that it's party suicide today, which was not necessarily true quarter century ago.
The 7% of GOP that are unpledged and the 20% Dem superdelegates are a relic of bygone days. I expect that in 2009, the Democrats will begin to change superdelegates into honorary delegates without votes. They may also trend to winner-take-all voting, while the GOP goes the other way.