Hugos V: E Paucibus Drama

[felice]

Since the last thread was locked, can we discuss Hugo nominating and nominees here without getting in to general political debates/trolling?

EPH isnt bad but from a quick look, i think there might be situations where a small bloc could concentrate on one work and get it nominated by selecting no other works rather than selecting other good possibilities. Maybe i dont understand it but that doesnt really matter.

Yes, in some categories a few dozen people collaborating would be enough to get a single work on the final ballot; that's pretty much unchanged from under the current system. EPH could potentially lower the bar a little bit for such a single work bloc in some situations, depending on how much overlap there is between the other top five potential finalists, but the chances of that making a difference are pretty remote. There have been single bloc-voted works on the final ballot before, including in 1984; it's much less of a big deal than a slate sweeping entire categories. A longlist is probably the most effective way to combat this.

[Darth Richard II]

Since the last thread was locked, can we discuss Hugo nominating and nominees here without getting in to general political debates/trolling?

Probably not, but I wish you luck anyways.

[Lord of Rhinos]

The more you talk about EPH the more I like it. :thumbsup:

[Lily Valley]

Felice,

I really appreciate your coverage on EPH very much. I am very disturbed that there's a chance that a nominee could get bumped with less nominations than a competitor. I KNOW that this is the point, but it still is extremely upsetting. I'll be at SasQuan and I'll be at that miserable meeting (hopefully drunk), so I am giving it some good and hard thought. Thank you for keeping us posted.

BTW, we have pals on several Hugo committees. The business meeting is worth attending.

***I know, but I'll bring a flask and pass it around. Vote, suckers.

Mormont, I'm listening to you, also.

[peterbound]

Damn, missed the thread before the lock.

[felice]

I really appreciate your coverage on EPH very much. I am very disturbed that there's a chance that a nominee could get bumped with less nominations than a competitor. I KNOW that this is the point, but it still is extremely upsetting.

Thanks! 8) Would it help to think of it as five separate elections, rather than one election with five winners? So the winner of one election might have fewer votes than the runner up in another election, but that doesn't matter because they're not competing for the same spot. Eg if there were two popular candidates for one slot, and four for a second slot, whichever wins the second slot will probably have relatively few votes, but that's ok because it was up against more competition. This isn't exactly how EPH works, but I think the basic principle is relevant.

What do you think of the longlist idea?

[Lily Valley]

I think the long list will be off putting to a lot of readers. I think I read more short fiction than most, so for that category it'd be great for me. So what do I do if I'm feeling lazy in the other categories? If I'm worried about slates from unpopular groups, do I go ahead and fill in with "recommendations"? I would not do this and large portions of my ballot have been blank in the past. With the new system gaming, I don't think everyone will be as scrupulous.

On the other hand, it's really nice to think that on a good year you could nominate ALL the works you thought worthy.

While I understand EPH and I do think it's a pretty good system, I honestly think the best solution is more participation from everybody under the current system.

I plan to be at the meeting provided it doesn't interfere with my hangover. I have a lot more thinking to do before I make up my mind.

Keep talking smart people, the discussion really is helping me to clarify my decision.

[felice]

I think the long list will be off putting to a lot of readers. I think I read more short fiction than most, so for that category it'd be great for me. So what do I do if I'm feeling lazy in the other categories? If I'm worried about slates from unpopular groups, do I go ahead and fill in with "recommendations"? I would not do this and large portions of my ballot have been blank in the past. With the new system gaming, I don't think everyone will be as scrupulous.

Ideally people should only nominate works they've read, and it would be completely impractical for most people to read even close to everything on the longlist (even aside from the lack of a voter packet). But you don't need to have read everything that was published in the previous year to nominate works you did read and liked under the current system, and you wouldn't need to read everything on the longlist to nominate some of the works on the longlist. In some categories, like the short fictions and dramatic presentations, reading/viewing many new works from the longlist will be a viable option for a lot of people. In other categories like novel, the main advantage to the longlist would be people nominating works they had already read but not originally nominated, eg "I thought Nugs was the best book of the year, but it looks like not enough people agreed with me. Oh well, Direwolves was pretty good too, and it has a chance of winning, so I'll nominate that instead. It's certain better than any of those ridiculous Dragons and Wyverns books."

The longlist doesn't make any real difference to people who want to follow an anti-puppy recommendation slate; they can and will do that anyway.

On the other hand, it's really nice to think that on a good year you could nominate ALL the works you thought worthy.

The longlist proposal wouldn't do that; indeed, it's incompatible with an increase in number of nominations per person, because that would let a voting bloc nominate a slate of 15 works and sweep the entire longlist just like they swept the final ballot in many categories this year. EPH, on the other hand, could work fine with an amendment to allow more than five nominations per person, but it probably wouldn't help much, because most people already nominate fewer than five works per category.

While I understand EPH and I do think it's a pretty good system, I honestly think the best solution is more participation from everybody under the current system.

And the best solution to the crime problem is for everyone to stop breaking the law. The hard part is how to bring that about... And with the Hugos specifically, even if we did somehow get a thousand extra people nominating, the puppies would only need to get an extra hundred or less to put us back where we started. The problem is that bloc voters have vastly disproportionate power under the current system.

[mormont]

The problem is that bloc voters have vastly disproportionate power under the current system.

This isn't really true, because the problem is not with the system. The system works fine. The problem is twofold:

1. Not enough people participate in nominating.

2. Those nominations that are received have become very spread out, due to the enormous range of stuff being published.

The EPH proposal doesn't address either of these problems. Instead it rigs the system so that nominations for works that don't have much support get part of their support reassigned to boost the vote of more 'popular' nominees. This is rather artificial, arbitrary and doesn't really address either of the above problems - instead it masks them in an attempt to block any repeat of a particular result that upset a lot of people this year but is actually unlikely to be repeated. It's a classic case of fighting the last war.

[felice]

This isn't really true, because the problem is not with the system. The system works fine. The problem is twofold:

1. Not enough people participate in nominating.

I don't think that's case. While more people nominating would be a good thing, the non-puppy participants already far outnumber the puppies, yet the system allows a result that the majority finds to be extremely unsatisfactory. I don't see how that can be anything other than a flaw in the system.

2. Those nominations that are received have become very spread out, due to the enormous range of stuff being published.

I don't think that's a problem in itself (reducing the range of stuff published would not be a desirable solution even if it were achievable!); the problem is that the current system doesn't handle that range well. I think some kind of longlist is probably the best way to address this issue, allowing for a more gradual winnowing down of the field.

The EPH proposal doesn't address either of these problems. Instead it rigs the system so that nominations for works that don't have much support get part of their support reassigned to boost the vote of more 'popular' nominees. This is rather artificial, arbitrary and doesn't really address either of the above problems - instead it masks them in an attempt to block any repeat of a particular result that upset a lot of people this year but is actually unlikely to be repeated. It's a classic case of fighting the last war.

EPH does address the second issue to a significant extent, by requiring different subsets of the nominators to support each finalist, instead of letting the same small subset win five times. I don't see that as rigging the system; it's simply a more democratic system. It's certainly not arbitrary, and I don't think your description of support being reassigned is accurate; every elimination comparison is based on the total unmodified number of nominations for each of the works being compared, and the work with the fewest nominations is eliminated.

What makes you think it's unlikely to be repeated? It was tried last year, it was repeated more successfully this year, and they've stated their intent to do it again next year. And even if the puppies go away of their own accord, EPH does no harm, and prevents any other group from using the same tactic in future. Do you have any idea what the next war might be, if not further bloc voting?

[mormont]

I don't think that's case. While more people nominating would be a good thing, the non-puppy participants already far outnumber the puppies, yet the system allows a result that the majority finds to be extremely unsatisfactory. I don't see how that can be anything other than a flaw in the system.

If it's a flaw, it's how the system has always worked. Looking at 2012, for example, Leviathan Wakes (a novel I both nominated and voted for, BTW) got onto the ballot with 71 nominations out of a total of 958, while The Quantum Thief missed out with 70. Ray of Light (by one Brad Torgersen) got onto the novelette ballot with 37 nominations from 506, while A Long Walk Home by Jay Lake missed out with 36.

Who can say whether 'the majority' found these nominations to be 'unsatisfactory' and would have preferred the narrowly-eliminated contenders to be on the ballot instead? The people who nominated Leviathan Wakes were obviously outnumbered by the people who didn't. (By more than 13 to one, actually.) Should we place asterisks by these nominations, because clearly the system they were nominated under was 'flawed'? Obviously not. It was a fair system. Eligible voters nominate and the five contenders with most nominations are the shortlist. What's flawed or unfair about that? Nothing, apparently: nobody complained about these results, did they?

The problem is not with the system. It's with the outcome the system produced, this one time. Hard cases make bad law.

EPH does address the second issue to a significant extent, by requiring different subsets of the nominators to support each finalist, instead of letting the same small subset win five times. I don't see that as rigging the system; it's simply a more democratic system. It's certainly not arbitrary, and I don't think your description of support being reassigned is accurate; every elimination comparison is based on the total unmodified number of nominations for each of the works being compared, and the work with the fewest nominations is eliminated.

Heck, I'm still not sure how the damn system works, so I'll take your word for that point. It's still arbitrary, and still not noticeably more democratic. If I nominated five works, and two of them are really obscure, any system that gives more weight - in any way, under any circumstances - to the other three nominations is being arbitrary.

A nominee supported by different subsets of the nominators is not necessarily a better supported nominee, by the way. It may instead represent the lowest common denominator. This is the system indulging in mind-reading, always to be avoided in any voting system.

What makes you think it's unlikely to be repeated?

Several of the Sad Puppies have come out and said so. I genuinely don't think they intended to sweep the ballot in the way that they did, and even if they did, the reaction has clearly put quite a lot of them off.

The Rabids might try again, but they succeeded in large part due to surprise - the numbers of people nominating next year will be considerably increased, I expect, and yes, I believe this will drown most, if not all, of the Rabid Puppies in genuine votes.

And even if the puppies go away of their own accord, EPH does no harm

Obviously, I disagree.

[Lily Valley]

Felice, I'm trying to find a link to the long list idea. I responded quickly without thinking. The long list proposal would have a runoff election before the final nominees were announced, yes?

Google is not helping me today.

Lastly, I hate 4+6.

[felice]

If it's a flaw, it's how the system has always worked.

That's right. The flaw hasn't been so dramatically exploited before now, but it's always been there.

Looking at 2012, for example, Leviathan Wakes (a novel I both nominated and voted for, BTW) got onto the ballot with 71 nominations out of a total of 958, while The Quantum Thief missed out with 70. Ray of Light (by one Brad Torgersen) got onto the novelette ballot with 37 nominations from 506, while A Long Walk Home by Jay Lake missed out with 36. Who can say whether 'the majority' found these nominations to be 'unsatisfactory' and would have preferred the narrowly-eliminated contenders to be on the ballot instead?

It's not about single finalists, it's about the set of finalists in a category as a whole. If the majority didn't like Among Others, Embassytown, A Dance With Dragons, Deadline, or Leviathan Wakes, then there could be a problem; but at least 175 people nominated one or more of them, and probably closer to 620 out of 958 (though the actual number will be somewhere in between, depending on how much overlap there was between nominators of the different finalists - and the lower the number, the more chance EPH would have produced a different outcome). Other people won't have nominated any of the finalists, but accept that at least some of them are decent works worthy of consideration even if they're not the finalists they personally would have chosen. It might or might not technically be majority satisfaction, but it's probably about as close to it as practically possible.

This year, a group of 15% or so of nominators have taken all the finalist slots in many categories, for works which pretty much everyone else consider to be largely unworthy. That's not even close to optimal satisfaction.

The problem is not with the system. It's with the outcome the system produced, this one time. Hard cases make bad law.

A system that predictably produces a bad outcome in response to an easily repeatable set of inputs is a flawed system.

Heck, I'm still not sure how the damn system works, so I'll take your word for that point. It's still arbitrary, and still not noticeably more democratic. If I nominated five works, and two of them are really obscure, any system that gives more weight - in any way, under any circumstances - to the other three nominations is being arbitrary.

You didn't comment on my dragons and direwolves example in the previous thread. Did it make sense to you? If not, is there any specific step you didn't understand?

What definition of "arbitrary" are you using? The weighting is entirely deterministic, and has a clear democratic purpose. For the ranking step, everyone's ballot has exactly the same total weight, divided equally amongst their surviving nominations. For the elimination step, every nomination has exactly the same weight, irrespective of how many surviving works are on the same ballot. And eliminating from the bottom up means someone who starts off with five nominations on their ballot isn't disadvantaged in the long run against someone who only nominates a single work.

A nominee supported by different subsets of the nominators is not necessarily a better supported nominee, by the way. It may instead represent the lowest common denominator.

Rather less so than the current system! The current system returns the five works with the broadest support (no matter how narrow "broadest" is - well under 20% for most finalists), even if it's the same small subset of nominators that support all five lowest common denominator finalists. EPH makes it easier for a controversial or relatively-obscure-but-highly-regarded-by-those-who've-read-it work to become a finalist, by "using up" some of the populist vote on the other finalists.

Several of the Sad Puppies have come out and said so. I genuinely don't think they intended to sweep the ballot in the way that they did, and even if they did, the reaction has clearly put quite a lot of them off.

The Rabids might try again, but they succeeded in large part due to surprise - the numbers of people nominating next year will be considerably increased, I expect, and yes, I believe this will drown most, if not all, of the Rabid Puppies in genuine votes.

The Sads aren't that relevant. And will the surge in non-puppy nominations be proportionally greater than the surge in Rabids bolstered by their success this year? Even if it is, will it be enough? There were enough puppies this time to get third place in novel, which suggests there were well over a hundred of them; even doubling the number of non-puppy nominations might still let the puppies dominate some categories. And even if the puppies all quit, their method has been demonstrated to be effective and could be taken up by other groups in future (I expect next year's finalists to be dominated by one or more anti-puppy slates, for a start).

[felice]

Felice, I'm trying to find a link to the long list idea. I responded quickly without thinking. The long list proposal would have a runoff election before the final nominees were announced, yes?

There isn't a formal proposal as yet, and there are multiple ways it could be done. My preferred option doesn't involved a runoff election. Instead, half way through the nomination period, just announce "these are the current top 15 nominations in each category" (in random or alphabetical order). If people want to change their nominations in response to that information, they can (just like you can currently update your final vote whenever you like via the Sasquan website); but if they don't, their original nominations are still counted for determining the finalists. Most of the time, the finalists would be the most popular works out of the announced top 15, but it's still theoretically possible for a work that isn't in the top 15 to become a finalist (eg lots of people responding to the announcement with "What, Wet for Nessie isn't in the top 15?!? That's outrageous, I must nominate it myself!").

There's been some discussion in the various massive threads on Making Light, but it's probably not worth digging through those.

Any such proposal would need someone who can actually attend the Business Meeting(s) to champion it; there's no way I can make it (and I'm a really awful public speaker anyway).

Lastly, I hate 4+6.

Good 8)

[mormont]

That's right. The flaw hasn't been so dramatically exploited before now, but it's always been there.

It's not about single finalists, it's about the set of finalists in a category as a whole.

This is not very coherent, I'm afraid. If it's about the set of finalists being unacceptable when considered as a whole, not single finalists, then that has never happened before, to my knowledge. The flaw might therefore have been 'there all along', but this is still a one-off result. You don't make the rules based on one-off results. That would be daft.

A system that predictably produces a bad outcome in response to an easily repeatable set of inputs is a flawed system.

A system that is made needlessly complex and opaque to safeguard against a result that has happened once only in the lifetime of the system, is a bad system.

What definition of "arbitrary" are you using?

I've explained above why the system is arbitrary IMO. It's assigning additional weight to some of my picks, based on factors other than my preference. That's arbitrary.

EPH makes it easier for a controversial or relatively-obscure-but-highly-regarded-by-those-who've-read-it work to become a finalist, by "using up" some of the populist vote on the other finalists.

Why is that a good thing?

The Sads aren't that relevant.

So it has been argued, but the truth is nobody actually knows how relevant each Puppies campaign was, because of the huge overlap (a deliberate tactic by Day, I believe, to magnify his impact). My own feeling is that Day on his own can't muster enough votes to repeat his success, if fandom gets up and nominates in large numbers, and if the Sads deliberately distance themselves from Day and do not propose an actual slate next year. I'm taking them at their word on that, largely because I prefer to wait and see how big the problem actually is before trying to fix it. The EPH proposal, on the other hand, is a rush to fix a problem right away by a complicated method that appeals to people who instinctively view voting as being more about maths than psychology.

[felice]

The flaw might therefore have been 'there all along', but this is still a one-off result. You don't make the rules based on one-off results. That would be daft.

"This security hole might have been there all along, but it's only been exploited once. You don't patch security holes based on one-off theft of all our customers' credit card details. That would be daft."

A system that is made needlessly complex and opaque

You didn't comment on my dragons and direwolves example in the previous thread. Did it make sense to you? If not, is there any specific step you didn't understand?

I've explained above why the system is arbitrary IMO. It's assigning additional weight to some of my picks, based on factors other than my preference. That's arbitrary.

It assigns equal weight to all your surviving picks. Any of your picks that have already been eliminated didn't survive because they have fewer unweighted nominations than the works that eliminated them. That's not arbitrary.

Why is that a good thing?

It's the opposite of lowest common denominator, which you raised as a bad thing.

So it has been argued, but the truth is nobody actually knows how relevant each Puppies campaign was, because of the huge overlap...

I prefer to wait and see how big the problem actually is before trying to fix it. The EPH proposal, on the other hand, is a rush to fix a problem right away by a complicated method that appeals to people who instinctively view voting as being more about maths than psychology.

The Rabids nominated several works that made the final ballot without Sad help. None of the Sad nominations that weren't on the Rabid slate made the final ballot. That seems fairly conclusive to me.

The EPH proposal will take at least two years to pass; why not support it this year just in case, and see what happens next year before deciding whether or not it should be ratified? If you're wrong and next year's nominations are swamped by slate works again, then EPH can be ratified in time to help for 2017. If you're right, you can vote against ratification. But if we hold off another year before even starting the process, the puppies could take over the Hugos for three years instead of just two.

[mormont]

"This security hole might have been there all along, but it's only been exploited once. You don't patch security holes based on one-off theft of all our customers' credit card details. That would be daft."

Apples and oranges. A credit card system is not an electoral system: the issues of complexity, legitimacy etc. aren't relevant, for a start. For another thing, this comparison inherently assumes that the Puppies sweep is illegitimate - a democratic fraud of some sort. I'm not necessarily willing to concede that: this whole problem is, after all, not so much about the votes as it is about the intentions of the voters. As the Puppies have pointed out, they didn't technically break any rules - they just failed to respect the spirit of the award. Trying to rig a democratic system to account for the intentions of the voters, as I have said, is a misguided enterprise IMO: but in any case, the issue here is clearly not the same as a security breach.

You didn't comment on my dragons and direwolves example in the previous thread. Did it make sense to you?

You keep asking me that, and I keep not answering. Why? Because it doesn't matter to the issues we're disagreeing on. Honestly, as long as we are not agreeing whether the basic assumptions of the system are even valid, and agree that it requires explanation even to someone familiar with AV elections, your example is beside the point, at least as far as I'm concerned.

It assigns equal weight to all your surviving picks. Any of your picks that have already been eliminated didn't survive because they have fewer unweighted nominations than the works that eliminated them. That's not arbitrary.

My surviving picks =/= my picks. The distinction is, yes, arbitrary, from my point of view as the voter.

The Rabids nominated several works that made the final ballot without Sad help. None of the Sad nominations that weren't on the Rabid slate made the final ballot. That seems fairly conclusive to me.

The assumption that the overlapping noms made it on because of the Rabids, though, is just that - an assumption. It may seem logical, but it's an assumption, not a fact. It's equally possible that this is an artefact of how the Sad voters rated Torgersen's picks. The issue here is one that the EPH crowd overlook but is crucial to the whole issue - insufficient data. One unexpected sweep is not a good enough data set to justify radical change to the nomination process.

Why not support EPH pro tem while we gather more data? Because the proposal still has fundamental problems, for me, even if there is another sweep. Even if I were to be persuaded that something like this is necessary, I'd rather not see this enacted as it stands.

[felice]

You keep asking me that, and I keep not answering. Why? Because it doesn't matter to the issues we're disagreeing on. Honestly, as long as we are not agreeing whether the basic assumptions of the system are even valid, and agree that it requires explanation even to someone familiar with AV elections, your example is beside the point, at least as far as I'm concerned.

You keep insisting that the system is excessively complex and opaque and you don't understand it. I've provided what seems to me like a nice simple example that demonstrates how it works, how the results could be different from under the current system, and why that's a good thing. If there's a problem with the example, I'd like to know what it is so I can fix it. If there's not a problem, then your continued objections could be seen as disingenuous.

It requires explanation because it's new. I don't think it's any more complicated than instant runoff.

Why not support EPH pro tem while we gather more data? Because the proposal still has fundamental problems, for me, even if there is another sweep. Even if I were to be persuaded that something like this is necessary, I'd rather not see this enacted as it stands.

What would you rather see enacted instead, then, if it turned out that the puppies were here to stay?

[mormont]

You keep insisting that the system is excessively complex and opaque and you don't understand it.

Sure. But my point is that any unnecessary complexity is excessive, and the case for this additional complexity being necessary has not (IMO) been made.

On that, we're clearly not going to agree, so let's agree to disagree instead. :)

What would you rather see enacted instead, then, if it turned out that the puppies were here to stay?

If things pan out this way, I've already said I think the best response is to drown the Puppies in legitimate votes - to actively encourage people to nominate. If necessary, that might include lowering the barriers to nominating, but based on the number of people already provoked to get a supporting membership for Sasquan, I think that won't be necessary.

Any other tweaks should be simple, minimal, and easy to explain to even a complete newcomer to the awards. I've already said I like four and six for those reasons. Another interesting possibility would be infinite nominations: allowing voters to nominate as many works as they like. In principle there's no reason to limit nominations, other than the practical one of making more work for the organisers: allowing infinite noms would mean more nominations, likely boosting the vote for many stories and making the race more competitive.

[felice]

Sure. But my point is that any unnecessary complexity is excessive

But it's not complex. If you disagree, point to the part of the example that's hard to follow.

Compare the two works with lowest Point totals, and eliminate the one appearing on the fewest Ballots; repeat until there are only five left. Each Ballot is worth one Point, divided equally amongst the not-yet-eliminated works nominated on that Ballot.

If things pan out this way, I've already said I think the best response is to drown the Puppies in legitimate votes - to actively encourage people to nominate. If necessary, that might include lowering the barriers to nominating, but based on the number of people already provoked to get a supporting membership for Sasquan, I think that won't be necessary.

I expect there'll be a lot more votes than usual this year due to extra supporting members. How many extra nominations that will lead to next year remains to be seen; nominating is a lot harder than voting. And more nominations only help to the extent that the new participants are reading and liking the same works; broadening the range of works nominated does nothing to combat slates. Lowering barriers has problems of its own - you'd be lowering the barriers for potential (or sockpuppet) puppies, too, and supporting a voting bloc is much easier than reading and evaluating a lot of newly released works to decide which ones to nominate.

Any other tweaks should be simple, minimal, and easy to explain to even a complete newcomer to the awards. I've already said I like four and six for those reasons. Another interesting possibility would be infinite nominations: allowing voters to nominate as many works as they like. In principle there's no reason to limit nominations, other than the practical one of making more work for the organisers: allowing infinite noms would mean more nominations, likely boosting the vote for many stories and making the race more competitive.

Four and Six is simple, minimal, easy to explain, and hopelessly ineffectual. It also increases the work voters need to put in to evaluate all the finalists by 20%.

The problem with unlimited nominations is most people already nominate fewer than five works per category, so raising or eliminating the limit would make little difference. Ranked nominations would encourage people to nominate a greater number of works, since there'd no longer be any harm in nominating works you think are good but not as good as you first choices; but that would be a major change to how people go about nominating (something EPH leaves unchanged), and require a new tallying system at least as complicated as EPH.