Reviewers vs Honesty

[Contrarius]

Ratings systems are obviously far from perfect, but I do think they're helpful. After all -- if ratings weren't helpful in real life, we wouldn't have grades in schools or scores from Consumer Reports or any of our other ubiquitous systems of assessing overall quality/competence.

When I see a 1-5 scale, I don't think of percentages (as Pat keeps referring to). Instead, I think of letter grades: 1=F, 2=D, 3=C, 4=B, 5=A. When I rate books myself, I use the general and imprecise rule:

5=I really loved it, and it had something to say beyond the obvious plot line

4=I enjoyed reading it; it was engrossing

3=it seemed average to me; nothing terribly special, nothing especially horrible

2=it may have had some merits, but it also had serious problems

1=it was awful, and it really offended me in some way

That "1" is there for a reason -- it's not a 2-5 scale or a 3-5 scale, after all -- so there's no good rationale for never using it at all.

Also -- it's logical that reviewers should average above a "3" on their reviews, even if those reviews are perfectly honest. After all, we are all more likely to read books that we expect to like -- and to avoid books that we expect to not like. So our sampling techniques are biased. If we were forced to put on blindfolds when we pick out books, THEN we might expect to see a more bell-shaped distribution of our rating curves. Sampling bias can also account for the rising ratings often seen with books in a series -- if folks enjoyed the first book they are more likely to read the second book, so you are likely to get a higher percentage of fans as reviewers for each subsequent book.

When I look at reviews before reading a book, I usually pay more attention to the 3 and 4 star reviews. Those will usually give me a more realistic overview of both the strengths and weaknesses of the book in question, as opposed to either extreme (1 or 5). And I love the "free sample" feature at Amazon! Great way to check out the first 10% of the book without commitment. :)

[PaulineMRoss]

Pauline, I think you mean bought reviews, rather than biased reviews.

Not at all, that's a very different problem. Reviewers aren't 'bought', exactly, more in bed with the publishers. It's an uneasy alliance; the publishers dish out review copies of books and perhaps author interviews and the like, in the hopes of a favourable review; the reviewers want to build and maintain their blogs, and be the first with the newest and hottest and most controversial. Inevitably that creates pressure for a certain bias in the reviews (in addition to their personal bias, of course). It's not exactly dishonest, but it's not necessarily very helpful either.

I'll be honest, I've more or less given up on reading the semi-professional review blogs, because I've found them to be too uncritical, or else stuffed with giveaways and interviews and other fluff, or else they cover too broad a genre base for my taste. In some cases, they're so formulaic that they give no real flavour of the book under review. I far prefer reading the less polished but more heartfelt reviews on Goodreads, which has a lot of trash but also some genuinely useful reviews.

[TrackerNeil]

You see, I think this is actually pretty overblown. Especially with blogger/reviewers who have been around longer. If anything, I think sometimes bloggers go too far to show that this isn't the case. The bigger issue I see is more of a groupthink one. In many ways we bloggers can end being kind of clique-ish. We all like very similar things. When we see someone we 'know' and trust rate a book highly, we rush to read ourselves because we think we'll like it too. I think that it's pretty common for us to not look as closely as we would have in the absence of the earlier opinion from one we trust and too often echo those thoughts.

Well, I would certainly agree that I usually see the same books reviewed on just about every site, which IMO isn't very useful. If three reviewers have said they really like The King's Blood, I very much doubt that another five saying the same thing are going to impact my decision to buy. I get that reviewers have no incentive to review obscure novels but popular ones, but oftentimes I find myself skimming sites asking myself, "Can someone review something I haven't already heard of?"

[kcf]

Not at all, that's a very different problem. Reviewers aren't 'bought', exactly, more in bed with the publishers. It's an uneasy alliance; the publishers dish out review copies of books and perhaps author interviews and the like, in the hopes of a favourable review; the reviewers want to build and maintain their blogs, and be the first with the newest and hottest and most controversial. Inevitably that creates pressure for a certain bias in the reviews (in addition to their personal bias, of course). It's not exactly dishonest, but it's not necessarily very helpful either.

As I said above, I think this is less a problem than most think. At least among the more established blogger/reviewers. (or at least among the few I choose follow closely)

I'll be honest, I've more or less given up on reading the semi-professional review blogs, because I've found them to be too uncritical, or else stuffed with giveaways and interviews and other fluff, or else they cover too broad a genre base for my taste. In some cases, they're so formulaic that they give no real flavour of the book under review. I far prefer reading the less polished but more heartfelt reviews on Goodreads, which has a lot of trash but also some genuinely useful reviews.

It's all about finding which reviews/reviewers work for you. I have serious trust issues with Goodreads and Amazon, so I ignore them. I follow the opinions of a few reviewers that I trust and know well enough to balance my tastes against.

But I'm curious, how do you define a semi-pro review blog?

[TrackerNeil]

Not at all, that's a very different problem. Reviewers aren't 'bought', exactly, more in bed with the publishers. It's an uneasy alliance; the publishers dish out review copies of books and perhaps author interviews and the like, in the hopes of a favourable review; the reviewers want to build and maintain their blogs, and be the first with the newest and hottest and most controversial. Inevitably that creates pressure for a certain bias in the reviews (in addition to their personal bias, of course). It's not exactly dishonest, but it's not necessarily very helpful either.

This is IMO an excellent point. The parts of the system, whether by design or happenstance, work together. Authors who aren't backed by a significant publisher can't get reviewed by most of the amateurs and any of the professionals, which of course makes their work seem less relevant, which makes it that much harder for them to get attention, and down, down, down it spirals.

[kcf]

I get that reviewers have no incentive to review obscure novels but popular ones, but oftentimes I find myself skimming sites asking myself, "Can someone review something I haven't heard already of?"

Incentive is the wrong way to look at it and that makes it sound much more complicated and underhanded than it really is. Simply put, the typical blogger/reviewer is a fan first and the whole blogging thing merely a hobby that helps amplify the whole fun of it. As fans, we all have books we are looking forward to reading. We all have limited reading time. So, we naturally choose the books that we are most looking forward to. Most of the time I'm looking forward to reading the same books as Pat, Aidan, RobB, Justin, Adam and all the others.

With that in mind, I often do try and avoid the hot books at the moment (at least if others have gotten to them first - such as King's Blood). And I do make an effort to regularly read books that I'm not seeing a lot of coverage for on other blogs that I follow - for both variety on my blog and the blogosphere and for my own reading enjoyment (because if I didn't perceive it as a benefit to my own reading enjoyment, I wouldn't 'waste' my time doing it).

[TrackerNeil]

And I do make an effort to regularly read books that I'm not seeing a lot of coverage for on other blogs that I follow - for both variety on my blog and the blogosphere and for my own reading enjoyment (because if I didn't perceive it as a benefit to my own reading enjoyment, I wouldn't 'waste' my time doing it).

Well, reviewing aside, would you read ANY book in which you had interest? Any at all?

[kcf]

Authors who aren't backed by a significant publisher can't get reviewed by most of the amateurs and any of the professionals, which of course makes their work seem less relevant, which makes it that much harder for them to get attention, and down, down, down it spirals.

I actually disagree with this. Especially with the advent of ebooks and electronic review copies, it no longer takes such a large marketing budget to get books in the hands of reviewers (though personally, I'm an old school guy who wants the paper book). Smaller publishers like Night Shade and Angry Robot are having more of their books reviewed than ever because they can distribute electronic copies.

But it also depends on how you are defining significant publisher. It's certainly the case for vanity/self publish/so-called independent publishing to not be able to get their books into the hands of the the more 'significant' bloggers and other reviewers. We simply don't risk it because there is so much bad out there and we don't have the time or inclination to try and sort through. (At least for the time being, it seems that when a quality self-published book comes a long, if finds it's way into traditional publishing and then I'll take a shot.) My inbox/spam filter literally receives dozens of requests every day for me to review. I'm forced to draw a line someplace - right now that line is 'traditional' publisher (note, that doesn't necessarily mean big 6) and paper copy of a book. I find that cuts things down quite a bit (though I still receive 300+ books a year and only have time to read/review about 10% of those).

[TrackerNeil]

I actually disagree with this. Especially with the advent of ebooks and electronic review copies, it no longer takes such a large marketing budget to get books in the hands of reviewers (though personally, I'm an old school guy who wants the paper book). Smaller publishers like Night Shade and Angry Robot are having more of their books reviewed than ever because they can distribute electronic copies. But it also depends on how you are defining significant publisher. It's certainly the case for vanity/self publish/so-called independent publishing to not be able to get their books into the hands of the the more 'significant' bloggers and other reviewers. We simply don't risk it because there is so much bad out there and we don't have the time or inclination to try and sort through. (At least for the time being, it seems that when a quality self-published book comes a long, if finds it's way into traditional publishing and then I'll take a shot.) My inbox/spam filter literally receives dozens of requests every day for me to review. I'm forced to draw a line someplace - right now that line is 'traditional' publisher (note, that doesn't necessarily mean big 6) and paper copy of a book. I find that cuts things down quite a bit (though I still receive 300+ books a year and only have time to read/review about 10% of those).

It may not take a marketing budget to get attention from reviewers, but it takes a publisher's name on the book spine. I don't understand the "risk" of reviewing an independently published book, nor the blithe assumption that all good writers eventually get book deals. I attended a reading last weekend where three of the four books in question were self-published, and for the most part I couldn't tell the difference.

[kcf]

Well, reviewing aside, would you read ANY book in which you had interest? Any at all?

No. I don't have anywhere near the time to read every book that I have any interest in reading. I have well over 1000 books in my office waiting to be read. Realistically, probably only 250 of those books really appeal to me. Right now balancing work, family and everything else life throws at me, I only read around 30 books a year. So, I have nearly a 10 year supply of books to read right now, and more books arrive in the mail every day.

[kcf]

I don't understand the "risk" of reviewing an independently published book, nor the blithe assumption that all good writers eventually get book deals.

The risk - it's really pretty simple. Let's say that I have 10 hours a week for reading (I wish it were that high, but it's not). If I spend even one of those hours reading a book that is I really dislike, then I'm using my time very ineffienctly. To put it another way, if my time is worth $100/hour, then I've wasted $10 by reading a book I dislike. The vetting process of traditional publishing lowers the risk of me reallly disliking a book. It doesn't elimiante it, but it lowers risk, and that's what it's about. I do take risks, but those are calculated risks of which I have information about. Gaining the information I would need to make self-published books a calculated risks, takes a lot more time in part because there is no transparent vetting process. That is time I don't have. So, I draw the line. Yes, a few gems will get by, but I have enough good books to read already. And yes a relatively higher percentage of those gems is at this time eventually finding its way into traditional publishing. So, I'll read it eventually.

OK, enough procrastinating today. Time to get some actual work done.

[Maithanet]

Also -- it's logical that reviewers should average above a "3" on their reviews, even if those reviews are perfectly honest. After all, we are all more likely to read books that we expect to like -- and to avoid books that we expect to not like. So our sampling techniques are biased. If we were forced to put on blindfolds when we pick out books, THEN we might expect to see a more bell-shaped distribution of our rating curves. Sampling bias can also account for the rising ratings often seen with books in a series -- if folks enjoyed the first book they are more likely to read the second book, so you are likely to get a higher percentage of fans as reviewers for each subsequent book.

I agree with this. I am very far on the amateur edge of book reviewing. It's just something that I do to talk about books that I've read and want to talk about, and as a way of participating in something my gf is interested in, since it's her book blog.

I would never review a book I didn't finish, and I'm not going to finish a 1 star book. So you could already say that the scale goes from 2 stars (poor, barely able to finish) to 5. And if a book is in the 2-2.5 star range, I would write a review if I wanted to, but in all likelihood I wouldn't want to. I write reviews because I want to talk about that particular book or subject, and it's unlikely that a poor or mediocre book is going to warrant such discussion. It's possible, certainly, if it was a book that maybe was strong in some aspects but weak elsewhere. But the majority of my reviews are going to be in the 3-4.5 star range. I'm leery of giving 5 stars to a book, just because there are very very few books that I would consider perfect or near perfect. I can think of maybe six I've read in my whole life. So my five star ratings system is mostly limited to books between 2.5 and 4.5 stars.

[TrackerNeil]

And yes a relatively higher percentage of those gems is at this time eventually finding its way into traditional publishing. So, I'll read it eventually.

Well, you won't read "The Mill River Recluse", because even after selling 400,000 copies, author Darcie Chan still couldn't get a book deal.

[PaulineMRoss]

I have serious trust issues with Goodreads and Amazon, so I ignore them.

That I understand. I have trust issues with Amazon too. Goodreads I'm used to, and I know my way round it, plus (big help) you get your (trustworthy) friends' reviews at the top of the list for any book, so that makes it easier to wade through the dross. I've just not found a review blogger yet who I regard as trustworthy, and is also on my wavelength. Mind you, my tastes are eccentric, so that's partly my own fault.

But I'm curious, how do you define a semi-pro review blog?

Anyone who gets free books, ARCs, big-name author interviews and the like. Free stuff, basically. It doesn't automatically mean their reviews are biased in favour of the publishers, obviously, but it does make them slightly suspect. Glowing reviews in particular are hard to take seriously.

I don't exactly blog (I write reviews for Goodreads and then dump them to a blog for ease of recall), but the first time I was offered a free book to review, I made a decision never to do that. I think it's the only way to be truly independent. I don't disparage those who do accept the free stuff, but I reserve the right to regard them with a degree of suspicion, that's all.

[sologdin]

ease of recall

that's the best reason i've got for participating in things like goodreads--drafting out a "review" is just an easily indexed way of organizing my thoughs on a writing. so it's more of a reading journal that's not private. if the "reviews" happen to extend the jurisdiction of my personal tastes to other persons, that's a perverse bonus.

[kcf]

Anyone who gets free books, ARCs, big-name author interviews and the like. Free stuff, basically. It doesn't automatically mean their reviews are biased in favour of the publishers, obviously, but it does make them slightly suspect. Glowing reviews in particular are hard to take seriously.

That's a good enough definition I suppose (though I cringe a bit thinking of my blog as semi-professinal). A healthy dose of skepticsim is generally a good thing.

[Seli]

This is IMO an excellent point. The parts of the system, whether by design or happenstance, work together. Authors who aren't backed by a significant publisher can't get reviewed by most of the amateurs and any of the professionals, which of course makes their work seem less relevant, which makes it that much harder for them to get attention, and down, down, down it spirals.

But the bar for significant publisher is not that high, although it probably depends on good networking and reputation. Smaller publisers as PsPublishing or Twelft Planet press do get reviews on the larger sites (for example Strange Horizons).

[pat5150]

Anyone who gets free books, ARCs, big-name author interviews and the like. Free stuff, basically. It doesn't automatically mean their reviews are biased in favour of the publishers, obviously, but it does make them slightly suspect. Glowing reviews in particular are hard to take seriously. I don't exactly blog (I write reviews for Goodreads and then dump them to a blog for ease of recall), but the first time I was offered a free book to review, I made a decision never to do that. I think it's the only way to be truly independent. I don't disparage those who do accept the free stuff, but I reserve the right to regard them with a degree of suspicion, that's all.

If you go to cons, ask any publicists and editors how my reviews are biased in favour of the publishers. :P I've been told that I'm one of the toughest guys to please, what with only a few titles getting a score of more than 7.5/10 every year. And based on the reviews I've read from them, I'm pretty sure the same can be said of people like Ken, Rob, Larry, Adam, yada yada yada.

We have all made a name for ourselves over the years, so I doubt that any of us would be stupid enough to compromise that credibility by getting in bed with certain publishers. Readers are not idiots. If we try to fuck with them, they'll realize it and call us out on that.

Newcomers excited about getting free stuff, perhaps. But us old-timers? I can't see that happening. . .

Patrick

[Werthead]

The original purpose behind my blog was as a storage place for my Amazon reviews, where a score is required. So the blog reviews end up with a score. If it wasn't for that requirement, I don't think I'd bother too much with them. I advocate simply ignoring them, if they are not to people's tastes.

It's an uneasy alliance; the publishers dish out review copies of books and perhaps author interviews and the like, in the hopes of a favourable review; the reviewers want to build and maintain their blogs, and be the first with the newest and hottest and most controversial

I cannot speak for others, but I like to read books and have zero interest in being the 'newest' and 'hottest' and 'most controversial'.

I don't understand the "risk" of reviewing an independently published book,

A good question, and one that is becoming more important as we are now seeing authors who are good enough to get a publisher's deal now self-publishing by choice, rather than by necessity. This dilutes the waters from the situation a few years back when we could confidently say that 99.999% of all self-published titles were unreadable shit (as opposed to maybe 80% of professionally-published titles). Now there's some decent stuff out there, but tracking it down is still a lot more work than going through professionally-published books. A better filter system for self-published titles (i.e. not Amazon reviews) is required before taking a risk on them, I think.

Glowing reviews in particular are hard to take seriously.

Why? There is stuff out there that is actually really, really good.

the first time I was offered a free book to review, I made a decision never to do that. I think it's the only way to be truly independent. I don't disparage those who do accept the free stuff, but I reserve the right to regard them with a degree of suspicion, that's all.

Fair comment. I don't draw the line at review copies (though have occasionally mused on simply cancelling them, but then this would also risk missing out on some of the more obscure, un-heralded books that TrackerNeil notes above) but I have refused all suggestions that I put adverts for publishers on my site, as I feel that putting up adverts for a publisher in return for money and then reviewing books from those same publishers creates a serious conflict of interest that I find unviable. IIRC, Pat got around this by hosting lots of adverts for Black Library books but never reviewing them, but I prefer not to do it at all. Neither do I do interviews, to prevent the often-embarrassing scenario where you have a great conversation with the author and then have to slag off the book (which I believe Pat has experienced with the infamous David Bilsborough episode), something which professional magazines do all the time and seriously annoys me.

[pat5150]

I have fond memories of the Bilsborough incident. . . ;)

But never again will I force myself to go through such a crappy novel of that size.

Patrick