"Any sufficiently advanced stupidity is indistinguishable from malice" -- minim_calibre
Expanded from a comment I made over at elishavah
's journal, since the whole matter of AmazonFail touches upon something that I've done professionally: run extensive database applications
-- not nearly of the size of Amazon's catalog, but big enough that the best a dutiful developer can do is spotcheck data to see that sorting and search functions are working correctly
I know how easy it is to fuck something up like a ratings system with just the tiniest of tweaks -- and how hard it can be for them to get noticed until it's a disaster. Thankfully, all of my egregious mistakes were for internal company databases affecting small subsets of users, but in one case a glitch that hit only a few affected documents went on for nine months before there were enough of them that there was a plethora of complaints and we realized that it wasn't "just" a user mistake.
(And thank goodness I had a great manager who really grokked 'innocent until proven guilty' and knew that the data we were dealing with was complex and quirky, so that I didn't have a whole load of work stress -- beyond my sincere self-administered kicking -- while I fixed things.)This is just my speculation
, pretty much free of Actual Knowledge -- except that I've read enough about the Amazon ranking system in the past to glean this -- an item's overal sales ranking is recalculated daily, weekly or monthly depending on where in the ranking hierarchy an item is currently at. So you have the low ranked ones -- in the hundred-thousands ranked monthly or less, because yeah, there's really not much reason to rerank A Consumer's History of Toe Lint
when it only sells 2 copies a year or less. But best-sellers who are selling hundreds of copies a day change fast enough that you might well be recalculating them more than once a day.
And then some book or author gets vaulted out of obscurity when she's suddenly in the news, or on the Daily Show or interviewed on NPR or whatever. So you've got to have allowances in your ranking algorithm for sudden popularity. Such as mentions of books in the press, press releases for upcoming guest appearances, book tours. And you probably want to fold in other sources of rankings, like LibraryThing or Shelfari (I don't know if they get used, but they're cool sources for aggregated data) and of course you need some official cataloging source like Library of Congress.
What you end up with is a very complex ranking system which weights all sorts of factors depending on recent movement and ratings, and like Archimedes' lever for moving the Earth, a very small change may end up majorly tweaking -- or, ahem, glitching -- rankings, which are used for customer searches.
Note that the AmazonFail LGBT glitch is especially glitchy in that it has not affected ALL GLBT content -- I suspect that somebody changed a ranking algorithm back in Jan or Feb, or added in a new rating system from libraries or groups (like some "pro-family" organization) that they didn't realize down-checks a whole lot of stuff. But because rankings are also calculated by time scale, it might have taken a while for the problem to snowball, as successive weeks go by and the more recent rankings get calculated into the overall ranking.
You'd practically need calculus to chart out what was happening. First couple of complaints in February were checked out and they could only find a few books affected, so it's a minor glitch.
And if their IT/Customer Support gets run the way I think it might be -- some poor schlub investigates, can only find a couple dozen books affected, and writes up the incident as a minor glitch that should get looked at the next time the master ratings system gets tweaked.
And then... every complaint from mid-Feb on gets the canned response, since it's in the system and it's going to be looked at. OR worse, since Amazon does have a reason to keep dildos and so on off the major ranking system, it's written up as a feature and nobody's looking at the fact that this occasional glitch is snowballing as the dozens of books goes into the hundreds, into the thousands.
A thousand books affected sounds like a lot, but given how many zillion titles Amazon carries, it IS a 'minor glitch' -- I'm betting that suddenly as some internal date passed and more recent rankings were weighted and/or the new input rating was newly added to the calculation, that the affected books soared into the hundreds of thousands, the big majority of LGBT titles.
And it does it just as we're sliding into a 3 day holiday weekend.
Oh, it's going to be an "interesting" morning at the Amazon.Com marketing and IT departments...