24 Aug 2009 2 pm eastern

Kindling

The process by which books are converted to Kindle format introduces errors which do not get corrected. Every publisher knows this, though none will say so on record.

To be clear, there is nothing wrong with Kindle or its format. The problem is one of economics. The cost of a printed book covers some degree of proofing and checking—not enough, but some. The cost of a Kindle book does not support editorial quality control, and the multi-step conversion process, handled in bulk by third parties, chops out content and creates other errors that no one fixes because no one is there to do QA.

I love the idea of Kindle. I love Kindle on my iPhone. As the economics of publishing continues to change, perhaps one day soon, a Kindle edition will contain the same text as the printed book. Until it does, Kindle is great for light reading. But if it’s critical that every word, comma, and code sample come through intact, for now, you’re better off with print.

Update

Two salient points I should have made in this post are covered in Kindling II, posted 27 August 2009.

ShortURL: zeldman.com/x/58

Filed under: Design, downloads, editorial, Formats, Publications, Publishing, writing

45 Responses to “Kindling”

  1. Nhan Nguyen said on

    I’m glad I inquired about if there would be differences between the Kindle and printed versions! I’m getting the printed version, of course, but was interested in a portable, Kindle reference version (on my iPhone) as well. Now, not so much on the Kindle.

  2. Bill Eager said on

    While I do think it’s unfortunate that Kindle can omit real blocks of content, I do see many similarities between this digitization of books and the digitization of music in the 1990’s. For the most part, music I came upon in the 1990’s was of pretty poor quality, in 56k or 112k MP3’s. This did not mean they were unlistenable; you could just hear imperfections. In the same way, books for Kindle have imperfections, but get their point across thoroughly.

    As this process evolves to consider the digital age in tandem with print, as the music industry has, we will see more parity of quality in the digital version and the print version. At that time, books will not need to be “ripped” anymore; they will simply be published initially in both formats.

  3. Marty said on

    I haven’t done Kindle because I like the feeling of having paper in my hands, but I DO think that having the ability to search for content would be a valuable addition (ever try to find a piece of content in a book?). However, I don’t think that I would ever review critical content — like code — in that format, even before reading this. You would think that technical volumes would get the extra money needed for the editing. So, are you running into this on a Kindle version of DWWS?

  4. Chris Johnson said on

    There are conversion errors yet Kindle books aren’t really much cheaper than real books (aside from the Amazon subsidized blockbuster hardcovers). I’d be more forgiving of errors if they were substantially cheaper.

    So you’re right. I love my Kindle, but I won’t buy anything fiction and non-fiction that’s straight text.

  5. Brad Larson said on

    It’s things like this that make the Pragmatic Programmers seem so far ahead of their peers. Not only do they provide DRM-free versions of their publications in a number of formats, each of these versions are auto-generated from the same internal master they use for the hardcopy versions. They describe the process in their online magazine.

    Their beta book program is something that I wish other technical publishers would emulate, as well.

  6. Darcy Murphy said on

    [With tongue suitably in cheek, I say...]

    It sounds like the Kindle needs some sort of flexible document format that can take the content of a book, and repurpose it for a digital medium.

    I wonder, do you know of any such document format? I would be a real boon if you could write a book on it.

    . . .

    That’s really interesting to know. I’ve never used a kindle, but that sounds like a major deficiency if the kindle edition of books aren’t accurate. I suppose this works well enough for fiction with it’s relatively simple needs, but a technical book could suffer from sloppy omissions. I hope it all turns out well for you in the end.

  7. Mark Trammell said on

    This really is too bad. I’ve been rereading Observing The User Experience on my Kindle and find it to be even more pleasurable than reading it in print with no noticeable errors. The images leave something to be desired, but that’s to be expected.

    Jeffrey, what would change the economics? It feels a bit chicken/egg. Maybe a print/digital package similar to how many Blu-Ray discs come with a digital copy to be imported into iTunes?

  8. Mandy Brown said on

    The economics are beginning to change—fully 30% of Amazon sales are for the Kindle version when it’s available—but there’s a lot of inertia in the industry to overcome. The entire production process is geared towards a print text, with ebooks the bastard stepchild. Even if ebook sales continue to rise, and it becomes economically viable to spend more money on them, we still have legions of editors and proofreaders and production managers and composition houses and the like who were trained on the print process. An entirely new workflow needs to be invented for ebooks; so far, there’s no sign of that happening.

  9. Florian said on

    Thanks for this insight!—I’ve never even considered Kindle could cause this kind of problems. I guess I’m still going to stick with the printed versions for a while longer.

  10. Jeffrey Zeldman said on

    I do see many similarities between this digitization of books and the digitization of music in the 1990’s

    Agreed. My 1990s CD of Brian Eno’s “Music For Films” sounds massively inferior to the vinyl. A subsequent pressing is of slightly higher quality, but the beginnings and endings of some pieces are unceremoniously lopped off, as if by an improperly set noise gate, and the sound is still dirty—like an eight-bit sampler.

    I’m glad I inquired about if there would be differences between the Kindle and printed versions! I’m getting the printed version, of course, but was interested in a portable, Kindle reference version (on my iPhone) as well. Now, not so much on the Kindle.

    If you have a printed version for reference, it’s fine to have a Kindle for light reading on a plane or train, or just as a back-up when you need to look something up quickly. That’s a great use of Kindle.

    So, are you running into this on a Kindle version of DWWS?

    The third edition is still being created, so there isn’t a Kindle version yet—or a printed version to compare it to.

    I bought a Kindle version of the previous version (2nd Edition), and found that some asides had been truncated—sentences began and never finished, paragraphs disappeared into the cosmos. I don’t mind for myself, but for I do mind for readers.

  11. Jeffrey Zeldman said on

    An entirely new workflow needs to be invented for ebooks; so far, there’s no sign of that happening.

    But it sounds like it will happen.

    Jeffrey, what would change the economics?

    It sounds from Mandy’s comment like they are already changing, but corporate processes, training, and personnel haven’t caught up yet.

    Currently Kindle versions are bastard offspring of e-books, which are bastards of books.

    If Kindle sales (and digital books generally) keep increasing as a proportion of overall sales—and if overall sales do not decline—then eventually we should see a change in workflow.

    Either that, or much smarter conversion algorithms.

  12. Scott Savage said on

    I guess I was a bit naive when I bought my Kindle2 earlier this year. I figured that the Kindle editions of books were the same files used to print regular books, just stripped down to the bare text elements (with photos and other non-text elements removed).

    Still, in the 30 or 40 books I’ve read on my Kindle now I’ve only noticed a handful of grammar/usage/capitalization errors. Overall I’m okay with this. Too bad that the Kindle can’t be proofread using a bit of cloud technology, eh? Allow users to submit proofreading data via the WhisperNet and if approved allow users to downloaded updated versions of the books with the errors corrected.

    Pipe dream? Probably.

  13. Jeffrey Zeldman said on

    Too bad that the Kindle can’t be proofread using a bit of cloud technology, eh? Allow users to submit proofreading data via the WhisperNet and if approved allow users to downloaded updated versions of the books with the errors corrected.

    “Crowdsourcing will save us.”

  14. Scott Savage said on

    Of course… the obvious issue with crowdsourcing is that some people will complain that they purchased a product and don’t feel like they should also have to be the ones to “fix” it. Rightfully so, but if there’s a substantial group of people who demand error-free books and the publishers won’t do it, why not crowdsource it? There’s no need to participate if you want and no need to download “fixed” books.

    I don’t know. I made the argument for it before and I’ve made the argument against it as well. Can’t have our cake and eat it.

  15. Mandy Brown said on

    Old-fashioned errata files are crowdsourced: readers notice errors as they are reading, then they write the author and/or publisher, who collects all the errors into an errata file for correcting in the next printing. The problem with ebooks is there is as yet no mechanism for correcting them, and without page or line numbers, it’s much more difficult for well-intentioned readers to report errors. Hopefully, this will change; and soon.

  16. Jeff Mackey said on

    I have noticed this discrepancy as well, but only for certain books. Those that have extensive asides, or blockquotes, or charts embedded within paragraphs, tend to get distorted, cut or otherwise cropped.

    That said, those publishers that actually make a Kindle version of a book (rather than have a print version converted) are doing it right. Those versions are incredibly detailed in every respect when compared to their print counter-part.

  17. Beau said on

    If only the Kindle platform would open up a bit more, perhaps “the community” could be involved in helping to edit books and correct mistakes a la Wikipedia.

  18. Caleb Clauset said on

    Yes, the typical reverse composition workflow (PDF to XML) use to create Kindle (and/or other electronic formats) tends to introduce differences (depending on the complexity of the layout). But this problem is not unique, it’s not uncommon to find differences between various print editions of the same title (e.g., hardbound first edition versus later mass-market editions). We (Typefi) are working to change this…

  19. Jeffrey Zeldman said on

    Old-fashioned errata files are crowdsourced: readers notice errors as they are reading, then they write the author and/or publisher, who collects all the errors into an errata file for correcting in the next printing.

    Yup!

  20. Jeffrey Zeldman said on

    We (Typefi) are working to change this…

    Neat!

  21. Jeffrey Zeldman said on

    It sounds like the Kindle needs some sort of flexible document format that can take the content of a book, and repurpose it for a digital medium.

    I wonder, do you know of any such document format? I would be a real boon if you could write a book on it.

    Hilarious!

  22. Keith Fahlgren said on

    Every publisher knows this, though none will say so on record.

    Um, O’Reilly has always shouted this fairly loudly:

    There were two main reasons we held our books back from sale on Kindle:

    1. Poor rendering of complex content.

    2. Compulsory DRM.

    To be clear, there is nothing wrong with Kindle or its format.

    No, there are real problems with Mobi, the format that Amazon uses internally (with some modifications) for the Kindle.

  23. God of Biscuits said on

    Are you saying that all Kindle Editions, from public domain (and in that category, all sources of public domain….Project Gutenberg, etc.) to previously published e-books, to major publishing houses that have electronic versions of their books already completed (including copy-edited, proofed, etc. versions that are used as electronic masters that comprise the printed version(s) suffer the same quality control issues?

  24. Jeffrey Zeldman said on

    O’Reilly has always shouted this fairly loudly

    I guess I should pay attention to him.

  25. Jeffrey Zeldman said on

    Are you saying that all Kindle Editions, from public domain (and in that category, all sources of public domain….Project Gutenberg, etc.) to previously published e-books, to major publishing houses that have electronic versions of their books already completed (including copy-edited, proofed, etc. versions that are used as electronic masters that comprise the printed version(s) suffer the same quality control issues?

    Pretty much.

    major publishing houses that have electronic versions of their books already completed (including copy-edited, proofed, etc. versions that are used as electronic masters that comprise the printed version(s)

    The thing is, those proofed, edited files get converted to e-books, and then the e-books get converted to Kindleschwabi. Errors are introduced in both conversions.

  26. Billee D. said on

    Thanks Jeffrey. I too am interested in the idea of Kindle, but I’m still too leery of the newness factor to drop down that kind of money on what is, as you noted, an e-book reader. Plus, I’ve read a few articles discussing the Kindle format and one author even noted the missing content dilemma.

    It’s a nascent technology with lots of promise, but I think it still has a few years yet to rival a printed book.

  27. Paul Berna said on

    The cost of a Kindle book does not support editorial quality control, and the multi-step conversion process, handled in bulk by third parties, chops out content and creates other errors that no one fixes because no one is there to do QA

    .

    Well, Kindle-Books are almost as expensive as the standard softcover books. So what does the price of a Kindle-Book cover, one may wonder?

    As the economics of publishing continues to change, perhaps one day soon, a Kindle edition will contain the same text as the printed book. Until it does, Kindle is great for light reading. But if it’s critical that every word, comma, and code sample come through intact, for now, you’re better off with print.

    I certainly agree with that.

    But the “problem” (but it doesn’t kill) with ordinary books still remains the same. If there is a typo, a misplaced comma, a grammatical error, it is in the book and will remain there in all the thousand books that will be sold in the fist version. You can correct in a subsequent version. But the errors remain in the previous versions. That’s where e-books could be more interesting. They could update constantly, like software (or the web). Like, hey, there is an update, and you just download the update. If there is a misplaced coma, you just change it and the change will spread to all your readers, not just your new readers. Some kind of books are actually bound to become like this. Imagine travel books that you just update each time you go the location you want to visit. But Amazon does not see the market that way. They force the old book format on a new medium.

  28. Keith Fahlgren said on

    They could update constantly, like software (or the web). Like, hey, there is an update, and you just download the update

    Yeah, that idea of frequent updates something we’re trying at O’Reilly to play up (and we’re getting much better with errata and formatting updates, so it really is happening):

    …free updates to reflect published changes in the books…

    …but the Kindle has a problem here: The user’s reading position, bookmarks, and annotations are stored in such a way that they can’t be saved across updates. Therefore, Amazon will never prompt a customer to actually update their copy of the file even if we fixed some HUGE ERRATUM (or that’s what they’ve told us). New customers would get the newest file we’ve sent Amazon, but not existing ones (there are workarounds that lose your bookmarks/annotations). Bummer.

    I guess I should pay attention to him.

    That is, the company O’Reilly Media, not necessarily the founder Tim O’Reilly. Unsurprisingly, the staff devoted to these sorts of issues (like me) are more interested in writing about it on a low level than Tim is, although Tim himself does write things like Why Kindle Should Be An Open Book for Forbes.

  29. Wendy Sharp said on

    “So what does the price of a Kindle-Book cover, one may wonder?”

    Well, it covers the cost of producing a Kindle edition. Think of it this way–the first person to buy a Kindle edition of a book gets the most incredible bargain–a book worth $1000 (a rough guess at the cost of converting the files) for a mere $29.99 (a rough guess at the cost of the book). The second person to buy gets less of a bargain, a book only worth $500, for the price of $29.99. Eventually there are enough buyers that the book is actually not costing the publisher ridiculously more to produce than they make off the sales but that day is not the day of the first purchase.

    E-books aren’t actually free to produce. Until the buyers are there to support the platform, publishers take a loss on every book sold. And adding editing and proof-reading to the cost of converting makes it even more of a risk to try to reach a new audience.

    That said, I know I speak for Peachpit New Riders when I say that if we learn of problems with a Kindle edition of a book, we will do our best to get them fixed ASAP. After all, we’ve paid for someone to convert the book–we have more right than anyone to be annoyed if they got it wrong!

  30. God of Biscuits said on

    The thing is, those proofed, edited files get converted to e-books, and then the e-books get converted to Kindleschwabi. Errors are introduced in both conversions.

    This is known? Admitted? Proven? Or just asserted?

    I’m not choosing sides, much less picking a fight, but it seems extremely unlikely that there aren’t ways of verifying that “those proofed, edited” files survived a conversion intact.

  31. James Lamb said on

    Some publishers use XML workflows so there shouldn’t be a problem. For example, Cambridge University Press use their CUP-XML workflow so that the copyediting, proofreading and indexing are done on the XML and then the paperback/hardback/html/cd-rom format are all produced automatically, so Kindle conversion should present no re-formatting problems. It will be comma-perfect. Of course, it costs more upfront, but represents an investment in future digital publishing, so I guess most publishers are not up to speed.

    I have no connection to CUP except that I index their books sometimes, so I see the extra work which goes into providing the future flexibility.

  32. Jeffrey Zeldman said on

    This is known? Admitted? Proven? Or just asserted?

    If it doesn’t happen during the conversions from publisher files to e-books and from e-books to Kindle, then the book fairies must be introducing the errors.

  33. Twitter Trackbacks for Kindling – Jeffrey Zeldman Presents The Daily Report [zeldman.com] on Topsy.com said on

    [...] Kindling – Jeffrey Zeldman Presents The Daily Report http://www.zeldman.com/2009/08/24/kindling – view page – cached Web design insights since 1995. Personal site of Jeffrey Zeldman, publisher of A List Apart Magazine, founder of Happy Cog Studios, co-founder of The Web Standards Project, co-founder of the Event Apart design conference, author of Designing With Web Standards. — From the page [...]

  34. Phil Stewart-Jones said on

    That’s a pretty all-encompassing statement there. Do you have more evidence than your personal experience? I’m not disputing it – it does seem all too believable that complex content such as sidebars, callouts, diagrams etc – could get lost (or corrupted) in translation. This is especially true given that the Kindle – unlike a book – is not a fixed format; the requirement to be able to change type size is bound to introduce a whole raft of problems for content with a specific layout that must be displayed on a screen of fixed size (and no scrollbars).

    However, I think it’s a reasonable assumption that the vast majority of people are using their Kindles for novels, short stories and so on; these are more or less straight text and as such are not nearly as prone to the trouble you describe. My wife – who is an editor by trade – has read hundreds of books on her Kindle and has very rarely encountered such issues, although (somewhat apropos) she’s currently reading a traditional book which has one signature bound upside-down.

    However, if this problem is as widespread as you imply, with the recent introduction of the bigger Kindle (aimed at the educational/text book market) this is Amazon’s problem to fix, and soon. Hopefully bringing light to the issue at least ensures that DWWS 3e ends up Kindle-intact.

  35. News Feeds - Welkom op www.ooldervesteherten.nl said on

    [...] Kindling The process by which books are converted to Kindle format introduces errors which do not get corrected. Every publisher knows this, though none will say so on record. The problem is one of economics. The cost of a printed book covers some degree of proofing and checking—not enough, but some. The cost of a Kindle book does not support quality control, and the multi-step conversion process, handled in bulk by third parties, chops out content and creates other errors that no one fixes because no one is there to do QA. [...]

  36. God of Biscuits said on

    If it doesn’t happen during the conversions from publisher files to e-books and from e-books to Kindle, then the book fairies must be introducing the errors.

    Then there’s evidence? Anecdotes? Failure statistics on a run-rate basis?

    I can see OCR’d stuff being pulled in from old texts and shoved into ebooks that get dumped into the market without much QC, but I find it hard to believe that HarperCollins isn’t going to be proofing its shipping product.

    Y’know, like they do for each printing of each edition of each format on paper?

  37. Jeffrey Zeldman said on

    @God of Biscuits:

    I’m speaking from personal experience, and from conversations I’ve had with book people. The personal experience prompted me to have those conversations. Like you, I was curious to find out if what I’d experienced was an anomaly or a common occurrence.

    Those conversations led me to believe that there are known problems with Kindle conversions, and that publishers don’t always have the resources to identify and fix the problems before the electronic file gets sent to Amazon and sold to customers.

    Publishers, as you may know, are not thriving at this time—not that they ever have. Especially at this time, publishers don’t have money to throw at lots of things, including salaries. Most people I know in publishing are in it for love. They are passionate about books and reading, about knowledge and sharing, and maybe even hopeful about the possibility of an informed democracy to bring some light and happiness to human life.

    The global economy shows signs of recovering (Wall Street, housing market), but people are still getting laid off, and some sectors of the economy are still hurting very badly.

    All of that is to say that it’s only to be expected that there aren’t necessarily staff proofreaders comparing Kindle editions to the original print files on a line-by-line and word-by-word basis. This might change as digital editions continue to grow as a percentage of a publisher’s market, and publishing companies reallocate budgets, but if you’ve worked in companies, you know that planning cycles lag—there may be a year or more between recognition of a change, acceptance of a plan to accommodate that change, and working implementation of that plan.

    To return to your question, this is a blog post, not a feature story in The Economist. I don’t have statistics (don’t know that anyone does—gathering statistics is also an expensive activity), haven’t spoken to dozens of publishers, haven’t hired teams to run tests on and perform an audit of, say, 30 random Kindle editions. Just a guy talking here.

    I can’t say what goes on at Harper-Collins. Wouldn’t know. I’ve had nice meetings with them (unrelated to this conversation), but never discussed Kindle editions with them.

    I also, truth to tell, haven’t discussed this with my own publisher. Which is weird, now that I come to think of it. If alerted to a problem in a Kindle edition, publishers fix it in a subsequent update, just as they do with print problems. My own publisher, who are like family and wizards to me, are checking for problems in Kindle editions.

  38. dandam said on

    Having designed (flowed, if you will) a number of books and designed for the screen (web and UI) too, I find my new Kindle a funny thing. It asks (begs?) for a new paradigm while trying to appease two existing and very popular paradigms (the page vs the electronic scroll) at the same time.

    Straight fiction flows well into either a scroll or a page. Paragraph break and punctuation conventions were developed by printers to accommodate the page and adapt well to a scroll and most divisions in straight fiction use those conventions.

    Guidebooks, how-tos, plays/screenplays, and other highly structured texts run into trouble on the Kindle (and on the web (a scroll)). I expected that, but I didn’t expect trouble with having multiple fonts in a single book. Code (if and when it appears in a Kindle book) appears in the same font as the paragraph text. This seems like a no-brainer, as does displaying boxed text, and pulls.

    The paradigm confusion is evident where and when the Kindle designers tried to accommodate the page or the scroll: the percentage of book read indicator is supposed to make up for not feeling the weight of the pages transfer from right to left as I read as much as the white jimmy mouse substitute is supposed to give me more control over what happens within the scroll.

    All that said, I’m glad that Amazon tried so hard to make room for the old ways, but I don’t think the e-book devices will succeed by simply matching the successes/requirements of all their predecessors, instead written works will need to adapt to the new medium as they always have from scrolls to illuminated manuscripts to the printing press to the web.

  39. James Lamb said on

    Jeffrey Zeldman asked:
    This is known? Admitted? Proven? Or just asserted?

    I am not sure what is being questioned. XML workflow is known, proven and documented. CUPs XML workflow is explained here.

    They say:

    The fully XML-coded master files enable us to repurpose the material for whatever publishing needs Cambridge and you may have for the content in the future. This could be just the book, or web material, e-books and smartpdfs (which are a sort of simple e-book) for display on the web, CDs, new editions or marketing material.

    XML is like more-powerful HTML (in fact HTML is a subset). In the same way that a (well-written) web-page will reflow and reformat when you resize your browser, so XML can automatically be reformatted and reflowed into any format.
    It requires more information to be recorded in the document originally (just like converting plain paper into a web page) but gives more flexibility later. Whether publishers of fiction, for example, would make such an investment is a moot point, but comma-perfect automatic kindling is possible.

  40. Jeffrey Zeldman said on

    @James Lamb:

    Jeffrey Zeldman asked:
    This is known? Admitted? Proven? Or just asserted?

    Jeffrey Zeldman didn’t ask that.

    Jeffrey Zeldman was quoting “God of Biscuits,” who was challenging Jeffrey Zeldman’s assertion that third-party conversion from publisher files to e-books to Kindle format introduced errors.

    “God of Biscuits” was implying that Jeffrey Zeldman’s observations were invalid unless Jeffrey Zeldman could support them with such things as “Failure statistics on a run-rate basis.”

    In quoting “God of Biscuits”‘s dismissal of Jeffrey Zeldman’s blog post, Jeffrey Zeldman was not questioning or referring to or thinking about XML workflow.

    Likewise, “God of Biscuits” was not questioning or referring to or thinking about XML workflow.

  41. Jeffrey Zeldman said on

    XML is like more-powerful HTML (in fact HTML is a subset).

    Not true.

    HTML is a subset of SGML.

    XHTML 1.0 is an XML application of XML, most often served as text/html.

    I wrote this book that talks some about these things.

  42. scott said on

    I don’t even own a Kindle.

  43. Kindling... (Jeffrey Zeldman Presents) » TechNews.AM said on

    [...] We are still in pre-alpha, so please bear with us. GA_googleFillSlot("TAM-Leaderboard"); KindlingJeffrey Zeldman Presents  Aug 24 09 The process by which books are converted to Kindle [...]

  44. All This ChittahChattah | ChittahChattah Quickies said on

    [...] The process of converting books to Kindle format introduces errors in the text – The cost of a printed book covers some degree of proofing and checking—not enough, but some. The cost of a Kindle book does not support editorial quality control, and the multi-step conversion process, handled in bulk by third parties, chops out content and creates other errors that no one fixes because no one is there to do QA. [...]

  45. Justus said on

    I know posting to ancient blog posts is bad form but I can’t help myself….

    I find it hard to believe that HarperCollins isn’t going to be proofing its shipping product.

    This statement appears to have been made by a non-Kindle owner. I can assure you that this is exactly what happens. I have purchased exactly 2 Kindle books from Amazon. One by Tor (Macmillan) and the other by Random House. Though not HarperCollins, I think they are sufficiently similar to extrapolate business practices. I can assure you they were not proofed in any way shape or form. I attempted to keep track of the number of errors I spotted while reading the Tor book and gave up when I crossed the 200 mark. I didn’t bother with the Random House one (and it is in much better shape, usually just extra or missing commas).

    The Kindle itself is a fine device but the e-books I have seen are pretty terrible.

Comments off.