The invention of the tablet PC has created a new medium for book publishing. Interactive books are everywhere, and have revolutionized the way people consume the printed word. With the recent software available to allow easy creation of interactive books and with the race to bring these products to market, there seems to be a more and more dilution of quality and a loss for the meaning of interactivity. When publishers create new eBook titles or convert a traditional printed book to a digital interactive eBook, they often miss the added value this new medium can provide.
It’s important to understand the distinction between apps and eBooks, as it’s something that often confuses both publishers and consumers. It basically comes down to formats; apps are mostly native iOS orAndroidsoftware, whereas eBooks are documents of a particular format, such as the open standards EPUB and Mobipocket (.mobi). And eBooks can be further distinguished from “enhanced eBooks,” which use formats such as ePUB3 for iBooks (Apple) and Kindle Format 8 (KF8) for Kindle Fire (Amazon).
eBooks were the first to appear on devices such as the Kindle, and have very limited interactivity. You are mainly able to flip the pages, search for content, or highlight words to see a dictionary definition. These devices also allowed font size to be increased to enable visually impaired readers enjoy books more easily. This gave publishers the unforeseen benefit of regaining a large population of users who couldn’t read printed books.
Enhanced eBooks (ePUB3) are a new digital publication standard that allows easy integration of video, audio, and interactivity. I expect this format to advance the future of textbooks and other educational material. Future textbooks might be able to “read themselves” with audio narration, perhaps preventing students from actually reading. But the benefits outweigh the downsides; for example, the new text books might also offer the ability to make and share annotations without destroying the book, interactive self-tests throughout the chapters, and generally a much more enjoyable learning experience.
Apple has recently released iBooks Author, a free eBook creation software that lets anyone with a Mac to create iBooks textbooks, cookbooks, history books, picture books, etc. iBooks Author generates a proprietary format for books that will only be available for sale on Apple devices. Adobe has also made available a Digital Publishing Suite via InDesign for the iPad, Android, and Blackberry platforms. Mag+ and Moglue are two other independent publishing platforms that are worth mentioning.
This video was released by Apple to promote the iBooks Author:
Interactive eBooks is a category for apps designed specifically to utilize the powers of tablets to enable users to interact with the storyline in sight, sound, and touch. I like to think of interactive eBooks as an evolution of the printed book with added interactivity in order to create an experience beyond the printed format. Examples of interactive eBooks include pop-up book apps for kids, interactive travel guides that utilize the device GPS capabilities, cookbooks with built-in timers and video recipes, or any traditional book that now uses the tablet to enhance the experience with interactivity.
Grimm’s Rapunzel ~ 3D Interactive Pop-up Book
On a touch device, interactivity is the ability to engage with the user interface, including the ways you move your fingers on the screen, the way you to select an app, or how you browse the Web. Interactive eBooks are, by definition, an enhanced book-like experience that have a different core premise than other types of apps (with the exception of games perhaps). Whereas in most applications, interactivity focuses on menu navigation and interaction with the user interface as means to achieve a goal (view an image, find an address, read an email), interactive eBooks provide interaction with the content and storyline, and therefore offer a unique experience each time. A good example of is Richard Dawkins’ The Magic of Reality, where you interact with the storyline through interactive demonstrations and games that allow you to get hands-on with the science discussed in the book by, for example, letting you simulate the effects of heat, pressure, and gravity on different states of matter.
The experience of interactive eBooks should not be confined to animations based on touch-and-response interaction, or merely flipping the page; when designing these Books one must ask what is the enhancedexperience—why to move from print to digital, and how to create value and fun.
Interactivity for the Sake of Interactivity
If a book app does not use interactivity in order to enhance the reading experience, it does not belong in the interactive eBook category. In the race to bring interactive books to market, some of the books have only featured very superficial interactivity—what I call “interactivity for the sake of interactivity”—where, for example, touching an image activates a simple animation such as making a butterfly fly, or a tree drop leaves to the ground. These interactive experiences do not add value to the story, and are therefore somewhat meaningless.
There are a few exceptions where this type of interactivity is actually a success. For example. one of the first books published as an interactive app for the iPad was Alice in Wonderland. This book was a phenomenal success though offered nothing but eye-candy interactivity. When the app was first published, the reviews called it “a reinvention of reading” that made clever use of the accelerometer to make Alice grow as big as a house or to throw tarts at the Queen of Hearts and watch them bounce. Although these activities through the 52 pages of the book are fun, I think they distract from the actual story. The reason this book was such a success is due its having been published when the iPad was fairly new, and touch interactivity was still an exciting experience.
Another book that was fairly successful at the time was The Pedlar Lady of Gushing Cross, which offers narrated animation with very basic interactivity, but was considered revolutionary when it came out because reading the story while seeing the animation unfold was definitely an enhanced experience to the young reader. However, this book did not offer any real value through interactivity, and might as well be classified as a short animated movie. The limited interactivity of seeing letters animate while you tilt the device was merely a gimmick, as you can see in the video below
Cozmo’s Day Off is an interactive eBook that was on the top-seller list for many months, and is packed with interactive elements that made it a great success. It contains over 100 unique audio and animated interactions. However, this app would be better characterized as a game for young kids and not as an interactive storybook because the story seems secondary to all the bells and whistles, and it’s written in style not intended for young audiences. But perhaps this is a case where interaction simply for the sake of interaction can be the whole point of a book.
The image below shows all of the hotspots that trigger an animation sequence for one page of the book:
Interaction for Value
It is possible for interactivity to go beyond the superficial, to add value to the book and create an experience that would be impossible in print. Here are a few examples of such cases.
Al Gore’s Our Choice is a great example of how meaningful interactivity creates an engaging and fun learning experience. With clever use of interactive infographics, animations, documentary videos, and images, this book is a great example of what the future has in store for digital publishing.
The Martha Stewart Cookies iPad app is a wonderful example of an interactive recipe book. Besides just offering great recipes, it also allows you to search recipes based on ingredients and cookie type to find the perfect cookie for your needs. For example, you might use the app’s search wheel (below) to look for bars and biscotti-type cookies with oatmeal as the main flavor component. This is a great added value because this type of interaction is unmatched in print.
Paris: DK Eyewitness is probably the most complete travel guide you can find for the iPad. It features beautiful cutaways of buildings that can be explored by tapping and zooming, complete offline maps for all the central districts of the city, interactive city and park walks with “hotspots,” and extensive listings of the best sights relative to your current location. No more searching aimlessly for your location on a map or looking through index pages; the interactive app shows what’s around you within walking distance, making the iPad a must-carry on in your travel bag for an experience unparalleled in a traditional travel guides.
Bobo Explores Light is an educational experience for young adults. It puts a fully functional science museum in the palm of your hand, teaching you about lasers, telescopes, lightning, reflection, bioluminescence, and sunlight. This is great example of using simple interactivity to explain relatively complex topics through science experiments that you can actually perform on your iPad. Bobo, a friendly robot, serves as a guide, taking the young reader through space, land, and sea, to learn all about the science of light.
In my book, Timor the Alligator, kids participate in the story by picking toothpaste and helping Timor brush his teeth. This story could not have been told in a printed book because, without the use of interactivity, young kids would not be able to visually understand that brushing actually helps keep a clean mouth. The simple process of choosing a toothbrush, adding toothpaste, and brushing Timor’s teeth until they turn white serves as an educational experience for preschoolers and toddlers reading the book.
With the Numberlys app, kids (and adults) learn about the alphabet through a series of fun interactive games. This book probably has the most spectacular visuals I’ve seen to date. Its aesthetic is inspired by Fritz Lang’s silent film, Metropolis, so the app offers a unique cinematic experience and gameplay to engage users to learn about the (fictitious) “origin of the alphabet.”
As you can see from these examples, interactive eBooks are no longer just about a touch-to-animate type of interactivity, nor simply the touch interface controls. Rather, they are about adding value through interactivity by using the full capabilities of a touch device to engage the user and enhance the learning and reading experience. These engaging experiences are what I call a true reinvention of reading.
The average New York City taxi cab driver makes $90,747 in revenue per year. There are roughly 13,267 cabs in the city. In 2007, NYC forced cab drivers to begin taking credit cards, which involved installing a touch screen system for payment.
During payment, the user is presented with three default buttons for tipping: 20%, 25%, and 30%. When cabs were cash only, the average tip was roughly 10%. After the introduction of this system, the tip percentage jumped to 22%.
Those three buttons resulted in $144,146,165 of additional tips. Per year. Those are some very valuable buttons.
In the Markdown Mark repository on Github, I had initially included only 12 static PNG images of the mark at different specific sizes. Several people asked me why I had done this instead of including a vector graphic, and the reason is actually extremely important. This is something everyone should know, especially people who are not designers.
Most icons and logos are stored in vector image files that specify perfect mathematical representations of their shapes. Unlike JPG and PNG images, vector graphic files don’t tell the computer how to display their contents in pixels; for a vector graphic to be displayed, the computer has to perform a translation from the mathematical vectors into something that can be displayed with pixels.
This translation process is relatively simple: the computer takes vector lines, lays them on top of a pixel canvas, and then fills in each pixel that the lines enclose. The result looks like this:
The S vector path is perfectly smooth because it is defined using math that is not constrained by display technology. The rasterized S has jagged edges because there are simply not enough pixels to faithfully show what the vector lines represent. Display technology is the limiting factor here, and it has been for many decades. To get around the problem, computers use a signal processing technique called antialiasing, which hacks your brain into perceiving more definition than can be displayed on the screen.
A gift from signal processing
Antialiasing repurposes pixels into building blocks that approximate whatshould be displayed on the screen instead of what is displayed on the screen. Instead of filling in boxes (pixels) with “on” or “off” states based on how they are enclosed by vector lines, antialiasing uses some visual and perceptual tricks to get closer to the real mathematical image. The most common trick uses things called half-pixels, which are pixels at the edges of paths displayed at different intensities around shape edges, to mimic the effect of having higher resolution. Here’s what antialiasing with half-pixels does to our S glyph:
As you can see, some of the pixels that were not quite enclosed by the vector path are now displayed as shades of gray. When you zoom out, those different intensities simulate the look of a sharp edge. These half-pixels can fill in gaps that solid “on” pixels cannot. But don’t let this trick fool you; the antialiased S is still not actually sharp. Antialiasing is a hack. It tricks the brain into perceiving a sharp shape when in fact it is looking at a gooey form. This is why reading on Retina displays feels so much better; you’re not looking at blurry shapes.
Until we have Retina displays everywhere, we’re going to have to live with antialiasing techniques. The problem is that computers are terrible at doing it automatically. Because the computer doesn’t know what is in the resized image, it doesn’t know where to optimize using half-pixels. This is why I included manually antialiased images with the Markdown Mark. Look at the demonstration below. The right side of the image shows icons that were automatically antialiased by Photoshop and the left side shows the manually fitted ones I included with the Markdown Mark:
The difference is subtle, but the comparison makes it very clear. There are several major problems with the computer-created icons that leaves them looking blurry and unimpressive. For example, in the transition from 66x40 to 48x30 pixels, the human version drops the border down to exactly 2 pixels but the computer drops it to 2.8 (that’s two solid black pixels, plus one half-pixel at 80% intensity), which gives it an incorrect soft edge. The computer has no way of knowing that the border should always be sharp, so it ignorantly tries to maintain the ratio of the originally-specified image at the expense of sharpness. The result is not great.
Let’s get a closer look so you can see exactly how the problem is presented in pixels on the screen. The top image in the figure below, which was fitted by a human, shows the correct border size of 3 pixels, while the computer-resized image shows 3.3 pixels. Zoomed out, the 30% half-pixel makes the edge look blurry (this is the 66x40 version of the Markdown Mark from above).
Also notice that similar issues occur when computers antialias typography. I pixel-fitted the M glyph to solid edges in my version, while the computer made poor choices in its attempt to maintain ratio at the expense of clarity.
Fitting
Now that you’ve seen the importance of pixel-fitting, you probably want to know how to do it yourself. If you’re not a designer, it’s not entirely straightforward and it requires a bit of patience and knowledge of the vector tools in image editing software like Photoshop. But the process is easy to describe: after you resize a vector image onto a raster plane (like a Photoshop canvas), you should zoom in to the pixel grid and manually refit the paths to pixel edges where it makes sense. In Photoshop, you should use the Direct Selection Tool to select the vector edges and then nudge them slowly until they fit perfectly to the edges of physical pixels. This only really works to sharpen straight lines and to define the furthest edges of rounded paths. To display a curve, you have to use antialiasing because there are simply not enough pixels to show a soft curve on the screen (see the very first, jagged S image above).
Sometimes, the difference is incredible. For example, here is Hulu.com’s logo, taken directly from the website. It is a badly antialiased graphic that is seen by millions of people every day:
The top image is the logo currently live at Hulu.com. The bottom version is a pixel-fitted mockup that I made (I probably overcompensated, which resulted in some lost some detail in the P and S). The difference is obvious. A closer look makes the human changes even more apparent:
The Hulu logo is a gooey, blurry mess that is seen by millions of people every day. Because it is at the top of every page on the entire site, it sets the tone for the quality of the experience. It should have been pixel-fitted.
What this means
Using automatic antialiasing when resizing vector graphics leaves a lot of important decisions up to computer graphics software. While the results are generally decent, they’re not perfect. So, for important icons and logos– really, for all rasterized vector images–you should fit the pixels to the grid and ensure they are as sharp as possible.
We’re slowly starting to see displays with higher pixel densities that can more faithfully reproduce vector paths on the screen, like Apple’s retina displays. But they’re not widespread, and most software doesn’t optimize for those kinds of screens. Until they are commonplace, pixel-fitting is an essential task.
LIST AND LINDER EXPLOITED THE CAPABILITIES OUR DEVICES ALREADY HAVE.
The message came courtesy of Natan Linder, a PhD student in the Fluid Interfaces group at the MIT Media Lab. Linder and undergraduate researcher Alexander List are developers of Swÿp, a piece of open-source software that facilitates “cross-app, cross-device data exchange using physical ‘swipe’ gestures,” they write on their website. “Our framework allows any number of touch-sensing and collocated devices to establish file-exchange and communications with no pairing other than a physical gesture.” Translation: Dragging files from a phone to a computer with a swipe of the finger isn’t just a cool, far-fetched idea, it’s reality. Watch the demo:
Here’s the amazing part: They didn’t hack the iPhone and iPad with IrDA transceivers or anything like that, which would’ve enabled the devices to detect each other in 3-D space, a la Sifteo cubes. Instead, List and Linder exploited the capabilities the devices already had.
Swÿp gathers information such as your phone and iPad’s approximate location (available via WiFi) and account details (via sites like Facebook or Gmail), then ties that information to a real-time gesture, the swipe (or Swÿp). Hold up two Swÿp-enabled devices next to each other, and they’re able to communicate in a language both understand: a hybrid of the digital and physical worlds.
SWŸP TAKES THE MYSTERIOUS PROCESS OF SHARING DATA AND EXTERNALIZES IT.
Why should we care? For one thing, it’s a hell of a lot easier to transfer files that way. I’d rather share a photo with a friend sitting next to me by dragging my finger across our screens than slog through a bunch of steps to send it via Dropbox, YouSendIt, or email. In a larger sense, Swÿp takes the mysterious computational process of sharing data—something we do all the time but never see—and externalizes it, giving it a tactile, intuitive interface. It allows users to “immediately grasp the concepts behind device-to-device communications,” Swÿp’s developers says.
“It’s a very smart way to use existing devices without any added technology,” says Ishac Bertran, who developed the UI concept we reported on last week. Though he points out that Swÿp doesn’t enable a completely seamless user interaction. For instance, he says, after swiping to create a connection between two gadgets, you can’t change their position, or they’ll lose their spatial link. A device equipped with spatially aware sensors, as Bertran envisions it, wouldn’t have that problem.
List started Swÿp, then Linder jumped on board. Linder was a member of the Media Lab’s LuminAR project, a desk lamp that can turn any surface into an interactive space (such as a guide for shoppers), and saw Swÿp as a “great fit with the stuff I was working on.”
Swÿp can be used for iOS and LuminAR, but it’s still part of ongoing research, so it’s not available in app form yet. “Our hope is that developers would jump in and contribute to the open-source project, make it better and that app makers will incorporate it into their apps making them Swÿp enabled,’” Linder says.
…
Swÿp and Luminar at work.
WE ARE STILL EARLY ON AND THERE IS LOTS TO DO TO FULFILL OUR VISION.
List and Linder picture a world in which the cumbersome process of sharing digital information with your neighbor is replaced by simple physical gestures. They’ve tried marrying Swÿp and LuminAR to create a new type of experience that lets users collaborate and create digital content together. “Most recently we’ve been deploying a website wherein any Internet-connected device would be able to Swÿp with any other,” Linder says. “Our target is everyone who uses touch- and gesture-enabled devices, counting laptops and iPads, but also screens with a Kinect setup. But we are still early on, and even though we demonstrated the working tech, there is lots to do to fulfill our vision of different devices chatting to each other using nothing but user-generated gestures.”
THIS CONCEPT BY ISHAC BERTRAN IS SO SIMPLE AND CLEVER THAT YOU WONDER WHY IT DOESN’T EXIST ALREADY.
A few weeks ago, Ishac Bertran wanted to pluck some articles from his web browser and slip them into his Kindle to read later (and more comfortably), but he was so daunted by the labyrinthine process of transferring data, he decided to skip it altogether. We’ve all been there in our own way.
“Our devices are well connected virtually, through services like DropBox or iCloud,” Bertran, an interaction designer, tells Co.Design in an email. “Those offer wireless synchronization for data, but the devices that contain this data still miss a tangible connection. I thought that a representation of a physical connection would facilitate a more intuitive interaction based on traditional mental models from the physical world.”
In other words: Transferring data really isn’t all that complicated. It’s just a few swipes or clicks of the mouse. But because it takes place behind the scenes—that is, we don’t see our files physically move from one device to the next—it feels difficult.
So Bertran tried to imagine a more natural interface, one that would help demystify the whole process by giving it a visual and tactile component.
As he conceptualizes it, users would hold their devices next to each other, and a half moon would materialize on each screen, together forming a full moon. That full moon visualizes the link between your hardware; it says, quite simply, “Your devices are now connected. Transfer away.”
Then to share a file, you’d drag it from one half moon to the other, using the swipes and pinches with which we’ve all grown familiar on iPads and iPhones. Here’s a nice animation of the idea:
Easy-peasy, right?
Bertran was inspired by spatially aware devices such as Sifteo Cubes, which turn user commands into jolts, tilts, and clicks, thus giving tactile form to invisible computational processes.
Of course, for Bertran’s moon concept to become a reality, the Apples and Amazons of the world would have to seriously revise their devices. As it stands, you can’t just hold your Kindle up to your MacBook and start swiping willy-nilly.
But Betran insists that new hardware wouldn’t be terribly hard to integrate: “Sifteo cubes use IrDA transceivers to detect other cubes,” he says. “Something similar could be placed in forthcoming devices to create this tangible connection. For the proposed interaction there is no need to detect other devices all around a device frame. It’s enough to have a particular position in which the devices physically recognize each other to enable more fluent and intuitive interactions.”
Fashion retailer C&A may be a fading brand in much of Europe, but its Brazilian arm is doing what it can to stay on the pulse of social media. A new initiative called Fashion Like allows people to ‘like’ certain items of clothing on the company’s Facebook page, and these clicks are collated and displayed on the relevant clothes rack in real-time. Customers are thereby able to view the item’s online popularity in the real world to help them make their decision.
It’s open to debate how valuable this will be to shoppers — we’ve seen the trivial natureof much that’s posted to Facebook, not to mention the dubious fashion sense of certain denizens, and it probably wouldn’t be hard to game the data. For terminally indecisive Brazilians, however, this seems like it could be a step forward.
I like to look at raw numbers every once in a while, without external influence, to recalibrate my ability to judge the magnitude of things. Here are some of the numbers from Facebook’s most recent S-1 filing (published on May 3rd) which I think are important as metrics to compare against when thinking about relative success and opportunity. (Sources are at the bottom.)
Total users and engagement
125,000 M Friendships 2,000 M Likes per day 1,000 M Comments posted per day 901 M Monthly active users 526 M Daily Active users 488 M Monthly active mobile users 302 M Photos uploaded each day
User penetration
85% Chile, Turkey, and Venezuela 60% India, the United Kingdom and the United States 30%-40% Brazil and Germany 20% Japan, Russia, and South Korea 0% China
Daily active users
526 MWorldwide total 152 M Europe 129 M US & Canada 126 M Rest of the world 119 M Asia
Monthly active users
901 MWorldwide total 242 M Rest of the world 241 M Europe 230 M Asia 188 M US & Canada
Revenue (last quarter)
$1,058 MWorldwide total $525 M US & Canada $328 M Europe $118 M Asia $87 M Rest of the world
82.42% Revenue from advertising 17.58% Revenue from payments (and other)
15% Revenue connected to Zynga
Ages of Facebook directors and officers
27 Mark Zuckerberg, Chairman and CEO 37 Mike Schroepfer, VP Engineering 39 David Fischer, VP Marketing 40 Marc Andreessen, Director 42 Sheryl Sandberg, Chief Operating Officer 42 David A. Ebersman, Chief Financial Officer 44 Theodore Ullyot, VP & General Counsel 44 Peter Thiel, Director 50 James Breyer, Director 51 Reed Hastings, Director 66 Erskine Bowles, Director 66 Donald Graham, Director
Executive compensation in 2011 (cash+stock)
$30,957,952 Sheryl Sandberg, COO $24,804,472 Mike Schroepfer, VP Engineering $18,761,293 David Ebersman, CFO $7,082,294 Theodore Ullyot, VP, General Counsel $1,712,362 Mark Zuckerberg, CEO
Board member compensation in 2011
$618,067 Marc Andreessen $610,067 Erskine Bowles $16,667 James Breyer $16,667 Donald Graham $16,667 Reed Hastings $16,667 Peter Thiel
Ownership after IPO
23.550% Mark Zuckerberg 7.632% James Breyer (and Accel Partners) 6.253% Dustin Moskovitz 4.912% DST Global Limited 3.258% Sean Parker 2.109% Mail.ru 1.729% Peter Thiel 1.660% Elevation Partners 1.560% Meritech Capital Partners 1.387% Greylock Partners 1.227% Microsoft Corporation 0.309% Marc Andreessen 0.201% Mark Pincus 0.112% David Ebersman 0.107% Mike Schroepfer 0.095% Theodore Ullyot 0.089% Sheryl Sandberg
Frequency of words appearing in S-1
404 Users 169 Mobile 124 Payments 38 Zynga 34 Microsoft 32 Games 18 Phone 14 Google 6 iOS 4 Android 2 Apple
Sources: All numbers are from Facebook’s most recent S-1, filed on May 3rd, 2012. Ownership percentages: Page 145, and calculated based on the number of beneficially owned shares against 2,138,085,037 total expected outstanding shares after the offering. Disregarded voting difference between Class A and Class B shares. Ages of Facebook directors and officers: Page 108. Executive compensation in 2011: Page 125. Board member compensation in 2011: Page 114. Ownership after IPO: Page 141. Total users and engagement: Page 2. User penetration: Page 48-49. Monthly active users: Page 48. Daily active users: Page 49. Revenue: Page 51.
Disclosure: I am considering purchasing Facebook stock after the IPO.