Episode #179 Luke Wroblewski - Designing Multi-Device User Experiences
Context is an important consideration in designing a mobile experience. As new devices enter the market, designers have to contend with new form factors and consider things such as ergonomics. Even things such as Apple’s retina displays affect approaches to design.
Context is an important consideration in designing a mobile experience. As new devices enter the market, designers have to contend with new form factors and consider things such as ergonomics. Even things such as Apple’s retina displays affect approaches to design.
Luke Wroblewski, author of Mobile First, is at the forefront of mobile design. He says that designers need to make sure their designs are fluid and flexible. Starting with a fluid grid at a foundational level ensures that your design can adapt to a variety of viewports.
In addition, Luke says you want to take multiple screen resolutions into account. Instead of relying on images, he suggests employing cascading style sheets and SVG. This will make sure that graphics scale appropriately to different sizes and devices.
Luke explores this topic further with Jared Spool in this podcast. You can also check out Luke’s daylong workshop from User Interface 17, now in our All You Can Learn Library.
Jared Spool: Hello, everyone. Welcome to another episode of the SpoolCast. I have with me today someone who we've talked to, oh, about 7,000 times before, Mr. Luke Wroblewski. Hey, Luke, how you doing?
Luke Wroblewski: For the 7,001st time, great.
Jared: [laughs] It's awesome. You're going to be presenting at our User Interface 17 conference this November in Boston, and you're going to be doing a workshop on designing mobile and multi-device experiences. Now, last year you did a session on designing for the mobile web, and this year we've got this multi-device thing here.
What does that mean, multi-device, and what's different about that than what you were talking about last year?
Luke: Yeah. A lot of the core things remain exactly the same, in terms of building on top of mobile. I guess to step back half a second, inside the workshop I'm still doing a very big concentration on the mobile experience and the mobile web experience, but now I'm really orienting it as a foundation for a broader series of experiences that you could build on top of that.
The primary reason for that is, I've noticed over the past year, year and a half of giving this workshop, and talking about this material, that people's questions really quickly orient to, "OK, great. This is awesome information about designing for mobile. But what about tablets? But what about if I also need this to work on X, Y and Z device?" Lots of other devices have entered people's vocabulary.
Luke: Yeah, refrigerators are probably coming very soon. I think it's primarily been driven by the onset of the iPad. The iPad is now two years old or so, getting to two-and-a-half-ish?
Jared: Three. Isn't it three?
Luke: Three? There's three generations.
Jared: Oh yeah, no. Yeah, that would mean it would be two years old, right?
Luke: I think so, because it came out, gen one, yeah, about two years old. Maybe two years, two months. I don't have my exact chronology.
Jared: No, no. But it should be having tantrums and refusing to get up in the morning.
Luke: Yeah, exactly. [laughs] That's right. It knows better than its parents.
Luke: Then what? Towards the end of 2011, which was only six months or so ago, Amazon released the Kindle Fire, which prior to that there wasn't really much of a competition to the Apple tablet. Since then, the Kindle Fire has actually dwindled a lot, but back in December of 2011, they were selling a million units a week for about two months or so.
Jared: Don't forget the RIM PlayBook.
Luke: [laughs] Yes, and all 200,000 of those sold worldwide.
Jared: [laughs] Yeah.
Luke: I was actually reading a very funny article. Maybe it was written tongue-in-cheek, but it was basically saying that the biggest competitive threat to the current iPad is the previous iPad. [laughs] It's the one that's hot on its heels.
Jared: Yeah. I just saw that. In fact, they're selling a ton of iPad 2s, because the price point actually is fairly competitive.
Luke: Yeah. I wrote a big article about this, actually, around how the competitive price point of the iPad and of some of the newer tablets is really actually pushing e-readers out. Amazon, which arguably sells the most e-readers because of the Kindle, they don't release their numbers, but lots of people do estimates based on information they can glean.
There's been this transition where Kindle sales and Kindle Fire sales have really started to fall off precipitously. If you also look at what Barnes and Noble is doing with their tablets, they've moved the Nook to the Nook Color, which is essentially a low-price tablet device.
Jared: I have a research technique that's based on my walk back from the lavatories at the back of the plane to the front of the plane, where I'm sitting. What I notice now is that there are less Kindles. There used to be a ton of Kindles, and now there are less Kindles and far more iPads, and a handful of Android tablets, than I remember seeing in the past.
Luke: Yeah. Your research maps with the research that I've got access to, which are analyst estimates. But they really show...
Jared: It's probably the same methodology.
Luke: [laughs] It's probably the same methodology, the bathroom walk.
Jared: Yes. It's a good method.
Luke: They just may be getting paid a little more for that than you. [laughs]
Jared: That's true. I need to revisit my rates.
Luke: Yes, exactly. Yeah. I think the tablet space has really been pushing things forward. Five or six months ago, the instant growth of a seven-inch tablet made a lot of people wake up to, "Hey, we really need something a bit more flexible on the multi-device front"
Because even if we optimize things for, let's say, the iPad, which is this tablet form factor, and our desktops and laptops and mobile, all of a sudden here comes this little in-between-state thing. What do we do about that?
Luke: If you haven't really put the work and effort upfront into being flexible with how your web content works across these devices, over the course of a month there's X million of these different form-factor devices, and you're caught with your pants down.
For reasons like that, I think this topic has started to come up more and more. Just to continue on this trend, there's also really supporting data that I took to mean as supporting data that I took to mean as supporting...maybe I'm reading too much into it. But ComScore released this stuff around what the Android tablet marketplace looks like.
Buried underneath that was this really fascinating data point, that as tablet screen size decreases, the average number of web pages people view decreases significantly.
Jared: Oh, interesting.
Luke: You had 10-inch tablets would view an average of 100-something web pages, and then, when you got down to 5-inch tablets, it was down to like 60.
Jared: Wow. Why do you think that might be?
Luke: Yeah. That led me to do a lot of thinking about this, "What's actually driving that?" One of the things I found was research from our friend Jakob Nielsen that he did on the Kindle Fire, which is the seven-inch tablet form factor.
His findings there was that usability on web pages was just abysmal, because everybody was serving up desktop pages to this tiny screen, and especially in landscape format, everything was just a usability nightmare. You basically had to use your fingernails to do tapping of links.
Luke: His conclusion on this was, "Look, 7-inch tablets are going to go nowhere unless people optimize their content for it, unless people are specifically designing for this form factor, repurposing what you have from desktop or from a mobile size is not going to cut the mustard in order to make this an attractive web browsing experience."
That was how I kind of looked at it, is that the browsing experience got worse and worse as the screen size decreased. Because essentially you're just scrunching these desktop sites into more and more awkward configurations. As a result, the cost of browsing the web goes up, so people do less of it. Sort of similar to when you increase page speed on a web site. Page views just go up.
I think you and I have talked about, why is that? I've been asking many people over the past year or so, and the most common explanation I get is that the cost of accessing another page goes down significantly. Therefore, people are more likely to hit another page, right? The page loads instantly. The odds of them hitting another one are, I don't know if "opportunity cost" is the right phrase here, but the cost of hitting another page goes down so much that x amount of people do, and therefore page views go up.
Jared: Yeah. Yeah. I think there is some conscious sense of hating for that in-between state. You've got a moment there where you've decided to move on. You've reacted to that and you've made a click. But the system isn't responding to that fast enough. You're still staring at this thing that you don't want to look at anymore.
If that happens enough times, it makes you resist clicking unless you absolutely have to. Whereas, if you click and you see the new thing and you decide, "Well, that's not what I wanted," you hit back, you see it immediately, then the cost is really, really low. It's not something that I think people pay much attention to. But I think it is something that plays a role in terms of how they interact with the system.
Luke: Yeah. The way the light bulb went off in my head was when somebody said, "Well, look. The inverse is absolutely true. If we make things slower and more painful, then you're going to view less stuff, right?" You're like, "Well, yeah, that's true." If I'm sitting there and waiting for a page to load for 30 seconds or something like that, the odds of me wanting to do that again on that site are greatly reduced because of the pain that I went through on that first page, right?
I think a similar thing applies here with the decreasing usability and visibility and functionality of the sites as the screen size adjusts. That may be what's responsible for the decreased level of web browsing. There could be a lot of other factors. All I have is that data point that says web page browsing goes down as average tablet size goes down.
Jared: Right. Right. There's a ton of things that probably happen there. But you were talking the Android operating system, so it's not like you're dealing with some weird proprietary browser that no one knows anything about, and has some weird usability issues in its own right.
Luke: Yeah. It should all be the same browser, right, standard?
Jared: It should be. What is that, Chrome for Android, is that what they run on that thing?
Luke: No, it's the Android web kit. Chrome is...
Jared: Oh, that's right. That's right.
Luke: Chrome is only something you can install on the new version of Android, Android 4.
Jared: Right. Right. I mean, that's really interesting. Of course, I don't know how much this is going to be important to people who listen to this podcast, but we're sitting here recording this at the beginning of June, and we're just days away from the WWDC, where Apple is, if they don't announce the new MacBook that I need to buy next week, I will be going on some sort of rage.
But the rumors are that they're going to be talking about new TV products. There's rumors that there's going to be new TV products out of Google at the end of the month for Google I/O. Sony and others are talking about TV products.
While for a lot of the folks who would listen to this, I think it's hard to imagine what their apps are going to be like in a TV setting, we could be looking at that down the road, a year, or two years from now, because a lot of us believed four years ago that most of the things that we were building would never be used on a phone, and that turns out not to be true.
Luke: Yup. Then just even smaller-scale stuff. There's all this swirl around the iPhone having a different screen size, therefore different resolution. There's still all this swirl around Apple putting out a seven-inch tablet. This is the world we live in. It was desktop, laptop for a long time. For many years, like 20 or so years, that was really what you had to contend with in terms of the web.
Then over the past, I would say, since 2008ish, even though the iPhone came out in 2007, the quantity of these devices didn't really get to a point where people started taking it very, very seriously until 2008, 2009ish. You had about three or so years of starting to deal with capable enough mobile devices in terms of web browsing. That's something you really have to take into account.
Jared: Yeah, what happens when Apple, as is reported through the rumor mills, announces Retina resolution displays for 11 and 13 inch or even 15 inch MacBooks? That's huge resolution. All of the sudden, all those little icons and all those little bits now are much tinier than they used to be.
Luke: Yep, and real crappy looking. All of a sudden, the beautiful brand presence you're trying to create on the web through all your hard work and art direction and everything looks like a bunch of pixilated mess because your imagery isn't Retina ready. That's another part to being multidevice, right?
Jared: It's not just imagery, too, right? Because I was reading an article the other day that talked about typography and how some fonts look really great at higher resolutions and other fonts look really crappy at higher resolutions.
Luke: Yep. We're just getting into more and more diversity, right? It's hitting more people front and center, right? I've actually, myself, come to this because of some of this stuff with seven-inch tablets and because of all this variety. I've come to the conclusion that some level of adaptability within the way you build designs is inevitable.
That adaptability used to be, for the web, make things fluid, make things flexible, allow things like fonts and stuff to scale. That was the level of adaptability that we had to deal with. Scalable design. Now we have another level of adaptability that we have to bake into our designs because the variants are more extreme.
That level of adaptability changes a lot because, for example, you have to deal with the resolution of the screen. You didn't have to do that kind of adaptation or optimization previously. You also have to deal with totally different device form factors.
You were mentioning things like TVs. Again, something that you didn't have to deal with. I don't know if you saw what Microsoft released at E3 with their smartglass for the XBox project.
Jared: No, I didn't see this.
Luke: It's really cool. Essentially, it's like Airplay for the XBox, but it will work on iOS and Android. You can kind of beam content around. They also have a version of Internet Explorer on the XBox which is voice and gesture controlled using the Kinect.
Jared: That's cool.
Luke: Now you have Internet Explorer 9, which is a pretty capable web browser, up on your TV and you're controlling it with your hands and your voice. This stuff isn't really that far forward. It's here today, but it's going to take a while before it actually hits people in the face, which is usually what happens with these things. The same thing happened with mobile, right? We knew it was coming, knew it was coming.
Jared: Oh, yeah. It was the technology of the year for about seven years.
Luke: Oh yeah. All these things are the next big thing in three to five years, right? But then, one day, you wake up and bam, there it is. It's sitting at your breakfast table.
Jared: That's right. Wow. With commercials on TV showing celebrities actually using it. It seems to me like there's so much more that we have to take into account. That's really what you're getting at, is this idea of developing a way of thinking, a philosophy, it sounds like, to make sure that you're not screwing yourself down the road by locking yourself into decisions that won't move to these other platforms when they need to.
Luke: Yeah, absolutely. Over...In 2011 and into 2012, as I've been talking about this topic, more and more questions have started arising in this area. I've expanded more and more on that. It's gotten to the point where it's, in the course of a full day workshop, it's half the day is really talking about all the things that you need to take into account for using a good mobile design as a foundation to reach all these different devices.
The reason why I say it's a foundation is there's a lot of things that you do to make a modern mobile web experience work that are underpinnings of anything you would want to do to reach multiple devices.
Jared: You're not going to go out and update your book, "Mobile First" to be "Multidevice First?"
Luke: No. Just to get really specific here, in terms of what foundational things, if you're going to go design a mobile web experience today, you want to make sure you use fluid grids. You want to take multiple resolution imagery into account. You want to try and cut down the use of images as much as possible.
Instead use anything that you can get away with, with cascading style sheets, maybe SVG, in terms of graphics. You want to utilize the device within the viewport to make the page render to the size that device needs to be. Just as an example of a couple things.
All of those are things that you would have in any baseline multidevice experience. You want the grid to be fluid. You want to have the device use the width it thinks it is to render your page. You want to take multiple screen resolutions into account. You want to use as much vector art as possible so it scales to all these different resolutions and sizes effectively.
That's what I mean. It's a foundation. Then, on top of that, you do additional things to make sure things work well on an increasing range of devices. I don't think anything's really changed with these mobile fundamentals. It's still arguably the most important experience, given the quantity of activity that's happening on it, and the number of devices.
Coincidentally, it gives you a really nice solid framework around how you can expand out from there, which you wouldn't have from building from your desktop site, as an example. Desktop site doesn't do anything with viewport device width. There's tons of desktop sites out that are fixed pixel layout. There's lots of desktop sites out there that are very raster image, pixel image heavy.
Jared: What's interesting to me is that all this stuff is changing but there's a ton of stuff that hasn't changed that much that we're still dealing with, right?
Luke: Well, as things change they stay the same or whatever that phrase is.
Jared: Right. The more things change, the more they stay the same.
Luke: That's right. I definitely think the methodology for building web experience has undergone, I would say, the most significant change I've seen since the introduction of web standards, if you will. You used to build web pages. It was very heavily table layout and pixel perfection early in the day.
Jared: Yeah, that David Siegel, make it look like print as much as possible.
Luke: That was initially how lots of people reacted to the web. The whole first generation, if you will, of websites were really built that way. Because you had very primitive tools back then, too. You also didn't have a lot of interactive widgets or anything like that.
Then, sort of the next big shift in here's how we build websites really came when you had web standards and CSS, and forced a lot of people to relearn the way they created websites. Then the next thing after that was really sort of the Ajax-y, if you will, world where things became a lot more rich and interactive and you didn't have the primitive DHTML world you had before then.
Jared: Yeah. Everything didn't require a page refresh so suddenly you could do sophisticated things all within the confines of the same page.
Luke: Yeah. It really changed a lot about how you build pages, too. Since then, I think the next biggest one is really this transition we're going through right now. I just reflected back. I've been burning the midnight oil working on building up a site recently and I was just reflecting back how everything I'm doing is different now.
It's, once again, a big transitional stage and there's lots of things to learn about and lots of things to fold into your existing workflow, toolkit, skill set, all that.
Jared: Right. How do people adapt their process, particular when they're working on teams? What are you seeing as a way for people to start thinking this way and to get the rest of their team to start thinking this way? What's working and what isn't?
Luke: Let's go back, for a second, to that quote of the more things change the more they stay the same. I'm finding that people still have the same challenges. I've been going and talking a lot to different companies around this mobile and multi-device stuff.
The biggest challenge they had 10 to 15 years ago is still the biggest challenge they have today which is their bosses want to see everything pixel perfect across all the screens, which was really damn hard back when we had IE6, we had Netscape Navigator 4 days. Now it's even more ludicrous in that how am I going to get this to render the exact same pixel position across different devices?
Especially given the quantity of different mobile devices and things like this. There's still this...I will call it, a naive expectation on the part of the business owners, in some cases marketing, in some cases brand teams that this has to look exactly the same because it's representing our brand. The web is inherently this fluid, dynamic medium.
That's an example of things are staying the same even though so much has changed over the years. I see a lot of people struggling to communicate to the teams that they work with this is how the web works. There's inherently this variability. We have to embrace that variability in order for us to be successful instead of fighting against it all the damn time, and trying to make sure our stuff looks like brand posters in every single screen.
That's one thing where I think educating and learning and communicating can help get everybody over this hump. It's not just the folks doing the product design that have to get over this hump and realize what's going on. Then the other big challenge, which, again, is a challenge that's existed for a long time but is now coming much more to a head is the designer/developer relationship.
The challenge that's always existed there is, "OK, I've made a mockup, now I'm going to throw it over a wall. Go implement it and I'll come back and look at it later." That was never a good process for building web pages, and today it's even more problematic because if you're doing things that adapt to multiple devices what are you going to do, make a mockup for every single sized device for every single screen and then send that over?
Even if you do, no matter how good a designer you are I guarantee you're going to be wrong in many ways because there's going to be things that you didn't think about in terms of the transition between those states.
The process is much more let's have some reference designs at the edges and then let's sit with our development partners or let's, ideally, do a little bit of development ourselves to figure out all of those in-between states and the variances of these pages and these designs. You just can't throw it over the wall.
Jared: This is actually really interesting to me because I've been working for the last year really trying to understand how this role of the designer is changing, and there's been a lot of resistance to designers who know about the technical side, folks who have been doing design for a long time.
I've gotten away through my career without knowing much about how these things are implemented and I don't really need to go learn that because it scares me, and it bothers me, and my plate is already full, and I've got a lot of things to do. I work with really talented developers so I'm going to let them do what they need to do from an implementation standpoint. I don't have to worry about that.
What I'm hearing from you is what I've been hearing in other places which is the designers do need to understand how these things transition and how they get implemented, just as much as the developers themselves need to understand the underlying principles and philosophies behind the design and what the design is trying to do.
Luke: Especially if they're separated. If they're sitting next to each other then they can talk about that the whole time and gradually they'll learn.
Jared: Exactly. That's what happens. Maybe that's the problem. Maybe these designers that are resisting this have lived in these siloed environments where they think that in order for them to understand this stuff they have to go to three years of school. Whereas the reality is, the way people are learning about this is they're sitting next to someone and they're watching them do it. They're learning through that osmosis process.
Luke: I think some level of responsive design is inevitable given the ecosystem we have. If you're putting in some responsive attributes into your web design work, there's all these things in-between that, frankly, you don't know until it's in the web browser and you start looking at it on a couple different screens and sizes.
A lot of the adjustments you want to make are driven by the design and the content. You don't want to drive it based on some technical consideration. You want to drive it based on, "OK, when you looking at it on this device this thing is off and awkward. Let's adjust that. How do we adjust that?" There's all these middle states, and in-between moments.
Again, you can't account for that in a series of mockups just because there's too many of those. The process is like you literally sit there and move the damn web browser from left to right changing the size and see what happens. Then you go and look at it on a bunch of different devices and make sure that what you saw on that web browser is actually the case on those devices.
There's just a lot more time spent in the prototype or production aspect of things that is design time as opposed to just development time. If you as a designer are not there for that design time somebody else is going to make those design decisions.
Jared: Right, and if they haven't absorbed the principles and the understanding that you have the odds of them making the right decisions are probably slim, so you've got to have that back and forth, it sounds to me.
Luke: Sure. I mean if you've got a developer who is essentially a designer then they can do a lot of that stuff. There's a lot of designer developers out there, people that will code the HTML and CSS and put together the whole package. In that case, it's the ideal situation.
This is what everyone calls the unicorn role or whatever...because that person is able to make both the design decisions and some of the technical decisions at the same time and fold it together into one system. The vast majority of people aren't in both sides of that that deeply, so you have people collaborating. When they're collaborating, it has to be a collaboration.
It can't be an over the wall kind of thing. I think Ethan Marcotte, who coined the phrase responsive web design, calls it, "designer developerism" or something. I can't remember the phrase.
Jared: He has a phrase for it where you're working back and forth. He's even developed tools where using a particular type of annotation in Photoshop that both the developer and designer understand, they can go and grab the right measurements and thoughts and stuff.
All of this talk is making me think...Years ago I was in a book store, and I came across the script to "Raiders of the Lost Ark". They were selling it in a bound edition. That was fine. I didn't really need the script but I picked it up and started flipping through it and I realized that the back third of the book was all the storyboard pictures that Spielberg had done of the movie.
They had these index card size drawings of practically every shot in the movie and it was fascinating to go through and actually watch the movie and watch the storyboards at the same time and see how much he hit on the mark. There were differences and, more importantly, there were things that were happening between each of those index cards that happened in the movie, that weren't there.
When Spielberg makes a movie, he doesn't draw out his index cards and then give it to the Director of Photography and say, "Go make this for me." He's sitting there watching the movie being filmed.
Luke: I think that's exactly the right analogy. He's not just sitting there watching the movie being filmed. He's, "take two, take three, change this angle." He's directing.
Jared: He's directing and they're looking at these storyboards and they're looking at the film they produce. They said OK, that didn't work out the way we expected. What if we try it this way? What if we try it that way? Film was revolutionized when they did something really clever a few years back.
They took a video camera and they mounted it to the side of the actual film camera. While they were filming, they were making a video and they could watch the playback of the video and come pretty close to what was going to show up on the film. Whereas before that they had to wait to get the film developed and then they would go to what they called dailies, where they would watch it and then decide if they have to retake a scene.
They could decide in the moment while the crew is still there and the set is still set up and the special effects are still in place, they could decide in that moment to redo that scene if they had to because they didn't get the effect they wanted. Now, with the advancements in digital cameras, they actually see what the lens sees for the films so they're even closer. I think that's what we're seeing here. I think we're seeing that progression.
Luke: I agree. The role of the director, I think you touched on this a little bit is, "That's not what we were thinking, let's try something different." That happens a lot. You have this initial vision in your head and it's based off of any number of assumptions that you're making at the time.
But then when you actually see, like in the case of the movies, when you see the shot on the set and you see the actor in there, and you see the zoom level, and you see the lighting or whatever it is you realize that's not going to work. Let's tilt this over here. Let's twist this over there, or whatever decisions you make.
There's all this improvisation and flexibility inherent in the process. Same thing with doing this type of design. You can't shut yourself out from that as the designer, I think.
Jared: Right. Right. All that time that you're doing that, the developer you're sitting with is learning what you like and what you don't like and slightly modifying their future work to match what they saw you liked in the past.
At the same time, you're learning what is possible, and what is not possible and modifying your future work. It becomes this process where minds are converging and talents are converging and it's no longer about who's the designer and who's the developer.
Luke: Yes, I love that. You're just working together on a product. I never understood this designer/developer separation.
Jared: Schism. Schism.
Luke: Schism or whatever it is. Some of the best design ideas I've ever heard have come from the developers I've been working with. They're in the guts of the code and they see this and they're like, "Hey, did you think about trying this?" You're like, "No, I haven't. Let me go build that into the design."
That idea may be really raw, may be very technology focused or whatever. It may need some crafting, in which case having a designer there to help you out is awesome. I could never imagine just cutting off all of that feedback and those ideas and say, "no, I'm doing the design." I don't know. It's like cutting yourself off at not just the knees, but at the hip.
Jared: Absolutely. Absolutely. Luke, this sounds really awesome. I'm very much looking forward to your workshop and learning all the things that I need to know, that I didn't know because I haven't been paying attention. [laughs] No, I think it's going to be great.
I think this multi-device stuff, it makes a lot of sense, this idea that we need to take into account all these different dimensions, and attributes, of the designs that we're creating, from the physical elements through the interaction components, to come up with a philosophical framework for producing that. I'm really looking forward to hearing you talk about that in the workshop.
Luke: Yeah, me too.
Jared: Excellent. For those of you at home who want to know more about what Luke is up to, you can listen to the other 7,000 podcasts that we've done with him, or you can save yourself a lot of time and you can read his fabulous book from A Book Apart, called "Mobile First," which really has a lot of insights in it.
Of course, you definitely want to attend his workshop in November at the User Interface 17 conference, and you can learn more about that at UIconf.com. Luke, thanks for taking this time today.
Luke: No, thank you. I appreciate it.
Jared: I want to thank our audience yet one more time for listening to us, and enjoying it, and sending in your feedback. Thank you very much for encouraging our behavior. We'll talk to you next time.