Episode #216 Adam Connor & Aaron Irizarry - Building Consensus in Critiques and Design Studios
Critique is often confused with being negative and critical. However, the basis of critique is communication. Having strongly grounded communication is necessary for any relationship in life, work related or not.
Adam Connor and Aaron Irizarry believe that critique is not just a design-centered skill that exists to make sure you’re doing things “right”. Instead, they see it as a living and breathing process of analysis and adjustment. Simply saying, “I don’t like blue” is not a helpful way to critique a design. Instead, they suggest framing it for better understanding of what objectives were trying to be met and what problems they were approaching in order to better iterate on the design.
Check out Adam and Aaron’s daylong workshop from the User Interface 18 conference, now in our All You Can Learn Library.
Jared Spool: Welcome, everyone, to yet another episode of the SpoolCast. We have done a ton of these. They've all been fun, but today is going to be the most fun because we have Adam Connor and Aaron Irizarry joining us.
You may not know this, but they're doing a full-day workshop at the User Interface 18 conference called "Building Consensus in Critique and Design Studios." Today, we're going to talk about critique and design studios and consensus and teamwork. I'm really excited about that.
Hey, Adam. Hey, Aaron. How are you guys?
Adam Connor: Doing good.
Aaron Irizarry: Good, thanks. Thanks for having us.
Jared: I thought we could talk about, today, something that I've noticed. I don't know if you guys have noticed. When you teach a team to really do critique well, it changes that team in some interesting ways. Adam, have you seen this?
Adam: Definitely. I think, if you understand critique as an analysis activity, where you're taking a design and you're comparing it to how it achieves objectives, and then you're taking that critique and you're making it a bigger part of the conversations you're having.
You see that the conversations you have with your teams actually start to be way more about the design and the objectives and that, than about personal opinion and about, "Oh, maybe we should use HTML5," or "Maybe this thing should be blue, and I like blue" -- things that really aren't necessarily material to the conversation you're having at that point in time. It really helps people stay focused, I think, rather than getting into parts that just aren't helpful in terms of progress.
Jared: What about you, Aaron? Have you seen teams shift when they learn really good critique technique?
Aaron: Yeah, I definitely have. I've seen teams go from a group of people who don't communicate to each other that well, and when they do it's often more each person almost trying to be heard, surrounding their opinions around design or design thinking and what's trying to be built. The more it becomes, just like Adam was saying, the objective side of it, where it's analysis, there's not the argument over who's right and who's wrong. It's all of us sitting down together and analyzing the progress of this product we're building.
The more it becomes that, everybody looking at it together, there becomes more of a team. The team actually starts to become more of a team, and you get that better conversation around design and what's being built. You just start to see everybody start talking about just the product better.
Which is pretty amazing, because we work so hard on products with our teams, you would think that we'd all just be able to talk about them with one another so well, but once we start critiquing better and understanding the analyzing and just looking at all the different goals and where we're going, and it's less about individuals and more about the product. Not only do we talk about critique better, but I've just noticed that teams communicate with each other, in general, just much better than they have before.
Adam: Another thing I've notice, too, is the frequency of communication changes. The environment of communication changes. As teams get more and more comfortable with this and talking to each other in this way, there's less reliance on these massive meetings where everybody gets together in a room, and the design review, so to speak.
Designers and developers and everybody get more comfortable with just reaching out and grabbing someone, much more frequently, to ask for their critique and their thoughts on what they're working on at that point. That kind of shift in how the communication is taking place, to Aaron's point, I think helps that collaborative spirit within the team.
Jared: It's interesting that you bring up this idea of frequency, because I wonder if some of that frequency comes because you've reduced the fear factor of critique. It changes from something that you were scared of getting.
It's like, "Oh, we have to prepare for this. We have to sort of bulk ourselves up and be ready, because it's going to be a complete affront on our ego and personality and skills," to something like, "Hey, dude! Let's collaborate on this. Let's work on this together. I have this idea, but before I go forward with it, I'd really like to know, am I half-baked here, or is this something I should pursue?"
Adam: Yeah, I think that's absolutely it. At this point, we've worked with a lot of different organizations on this skill, and also we've had the opportunity to talk to a lot of kids going through school who have various experiences with critique. What we've found is that the ones that really do get it and are doing it well, they'll start to talk about how they want critique.
As part of their creative process, they desire this feedback loop, and so they actively seek it out. We've also seen this go a little bit too far in the other direction, where if it starts happening again in a bad environment, some people can actually become fearful in the other direction, of making too many decisions and not having everybody else agree to them before going too far.
There's a nice middle ground that it should be your goal to reach. You don't want to ignore critique, but you don't want to become so dependent on it that you're scared to make a choice yourself.
Jared: How much is critique a way to check your gut feeling? Here in my gut I believe that this thing I just put out, this is the way we should go. I'm going to now ask for critique from people whose opinions I admire and who've been working with me on this so they understand where I'm coming from. They understand where I'm going.
Two things can happen. I can either find out that I got it right or I can either find that maybe I'm going in slightly the wrong direction or maybe majorly in the wrong direction and there are other ways to think about it.
I think the fear comes from this thing of, "I don't know if my gut is very good." I'm wondering if, over time, you've seen people go, "You know, I keep putting this out for critique and I keep finding out, hey, I'm actually...Most of the time I'm going in the right direction. I'm getting feedback about this that tells me I'm...My gut's pretty good. I'm getting good feedback that I actually have decent judgment of my own."
Aaron: One of the things that I've noticed is we talk about this as we're sharing and just discussing this idea of critique with people. Sometimes we focus so much on critique being, "This is everything that's not working in the design." Equal to that needs to be, "These are all the things that are really working," because you want to know what's working so you can measure that for yourself.
Like you're saying, with that gut instinct...I'm looking at the goals and what I'm building and what I'm working on. OK, based on everything and what we're doing, I think this is the right direction to go. And then you grab a team member or two and say, "Hey, let me bounce this off you. Here's what we're trying to do. Here's what I'm thinking. Am I right here? Am I a little crazy? What's going on?"
And then you find out, "Oh, it is. Good." You can start to build that confidence and you start to be able to really move the design forward because...I don't know. There's something about always hearing about what's not working that I think can start to create a little bit of that intimidation or that fear.
Or as Adam was talking about, you're worried then to make a decision on your own because you're becoming so reliant on critique to get all the right feedback because you're just, "Ah..." Your self doubt creeps in and a lot of those things. It's an interesting dynamic with critique because you really do end up putting yourself out there.
But I think it's great. I'm fortunate with the team I work with now at Nasdaq. It's not uncommon for my Skype to blow up two, three times a day. "Hey, if you've got time later can I run something by you?" Or even into our little group chat for the company. "Hey, anyone have time at this time today? I have some ideas. I just kind of would love to vet those with you guys. Can I just get some critique on some of this?"
It becomes a natural part of our process to discuss design and check our gut instincts with our peers and our coworkers. That's great. I think, as much as you can, then you really should try to do that. Maybe throwing caution to the wind like Adam said.
Not trusting your gut so much that you check every little thing and don't make the decisions for yourself but I'm a big fan of just reaching out and always asking and double checking with people when I'm feeling that I'm at that point where the work can be analyzed. "I think I'm at a good point. I'm going to ask for some feedback," and then move it from there. It's a great tool.
Adam: Yeah, and to build on that aspect of making sure that the strong points are articulated in a critique, a lot of the times it gets positioned at you're doing that so that the designer will feel more comfortable, feel better about what they've created. But it's also important to the design itself in that often when we find something that's working in a design there are ways to abstract what it is that's working and pull it into an aspect that isn't working.
I think that's what designers do all day long, really. They go out and they look at other work and they pick up on pieces that are effective and achieving certain objectives. They're like, "How can I use that to make this better?"
You can integrate that into your critique by identifying what's working. The designer can then step back and say, "Well, if it's working over here is there a way to build that in over here where I might not be as strong?"
Jared: I like this idea because it sounds to me that some of what happens is that critique is the opportunity when I get to learn what this design is really about. Sometimes when I'm designing something I put the design out there and I think I know what it's about.
I think I know what the users want from it or I think I know what we're trying to do here but it isn't until I get the critique and I start to talk to my peers, talk to stakeholders, talk to clients about what we're trying to do that in the process of both talking and listening I find out things about the design that I didn't realize until that moment. Does that happen?
Aaron: Yeah, I mean I think so. One thing I've found is that as a design community we're relatively spoiled, because of just the ramping up over the past so many years of social engagement, and this has allowed us to just have access to incredible amounts of content.
And for me, personally, one of the things I've found is that how many blogs or books or conversations via Twitter or anything else -- the conferences we attended -- we start to kind of think, "Well, OK, that's the way to solve this problem. These are goals. This is what we're trying to build here. Well, this is what I know to do in this situation."
And a lot of times critique allows me to vet that, right? So I start talking to other team members and then, "Oh, well, did you think about doing that this way?" or, "Why did you do that?" And I have to explain as I answer questions and critique and explain my thinking behind thing.
It's like you said it starts to re-familiarize myself sometimes with the design as opposed to just going with what I always know and then getting the insights and feedback and taking that back on my own and having time to digest that and then refocus my efforts on what's being built.
It allows me to explore a little more outside of just the typical, "Oh, yeah, it's this type of flow. We're going to use these types of patterns, because that's what we always use," or, "That's best practice, or whatever, on this blog," or whatever that scenario might be. It allows me to slow down and digest the design and continually keep the goals and the scenarios and the personas in the forefront of my mind, because those tools are constantly being used in analyzing the product. So at least for me that's -- if that answers the question -- that's how I've seen that.
Adam: So Aaron just mentioned goals, scenarios, personas, design principles. I think every critique should start with some kind of foundation, and those are typically the elements that we propose a project establish first in order to facilitate good critique, but as the project goes on and you're referring back to those goals and those principles, you can see kind of an evolution and a refinement.
Goals might start off being a little vague or even principles might start off being a little vague, and then as you go through these critiques you find that they become tighter and tighter and tighter, because you're talking about things more with the team, you're coming to this understanding a little bit more about what it is you're trying to achieve and exactly how you're trying to achieve it, and that's a natural progression.
That's consensus building, really, too, in fact. So, yes, things will evolve, but the important thing is that as they're evolving you're making sure everybody's still on the same page. Everyone's aware of that evolution and so you're updating your principles as necessary. You're moving the flag in the ground, basically, and making sure everyone still knows where it is.
Aaron: Yeah. One of the things we started doing with our team just to do exactly what Adam said right now, is that every maybe four or five sprints, or looking at our planning as we're working things out, we re-evaluate, because we're testing on a regular basis. We're testing almost weekly. We're critiquing even more frequently than that.
So we get to a point where we say, "OK, now let's step back, and let's take everything. Let's kind of gather and aggregate all this information we've been learning as we go through the product design process, and then now let's reevaluate. Do our personas still hold up? Is there edits that need to happen? Have we learned more about this user or this user group? Have we learned more about these scenarios and the different use cases?
And then we adjust from there keeping that same foundation and a lot of the same principles there, but there are adjustments that happen, because you learn as you go. If you're willing to critique, and you're willing to open up to analysis throughout your process, you're going to gather new information and learn new things. Then you can then kind of retool and adjust those scenarios and goals and personas and your principles and all those things based on what you're learning.
And like Adam said, it's like moving your flag forward a little bit, sticking it in the ground, working for a bit, moving it forward, and that really helps immensely to keep your team having shared understanding of what's being done in the process and what you're building.
Adam: So I actually want to turn it around a little bit now. We've used the word critique throughout this conversation quite a bit, and one of the things that's been on my mind lately is if you think about everything the three of us just talked about, we're really talking about all these conversations that happened throughout the life of a project.
And we keep using this label critique, but does it create resistance or maybe confusion that critique is often thought about as this specific activity that you go and do, right? And it's not really seen as it's just how we should be talking. That's something that I believe, but I wonder if that creates maybe discrepancies in the community and in teams in terms of what we mean when we say critique.
To me critique should be integrated into every conversation we have. We don't necessarily go and have these dedicated critique meetings where this is what we're going to do. It should just be part of the natural language. So I wonder if you guys actually see that happening in your teams?
Jared: I think that's interesting this idea that there isn't a dedicated moment where it's like, "OK, it's 10 o'clock. It's critique time, and it's going to go to 11," and then at 11:15, "We're no longer critiquing so go back to your old talking ways."
Adam: Exactly. [laughs]
Jared: Right? It's like, "OK, at 11:15 that color sucks. I don't like that color." I told you about the CEO who didn't like the green because it reminded him of the sweater that his ex-wife wore?
Jared: It's like, "Really? That's where we're going with this?" So this idea that critique changes the way we talk to each other, and allows us to have a dedicated method of approaching an idea. What's interesting is the teams that I see that start with critique about the design quickly turn to critique about their own process and about how they work with each other, and they just learn how to deliver constructive and affirmative feedback to each other much better.
And you hear them joking about it. It's like, "Oh, oh, yeah, you're giving me that feedback thing," but it turns out it works.
Aaron: Yes, I agree. I know critique isn't just this design thing, like you were saying like we're seeing who likes what color based on their personal experiences, or any personal opinions. Critique is a methodology, and we can apply it to anything.
We can apply it to, like you said, our process, how we even communicate with each other. Is it even working? Is it effective for the team? It almost becomes like the breathing of the design process. It should be a natural piece of what we do and how we talk and engage team members.
A lot of times Adam and I have both said in the past that it's almost like a life skill as well, because pretty soon many of us have mentors or spouses, relationships, where there's a level of feedback, good or bad, that comes with those sometimes, and just learning how to communicate gives you a chance to practice critique.
And so it's not just some design-centered activity that's done to make sure you're doing stuff right, you know? It's really this breathing, living process of analysis and adjustment and moving forward in the right direction, and I think that's it's really helpful if we do take it outside of design sometimes and look at how we're communicating with teams, or apply it to our process or our workflow. I mean, "hey, this is how I'm working through the day. Maybe I keep email open too long. Talk to people. Think about ways to improve everything by using this active iterative process to do that.
Jared: What, in your mind, because I'm wondering if there are people listening who are like, "This critiquing sounds cool, but I am not exactly sure what they mean by it." Can you have this short summary?
When someone is doing critique, how does it feel different than when just the normal type of criticism that you get in most projects?
Adam: Really, quickly, Aaron and I have this mental framework that we use where your first step is pretty much to identify what objectives you think the creator -- I'll use creator very loosely -- whoever it is that made the thing you're looking at, you're critiquing.
Jared: If I brought a design to the table and I said, "Hey, guys. This is what I've been working on for the last couple of days." I'm the creator in that case?
Adam: My first objective should be to try to understand what your objectives were in that creation. What were the goals that you were trying to meet or problems you were trying to solve?
I have one of two ways that I can do that. I can come up with a whole bunch of assumptions on my own or I can ask you explicitly. From there, now my thinking should be shifting to, "OK, so what choices did you make to try to achieve those things?"
Then, once I understand that, then I can say, "OK, so how effective are those choices in meeting the objectives that I think you have?" Then we can talk about that. Then you can start to go beyond that. You can start to talk about maybe challenges that might arise because of the choices that the designer or the creator has made. Maybe new challenges or maybe thinking about missed opportunities.
If you start to go into that realm, then you've just got to be prepared for the creator to say, "Nope, that's not what I'm focused on right now." That's out of scope, so to speak. It's really those first three steps. What were the objectives? What did you do to try to achieve those objectives? How effective were the choices you made in achieving those objectives?
Jared: I could say what I was trying to do was see if I got that status information on the same screen, so you didn't have to press a button to get it. Would it make it feel better? If you come back and say, "Well, I don't like the color or the font," I could say, "Well, yeah. But I wasn't ever thinking about the color or the font. I was just trying to figure out, could I get it to physically be in the same space."
I tried putting it above the button. It didn't quite work, so I ended up putting it below the button. What do you think? Now we could talk about that. Do I have that right?
Adam: Exactly. Yeah.
Aaron: You can even use, like we mentioned a few times, the goals, scenarios, personas. If you go into this critique knowing what we're trying to achieve with what we're building and someone does say like, "Well, you're status message is below the button, but I really don't like the color that you used or the typeface."
Then you can then say to the person receiving critique and then ask, "Well, you know our goal right here is to get this specific persona or user group information quickly about what they're trying to submit with this button.
How does that color choice or that typeface that you're concerned about affect that positively or negatively? Critique then starts to become more about the product, not about the individual. It's not like a me versus you type of thing.
You're actually providing questions back to the person providing the critique that will help them frame the information back to you in a way that you can actually do something with. It's something that's actionable and understood.
Adam: This really starts to speak to you the immense importance of having this agreed upon problem framing and foundation to work from. Everybody is coming into this conversation, this project, with a different set of goals in their heads, different set of scenarios, different design principles, different aesthetics that they like.
If that's what you're working with when they're having these conversations, you can see basically everybody's got a different project in their mind. You're not all working on the same thing. Building that agreed upon foundation to start with is really important to how your conversations are going to go for the remainder of the project.
Jared: That idea I think is huge. That idea that we use the critique to actually help us make sure that we are on the same page and that we're focused in the same area. We're all looking through the same lens at the design at the right time.
So many projects that I've been involved in, you're sitting in this meeting and I have this thought to myself, "Are we actually all in the same meeting right now?" Everybody seems to be talking about something different and everything goes in circles.
Aaron: Yep. Yeah, one thing...I'm pretty sure we've all experienced that. Then to the other end of that, one thing that I experienced, and actually experienced quite a bit last year, was we'd be in the meeting and people were using a lot of the same terms. They didn't all quite have the same definition of those terms.
Very much like a princess bride moment where it's that thing you're saying, "I don't think it means what you think it means." By reiterating goals and scenarios, adding a little bit of definition to that, and referring to it over and over again, people start saying, "Oh, wait a minute. You know what? Maybe I need to reframe what I just said right now because I thought it meant something different. Here's what I'm really trying to say."
You start to provide a framework, almost like a template per say and these guidelines for helping people communicate, because very rarely are we only in meetings with just designers.
We will have product folks, marketing people, executives, and various stakeholders. They're all going to have some different context and understanding of what we're discussing. Using these goals and scenarios and re-bringing them up every time and analyzing the design against those starts to help them understand that definition and solidify the shared understanding.
When you're in a situation where everyone seems to understand or they sound like they do, a lot of us at times will speak as though we have understanding. We want to make sure that we fit in and don't get called out. Then it really helps to unearth that a little bit and ensure that we all really do have the same understanding moving forward.
Adam: I think oftentimes teams get caught up in this mentality, this idea of, "Well, let's just start building something and then we can start talking about it and critiquing it." As much as I have total respect for agile, manifesto, or parts of it, and a lot of the lean stuff that's out there, and I agree with a lot of stuff that's out there, I think that there's a danger to taking it too far.
When you're working with the team and saying, "Let's just start making something." Good collaboration really comes from shared goals and some degree of shared understanding and objectives.
You need to take at least sometime to frame the problem that you're trying to solve in a way that you can all agree upon, so that as you are working together you can always come back to that agreement and focus your discussions there.
When Aaron and I start talking about principles and scenarios and stuff, sometimes we get pushed back and forth, "Oh, that's going to take too long. We don't have all that time. We need to just start moving."
I think if you do that, a lot of teams end up shooting themselves in the foot later on. Inevitably, at some point, they have to figure out what those agreements are. Doing it when you're months into a project is not necessarily as efficient as taking a little bit of time upfront to at least figure out what that seed is.
Aaron: Yeah. I think even you could take a lean or agile approach to some of these things. Personas don't always have to be like a short story about someone's life. They can be the key things about that user or that relate to the product you're building.
You don't have to really go into detail about this person that only has two point five kids, a white picket fence, a minivan, their dog. Likes to play on Saturdays, but doesn't really fetch very well.
Keep it simple, you know? Get in there and get the information. Use critique to critique your information and make sure that it's right. I think that if you could approach that correctly, like Adam is saying, you will actually have the ability to work quicker as a result.
Because your shared understanding is established, you're off to the races after that. There are lots of ways. Even with some of the lean stuff, I know that Jeff has provided lean templates for personas. It's not that these things contradict, but it's understanding how to use them in the context of your business and with your clients.
Jared: These ideas of just gathering this stuff and not making it a big production, but just saying, "OK. If I'm working on this dialogue box right now, who are the different people who will need this dialogue box? How will they behave differently when they come to it?"
That's really what we need from a persona. We don't need to know what they have for pets and for what they drive. We just need to know that the system admin is probably going to do batch entry, but the end user is just putting the one thing in. They may not understand all the words.
At the one time I got the assist admin who knows the words and understands the fancy language, but they need to get in and out really quick. At the same time, I need to support this end user who's coming in at this and this may be the first time they've been introduced to these concepts. How do I get them to make sure they put in the right answer?
That's really all I need to know, right? I got two personas. I got scenarios around that. We're not talking about a whole lot more detail than that, are we?
Adam: No, it doesn't have to be. Sure, you can add in all that back story to Aaron's point, but how does that change the design decisions you're going to make? How does knowing that so and so has a dog or doesn't have a dog, how is that actually affecting your decision making?
That's the key thing. It's understanding what information it is that will allow you to make different decisions. That's the information you need to know. Anything else is superfluous.
Aaron: Sometimes you may come into the project with this information provided already from a research team. One of the things we talked about is the mini or the short form creative brief and providing these small summaries of this information based on the context of what you're doing with the product at that moment.
You can use those things, summarize that, and then use that summary to be more lean, precise, effective, or however you want to put it and move things forward from there. That way, you don't have to have these larger, if you have these larger, more comprehensive documentation, specifications, and personas that are a little more than what you need.
Then you can just pull the summaries from these things and use them as a great little measuring tool each time you meet and as you move forward. It doesn't have to be so in depth.
Adam: A great way to understand if something is important material to your decision making is imagine yourself in a critique later on, asking yourself how some aspect of a design helps or doesn't help. Or works or doesn't work to the extent that the user has a dog.
If it doesn't matter, if you imagine that being the most ridiculous thing in the world, that thing doesn't need to be in the persona. If you can imagine yourself asking that question and it being important, then yeah, you probably want to include it in there.
Jared: I have an example. We were working with HGTV, the home and garden television network. They were working on a prototype search engine that would let you find projects that you could work on.
Their thought was that based on some of the customers they talked to, that people are concerned when they own pets whether the project itself is pet friendly. Like if the dog gets into the construction materials, will bad things happen?
That turned out to be it didn't matter what type of dog they had, but the fact that the owner had a pet actually did make a difference to the design. They wanted to be able to have pet-friendly search criteria. Show me pet-friendly projects.
Adam: There you go.
Aaron: Yeah, that's a perfect example.
Jared: Yeah. You know, guys, I had so much fun talking to you about this stuff. I am so excited that you guys are going to talk for a full day about this at the User Interface conference. This was one of the most well received sessions from last year's conference. We're very excited that you guys could come back and talk about it.
It's called Building Consensus Critiques and Design Studio. It's a full day. It's going to be October 23rd, in Boston. If people are interested they can find out more about it at uiconf.com. It' fun. Right, because you guys into design studios. You get into critique. Everybody walks away being way smarter about the process and design.
Adam: There will be a lot of hands on activities.
Adam: People will come away with lots of actual experience working through this and thinking through it.
Jared: That is so cool. People loved it last year. It was so much fun. I remember going into the room and there was so much energy. It was just fabulous.
Adam: I promise to try to have a voice this year.
Jared: Oh that's right. You'd lost your voice.
Jared: You were able to critique without a voice. That is magical.
Adam: All hand gestures.
Jared: It was an interpretive dance.
Jared: [laughs] It was fabulous. OK guys. Thank you so much. Adam, Aaron, thank you so much for spending time with us today. You're so smart.
Adam: Thanks for having us.
Aaron: Thank you for having us.
Jared: I want to thank our audience who was smart enough to listen. We love talking to you guys. If you like our pod cast and you are on the Twitters. It would be awesome if you'd just posted a little tweet and let us know. We love hearing from you. So please do that. You can just tweet to UIE and we'll forward it on to everybody else. You guys are on the Twitter, right?
Adam: Yep, I can be found @adamconnor.
Aaron: Mine's just Aaroni.
Jared: Cool. You can follow them there. And of course catch them at our conference and see them on the web. You have a web site, right? Discussing...?
Adam: Discussing Design. Right, where we host a lot of expansion on the ideas that we cover in our talks around critique and how to integrate it and use it to build collaboration.
Jared: OK. Check that out. It's a fabulous, fabulous blog article, posting, thing.
Jared: In a future podcast, we'll critique my description of your blog.
Jared: Thanks guys.
Aaron: Thank you.
Adam: Thanks a lot.
Jared: And for our audience thank you for encouraging our behavior. We'll talk you again at a future time. Take care.