Rian: But switch interviews for us are based on the framework, and particularly there’s a model called the product forces, which, are you familiar with that?
Rian: Okay, let me find a quick link here. Oh. Funny, my blog comes up when I search for it. Nice. So, I’m going to do the total douche move and send you a link to my own site. Apologies.
Matt: No, it’s good.
Rian: But I’m going to blame Google. So that diagram there is called a product forces diagram. It basically says when someone changes from existing behavior to new behavior, there’s a couple of progress-making forces that works in your favor, and then there’s progress hindering forces that work against you. So progress making forces, push the situation, so they’re unhappy with something in their current product, and the pull of the new idea, they look at you, and they go, oh that looks nice. But then the hindering forces are, there’s allegiance to the current behavior. Like oh there’s this thing, I know how this works, I don’t know if I should change right now. And their anxiety of the new solution, of saying what will I miss? What will I lose when I go to you.
And switch interviews is about interviewing people who have made the jump from some existing behavior to some near behavior and filling out those forces. Understanding what all four of those forces were. In our case, there’s two things that are really important. 1 is people who recently switched to Postmark, because obviously for them the progress-making forces were stronger.But then also people who have switched away from us, to try to understand what was the compelling thing about our competitors that came up.
In our case, what we found was, you’re exactly right, there’s an allegiance to the current behavior, because people don’t understand, the pull of the new idea isn’t strong enough. They don’t switch until there’s a stronger push of the situation they’re in, which is that they realize there’s a deliverability issue. Because what we found is, and this is where it goes into marketing, is we say on our site best deliverability, all that stuff, but then when I talk to people they tell me, well we just assume deliverability is the same everywhere. Because everyone says deliverability is good.One guy even said they looked at our status page, which showed 100% deliverability, and he’s like that can’t be right.
So, these guys are full of it. I’m not going to believe them. This is so weird, I’m going to go with the company that promises 97% deliverability, because at least they’re being honest about it. It’s the weirdest thing, and we made some changes on our status page to be more clear about where does this data come from, can you really trust it, that kind of thing. We learned a great deal from this, and the main thing is that unfortunately, postmark is an excellent 2nd choice, but it’s, right now, not a good enough first choice. And the issue for people around not understanding really how deliverability works and that they’re going to have issues with deliverability, and also not understanding why we are a little bit more expensive than other providers, and because of that, because I’ve sent so much time on deliverability.
The switch interviews, they’re on going, they’re part of this larger jobs to be done, framework that we try to follow, but this diagram is central to my understanding of our product, and how to get people to switch to us, or to prevent them from switching away from us. We obviously need to make sure that we don’t push them away, and that they have strong allegiance to our product.
Matt: When one of your main differentiators is deliverability, how do you defend against competitors who either misquote or misrepresent their deliverability. Let’s say yours is showing 98% and it’s honest and accurate and really good, what if they show higher? Whether or not it’s the real case.
We’ve talked about that a lot. We check deliverability on our competitors as well, so we know what’s going on. I mean they’re smart put out a web paper, where they just conveniently left us off their comparison, and just compared them with people who were worse than they were. Which is infuriating, but that’s the way it goes, this isn’t a charity, so I get that. But what we try to do, is to tell people set up, or try, of if they were somewhere else, try us with something small. Try us for your password resetting emails, just switch one email over, and compare it, and then see what happens.
I would say 90% of cases when we talk to switchers, the answer is always there was some kind of catastrophe. Suddenly, something went down, support went nuts, and they couldn’t do anything. Support was non-existent, whatever, so there’s always this thing that happens and then they frantically look around, and like oh, okay, maybe this. Our new homepage has been super helpful, we get a lot of comments about, okay, seeing that, those inbox metric on the homepage is useful, because a lot of people when they’re in the mindset of switching, that is what they’re looking for. My email is always delayed by 30 minutes, oh thee guys deliver in 10 seconds or less, okay, that’s good.
Rian: Yeah. Were you at Micro-conf.
Matt: I wasn’t, no, I missed that.
Rian: When I was there, Natalie, that was back when the old set was live, and Natalie was talking about, in her talk, about the time to inbox metric, and I didn’t quite get it. It hadn’t been a problem for us, and it wasn’t pervasive through all your marketing yet, so the rallying around that metric didn’t quite make sense to me, but when the new site rolled out, it makes perfect sense, and it fits in so well with everything else that you guys report on and that you promise.
Rian: When people switch away from Postmark, I have a pretty good idea of why they might do it, my gut would be that they’re, even though it might perform more looking to consolidate marketing and transaction emails, normally the are the reasons they switch away.
Matt: That’s absolutely right, did you do these interviews for us?
Rian: I’m telling you, is exactly the same-
Matt: That’s crazy.
Rian: We’ll see people where even though more performant they’ll switch to another app, because they have everything in one place and there’s one less log-in, which is a pretty good indicator that they’re not a volume where performance really matters, but it’s still frustrating for us to see people switch, and in our case it’s tied directly to revenue. So, start recovering less revenue to have one less log-in.
Matt: Yeah, in our case too, it’s frustrating because it’s worse for them to be on the single provider, because, like we even tell them, okay, fine you’re leaving, that’s fine, but please do us this favor, and still send your transaction email from a different sublimate, because you need to send transactional and marketing from different domains, otherwise they’re going to share the same reputation, and you don’t want that, because your marketing emails aren’t going to have the same engagement. That’s one of the reasons, we are thinking about how can we do bulk in our own way, which would definitely mean different domain, different IPs, so that we don’t mess with what we have. Because it’s such a point even if we tell people it’s actually a good thing from a a deliverability perspective to split it up, but they still want it under the same account.
Rian: And beyond marketing and transactional being combined, seems like one of the other risk areas is that other postmark competitors like Singrid really push larger sender to be on a dedicated IP, which comes with its own negatives, right?
Matt: It does, and that’s why we have such a strong opinion on the value of shared IPs on our platform and why we have a manual approval process for everyone. We were like, no, we want to know your neighbors. I like this phrase, we want to make sure you have good neighbors. I think that’s somewhere in our blog post somewhere, and that’s what we do. We police the community in a way that’s okay, we want to have good senders, because then everyone helps everyone else, and if one of your neighbors gets attacked or spam attack or whatever that’s out of their control, the volume is actually such a drop in the bucket against everyone else that it doesn’t hurt everyone else, and at the same time they then benefit from everyone else’s good behavior around them.
It’s kind of like buying insurance, it’s like we’re all in this together, and we help each other out when needed, whereas we do have dedicated IPS for extremely large senders because there’s some benefits to that if you get to a certain volume, but for the most part, and the other thing that’s frustrating is they charge extra for that even though it doesn’t always guarantee you better deliverability. We just had a switcher that came over from a provider that kept switching them from IP to IP, until they finally said forget it, we’re just going to try this out, and now they’re really happy.
Rian: So, here’s another parallel, it’s not just Singrid offering that, if you search google, that dedicated IP, you can see thousands of people wondering where they can get a dedicated IP, so there’s demand for it. People want to send emails to customers telling them their cards are going to expire, even though Visa and other providers now offer card updater, which fixes most of those problems in the background, so we know that if you’re sending emails saying your cards going to expire, that 7 out of 10 people, maybe more now, won’t actually have a card issue.
Rian: But people still set that up on their own, and they still want to see providers offer it. How do you educt and industry to know that dedicated IPS aren’t good, because I think that’s key to someone considering Postmark as a first choice rather than a second.
Rian: How do you get in front of them before they-
Matt: Chris has written a couple of blog posts about this that tend to rank fairly high, and we get quite a lot of leads from that. But then they have to search not, I want a dedicated IP, they need to search for should I get a dedicated IP. Then they’ll land on Chris’s post, and one of them, I think he’s written 2 over the years, and that’s been really useful, but it’s also slipped into our marketing too. I think, if I remember quickly, I should know this, right? But it’s on one of our, if you go to our wire pages the features there, I think at some point there’s going to be … yeah. Right on the first page of why, there’s a section that says looking for dedicated IPS? We think there’s a better way. And we explain right there on the first page of features why they don’t need it.
Rian: So, transitioning a bit. How do you, I imagine that in your role these switch interviews and jobs to be done framework are pretty core to everything that you do, how do you involved the rest of the team in that?
Matt: That’s a fascinating story. We’re running quite an experiment right now, and I need to talk about it in a development process to answer your question, so I promise I’ll get back to it. On our retreat this year in April we decided to do an experiment where we were working 6-week cycles followed by 2-week off cycles. The idea is that we create these small teams that work on a feature and release it within 6 weeks. If it’s a bigger thing it can get split up or whatever, but that team made up of a developer, and a designer, and me and sometimes just the backend developers, are responsible for the timeline and for making sure it ships. And then ideally, although we’re still trying to get there, we then have 2 weeks to do whatever, fix bugs, they can work on side projects, they can do whatever the hell they want. Because we wanted to bring back this idea of just hack the product and see what comes out of it.
Ano that 2 week off cycle is coming back to your question, also is our planning cycle, and how our planning cycle works is we had an area in base camp called the idea zone, because we’re terrible at naming things, I don’t know. Where anyone can post stuff, it usually follows, there’s no real format, but usually we try to follow a format of what is it, why should we do it, and why should we do it now? So you kind of answer those three questions, and then during that 2, like I monitor this, and we comment and have some discussions, but during that two weeks, we have a dedicated time where we go through it in detail, and I ask everyone to give their comments, vote on stuff, say I really want this, this is really important, and then so we take that input along with what I know form our interviews, from talking to costumers, as well as what Chris and Natalie are doing form the marketing and strategy perspective, and we come up with an initial set of, okay these are the 5 or 6 things we want to do in these 6 weeks, what do you think team?
Then there’s a little more discussion around that, and then we lock you done. We take that 2 weeks … you can submit and idea at any time, but that 2 weeks is really a, okay what’s most important for us to work on right now? Let’s consolidate down to things that everyone wants to work on, then we split into teams, into what makes sense for front-need developers, and back-end developers to work on what, and off we go.
There’s a lot of input from, all throughout, from customer success and from developers and from our interviews, and from me and form Chris and Natalie, and then that 2 weeks is really a dedicated time for everyone to get together and give their input on what they think we should work on. What I like about that is actually driven by customers, because my initial thought was, oh man that’s very inward focused, but I realized, that no it’s actually not, because so many of us are so steeped in customer needs, because that’s what we do every day, even developers spend a lot of time in support talking to customers. All those ideas and votes are representative of our customers because we spend so much time with them. A customer outreach isn’t something that’s just one person, it’s from all the way across the organization.
So, that’s kind of a long answer, I’m not sure if that is what you need or makes sense.
Rian: Yeah. I’m curious, so Basecamp’s another company that has these on and off cycles, and they last what seems to me to be pretty long. We’ve struggled with, in the past, being able to keep our shipping frequency up while managing long-term, big projects that are high value, and short-term easier projects that are lower value, and the big projects are always more exciting for us for customers, and they usually are what we focus on but it leads us to go into a development whole for over a month, and then our frequency talking with customers, shipping and features, slows to a crawl. How do you manage that? Are people’s cycles staggered so that they’re 2-week bug and small improvement times are staggered, or is everyone in the same 6 week batch working big ideas at the same time.
Matt: the same, but there’s 2 things I would to say to that, because I’m concerned about that as well. What about long-term strategy, how do we make sure we’re not just looking 6 weeks ahead, or 8 weeks ahead and not thinking about the future there’s a couple things I would say to that. 1 is we try to make experimentation for new ideas part of that 6 weeks, so let’s say for example we take on something big. What we would do is, we wouldn’t say, okay let’s send bulk email within 6 weeks. We would say, okay, one developer is going to spend time building out a concept or looking and understanding what this would actually look like if we wanted to do it on different IPS, things like that.
Almost like an experimentation, not an MVP, because it’s not being released to customers, but it’s an experiment that we ran in that 6 weeks internally, and then at the end of that 6 weeks, they what would it take for this to become live. The second part to that is then, if it’s a really long project, like oh many this is going to take us 4 months or whatever, how can we break that up into smaller 6 week cycles, still with that 2 weeks in the middle, because we feel that’s really important to fix bus, and let that release … There’s always little follow-up after release, no matter how good it is. So, we don’t want to just string the 6 week’s releases together.
In that case we would say let’s work on these in 6 week chunks, and try to release something every 6 weeks, even if it’s not released front-end to customers, but at least release something to postreduction to make it less daunting as we go. Then we see progress.
Rian: I see that you guys recently launched these migration guides for people switching form other providers. Are those largely based off the interviews that you conducted?
Matt: They area, no, they’re based on , so Patrick who wrote them spent a ridiculous amount of time doing it. Like he made the migration, and he read the documents, and all that stuff, and he just followed this same format for everything. Like what’s different in the API, how do I do this in Postmark, it was just a slog and long hard work to just go through all their documents and ours and make the comparisons. If the question is what’s the reason for doing it, yes. Then one of the big reasons for doing it is, I don’t want to switch right now until I have an issue, switching is too hard, so that takes away some of the … going back to that forced stagger, takes away from of the new solution. If you can show them, oh this is actually quite doable.
Rian: Going back to the cycles that you were speaking about a second ago. Does that apply beyond just the product team is this an example of a 6 week project, built migration guide for SES?
Matt: This wasn’t, but what’s interesting about that is I would say it’s not exclusive to product team, but it is exclusive to product work. As an example, Garrett is … The marketing designer is currently going through some on-boarding changes that’s totally going to touch the app. Even though they’re in marketing and usually not part of our cycles, we’re pulling them in, into these iterations, and saying, okay because you’re working in the app, you’re going to need a back-end developer. Let’s make them part of our 6 week iterations for as long as it needs, and then they go back to just doing their thing. They try to … I think they plan more weekly or bi-weekly, and just kept chugging along at things, but whenever something touches the product and needs QA and needs developers, then they just get sucked into our iterations until that’s done.
Rian: Have you noticed a difference with customers coming in through this new website that has the new migration guides and a lot more information about what sets you apart through the past few months?
Matt: The migration guides are fairly new, and we’re really trying to get feedback on those, so there’s a link there for feedback, and we’re trying to flag people who come from other places, but it’s been really new so not much yet. The website itself, though, definitely we do get … We have received unprompted comments about this is exactly what we needed to know before choosing you. Particularly the TTI stuff, the time to inbox stuff on the homepage is just … has been surprisingly effective to help people make that decision.
Rian: I think what it does great is it teaches people about what you do, and it build trust, which is something that’s been difficult for us in the past. I know we’ve kept a lot hidden, because we didn’t want to share it with competition, but the side effect of that is a less-informed customer, and a less trusting customer.
Matt: Mm-hmm (affirmative)
Rian: One of the best things we did was a blog post where we broke down exactly what sets our campaigns apart and how to build them yourself, and we saw a big difference, too with the people who would book a sales call after reading that, or they trusted us, they were ready to hand off this problem for us to solve, whereas when people hadn’t gone in and seen the details, we had to earn that trust on the call.
Matt: Yeah. The big issue is that you can’t solve a problem for someone that they don’t think they have. That was an interesting realization for us is that it doesn’t matter that we tell people we’re fast to the inbox if they think everyone’s fast to the inbox. Because then why, that’s not a differentiator. We need to find a way to show people, we do live deliverability data, does anyone else do that? So we need to find a way to make that contrast clearer to say, not to tell people there’s a problem that you don’t know you have, and you should think about this problem, but that’s basically what we’re doing. We have to first make people aware of the problem and then tell them how we solve it.
Rian: I’m curious with the … you guys recently released an update activity page.
Rian: That was the one are … we use Postmark for emails, and that was the one area that we used really heavily, and that I could not stand whenever I went in to look at it, but I just accepted it-
Matt: You’re not the only one.
Rian: As the way it was, and I know everyone else on our team, who also uses postmark, had the same reaction when it came out. Every single pice of it that we looked at, it was like this is exactly what we needed.
Rian: Is that the result of switch interviews, or just develoPing a much better understanding of how people use the product?
Matt: It had a lot phases, the first was understanding what the problems were, and we tagged these things in help scout, and everywhere so that when we went in, and said okay we want to solve this problem, it was easy for us me to go consolidate all that stuff, and say, okay this is the problem we’re trying to solve. First of all, we had 2 main issues, and I mentioned this in the blog post as well, which is cool, because this is really how we solved the problem. 2 main issues were wasn’t scannable enough, so it was really hard to find what you were looking for, and second it was … that has to do with the information design and the clutter and stuff, but then secondly, the filtering was very limited, and the main thing people want tot do in activity is find a specific email, that’s 905 of the use case. This email address or this bounce type happened and I need to find that quickly.
That’s what we ended up setting up doing was first solve a lot of the visual clutter and understandability issues, and then secondly try to make the filters better. This was, in transparency, a really hard project because it took longer than we expected, it took two iterations, because there were some things that we couldn’t tod in the date that we really wanted to do, so it’s not really where we wanted it to be. I could tell you, where we really want it to be is to only show the current status of the email on that main activity page. Right now you see an event stream, you see it was processed, and you see it was delivered so you see from most emails to events. What we would like to do is show a table with all your messages and just show you the most recent event. Then, if you click into the message it can show you everything that happened on that message.
But because of the way our data is structured, we can’t do this right now, if you’re going to write this up, maybe don’t put this in there. This part probably shouldn’t make it into whatever you write up about this. That’s some work that we have to do in search to create a new index that we’re able to do that. But we really wanted to push through and say, no, we know we can solve this, and this is how we solved it for now. So, it was first that, what are the problems we’re trying to solve. But the we also did usability testing on it. We created prototypes, and I did interviews with people on Zoom where we share our screen, and they click through it, and we made changes based on how they went through it and got feedback on the general idea before we went into development.
Rian: For a page like this, it looks simple at first glance, but when you dig into filters, date ranges, different bounce types, how do you work across the team to work quickly on something like this when so many different concerns are involved. There’s certain things that may be great form a UI perspective, but forma data perspective, they’ll take 3 months to pull off.
Matt: Yeah, that’s hard, we’re remote, too, which makes it even harder. How we try to do it is , like I said, work in small teams so in this case, and we have these concentric circles. So the core team would be me and the designer and the developer, and we would chat daily, not in person but in Vision and in a dedicate room, and we would ask questions, can the data do this? And then would say yes, but it’s rally expensive, it’ll take 6 months, and then we go back< but we try to actually not put too many guardrails on it. We try to create an ideal solution, because the developers don’t, also don’t want to just say no, they just want a little time to figure it out.
So they might look at something and go, man this is going to be expensive, but I don’t want to say no right away, can you give me a day or two to just think about this, because maybe I can come back with something that is not as expensive, but almost gets us there. Form there … We kind of just feel our way through this, at certain points, we throw that open a little larger and have a meeting with Chris, the CTO, and the lead developer and Garrett, and say here’s the direction we’re thinking, do you have feedback? We do that on a call, then we go back to working in our small team. We try to keep it really small for iterations, and then come up for air and get feedback form a larger team. We really have to trust each other on that, and there were some issues on this one for sure, where we thought we could do something and didn’t realize until later that we couldn’t do it, which is why our estimate was so off.
But we learned from that too, and we won’t make that mistake again. Right now, we’re working on redesigning the DNA settings page, where you enter your and all that. We’ve made changes to the way we work, so this time, we’re actually doing 2 releases: we’re doing a back-end release today or tomorrow, and then the front-end release will follow next week, and the developer and designer are much more in-line with what’s possible and what’s not, so we try to learn our way through the mistakes, too.
Rian: As product manager for any of these features, what’s your role look like? Are you involved initially and then you switch to more of a review position as the work gets done? Or are you really involved throughout?
Matt: I’m really involved throughout, so I do the initial problemitization, and understanding of the problem we solve. I still believe in specs, so not huge, but a short spec that includes, what we call a job story, which is different from the … their normal, agile stories, so it’s based on the jobs to be done. Let me actually share with you if aI can find it really quick, just as an example our job story for the activity thing. So our job story for that is when issues come up related to specific transactional emails I sent or receive, I want to find information and events about effected emails quickly, so I can act on and resolve and problems and get on with running my business.
That is the one sentence that drives every features, and then I go through and do a short, what’s called a product opportunity assessment, so I answer questions like what problem will this solve, for who are we solving a problem? How will we measure success? What alternatives are already out there? What factors are critical to success, things like that. Then we start working with the team on what the solution will look like, and I typically, I’m involved through all of that, providing guidance on the design, since I come from a design background. Then I switch more to a QA feedback trade-off person. Next we do this or this, because there’s often like, this is expensive, should we do this or not? Later on I flow more into an execution mode on that side.
Rian: Are the jobs stories then bubbled up to sit under a type of customer? It seems like a lot of the job stories would end with the same ending to go on about their day and be a better business person. Do you have an understanding of the types of people who use Postmark, because I’m sure at your scale there’s totally different use-cases that you have to balance out even if zoomed in on a job story, it sounds good.
Matt: We have personas that we create, and then actually use, I know a lot of people don’t use personas, particularly we use it of saying this is who we’re not targeting, and this is who we are targeting, but in this case … The second part of the sentence doesn’t always look the same, I think there are … For the DNS settings page, for example, it’s more around understanding why should I add these things to my DNS, and what will I gain from it. It’s a little bit different every time, but we have a good understanding of the type of person we’re going after, which is mostly on the development side.
Rian: Has focusing on developers been a successful content approach for you guys? Is that really who your trying to speak to with the website?
Matt: Yeah it’s helped immensely, particularly, you’ll notice small things on the marketing page. I think at some point we say, we’re built for developers, not marketers, something like that, unless we chickened out and changed that at the last minute. But I think it’s still there somewhere, built for, oh I think it’s in the API section. I think it’s still somewhere, I’m not exactly sure where it is now, but yeah. And then our newsletters for example, which I write now, is very much focused on what would a developer want to know about email and about what’s going on in the world of email, not related to 10 hacks to get more people to open your marketing email, but here’s … like we would and things like that. Things that a developer should be interested in in helping them be better. That comes back to mission, which is to make tools to make developers be better. Anything we can do on Postmark that helps with that even if, we don’t really get anything out of me writing this newsletter, but that’s part of our mission, so that’s what we do.
Rian: Yeah. I think that’s a great place to end it.