Defining “Exit”

Radios, checkboxes, and the nuances of copy: the building of another Customer.io feature

When customers build messaging campaigns in Customer.io, they have fine-grained control over when people enter those campaigns. But until now, we didn’t offer the same control for when people should leave a campaign, and stop receiving messages. We had one default behaviour: to remove users when they stop matching the campaign trigger.

For me as UX designer, this was an interesting look into how our customers understand a key feature of ours, and how their understanding shapes their expectations of our app. It was also an opportunity for our team to make our app more transparent, surfacing default behaviours to enhance understanding, and allowing our customers to make changes to those behaviours without making assumptions.

Defining

When setting up a Customer.io campaign, you define a few things:

  • a trigger condition, checked at the start: something like entering a segment or performing an event
  • an optional filter condition, checked when it comes time to send a message (an “only message people if”).

Then, you build that campaign’s workflow and messages.

But what about when it comes time to leave a campaign? When do people leave it? When should they? We knew from our support requests that folks had one of three expectations, thinking that end users would exit:

  1. When the campaign finished, and they got all of its messages
  2. When they converted (that is, met a customer-defined conversion goal)
  3. When they stopped matching the trigger and filter conditions

The third option is how Customer.io worked by default, but our app didn’t tell customers that. As a result, there was trouble when their expectation was (1) or (2). We had folks writing in asking questions like, “Why are my users not receiving these messages? They should never have left the campaign!”

So the problem was two-fold: the default exit behaviour wasn’t clear, and couldn’t be changed.

Scoping, sketching, wireframing

What we wanted to do with a first version of this feature (henceforth referred to as “exit conditions”) was make the app’s default more visible to our customers, and allow them to make changes to it as part of setting up their campaign. In the process, if we could make the campaign workflow clearer, all the better!

So I started sketching, looking at how we could present a campaign trigger, filter, and exit during the campaign creation process. We wanted to present the right interface to define the exit conditions, but also present it in the right place within the context of the campaign (and afterwards). Here’s some of that:

Iterations on what it means to leave a campaign. Thanks to Jóhann for the right-most effort!

I posted various sketches to Basecamp for the team to give feedback on, went back, drew some more, and in the end, this is what I landed on for first iteration, below.

Greatly simplified, just “Start” and “Leave”

The biggest change here was that filters became part of exit conditions. We also changed the default behaviour (to “People will move all the way through the campaign unless you decide otherwise.”) to better line up with folks’ mental model.

We went back and forth on filters as part of exit conditions, rather than as their own component. One of my comments in the feedback thread was the following:

“I can still be convinced on the filters as their own component, but I’m still very much leaning towards keeping them in the Exit box.”

I thought this made sense. If the users don’t meet this rule when it’s time to send them a message, they leave the campaign. It’s just a specific exit condition! This simplified the interface and made it a bit easier to understand (or so I thought).

Making a mistake (oops!), and moving forward

At the end of this first round, we thought some more on what the ‘Exit’ input should be— radio boxes, checkboxes, dropdowns (more on that later). But when we arrived at a good place, I built out some wireframes.

We grouped the conversion goal and exit condition together, because having them on the same screen made sense especially if meeting that goal is a condition for leaving the campaign.

“Start,” and “Goal & Exit” were now all together, grouping both a campaign goal and exit together.

There were a number of small phrasing and interface nuances here that imply big changes, including the fact that filters are now part of exit, and the default behaviour changed, specifically:

“A user will leave the campaign before the end if…”

This was a mistake. But an instructive one!

When we posted the wireframes and others for broader feedback from the team, we realised that Customer.io’s trigger and filter functionality is much more complicated than expected, and filters cannot be coupled together with exit. They work in a much more subtle way, controlling the timing of messages in a way that I hadn’t understood (and I wasn’t the only one!). I needed to make the mistake in order to understand the nuance, and I’m grateful to the folks in the team who pointed it out.

So it was a useful mistake. It leveled up my product knowledge and, as a result, I rewound the UX a little bit, to one of the earlier sketched iterations.

We reposted again, and I reworked and iterated on the wireframes with the team, with more frequent feedback from a few outside folks. Soon enough, we arrived at something we were happy with. We also decided against changing the default behaviour. It was ambitious but unnecessary at this stage.

Prototyping

Once we settled on the above, we started prototyping. The first part of this went quickly. We used our front-end framework to get the above wireframes onto staging, so we could test and see if the feature worked as expected, and get our support team’s eyes on it.

The feedback quickly became detailed as we moved towards a first release, and I want to delve into those details a little bit more.

The last 5%

As prototyping wrapped up, this is what we had for Goal & Exit:

This ‘Exit the campaign’ interface, while it seems a little too complicated on its surface, took a lot of tinkering to figure out, especially when it came to input, errors, and fallbacks.

Radio vs. Checkbox vs. Dropdown

We wanted a user to be able to select if a person leaves their campaign:

  • when they complete it
  • when they meet a conversion goal
  • when they stop matching trigger and filter conditions

And we went through quite a few iterations for this, each of which had their issues:

  1. Dropdown: Users can only select one. What if users wanted people to exit when they stopped matching OR met a conversion goal?
  2. Two checkboxes: This was fine, but predicated on changing the default behaviour. If we kept the default, a user would have to read the instruction, understand the new default, and then leave this empty, which made for an interaction with high cognitive load. Not ideal.
  3. Option cards: If ‘convert’ is selected, the default behaviour is unclear. Will they also leave once they stop matching? Or will they continue? We could select multiples here, too, but the UI didn’t make that clear.
  4. Four radios: A list of the four options actually available. This created issues with fallbacks: What if a user selects “either stop matching or convert” or even “convert”, but changes their Goal settings, what do we fall back to?

We got some great feedback on the copy and options here from Brandi on our support team. The below interface best fit the goals of visibility and control:

The current implementation, with the default selected.

The default is chosen and clear (remember, it wasn’t visible before), and the user can change it. If they try to choose “exit on conversion” without setting a goal, they see an error; we don’t make an assumption and fall back anywhere.

The importance of copy

Brandi also gave us some great copy feedback which aligns to her constant conversations with customers and how they talk about people triggering and leaving campaigns.

A few examples of the copy we figured out here:

“it’s time to send them a message and they don’t match your triggers and filters” The ‘it’s time to send them a message’ here is critical, because it indicates to our customers when the condition is evaluated— when it’s time to send a user a message. People proceed through other actions in the Customer.io workflow, but won’t get messages.

“Meet the conversion goal” is used instead of “converted” because there can be subtle differences in what it means to be marked as converted in Customer.io and meeting a conversion condition. Again, we wanted to be explicit about when people would leave, and not interchange the two.

“People exit when any of the following are true” is used to show that either or both of the following options can be selected. This indicates that the options below are connected by an or operator, not an and.

We went through several rounds of this for that last 5%, and when we finally shipped, were really happy about the feature’s clarity. It might seem like overkill at times, but small differences in Customer.io can have huge consequences— and we want to be as clear as possible, sometimes at the expense of initial simplicity.

Next up!

We released the feature on July 3rd. Going forward, we want to track how folks interact with this feature, both in the app and in their questions to support. Do they keep the default behaviour? Do they change it— and if so, to what?

Furthermore, I’m going to be writing up some documentation and giving an internal lunch and learn on how filters work, because our app is Complicated, but awesome.


Hope you enjoyed this post! It’s the second of an ongoing series of UX and product design process posts at Customer.io. The first was on the building of our Journeys feature.

Join our mailing list for updates on our content and more!