6x speed improvement for building segments

Outside my window is a construction site. There are steel beams going in to concrete pilings. When you construct a building, you have one shot to get the foundation right.

Not so with an internet product.

I’m pleased to announce that we’ve improved the foundation of Customer.io, and segments are now processed at least 6X faster.

The promise of how we process your data

One of the things people say they love about Customer.io is that you can throw your [mega / giga / tera]bytes of data at us and we’ll do the heavy lifting.

You define a segment, and we’ll process all your historical data and incoming data in real-time.

Segment processing is a core part of what we do for you.

Recently, segment processing has been taking too long

Our original technology choice scaled poorly. It wasn’t parallel, so we could only rebuild one segment at a time, and the processing of a single segment was also too slow. Many of you expressed frustration and we were frustrated too.

John spent a lot of time over the past few months evaluating technologies to help with the problem. We’ve made some headway on performance that I think you’re going to love.

At least a 6x improvement today — more to come

If you have data for a million people in Customer.io, we can now process it 6 times as fast. That’s what a benchmark we did yesterday suggests. We cut down processing time from 3 hrs to 30 minutes on a large segment.

Most often you’ll see a 100x improvement

This will be even faster in most cases since we’re now processing segments in parallel. You’re no longer blocked by people in front of you. We’ll also show you what % complete it is.

The technology and design we’ve chosen (Go + Riak) will allow us to scale this service well. As we grow, we can add more machines to keep performance high and make sure your requests are dealt with quickly.

What do you need to know?

You don’t need to change anything in your current segments.

If you’re creating a new segment or updating an existing segment, give us feedback. These changes are well tested, but there’s no substitute for the real world.

Thanks for trusting us with your business. Please let me know what else we can do to help,

Sincerely,
Colin, John, Asha