Inside Quizlet's incredible feedback center

Inside Quizlet's incredible feedback center(quizlet.com)

49 points by asuth 12 years ago | 11 comments

conorh 12 years ago |

I love these tools, can't wait until my next chance to build another one. I've seen several of these at various startups that have been built internally with similar features (although not with this much polish!) I built the first version of one at StreetEasy to handle a large volume of support queries - mostly driven by the fact that it was us developers answering all the support queries at that point. As pointed out in this post much of the benefit comes from tightly tying in with internal tools - billing, logging, logins, site visits etc.

timdorr 12 years ago |

This looks good enough to be a product on it's own. It's got a crazy amount of polish.

I know it's highly-coupled to your particular infrastructure, but would it be possible to open source?

asuth 12 years ago | |

Thanks a lot! The polish came from a hack night we did to redesign it.

I don't think there's much to open source -- it's about 2,000 lines of code. It's mostly just writing custom sql and elasticsearch queries to show data related to the users we're about to respond to.

Re: the log integration, all our application logs get written to logstash/elasticsearch/kibana, and one of the fields on the logs is the person_id (they're unique id whether logged in or out). When we show the logs in the feedback center, we just query elastic search and group by the different log types.

silverlight 12 years ago | |

When I saw this, that was my first thought as well. Wow, if I had this, it would be so easy to answer a lot of our technical questions especially -- they're absolutely right, those typically require a ton of back-and-forth to dig into the issue, often resulting in us asking the end-user to provide a console log/dump.

purephase 12 years ago |

Very nice. I like the idea of tying logs to users like that.

Seems like there would be a lot of overhead for that amount of logging though. Could you elaborate a bit more on how it all works? Do you run an internally developed mixpanel/GA type tool? Or, is this server side logging (the JS issue speaks otherwise).

mirashii 12 years ago | |

Quizlet employee here.

For the JS logging you see in some of the screenshots, we use some JS on the client side that watches for errors and fires off a request to us with as much data as on the error as we can collect (in most cases we can get stack traces, which is pretty nice).

In terms of where this all gets stored and aggregated, all of our application level logs are shipped into Elasticsearch, and that's where the aggregations and counts you see in the screenshots come from. All of the application level logs are sent in with a unique identifier per-client so that we can keep track of them that way, as well as tagged by username, IP address, and anything else that might be useful in cross-correlating with logs from the rest of our stack.

If there's any more specific details you're curious about, I'd be happy to elaborate.

purephase 12 years ago | | |

This is excellent, thanks for sharing.

What do you use to ship the logs into ES? I'm not familiar with the technologies you're using, but is this in-sourced developed tools? Or, are there OS options for log aggregating in ES that you would recommend?

philfreo 12 years ago | | |

You should write a blog post about your usage of elasticsearch/kibana/logstash

curiouscat321 12 years ago |

It's shocking to see something so heavily polished that's accessed by so few people. Well done!

Maybe not the best place to ask, but are you guys hiring summer interns?

mirashii 12 years ago | |

We're certainly hiring full time, and may be looking for interns more as summer approaches. Feel free to send a github/resume/etc. to jobs at our .com and mention the post.

lpolovets 12 years ago |

Lots of great nuggets here. Connecting logging data to customer feedback seems especially worthwhile.

There was one line that really surprised me: "We personally respond to almost every one of [the messages], without using response templates." Is that a feature? I think you can sound fairly personal even while using templates, and they could save a lot of time (30 seconds saved per message * 20k messages per month = 1 full time employee).