Friday, November 6, 2009

Dan Siroker: CarrotSticks, Google Chrome, and Obama

Dan Siroker is the founder of kid’s learning games site CarrotSticks.com. Before that, he was involved with the Obama campaign transition as deputy new media director; prior to that, Dan was a product manager for Google Chrome.

This email interview was made possible with the friendly help of the Search Engine Strategies Chicago conference (December 7-11 this year), where Dan Siroker – as well as Jeff Jarvis, author of What Would Google Do – are keynote speakers. [Thanks Byron!]

Can you explain your role at CarrotSticks.com?

CarrotSticks is an online multiplayer game where kids practice math through real-time social competition with their peers. Our mission at CarrotSticks is to provide parents with solutions to make learning a more exciting and interactive experience for their children. Research shows that peers have a tremendous influence on a child’s attitudes towards learning. We want to harness this influence in a positive way by using technology to make learning social. My role at CarrotSticks has two aspects to it: customer development and product development. Through customer development we listen to our customers and potential customers and identify their pain points and what we can build to help solve them. Once we have a clear picture of the problem we are trying to solve, we develop the product to help solve it. Our goal is to make something people want and the only way we know how to do that is to listen to a lot of parents and teachers.

You probably know the market of learning games, what was the thing you wanted to do differently with CarrotSticks – what was the need you saw?

We built CarrotSticks after meeting with dozens of parents and hearing two pain points over and over again: my child’s school isn’t rigorous enough when it comes to math, and my child doesn’t enjoy supplemental math programs. Given these two pain points our goal was to build a rigorous supplemental math program that is so fun kids love doing it. CarrotSticks is a rigorous way for kids to practice math because we’ve built a step-by-step interface that gives immediate feedback at every step of the problem when the child answers correctly or incorrectly. If they get it wrong, they get a helpful hint. If they get it right, they are rewarded with carrots. What makes CarrotSticks so fun is that we allow kids to compete with other kids in real-time competitions. This transformed ordinary math problems into a fun game in which the objective is to get as many carrots as you can in 30 seconds.

Do you do usability testing with kids? Could you give us some examples of revealing findings you stumbled upon during testing?

We’ve done several classroom trials in which we watch 20-30 kids at a time playing CarrotSticks. Through this we’ve learned a lot about the way young kids use our product. I’ll describe two findings we stumbled upon through this testing.

The first finding was that kids don’t navigate. The very first version of CarrotSticks we showed kids had a lot of navigational elements to it. We had separate tabs for the leaderboard, saving your account, etc. Turns out kids don’t click on any of this. They just play what’s directly in front of them. This forced us to re-design CarrotSticks to put everything on the front page. Now the leaderboard shows up on the right-hand side of the screen as you practice and you get automatically prompted to save your account after you complete a certain number of problems.

The second finding was that kids love interacting with each other in a social way. We were blown away the first time we showed kids the avatar and social competition features. We were in a 3rd grade class in Palo Alto and intended to have the kids use it for at most half an hour. The kids were hooked for almost an hour straight until the bell for recess rang. To our surprise, the teacher gave the kids a choice: you can stay in the classroom and keep playing CarrotSticks, or you can go to recess. Not a single kid went to recess!

Many parts of your site can be entered as guest, with no registration. How important is that?

Anyone can use CarrotSticks anonymously without signing up for as long as they want. Forcing users to register before they even try a product is silly. It’s annoying to users because most of them came to our site through an ad or from a word-of-mouth recommendation. Almost all of them are here just to try it out and see if it is worth buying. The more steps we put between them showing up to our home page and them realizing the value in our product the worse it is for our business because users will just bounce. We know that if the user actually tries our product they will be hooked so we want to get them engaged as soon as possible.

Who designs the games on your site, and how do you go about it?

We have built all the games ourselves. We have done this because the user experience is so important when it comes to making this engaging and educational. We wanted to provide a step-by-step way to solve each kind of problem, and not simply a fill-in-the-blank or multiple choice option. We worked with the Stanford School of Education to come up with a pedagogically sound definition of the problems and difficulty of problems and have implemented the games based on this. We also wanted to integrate the notion of earning carrots every step of the way to make it engaging for kids. That said, we certainly hope in the future to partner with others to provide more games and content for our site. If you are interested in helping out, let me know!

You don’t allow free chat on the site, to make it a safer environment. How do you let kids interact?

Kids interact by playing with each other. That’s a natural way for kids to interact. Any kid can challenge any other kid to a competition at the difficulty of their choosing. The other kid can choose to accept or reject that challenge. CarrotSticks is completely safe for young kids because we don’t allow free chat. We want CarrotSticks to be a safe place for kids to learn with each other.

You worked as product manager for Google Chrome before. Please tell us one interesting thing about the experience of building Chrome which the world doesn’t know about yet!

While I was a product manager for Google Chrome I learned a lot. I was responsible for the backend aspects of the product including compatibility, stability, and performance. One interesting thing we did while building Chrome was to use Google’s massive distributed infrastructure to test stability. Since Chrome is used by users to load billions of different web pages we wanted to make sure changes to the product would still allow these pages to load correctly. So we built “Chrome Bot”. Every time a new build of Chrome is pushed, which happens multiple times a day, Chrome Bot tests the build on tens of thousands of pages and reports the results to our developers. This allows developers to catch problems as early as possible without relying on large external betas to report bugs.

What kind of errors was the Chrome Bot able to spot – and which ones were harder to spot? Did you compare visual output to specified “correct” target images to test CSS rendering?

Chrome Bot was really good at identifying crashes. One of the things we cared a lot about was stability. Nobody wants their browser to crash or hang. This led us to implement an infrastructure in which each tab is sandboxed from one another and live in their own process. If one tab crashes, all of the other tabs remain unscathed. We used Chrome Bot to help identify which websites out there caused crashes. Once we were able to identify a website that consistently caused Chrome to crash, we’d pick apart the website and come up with a reproducible test case which we could then use as a basis for identifying what we could do to change the browser so that that page would no longer crash.

Was the recently announced Chrome OS part of the original design, or was it something that Google started to develop after you left?

[This question remained unanswered.]

When did you leave Google, after which project?

I saw Barack Obama speak at Google back in November of 2007. At the end of his talk he said, “I want you to be involved,” so I took him literally and within two weeks I was volunteering at Obama headquarters in Chicago. I went back to Google in January and finished work on Google Chrome and then I left Google for good in July of 2008 to join Obama campaign as the Director of Analytics.

What did you do as Director of Analytics for the Obama presidential campaign?

I led a team of software engineers and analysts responsible for optimizing the effectiveness of the campaign. We worked as part of the “New Media” department which is what campaigns call the folks who do things that don’t fit the mold of traditional campaigns – the guys who SMS, Twitter, Facebook, Google, and email people. Our team ended up raising over half a billion dollars online, registering over 2 million new voters online, and helping persuade millions more to make phone calls and go vote on election day. After the election I went on to serve as Deputy New Media Director for the presidential transition, where I was responsible for strategic planning of the administration’s use of the internet and technology.

While working with the New Media department, did Obama get personally involved, did you speak with him?

Common wisdom in political campaigns is that if you see the candidate around the office then things aren’t going well. The candidate’s job is to get out in front of voters face-to-face. Every minute Barack Obama spends with us is a minute he could be spending with potential voters making his case for change. We did have a “road team” within New Media that was responsible for traveling with the candidate and other surrogates and did a great job writing blog posts, sharing photographs, and producing videos showing an insider’s view of the candidate on the campaign trail.

Google doesn’t seem to do much in the area of games. A somewhat more playful product, Lively 3D, got cancelled shortly after release. Why do you think Google isn’t too much interested in games?

Google’s core business is search and ads. Almost everyone at Google is focused on these two core aspects of Google’s business in some way or another. Most people wouldn’t consider Gmail under this umbrella but even though Gmail is free for most users, it does add multiplicative value to search and ads. If you are a Gmail user, then you are more likely to search using Google, which means they are more likely to click on a Google ad. Giving away most of Gmail isn’t as altruistic as many people might think because it still does drive increased searches which drives increased revenue. That said, Google is always trying new products and exploring new ideas to see if they might stick.

Dan Siroker: CarrotSticks ... by Philipp Lenssen | Comments (10)

>> More posts

Advertisement