{"id":7804,"date":"2015-08-14T17:49:21","date_gmt":"2015-08-15T00:49:21","guid":{"rendered":"http:\/\/intercom.com\/blog\/?p=7804"},"modified":"2020-07-30T13:02:20","modified_gmt":"2020-07-30T12:02:20","slug":"better-goals-help-magoosh-improve-message-performance-by-11","status":"publish","type":"post","link":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/","title":{"rendered":"Better goals help Magoosh improve message performance by 11%"},"content":{"rendered":"

What are you hoping to achieve when you message customers? Do you want them to simply open the email, and maybe click a link? Or do you actually want them to take some real action in your product?<\/p>\n

Magoosh was one of our earliest and heaviest users of Intercom\u2019s Message Goals<\/a> feature while it was in beta.<\/p>\n

Magoosh is a web and mobile application that helps students prepare for standardized tests. The application allows students to learn at their own pace, on their own time, and in an adaptive assessment environment.<\/p>\n

What follows is a post by Kevin Rocci<\/a>, on how Magoosh<\/a>\u00a0use Message Goals and A\/B testing to successfully get more students to engage with their learning materials.<\/p>\n

\n

Let\u2019s be frank \u2013 standardized testing is not fun. Our perception of preparing for one of these tests usually conjures images of thick expensive books, Red Bull fueled 8-hour study sessions, long classes with instructors of varying skill, or a painfully solo journey of rote memorizing.<\/p>\n

Magoosh exists to change all that. We want to shatter those expectations by offering materials and products that are accessible, effective, and engaging. We believe that learning can be fun, even if it\u2019s preparing for a standardized test.<\/p>\n

The core value of meaningful data<\/h2>\n

We value meaningful data. One of our core values<\/a> is Data > Intuition, so we knew that standard metrics \u2014 like email open rates and click through rates \u2014 weren\u2019t enough to tell us if we were successful. Those numbers are easy to game, and they don\u2019t achieve the real goal: to help students learn.<\/p>\n

Since we have a healthy culture of experimenting and measuring success towards goals with metrics, we instantly gravitated to the simple and easy A\/B testing and Goal-setting features in Intercom. They\u2019ve become a regular part of any message we send to our students because it\u2019s so easy to track metrics. It also ensures that any message we send will be meaningful and effective. We\u2019ve gotten to the point where if we don\u2019t A\/B test or don\u2019t have a goal set, we re-evaluate why we are even sending the message.<\/p>\n

For example, we help high school students prepare for the Standardized Achievement Test (SAT), a common requirement on university applications. We noticed high schoolers tended to not be a strongly self-motivated bunch, so our goal for the quarter was to increase engagement by encouraging students to answer more online questions and watch more interactive lessons.<\/p>\n

We created an Intercom message to send out to students who had been studying, answering questions, watching lessons, but stopped. After 5 days of inactivity, they would get a message that would encourage them to continue studying.<\/p>\n

But how do you motivate a high schooler to study? What sort of message should you write? What should you talk about? Having the A\/B Test and Goal features in Intercom made it easy to experiment with a message that really motivated students.<\/p>\n

Messages that motivate<\/h2>\n

We started our A\/B test with two messages: one that focused on getting back to studying for the SAT and one focused on getting to college. We had a hunch that students would be more motivated to study if we tied success on the SAT to college and life goals\u2014not success on the SAT itself. So we crafted one message to link success on the SAT to larger life goals. In the message, we said:<\/p>\n

\"\"<\/p>\n

The other message focused on studying and preparing in order to succeed on the SAT:<\/p>\n

\"\"<\/p>\n

Our goal was for students to login and practice so we added a Goal for \u2018web sessions\u2019 to increase and set it live.<\/p>\n

After letting the experiment run for a few weeks, we found that the message that tied success to future goals won. It was nice to see that one message had a better open rate than the other \u2014 57% to 60% \u2014 but the real win was the seeing more students log in and study.<\/p>\n

With Goals, it was easy to see that the message about life goals had 236 students logging in and studying versus 227 students for the other message. This doesn\u2019t look like a significant difference\u2014a difference of 9 between the two tests. How could this be a win?<\/p>\n

We use isvalid<\/a> to help. This tool allows us to quickly check the significance of our numbers that we get back from the goal feature in Intercom. We want to know if one message is directionally better than another. This allows us to turn off the test and move to the next one quickly. We found that we were at 95.8% significance with our message with the winner performing 11% better than the other. This was good enough for us to turn off the experiment and choose a winner.<\/p>\n

Goals and A\/B testing make it so easy for us to commit to our core value of Data > Intuition. Not only can we default to testing and experimenting to find what really works, but also Intercom makes the features very easy to use so new team members at Magoosh can jump in right away and start testing.<\/p>\n","protected":false},"excerpt":{"rendered":"

What are you hoping to achieve when you message customers? Do you want them to simply open the email, and maybe click a link? Or do you actually want them to take some real action in…<\/p>\n","protected":false},"author":141,"featured_media":7806,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"category":[3],"tags":[207,213,318,333,200],"coauthors":[440],"class_list":["post-7804","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-customer-support","tag-ab-testing","tag-customer-engagement","tag-customer-story","tag-message-goals","tag-messaging"],"yoast_head":"\nBetter goals help Magoosh improve message performance by 11%<\/title>\n<meta name=\"description\" content=\"Tracking Message Goals in Intercom rather than clicks allowed Magoosh chose a message that performed 11% better in getting users to take action.\" \/>\n<meta name=\"robots\" content=\"noindex, follow\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Better goals help Magoosh improve message performance by 11%\" \/>\n<meta property=\"og:description\" content=\"Tracking Message Goals in Intercom rather than clicks allowed Magoosh chose a message that performed 11% better in getting users to take action.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/\" \/>\n<meta property=\"og:site_name\" content=\"The Intercom Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/intercominc\" \/>\n<meta property=\"article:published_time\" content=\"2015-08-15T00:49:21+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2020-07-30T12:02:20+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png\" \/>\n\t<meta property=\"og:image:width\" content=\"984\" \/>\n\t<meta property=\"og:image:height\" content=\"466\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/png\" \/>\n<meta name=\"author\" content=\"Kevin Rocci\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@kevinrocci\" \/>\n<meta name=\"twitter:site\" content=\"@intercom\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Kevin Rocci\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"4 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/\"},\"author\":{\"name\":\"Kevin Rocci\",\"@id\":\"https:\/\/www.intercom.com\/blog\/#\/schema\/person\/c3cdbd6608cba0ee3da5e5dbcadffb39\"},\"headline\":\"Better goals help Magoosh improve message performance by 11%\",\"datePublished\":\"2015-08-15T00:49:21+00:00\",\"dateModified\":\"2020-07-30T12:02:20+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/\"},\"wordCount\":867,\"publisher\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/#organization\"},\"image\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png\",\"keywords\":[\"A\/B testing\",\"customer engagement\",\"customer story\",\"message goals\",\"messaging\"],\"articleSection\":[\"Customer Service\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/\",\"url\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/\",\"name\":\"Better goals help Magoosh improve message performance by 11%\",\"isPartOf\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage\"},\"image\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage\"},\"thumbnailUrl\":\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png\",\"datePublished\":\"2015-08-15T00:49:21+00:00\",\"dateModified\":\"2020-07-30T12:02:20+00:00\",\"description\":\"Tracking Message Goals in Intercom rather than clicks allowed Magoosh chose a message that performed 11% better in getting users to take action.\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage\",\"url\":\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png\",\"contentUrl\":\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png\",\"width\":984,\"height\":466},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/www.intercom.com\/blog\/#website\",\"url\":\"https:\/\/www.intercom.com\/blog\/\",\"name\":\"The Intercom Blog\",\"description\":\"Articles and Podcasts on Customer Service, AI and Automation, Product, and more\",\"publisher\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/www.intercom.com\/blog\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/www.intercom.com\/blog\/#organization\",\"name\":\"The Intercom Blog\",\"url\":\"https:\/\/www.intercom.com\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/www.intercom.com\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2019\/08\/Intercom-logo-sq-black-trans.png\",\"contentUrl\":\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2019\/08\/Intercom-logo-sq-black-trans.png\",\"width\":1000,\"height\":1000,\"caption\":\"The Intercom Blog\"},\"image\":{\"@id\":\"https:\/\/www.intercom.com\/blog\/#\/schema\/logo\/image\/\"},\"sameAs\":[\"https:\/\/www.facebook.com\/intercominc\",\"https:\/\/x.com\/intercom\",\"https:\/\/www.instagram.com\/intercom\/\",\"https:\/\/www.linkedin.com\/company\/2491343\",\"https:\/\/www.pinterest.ie\/intercom\/\",\"https:\/\/www.youtube.com\/channel\/UCJG0MvLP03kyzzAkD-w98aQ\",\"https:\/\/en.wikipedia.org\/wiki\/Intercom_(company)\"]},{\"@type\":\"Person\",\"@id\":\"https:\/\/www.intercom.com\/blog\/#\/schema\/person\/c3cdbd6608cba0ee3da5e5dbcadffb39\",\"name\":\"Kevin Rocci\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/secure.gravatar.com\/avatar\/89d9b01fe386b724f6058fad1879837c2d3389168bf515f6f29018487be8a2d3?s=96&d=mm&r=pgf30a6ea9541b0d81855da32f44ab5fe3\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/89d9b01fe386b724f6058fad1879837c2d3389168bf515f6f29018487be8a2d3?s=96&d=mm&r=pg\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/89d9b01fe386b724f6058fad1879837c2d3389168bf515f6f29018487be8a2d3?s=96&d=mm&r=pg\",\"caption\":\"Kevin Rocci\"},\"sameAs\":[\"https:\/\/x.com\/kevinrocci\"],\"url\":\"https:\/\/www.intercom.com\/blog\/author\/kevinrocci\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Better goals help Magoosh improve message performance by 11%","description":"Tracking Message Goals in Intercom rather than clicks allowed Magoosh chose a message that performed 11% better in getting users to take action.","robots":{"index":"noindex","follow":"follow"},"og_locale":"en_US","og_type":"article","og_title":"Better goals help Magoosh improve message performance by 11%","og_description":"Tracking Message Goals in Intercom rather than clicks allowed Magoosh chose a message that performed 11% better in getting users to take action.","og_url":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/","og_site_name":"The Intercom Blog","article_publisher":"https:\/\/www.facebook.com\/intercominc","article_published_time":"2015-08-15T00:49:21+00:00","article_modified_time":"2020-07-30T12:02:20+00:00","og_image":[{"width":984,"height":466,"url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png","type":"image\/png"}],"author":"Kevin Rocci","twitter_card":"summary_large_image","twitter_creator":"@kevinrocci","twitter_site":"@intercom","twitter_misc":{"Written by":"Kevin Rocci","Est. reading time":"4 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#article","isPartOf":{"@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/"},"author":{"name":"Kevin Rocci","@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/person\/c3cdbd6608cba0ee3da5e5dbcadffb39"},"headline":"Better goals help Magoosh improve message performance by 11%","datePublished":"2015-08-15T00:49:21+00:00","dateModified":"2020-07-30T12:02:20+00:00","mainEntityOfPage":{"@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/"},"wordCount":867,"publisher":{"@id":"https:\/\/www.intercom.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage"},"thumbnailUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png","keywords":["A\/B testing","customer engagement","customer story","message goals","messaging"],"articleSection":["Customer Service"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/","url":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/","name":"Better goals help Magoosh improve message performance by 11%","isPartOf":{"@id":"https:\/\/www.intercom.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage"},"image":{"@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage"},"thumbnailUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png","datePublished":"2015-08-15T00:49:21+00:00","dateModified":"2020-07-30T12:02:20+00:00","description":"Tracking Message Goals in Intercom rather than clicks allowed Magoosh chose a message that performed 11% better in getting users to take action.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.intercom.com\/blog\/better-goals-help-magoosh-improve-message-performance-by-11\/#primaryimage","url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png","contentUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png","width":984,"height":466},{"@type":"WebSite","@id":"https:\/\/www.intercom.com\/blog\/#website","url":"https:\/\/www.intercom.com\/blog\/","name":"The Intercom Blog","description":"Articles and Podcasts on Customer Service, AI and Automation, Product, and more","publisher":{"@id":"https:\/\/www.intercom.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.intercom.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.intercom.com\/blog\/#organization","name":"The Intercom Blog","url":"https:\/\/www.intercom.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2019\/08\/Intercom-logo-sq-black-trans.png","contentUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2019\/08\/Intercom-logo-sq-black-trans.png","width":1000,"height":1000,"caption":"The Intercom Blog"},"image":{"@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/intercominc","https:\/\/x.com\/intercom","https:\/\/www.instagram.com\/intercom\/","https:\/\/www.linkedin.com\/company\/2491343","https:\/\/www.pinterest.ie\/intercom\/","https:\/\/www.youtube.com\/channel\/UCJG0MvLP03kyzzAkD-w98aQ","https:\/\/en.wikipedia.org\/wiki\/Intercom_(company)"]},{"@type":"Person","@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/person\/c3cdbd6608cba0ee3da5e5dbcadffb39","name":"Kevin Rocci","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/89d9b01fe386b724f6058fad1879837c2d3389168bf515f6f29018487be8a2d3?s=96&d=mm&r=pgf30a6ea9541b0d81855da32f44ab5fe3","url":"https:\/\/secure.gravatar.com\/avatar\/89d9b01fe386b724f6058fad1879837c2d3389168bf515f6f29018487be8a2d3?s=96&d=mm&r=pg","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/89d9b01fe386b724f6058fad1879837c2d3389168bf515f6f29018487be8a2d3?s=96&d=mm&r=pg","caption":"Kevin Rocci"},"sameAs":["https:\/\/x.com\/kevinrocci"],"url":"https:\/\/www.intercom.com\/blog\/author\/kevinrocci\/"}]}},"jetpack_featured_media_url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2015\/08\/Magoosh-message-goals-hero.png","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/posts\/7804","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/users\/141"}],"replies":[{"embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/comments?post=7804"}],"version-history":[{"count":0,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/posts\/7804\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/media\/7806"}],"wp:attachment":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/media?parent=7804"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/category?post=7804"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/tags?post=7804"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/coauthors?post=7804"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}