{"id":24220,"date":"2020-10-13T17:55:07","date_gmt":"2020-10-13T16:55:07","guid":{"rendered":"https:\/\/www.intercom.com\/blog\/?p=24220"},"modified":"2024-10-22T18:49:16","modified_gmt":"2024-10-22T17:49:16","slug":"building-resolution-bot-machine-learning-in-product-development","status":"publish","type":"post","link":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/","title":{"rendered":"Building Resolution Bot: How to apply machine learning in product development"},"content":{"rendered":"<p>We are at the start of a revolution in customer communication, powered by machine learning and artificial intelligence.<\/p>\n<p>At Intercom, we have taken advantage of these technologies relatively early. Our <a href=\"https:\/\/www.intercom.com\/blog\/customizable-bots\" target=\"_blank\" rel=\"noopener noreferrer\">Custom Bots<\/a> and <a href=\"https:\/\/www.intercom.com\/blog\/campaign\/resolution-bot\" target=\"_blank\" rel=\"noopener noreferrer\">Resolution Bot<\/a> already work for thousands of businesses every day. These bots help businesses deliver both radical efficiencies and better, faster support experiences.<\/p>\n<p>So, modern machine learning opens up vast possibilities \u2013 but how do you harness this technology to make an actual customer-facing product?<\/p>\n<p>Late last year, I <a href=\"https:\/\/youtu.be\/RPs32s_3e6k\" target=\"_blank\" rel=\"noopener noreferrer\">spoke<\/a> at the <a href=\"https:\/\/www.predictconference.com\/\" target=\"_blank\" rel=\"noopener noreferrer\">Predict Conference<\/a> on how we built Resolution Bot, our intelligent support chatbot that can instantly resolve common questions. This post is based on that talk, and details our journey from early experimentation to release, as well as some valuable lessons we learned about how to implement machine learning (ML) in a real-world product.<\/p>\n<h2 id=\"the-cupcake-approach-to-building-bots\">The cupcake approach to building bots<\/h2>\n<p>Reading papers about how Google or IBM build their ML products, it\u2019s easy to think only the very largest companies can afford to productize machine learning. Those companies need to spend a lot of time considering problems that will occur when a system has millions of users, and have to think carefully about ML tech debt given their scale. However, for smaller companies interested in delivering successful ML products, a lean approach can bring a lot of rewards.<\/p>\n<p>At Intercom, we generally follow the <a href=\"https:\/\/www.intercom.com\/blog\/start-with-a-cupcake\/\" target=\"_blank\" rel=\"noopener noreferrer\">cupcake approach<\/a> to building product \u2013 start with the smallest functioning version and iterate fast from there.<\/p>\n<blockquote class=\"pullquote-style-one\"><p>\u201cA complex system that works is invariably found to have evolved from a simple system that worked\u201d<\/p><\/blockquote>\n<p>When it comes to machine learning, Gall\u2019s Law applies: \u201cA complex system that works is invariably found to have evolved from a simple system that worked.\u201d ML products also require us to manage relatively large technology risks \u2013 this is an area where, unlike in most other product development, technical limitations might render the entire design impossible. We can\u2019t assume the ML will always perfectly do what we want.<\/p>\n<p>New software engineers quickly learn that a lot of complexity arises from error handling. Similarly, much of the complexity of ML products involves handling what happens when they make a mistake and in designing around imperfect performance. We can\u2019t just build for the good path.<\/p>\n<p>It&#8217;s also easy to over- or under- invest in the technology. It\u2019s hard to balance investment optimally across ML, product, engineering, and design.<\/p>\n<p>To address these challenges, as well as building &#8220;cupcakes,&#8221; we like to use a metaphor from computer science: ML products should be built &#8220;breadth-first&#8221;.<\/p>\n<p>Don\u2019t build three generations of ML tech before designing the first shippable design. Similarly, don\u2019t design far beyond what the machine learning can deliver \u2013 instead, traverse the product development tree, one level of depth at a time.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/10\/Machine-learning-Breadth-vs-depth-scaled.jpg\" \/><\/p>\n<p>Achieving this balance in practice requires a lot of work.<\/p>\n<h2 id=\"case-study-resolution-bot\">Case study: Resolution Bot<\/h2>\n<p><a href=\"https:\/\/www.intercom.com\/blog\/announcing-resolution-bot\/\" target=\"_blank\" rel=\"noopener noreferrer\">Resolution Bot<\/a> is a product that automatically answers the repetitive questions faced by customer support teams. Support teams get a lot of repeated inbound questions, which get tedious to answer. Automating the resolution of those questions helps companies to scale their support \u2013 operating online might mean you can sell to anyone in the world, but you better be able to support all those people if you want to retain your brand.<\/p>\n<p><script src=\"https:\/\/fast.wistia.com\/embed\/medias\/8q9uw4tibd.jsonp\" async><span data-mce-type=\"bookmark\" style=\"display: inline-block; width: 0px; overflow: hidden; line-height: 0;\" class=\"mce_SELRES_start\">\ufeff<\/span><\/script><script src=\"https:\/\/fast.wistia.com\/assets\/external\/E-v1.js\" async><\/script><\/p>\n<div class=\"wistia_embed wistia_async_8q9uw4tibd aligncenter\" style=\"height: 461px; position: relative; width: 500px;\"><\/div>\n<p>A big risk with a project like this is always end user experience. You might think that for an ML product, the UX is less important than usual \u2013 but actually, it\u2019s worth going particularly deep on UX when starting an ML project.<\/p>\n<p>An example of an exceptional end user experience for an ML product is the keyboard suggestions feature on smartphones. Consider how the suggestions subtly appear above the keyboard, without causing friction or irritation if they\u2019re not helpful.<\/p>\n<div class=\"wistia_responsive_padding\" style=\"padding: 56.25% 0 0 0; position: relative;\">\n<div class=\"wistia_responsive_wrapper\" style=\"height: 100%; left: 0; position: absolute; top: 0; width: 100%;\"><iframe loading=\"lazy\" class=\"wistia_embed\" title=\"Keyboard-suggestions Video\" src=\"https:\/\/fast.wistia.net\/embed\/iframe\/9nooyov3y2?videoFoam=true\" name=\"wistia_embed\" width=\"100%\" height=\"100%\" frameborder=\"0\" scrolling=\"no\" allowfullscreen=\"allowfullscreen\"><\/iframe><\/div>\n<\/div>\n<p><script src=\"https:\/\/fast.wistia.net\/assets\/external\/E-v1.js\" async><\/script><\/p>\n<p>A lot of design effort went into that subtlety, which results in suggestions that are simultaneously noticeable when they\u2019re relevant, but ignorable when they\u2019re irrelevant.<\/p>\n<p>This is what you want to design for \u2013 you can ship a first version with machine learning that&#8217;s not amazing, but it doesn&#8217;t really damage the end user experience, because the design makes the cost of false positives relatively low. Having a low cost of error makes it much easier to get an ML product off the ground \u2013 that\u2019s why it\u2019s often best to start with a version that makes suggestions, to mitigate the impact of errors. It might not be intuitive, but extra UX effort here can actually make the machine learning effort much more likely to deliver good results.<\/p>\n<blockquote class=\"pullquote-style-one\"><p>&#8220;We knew natural language processing is rapidly improving, but was it now good enough to produce the great user experience we needed?&#8221;<\/p><\/blockquote>\n<p>With this heuristic in mind, we were a little bit nervous when we started work on Resolution Bot. This bot was going to have to unexpectedly interject its way into support conversations. It\u2019s easy to annoy users by interrupting them. What was the cost of a false positive going to be? Machine learning is inherently probabilistic, so there will always be times it interjects with an answer that\u2019s irrelevant. Would end users be glad to see the bot? Would it hurt their experience if the bot was irrelevant? We knew natural language processing is rapidly improving, but was it now good enough to produce the great user experience we needed? That\u2019s a very fine judgment to make, and so very hard to know without prototyping.<\/p>\n<p>Finally, a huge worry for us was the customer set-up experience. Intercom has tens of thousands of customers, and they needed to be able to set up and configure Resolution Bot for each of their businesses without any intervention from us to make this work as a product.<\/p>\n<h2 id=\"balancing-risks-in-machine-learning-development\">Balancing risks in machine learning development<\/h2>\n<p>I believe that for any ML product, you&#8217;ve got to find the path that cheaply reduces these different types of risks in a balanced fashion. If you find yourself six months in, and there\u2019s a team working deeply on the machine learning, but haven\u2019t yet shipped versions that customers are using, that\u2019s a really bad sign. You might easily be over-engineering your solution or solving slightly the wrong problem.<\/p>\n<p>Equally, if you spend six months producing a great design, but without reducing technical risk, you can produce a great design which depends on unattainable ML performance, and so never ships. One of our <a href=\"https:\/\/www.intercom.com\/blog\/intercom-product-principles\/\" target=\"_blank\" rel=\"noopener noreferrer\">design principles<\/a> is &#8220;what you ship is what matters,&#8221; and rightly so.<\/p>\n<p>Instead, you want to carve out and validate an acceptable product envelope as soon as possible, in the breadth-first way described above. Do whatever it takes to define a version of your feature which can deliver value with the simplest machine learning possible. Ship that to customers in some alpha form as quickly as possible, and get the iteration flywheel spinning.<\/p>\n<h2 id=\"a-year-in-the-life-of-a-bot\">A year in the life of a bot<\/h2>\n<p>Resolution Bot took us just under a year to build, from first prototype in November 2017 to launch. I think the ups and downs of that year \u2013 the things we are proud of, and the things we messed up \u2013 provide a good case study for how to approach an ML project.<\/p>\n<p>During the first six weeks of the project, we built several waves of fast prototypes. Prototype one was just something we hacked together in a week, using Jupyter Notebooks and regular expressions. This enabled us to put something in front of stakeholders internally and check we were broadly aligned as to the project goals. An early working prototype you can interact with provides clarity and definition to ML projects, which often suffer from fuzziness.<\/p>\n<blockquote class=\"pullquote-style-one\"><p>&#8220;We had seen the magic of a couple of real end users having their questions successfully answered by a bot, and we knew time spent improving accuracy with real machine learning wouldn\u2019t be wasted&#8221;<\/p><\/blockquote>\n<p>For the next few prototypes, we still didn&#8217;t build any machine learning. We took an off-the-shelf Python search engine and built another test bot that could answer our own inbound support queries. We used this off-the-shelf technology for as long as possible, and focused on trying to identify and reduce other types of risk in a balanced fashion.<\/p>\n<p>After six weeks, we had reached prototype four \u2013 you could dialogue with it, and our colleagues could give helpful directional feedback. We still hadn\u2019t done any \u201creal machine learning,\u201d but we had seen the magic of a couple of real end users having their questions successfully answered by a bot, and we knew time spent improving accuracy with real machine learning wouldn\u2019t be wasted.<\/p>\n<h2 id=\"the-initial-core-challenge\">The initial core challenge<\/h2>\n<p>Moving fast, and not over-building the technology, allowed us to identify our first real challenge: given a bot that can answer customer questions, how does the bot get content? How does it get the answers to provide to customers? By proceeding in a cupcake fashion, in this breadth-first way, we flushed out this challenge earlier than we otherwise might have.<\/p>\n<p>The solution we reached for was to train the bot by mining historical questions. We prototyped a very quick and dirty JavaScript tool which would put a bank of historical questions through a simple off-the-shelf clustering algorithm. Seeing these clusters helped us figure out what questions we needed to teach the bot how to answer.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/intercom-origin.pagelyapps.com\/blog\/wp-content\/uploads\/2018\/11\/AnswerBotClusters.jpg.optimal.jpg\" \/><\/p>\n<p>At this point, we had a prototype version of every piece we needed to build a simple bot. No piece was brilliant, but we had identified most risk categories. This enabled us to start a broad internal alpha and we put it live \u2013 within two months of starting the project, it was answering real questions for real customers on Intercom.com. We could now iterate from a position of strength, observing live performance as we went. Getting to this point as fast as possible is the best way I know to accelerate a complex machine-learning product.<\/p>\n<p>Executing this whole product loop made us aware of several key constraints:<\/p>\n<ul>\n<li>First, curating the bot\u2019s replies, so it actually can answer useful questions, takes iteration to get right.<\/li>\n<li>Second, while our users were very patient with \u201cfalse positives\u201d \u2013 irrelevant suggestions \u2013 the Search technology we were using generated a lot of them.<\/li>\n<\/ul>\n<p>Overall, our biggest learning was that it felt like magic when the bot perfectly answered questions, and the many examples of people instantly getting answers to their questions really motivated us to continue pushing on this product.<\/p>\n<h2 id=\"curation-poses-a-design-challenge\">Curation poses a design challenge<\/h2>\n<p>With these learnings, we were confident enough to move on to full production with the envelope we had carved out, but this was when we really had to confront an incredibly difficult design challenge \u2013 the set-up experience for our customers.<\/p>\n<p>Up to this point, we were training the bot ourselves and were sure our users liked it. But could we design a curation interface that our customers could successfully use without understanding how it all worked under the hood?<\/p>\n<p>We hoped we could solve the false-positive issue by having people curate keywords, but several rounds of user research showed this approach didn\u2019t work. We spent many weeks of design iteration trying to make it easy to curate required keywords, but, no matter what we tried, too many customers struggled, frequently ending up with overly restrictive keywords, or sometimes misunderstanding them entirely. At this point, the wheels could have come entirely off the project.<\/p>\n<blockquote class=\"pullquote-style-one\"><p>&#8220;We started work on a custom machine learning algorithm to replace the off-the-shelf search engine with something more sophisticated&#8221;<\/p><\/blockquote>\n<p>But we were lucky \u2013 this was where our cupcake approach really paid off. By proving the utility of the concept early, we knew that we could afford to invest in solving this problem, and that the effort building a custom solution here wouldn\u2019t be wasted. So we started work on a custom machine learning algorithm to replace the off-the-shelf search engine with something more sophisticated \u2013 a custom algorithm would be much more accurate at deciding when it had a good match. We initially used word vectors and a lot of custom code, and now use cutting-edge deep learning.<\/p>\n<p>This worked very successfully. It\u2019s expensive and time consuming to build in-house ML, but this is the best time to spend those resources: when you\u2019ve proven the ML is the bottleneck on a real product problem and know the exact performance thresholds the ML solution has to hit.<\/p>\n<p>We also had another ML challenge on this project. For internal use we successfully used standard topic modelling algorithms to suggest common questions on which to train the bot, but these off-the-self algorithms are difficult to productize.<\/p>\n<p>Often, more theoretical clustering algorithms try to find all possible clusters of data, and to cluster every data point. But when building products, we often want to show customers just the best clusters. So we again ended up writing custom algorithms, designed to be more cautious and selective. We put the output of these algorithms into the product in a very slimmed-down interface, where we just showed the best single example of each cluster. Having a thin interface to the product like this allowed us to iterate rapidly on the underlying ML algorithm, even after launch, without needing to redesign each time. Being very thoughtful about the interface the ML system exposes can really help iteration speed.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/intercom-origin.pagelyapps.com\/blog\/wp-content\/uploads\/2020\/03\/how-to-train-a-chatbot-common-questions.png\" \/><\/p>\n<p><img loading=\"lazy\" decoding=\"async\" class=\"alignnone\" src=\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/03\/chatbot-common-customer-question-resolution-bot.png\" alt=\"Training Resolution Bot\" width=\"1600\" height=\"415\" \/><\/p>\n<p>Our metrics showed these suggestions did a great job at reducing initial friction, by helping customers curate their first few answers.<\/p>\n<h2 id=\"key-takeaways\">Key takeaways<\/h2>\n<p>Resolution Bot launched in October 2018 and went through a significant update (and name change) early this year, and now uses some of the most sophisticated neural network algorithms there are.<\/p>\n<p>But building machine learning products is always risky. You have to balance product and technology risk. My key takeaways from this project are:<\/p>\n<ul>\n<li>Don&#8217;t go and spend six or 12 months and build a beautiful design that just needs an amazing machine learning feature that can&#8217;t be delivered, or the other way around.<\/li>\n<li>Start prototyping as soon as possible with minimal machine learning.<\/li>\n<li>Test it with as many types of users as you can as soon as possible.<\/li>\n<li>Resist the temptation to wait until an unrealistically high accuracy threshold has been reached before shipping \u2013 shipping early and learning fast is crucial in order to reach that accuracy threshold.<\/li>\n<\/ul>\n<p>Across the whole industry, we think the ML technology that is now available, is way ahead of application. It\u2019s a very exciting time to be working on machine learning, as there are so many breakthroughs just waiting to be productized and brought to customers.<\/p>\n<p>Resolution Bot is just the start of our vision for this space. We\u2019re incredibly excited about the iterations and innovations that are on the way.<\/p>\n<p>Interested in joining the Intercom engineering team? Take a look at our <a href=\"https:\/\/www.intercom.com\/blog\/careers-engineering\">job openings<\/a>.<\/p>\n\n","protected":false},"excerpt":{"rendered":"<p>Building Machine Learning into viable products is a very different process than usual product development. Here are the key lessons we learned when we built Resolution Bot.<\/p>\n","protected":false},"author":451,"featured_media":24225,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"_jetpack_memberships_contains_paid_content":false,"footnotes":""},"category":[25198,12898],"tags":[604,11948,9980,158,21425,24865],"coauthors":[23690],"class_list":["post-24220","post","type-post","status-publish","format-standard","has-post-thumbnail","hentry","category-ai-ml","category-engineering","tag-ai","tag-automation","tag-chatbot","tag-machine-learning","tag-resolution-bot","tag-self-serve-support"],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.3 (Yoast SEO v27.3) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>How to Apply Machine Learning in Product Development<\/title>\n<meta name=\"description\" content=\"How do you apply machine learning in the development of actual products? Here are the key lessons we learned when we built Resolution Bot.\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Building Resolution Bot: How to apply machine learning in product development\" \/>\n<meta property=\"og:description\" content=\"How do you apply machine learning in the development of actual products? Here are the key lessons we learned when we built Resolution Bot.\" \/>\n<meta property=\"og:url\" content=\"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/\" \/>\n<meta property=\"og:site_name\" content=\"The Intercom Blog\" \/>\n<meta property=\"article:publisher\" content=\"https:\/\/www.facebook.com\/intercominc\" \/>\n<meta property=\"article:published_time\" content=\"2020-10-13T16:55:07+00:00\" \/>\n<meta property=\"article:modified_time\" content=\"2024-10-22T17:49:16+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/07\/Machine_learning_Hero.jpg\" \/>\n\t<meta property=\"og:image:width\" content=\"1968\" \/>\n\t<meta property=\"og:image:height\" content=\"921\" \/>\n\t<meta property=\"og:image:type\" content=\"image\/jpeg\" \/>\n<meta name=\"author\" content=\"Fergal Reid\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:creator\" content=\"@fergal_reid\" \/>\n<meta name=\"twitter:site\" content=\"@intercom\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Fergal Reid\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"12 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/#article\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/\"},\"author\":{\"name\":\"Fergal Reid\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#\\\/schema\\\/person\\\/f9403554a5f5d0b6a685a9ecc424cd6d\"},\"headline\":\"Building Resolution Bot: How to apply machine learning in product development\",\"datePublished\":\"2020-10-13T16:55:07+00:00\",\"dateModified\":\"2024-10-22T17:49:16+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/\"},\"wordCount\":2485,\"publisher\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#organization\"},\"image\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/Machine_learning_Hero.jpg\",\"keywords\":[\"AI\",\"automation\",\"chatbot\",\"machine learning\",\"resolution bot\",\"self-serve support\"],\"articleSection\":[\"AI &amp; Automation\",\"Engineering\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/\",\"url\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/\",\"name\":\"How to Apply Machine Learning in Product Development\",\"isPartOf\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/Machine_learning_Hero.jpg\",\"datePublished\":\"2020-10-13T16:55:07+00:00\",\"dateModified\":\"2024-10-22T17:49:16+00:00\",\"description\":\"How do you apply machine learning in the development of actual products? Here are the key lessons we learned when we built Resolution Bot.\",\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/building-resolution-bot-machine-learning-in-product-development\\\/#primaryimage\",\"url\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/Machine_learning_Hero.jpg\",\"contentUrl\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/wp-content\\\/uploads\\\/2020\\\/07\\\/Machine_learning_Hero.jpg\",\"width\":1968,\"height\":921},{\"@type\":\"WebSite\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#website\",\"url\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/\",\"name\":\"The Intercom Blog\",\"description\":\"Articles and Podcasts on Customer Service, AI and Automation, Product, and more\",\"publisher\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#organization\",\"name\":\"The Intercom Blog\",\"url\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/08\\\/Intercom-logo-sq-black-trans.png\",\"contentUrl\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/wp-content\\\/uploads\\\/2019\\\/08\\\/Intercom-logo-sq-black-trans.png\",\"width\":1000,\"height\":1000,\"caption\":\"The Intercom Blog\"},\"image\":{\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/www.facebook.com\\\/intercominc\",\"https:\\\/\\\/x.com\\\/intercom\",\"https:\\\/\\\/www.instagram.com\\\/intercom\\\/\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/2491343\",\"https:\\\/\\\/www.pinterest.ie\\\/intercom\\\/\",\"https:\\\/\\\/www.youtube.com\\\/channel\\\/UCJG0MvLP03kyzzAkD-w98aQ\",\"https:\\\/\\\/en.wikipedia.org\\\/wiki\\\/Intercom_(company)\"]},{\"@type\":\"Person\",\"@id\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/#\\\/schema\\\/person\\\/f9403554a5f5d0b6a685a9ecc424cd6d\",\"name\":\"Fergal Reid\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ad8a9a3ec34ea2796fed5ef6a6a9fd0423e5b3025cec8bdc91f42d921767aedb?s=96&d=mm&r=pg9d92e2d2427cd72bf3b3f43021666daf\",\"url\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ad8a9a3ec34ea2796fed5ef6a6a9fd0423e5b3025cec8bdc91f42d921767aedb?s=96&d=mm&r=pg\",\"contentUrl\":\"https:\\\/\\\/secure.gravatar.com\\\/avatar\\\/ad8a9a3ec34ea2796fed5ef6a6a9fd0423e5b3025cec8bdc91f42d921767aedb?s=96&d=mm&r=pg\",\"caption\":\"Fergal Reid\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/fergal_reid\"],\"url\":\"https:\\\/\\\/www.intercom.com\\\/blog\\\/author\\\/fergal_reid\\\/\"}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"How to Apply Machine Learning in Product Development","description":"How do you apply machine learning in the development of actual products? Here are the key lessons we learned when we built Resolution Bot.","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/","og_locale":"en_US","og_type":"article","og_title":"Building Resolution Bot: How to apply machine learning in product development","og_description":"How do you apply machine learning in the development of actual products? Here are the key lessons we learned when we built Resolution Bot.","og_url":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/","og_site_name":"The Intercom Blog","article_publisher":"https:\/\/www.facebook.com\/intercominc","article_published_time":"2020-10-13T16:55:07+00:00","article_modified_time":"2024-10-22T17:49:16+00:00","og_image":[{"width":1968,"height":921,"url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/07\/Machine_learning_Hero.jpg","type":"image\/jpeg"}],"author":"Fergal Reid","twitter_card":"summary_large_image","twitter_creator":"@fergal_reid","twitter_site":"@intercom","twitter_misc":{"Written by":"Fergal Reid","Est. reading time":"12 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/#article","isPartOf":{"@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/"},"author":{"name":"Fergal Reid","@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/person\/f9403554a5f5d0b6a685a9ecc424cd6d"},"headline":"Building Resolution Bot: How to apply machine learning in product development","datePublished":"2020-10-13T16:55:07+00:00","dateModified":"2024-10-22T17:49:16+00:00","mainEntityOfPage":{"@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/"},"wordCount":2485,"publisher":{"@id":"https:\/\/www.intercom.com\/blog\/#organization"},"image":{"@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/#primaryimage"},"thumbnailUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/07\/Machine_learning_Hero.jpg","keywords":["AI","automation","chatbot","machine learning","resolution bot","self-serve support"],"articleSection":["AI &amp; Automation","Engineering"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/","url":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/","name":"How to Apply Machine Learning in Product Development","isPartOf":{"@id":"https:\/\/www.intercom.com\/blog\/#website"},"primaryImageOfPage":{"@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/#primaryimage"},"image":{"@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/#primaryimage"},"thumbnailUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/07\/Machine_learning_Hero.jpg","datePublished":"2020-10-13T16:55:07+00:00","dateModified":"2024-10-22T17:49:16+00:00","description":"How do you apply machine learning in the development of actual products? Here are the key lessons we learned when we built Resolution Bot.","inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/"]}]},{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.intercom.com\/blog\/building-resolution-bot-machine-learning-in-product-development\/#primaryimage","url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/07\/Machine_learning_Hero.jpg","contentUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/07\/Machine_learning_Hero.jpg","width":1968,"height":921},{"@type":"WebSite","@id":"https:\/\/www.intercom.com\/blog\/#website","url":"https:\/\/www.intercom.com\/blog\/","name":"The Intercom Blog","description":"Articles and Podcasts on Customer Service, AI and Automation, Product, and more","publisher":{"@id":"https:\/\/www.intercom.com\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/www.intercom.com\/blog\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/www.intercom.com\/blog\/#organization","name":"The Intercom Blog","url":"https:\/\/www.intercom.com\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2019\/08\/Intercom-logo-sq-black-trans.png","contentUrl":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2019\/08\/Intercom-logo-sq-black-trans.png","width":1000,"height":1000,"caption":"The Intercom Blog"},"image":{"@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/www.facebook.com\/intercominc","https:\/\/x.com\/intercom","https:\/\/www.instagram.com\/intercom\/","https:\/\/www.linkedin.com\/company\/2491343","https:\/\/www.pinterest.ie\/intercom\/","https:\/\/www.youtube.com\/channel\/UCJG0MvLP03kyzzAkD-w98aQ","https:\/\/en.wikipedia.org\/wiki\/Intercom_(company)"]},{"@type":"Person","@id":"https:\/\/www.intercom.com\/blog\/#\/schema\/person\/f9403554a5f5d0b6a685a9ecc424cd6d","name":"Fergal Reid","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/secure.gravatar.com\/avatar\/ad8a9a3ec34ea2796fed5ef6a6a9fd0423e5b3025cec8bdc91f42d921767aedb?s=96&d=mm&r=pg9d92e2d2427cd72bf3b3f43021666daf","url":"https:\/\/secure.gravatar.com\/avatar\/ad8a9a3ec34ea2796fed5ef6a6a9fd0423e5b3025cec8bdc91f42d921767aedb?s=96&d=mm&r=pg","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/ad8a9a3ec34ea2796fed5ef6a6a9fd0423e5b3025cec8bdc91f42d921767aedb?s=96&d=mm&r=pg","caption":"Fergal Reid"},"sameAs":["https:\/\/x.com\/fergal_reid"],"url":"https:\/\/www.intercom.com\/blog\/author\/fergal_reid\/"}]}},"jetpack_featured_media_url":"https:\/\/www.intercom.com\/blog\/wp-content\/uploads\/2020\/07\/Machine_learning_Hero.jpg","jetpack_sharing_enabled":true,"_links":{"self":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/posts\/24220","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/users\/451"}],"replies":[{"embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/comments?post=24220"}],"version-history":[{"count":0,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/posts\/24220\/revisions"}],"wp:featuredmedia":[{"embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/media\/24225"}],"wp:attachment":[{"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/media?parent=24220"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/category?post=24220"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/tags?post=24220"},{"taxonomy":"author","embeddable":true,"href":"https:\/\/www.intercom.com\/blog\/wp-json\/wp\/v2\/coauthors?post=24220"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}