Overview
Hulubot is moving from Salesforce Einstein to the custom built Chatbot platform that is integrated with Google Dialogflow. We expect to gain efficiencies with maintaining Hulubot while piloting the chatbot platform on a non-DDEG product.
Relevant Docs
MVP Hulu intents - WIP
Hulubot - Conversation Design project from Braulio (accessible with disneystreaming account)
Testing pages
Hulu stage (homepage only)
Bernard testing page (must be on VPN)**currently intents are only in Dev
Meeting Notes
Attendees
PMO: Joe
PM: Jesse, Katie
Tech: Darryl, Dylan, Horven, Drew
Analytics, Derek, Julia, Heather
Actions:
- Reporting: Julia and team to review logistics of Domo vs Tableau and confirm possible timeline
- Advocate impact: Dylan, Katie sync with support ops on impact (Danny G and Courtney D)
- Dylan: organize tech deep dive for using current CSAT/Snap-in solutions
- Katie/Jesse: Analytics brief, discuss implications if we do not have a CSAT for the chatbot experience (possibly build into Dialogflow as an intent instead)
- Katie: reach out to Alissa to confirm the unauth bot experience will not impact the overall AB testing of the auth/unauth HC project slated for Apr/May
Overview:
Live chat - use of Snap-in chat is preferred route to keep launch on track
- Dylan, Drew, Darryl to discuss tech details and implications (eg chatbot session transcript)
CSAT - Q: can we use the current Salesforce CSAT experience
- If we use the snap-in, would we be able to use the CSAT for the live agent experience?
- Do we need a CSAT for the chatbot session
- prefer that we do as it is used as a measure of deflection success
Reporting - Where will it go, are there requirements/restrictions of keeping Hulu data separate (Darryl confirmed this is possible)
Current conversations
Possibility of using snap-in chat to transfer to Salesforce
Using existing feature
May not be able to pass through transcript
Possibility of using existing Salesforce CSAT
Using existing feature
- TBD Tableau or Domo
- Data coming from Chatbot server and Salesforce
Jira
From the Chatbot Feature Comparison Table
| Platform | Einstein | ||||
| Client | Bernardbot | Salesforce | |||
| Feature Category | Feature Name | Description | CURRENT STATE | CURRENT STATE | PRIORITY |
| Overview | Authenticated State | Bot version for authenticated users. | NOT MVP | TRUE | 0 |
| Overview | Unauthenticated State | Bot version for unauthenticated users. | AVAILABLE | TRUE | 0 |
| Overview | NLP Enabled | Bot has NLP capability. | AVAILABLE | TRUE | 0 |
| Overview | Text Field Enabled | Bot has text field enabled permanently. | AVAILABLE | TRUE | 0 |
| Overview | Agent Escalation | Bot can escalate user to a human agent. | IN PROGRESS | TRUE | 0 |
| Overview | Multi-Lang Support | Bot has multi-language support. | NOT MVP | FALSE | 3 |
| Overview | Resolution Confirmation / Ratables | Bot presents question that asks if the bot resolved user's issue. | AVAILABLE | TRUE | 1 |
| Overview | Rich Text Formatting | Bot platform allows for rich text formatting (bold, italics, etc.) OOTB. | AVAILABLE | FALSE | N/A |
| Overview | Rich Media | Bot platform allows for rich media (carousels, videos, etc.) OOTB. | REQ DEV | TRUE | 2 |
| Overview | Authentication via Bot | User is able to authenticate via the bot. | NOT MVP | FALSE | N/A |
| Overview | Main Menu | User is presented with a main menu (or quick replies) at the start of the interaction. | AVAILABLE | TRUE | 0 |
| Overview | Backend Triggers | Bot leverages backend triggers for set conditions. | REQ DEV | TRUE | 1 |
| Overview | Custom Styling | Bot UI leverages custom styles. | AVAILABLE | TRUE | 1 |
| Overview | End-of-chat survey | Bot serves an experience survey at the end of the interaction. | IN PROGRESS | TRUE | 1 |
| System Integrations | Account Management | User is able to update PII within bot experience (email address, mailing address etc) | NOT MVP | FALSE | N/A |
| System Integrations | Subscription Management / Billing | User is able to view/edit subscriptions, view previous billing or current access that is non-recurring | NOT MVP | FALSE | N/A |
| System Integrations | Library Management | User is able to view/edit library content (e.g. watch list, delete content from library?) | NOT MVP | FALSE | N/A |
| System Integrations | Device Management | User is able to view/delete devices activate on their account (possibly also view useage per device?) | NOT MVP | FALSE | N/A |
| System Integrations | Subscription Creation | User is able to create a new account and complete purchase flow of subscription/one time billing | NOT MVP | FALSE | N/A |
| NLP | Number of Intents | Number of intents currently live in bot. | TBD | 48 | |
| NLP | Minimum Training Phrases per Intent | The minimum number of training phrases required per intent for the model to function. | 10-15 (can function with less) | 150-200 | |
| NLP | Model Build Time | Time it takes for the NLP model to be "trained". | seconds | ~1.5 hr | |
| NLP | Training Phrase Generation Assistance | Platform provides training phrase generation assistance using past user utterances. | TRUE Think GAP work Jesse has set up is more refined than OTB DF | TRUE | 3 |
| NLP | Holistic NLP Model Assessment | Platform provides holistic model assessment (confusion between intents, etc.). | TRUE | FALSE | N/A |
| NLP | "Follow-Up" Intents | Bot leverages follow-up intents in conversation design. | TRUE | FALSE | N/A |
| NLP | Fallback Intents | Bot leverages fallback intents. | TRUE | TRUE | 0 |
| NLP | Small Talk Capability | Bot has small talk capabilities to emphasize conversational aspect of interaction. | TRUE | FALSE | N/A |
| NLP | Content Gap Identification | Platform has content gap identification capabilities, leveraging past user utterances. | TRUE Think GAP work Jesse has set up is more refined than OTB DF | FALSE | N/A |
| NLP | Escape Hatches | Bot conversation design contains escape hatches. | TRUE | TRUE | 0 |
| NLP | Context Retention | Bot leverage context retention in conversation design. | TRUE | FALSE | N/A |
| NLP | Duplicate Detection Within Intent | Platform is able to detect duplicate training phrases across intents. | TRUE | FALSE | N/A |
| Content | Account Management | Subscription Management | Bot guides users on how they can manage their subscription. | TRUE | TRUE | 1 |
| Content | Account Management | Cancellation | Bot guides users on how they can cancel their account. | TRUE | TRUE | 1 |
| Content | Account Management | Device Management | Bot guides users on how they can manage devices on their account. | TRUE | TRUE | 2 |
| Content | Account Management | User Management | Bot guides users on how they can manage the users on their account. | TRUE | TRUE | 2 |
| Content | Account Management | Logging In Assistance | Bot guides users on how they can resolve their login issues. | TRUE | TRUE | 1 |
| Content | Billing | Payment Update | Bot guides users on how they can update their payment information. | TRUE | TRUE | 1 |
| Content | Billing | Refunds | Bot guides users on how they can request / confirm a refund(s). | TRUE | TRUE | 1 |
| Content | Billing | Unexpected Charges | Bot guides users on what an unexpected charge(s) might mean and how they can get assistance. | TRUE | TRUE | 1 |
| Content | Product-Related | Troubleshooting | Bot guides users on how they can troubleshoot a product issue. | TRUE | TRUE | 1 |
| Content | Product-Related | Product Launches (What) | Bot informs users of product launches (ex. offline downloads feature, Oscars, etc.). | TRUE | TRUE | 2 |
| Content | Product-Related | Product Guides (How) | Bot guides users through how they can set up or use the product / feature (ex. live tv, offline downloads, etc.). | TRUE | TRUE | 1 |