Current Discussion/concerns
Topics:
- Chargeback model
- On going Launch issues
- Risk caching issue - future launches that have a breaking change, if so then we should add random queries string to the settings
- Ask Techops to see if we can get access to Akami to be able to see what is happening
- test location options in Backstage Pass to see if it is a specific location issue
- Smoke test seems to need to be run twice before agent can be promoted toProd
- Risk caching issue - future launches that have a breaking change, if so then we should add random queries string to the settings
- ABC onboarding - Help site only
- Next Steps - ABC feedback page
- Target Launch date: Jan 28, 2021
- Next Steps - ABC feedback page
- Movies Anywhere Title search solutions
- Bifrost dependency
- Chatbot 2.0
- Target Launch date: Feb 3, 2021
Feature Wish List
| Feature | Purpose |
|---|---|
| Product specific URLs for testing (current Bernard URL is all products) | To limit Product teams to their own Agents for UAT and ongoing continuous improvement |
| Assess PII APIs for listed use cases - GSTECH-3118 |
|
| Way we can do a data dump of Live Chat tickets to target the top topics and ways that guests ask questions | When on-boarding we can use existing live product volume to create training phrases for intents |
Recent Discussions
- Separate servers to scale
- probably worth separating out Disney+ and Hulu to their own
- possibly launch with ESPN+ using DDEG products
- Next onboarding
- ESPN+
- Hulubot
- When to move to web sockets
- need head count (work is 50/50 ish server client .. maybe a bit more server) (possibly 6 sprints)
- when we have 4-5k concurrent sessions we will benefit from moving to web sockets
- could have a cost benefit to move earlier, possible easier scaling
- Documentation of
- Updates - in readme files (server)
- review Chatbot 2.2/2.X
Attendees: Kevin, Darryl, Marc, Curtis, Katie
Overview:
- Title search will be triggered from Dialogflow/server to the client – Action Darryl/Katie to add in intents and triggers
- Design discussion:
- This will let the conversation continue in Dialogflow
- Confirm will send title as text into Dialogflow for next steps
- Go back will return to carousel will allow the viewer to view their initial search again
- Images can be a bit smaller
- Keep year visible
- Add info icon that will open to show full text that can be ‘x’ed out to close
- Remove looping to encourage viewers to scroll through options first
- Remove left scroll from initial card (prevent viewer from seeing the Movie not found card prior to scrolling through all the surfaced cards)
- Add end add Movie not found card
- Selecting a movie will trigger a request to Confirm the title OR Go back option
Was there anything missing from this list? If not I’ll go ahead and update both Confluence and Jira tickets.
Context for future designs: we are looking to handle as much as possible through the server and Dialogflow and keep the Client as simple as possible.
Question for future discussion: Does the background of the carousel impact Accessiblity? Do the thumbnails need an outline for readers to be more effective?
Archived Discussion
Q: how will Chatbot 2.0 handle determining language departments?
A: relies on the users' browser language and will also rely on the lang attribute set on the HTML tag in the header *standard in Zendesk for Help Sites, need to confirm if putting Chatbot on an external site
also during onboarding will need to determine a fallback language
Looking at V2 Fallback state
- can take out browser fallback code
- need to have fallback to Zendesk live chat in web widget if there is a failure to access the chatbot or our Live Chat
- Adding a table with expected outcome of each system state (GCS, GAE, ZD)
discussion about questions listed under "Differences" in ticket GSTECH-3307
- If product is using standalone chat need to disable CTO
- Start session tracking now on server
- Able to add additional data tracking in Client for:
Escalate (offline form)
- Escalate (offline form - cancel)
Escalate (online - in queue - cancel)
Escalate (online - 0 queue (connected to Rep))
- Initial data the Ted presented looks like The Good Bot is on par with the trends we saw with Lil Forward.
- Looks like there is an issue with CTO data going into Omniture
- ACTION - Darlesson and Ted to meet to try to problem solve the CTO mystery
Darlesson showing current features in v2
- solved v 1 issue of using back button by using iframe
- Name/title localizable
- app is localizable
- product (theme) is localizable (eg bot name can change per language)
- bot label beside name (very clear user is interacting with a bot vs a agent in live chat)
- Menu has a lot of flexibility
- Modals better model what an app would do
- Time stamp is available if you click on the message (can update)
- Responses that have multiple bubbles have a randomized amount of time to mimic a more natural chat experience
- Menu - contact button can change based on conditions (eg check dept and surface only available contact options)
- ACTION - Contact Support working session required Katie Ross to book
- Contact Support feedback - have the pre-chat form have the same 'blacked out" experience that happens when the menu is open
- Menu feedback - not clear about how to close the menu (possible confusion that the end chat is the traditional 'X' used to close out of the experience (closing the menu that was opened vs ending the chat)
- really like how clear it is when chatting with the bot vs an agent
- Label feedback - is it possible to have a different label colour or something between the bot vs agent?
- Want to remove risk of the bot interacting with a live chat once an agent is engaged and a live chat session is active - perhaps disable communication with ways to bot during an active live chat session?
- Contact support time zone questions - how should we best deal with the time zone difference
- also how to handle messaging when a product has taken a contact method offline during regular support times (eg shopDisney turning off live chat or phones)
Three features:
- flow from Chatbot to Zendesk
- pre-chat form
- offline form should have a success message
- canceling transfer to live chat
- need ample time for the person
- conversation is over in lc how do they get back to chatbot
- leaving rating as we currently have
- guest has action to end
- rating happens when Agent or Guest ends the chat, if Agent goes offline there is no rating
- when rating don't need to do a rating so the Guest can still see the chat but get rid of "type a message...". Guest needs to rate or skip
- modal vs in-line
Agent name would have name (agent/bot)
Actions for server for 2.0
- task about the Client send action request, then triggers
Yay
- It launched on the day that we said it would
- nice it was a small launch
- good to have deferred some issues
Meh
Nay
- linking did not go as expected due to caching issues with Akami - seemed to be linked to the office (worked on Backstage pass)
- request that Tech ops flush Akami- did not resolve
- Risk - future launches that have a breaking change
- Smoke test never updated the ticket, so wouldn't promote until a second smoke test was run
Actions:
Darryl Dekok check Darlesson's permissions
Recap from today’s sync:
- shopDisney
- Katie will give more details as to the reason for the change once they are shared
- deferred, looking at Q2
- 1.1.1
- Will decide by EOD if 1.1.1. needs to be pushed
- Moved GSTECH-2671 to 1.1.2
- Kevin confirmed sec review completed for 1.1.1.
- Include ABC theme/asset update to 1.1.2
- 2 outstanding bugs GSTECH-2878 and GSTECH-2883
- 1.1.2
- Require ABC icon and name by Oct 28 to hit Nov 5 launch
- Cody to share ABC theme assets with Darlesson
- Other
- Katie to intro with ABC contact and Darlesson to arrange for script and stage testing
Chitchat with Curtis and Katie about shopDisney Chatbot experience:
- third support method will be either live chat OR the chatbot depending on the article/section tags
- icon will update to reflect live chat or chatbot
RISK: if a guest has an active live chat session and moves to an article with the Chatbot, the session will follow them and they could choose to open the chatbot and start a second live chat session
this is due to the bug/feature in Zendesk that doesn't end a session if the rep/agent ends the chat
may be able to be resolved in 2.0
Attendees: Kevin, David, Matt, Curtis, Darryl, Katie
review ABC onboarding tickets
- review 1.1.1
- create timelines for ABC and shopDisney
Curtis, Katie, Darryl, Kevin
- about 70% of the work is done for the Zendesk portion of the switch to SDK
- need tickets for:
- flow for when the chat was ended by agent and department is offline
- guest sending Zendesk supported files to rep in live chat
- rep sending Zendesk support files to guest in live chat **low priority
- server work for starting the session before the Chatbot has been triggered
- SDK documentation for security
Attendees: Kevin, Curtis, Darryl, Katie
- Requires 2.0 launch
- Initial Admin service will allow changes without involving Tech Ops if a chatbot needs to be disabled
- future versions will be more user friendly and permissions can be extended to a greater number of the tech team (CRM team)
- ideally this service will live in Middleware/Krang/Core Services
@Ross, Katie to gather user requirements and set up meeting with larger group for feedback for the design kick off
Attendees: AJ, Darryl, Matt, Curtis, Darlesson, Dave, Katie
- Review and added to tables in the ABC on-boarding epic
- added sections for:
- security
- Zendesk
- Product (Katie Rossreview to see overlap with Product onboarding spreadsheet)
- Call outs
- update naming convention of GX CTO to avoid confusion if adding to an external site
- should update Zendesk master theme to implement health check fallback in manifest
- best practice when creating help sites is to use labels and use manifest settings as an override
- Katie Ross create gantt chart with timelines for upcoming Chatbot onboarding and releases
- added sections for:
Attendees: Darryl, Dave, Kevin, Katie
ABC or MA first?
ABC is easier based on Freeform than Movies Anywhere
MA site all in-house
Options of using Chatbot/Dialogflow solution or Middleware
similar Dev overhead
Con of MIddleware:
- need more time for scope additional use cases
- Dependencies on Tech Ops and Security work
Decision - to lower risk of sprawling timelines we will start with the Chatbot/Dialogflow solution
Action:
Katie
- follow up with Julia to see if they have a preference to ABC or MA launching first
- follow up with Ricardo to see if MA would want to launch earlier without the title API search feature
Attendees: Matt, Dave, Darryl, Darlesson, Curtis, Kevin, Katie
- reviewed bugs
- need to see if any can be released in 1.1.1
- Next Launch - reviewed tickets in 1.1.1 and 1.2
- keeping 1.2 as dedicated to OneID, moving other task/story to 1.1.1
- Had a good meeting with shopDisney, they are asking if there can be a version launched pre-November holiday on v1
- want to be able to customize which articles the chatbot is shown on
- Katie sync with Rick on GSTECH-2311
- look at timeline:
- Oct 10 - submit 1.1.1 for review
- Oct 23 - launch 1.1.1
- Oct 30 - ABC launch
- Oct 31 - Nov 8 - ABC hypercare
- Nov 14 - shopDisney launch
- Nov 14- 22 - shopDisney hypercare
- Dependencies
- Techops - co-ordinate more official kickoff
- Sec review - 1.1.1 should be quick, but will give the expected 2 week timeframe
- ABC team that controls feedback page
Attendees: Matt, Dave, Darryl, Darlesson, Curtis, Kevin, Katie
Yay
- we HAD a launch checklist
- good amount of time to QA before Zendesk releases
- went relatively smoothly (had to wait for things, but it was ok), planning for how external sites made it go well
- lots of collaboration between team
- pushing to get this done even with last minute changes before the Zendesk changes was great (woot woot Darlesson)
- liked the contingency plans (though 3 may have been a lot for future releases)
- Matt found a lot of issues, yay QA (x many) and team was able to resolve
- first major point release, learned a lot and went relatively well
Nay
- need to call out who is responsible for each action on the checklist
- not for us but not being able to use synergy is a PITA (QA takes way longer)
- Action: is there another solution to replace synergy?
- didn't break down as a project, just a bucket - need a better defined scope - by not doing this is made scope and timelines vague
- 3 hours before launch there wasn't a preview theme on Zendesk Stage- needs to be available (add to checklist) define when to move to the Product specific stage environment
- start how to test iteratively throughout the build (x 3) easier on whole team
- start building and working on products asap to allow for to spread out testing
- did not know what version was on which environment for QA
- Action: track using a versioning number add to build - see if we can match the numbers
- Darlesson found an issue at home that wasn't caught at all in the office/during testing - need to have time/equipment to test that wouldn't have cached info
- Security doesn't have good processes and inconsistent in message
- need to start design process before code for any infrastructure or more major change
- not a very efficient escalation process, so getting them involved as early as possible is key
Meh
- Launch checklist needs to be dialed in
- push client
- push server
- estimated times
- symlink took 15 mins!
- Product name (x all)
- Tech Ops a challenge (pipeline and DBA side)
- Google broke.. seriously?!?!?!
Attendees: Matt, Dave, Darryl, Darlesson, Curtis, Kevin, Katie
Google issue is preventing Sept 25 morning launch, team will regroup in the morning.
Katie will communicate out the contingency plans:
Plan A – if issue resolved by Wednesday morning
Wednesday September 25
10:00 am - Go/No go meeting
1:00 pm – 3:30 pm – launch
Plan B – if resolved during workday Wednesday
Wednesday September 25
EOD – go/no go
Thursday September 26
9:00 am – 10:30 am– launch
Plan C – if not launched Thursday
Thursday September 26
Deactivate the Chatbot on Freeform and launch the updated live chat widget **this will not require additional work from Storm Ideas
Time: TBD
Week of Sept 30 – Oct 4th
Launch Chatbot 1.1
Time: TBD
Attendees: Matt, Dave, Darryl, Darlesson, Curtis, Kevin, Katie
Meeting to discuss launch plan and bugs/concerns
Determined that some of the work for launch could be done prior to the 9 am bridge opening for a smoother launch
Action Items:
- follow up on whitelist request Darlesson Oliveira Katie Ross
- send scripts with CTO included to Dave at storm ideas (or confirm he was able to grab the previously used CTO scripts) Darlesson Oliveira
- add credentials to SFTP - Darryl Dekok
- go through process with Ted for accessing the files for reporting - Darryl Dekok
Day prior
- Push version 1.1 to prod (do not enable)
9am
- Open bridge - Katie Ross Katie
- update script including CTO tracking on https://freeform.go.com/feedback - Stormideas Dave
- turn on ICW - Curtis Dodd
- promote template
- promote server Darryl Dekok
- update symlink to 1.1 - Darryl Dekok
- QA - Matthew Spittal
Attendees: Dave, Darryl, Darlesson, Curtis, Kevin, Katie
Due to timelines of other product launches and dependencies for Movies Anywhere features requiring Darryl time to work on Middleware to use the Title APIs, we are going to launch 2.0 with polling, and upgrade to websockets ASAP after
WebSockets
- Architecture questions:
- is the Flexible environment security approved
- what is the impact on the deploy process/pipeline?
- Are there other impacts that may affect the timelines
- User impact
- What are the guest impacting pain points with using polling that will be alleviated with the move to WebSockets?
Attendees: Darryl, Matt, Darlesson, Curtis, Katie, Dave, Kevin
1.1 content review (pre-QA… so there will likely be more)
Current questions:
- Changing text on transfer page (asking about sending transcript)
- Simplify text in buttons and make instructions more specific
- Change text of message/buttons
- Increase size of buttons
- @Ross, Katie confirm text changes and add to ticket https://jira.wdig.com/browse/GSTECH-2440
- @Oliveira, Darlesson do the things
- @Ross, Katie to review and get final approval from stakeholders
- Issue:
- Potential solution:
- Action Items:
- Text/background colour
- @Oliveira, Darlesson/@Dodd, Curtis screenshots of at least two options – the current experience and something that is more neutral (like grays vs Freeform blues)
- Issue: Freeform blue with black text is hard to read
- Action Items
- Multi-bubble text responses
- @Dekok, Darryl is this possible to add?
- There is a note from @Oliveira, Darlesson on https://jira.wdig.com/browse/GSTECH-2438 that the animation isn’t possible until 2.0/SDK?
- Issue: missing typing animation
- Action Item:
- Character limit messaging – ok, I know we didn’t discuss this one in the meeting but I found it while writing these notes
- @Oliveira, Darlesson Is it possible to have a gray text message in the guest response window (like how Slack as “Message gxtext-chatbot” in our group?
- @Dodd, Curtis is there a different ‘industry standard’ solution to character limitation messaging?
- Issue: it is a poor user experience to potentially be typing a lot and then abruptly get the character limit message and have to redo work (or more likely send an incomplete message that will give a fallback response
- Action Items:
- IN ROOM DISCUSSION:
- Curtis: it is not standard to have pre-messaging about character limits or a descending character count - satisfied with current solution of having message appear if a guest types 256+ characters
- Higher impact is Guests that write that much are likely to get the fallback reply from the Chatbot - low likelihood to match with an intent
1.1 Launch Dependencies
- Need to determine if including SFTP work in 1.1
- Need to determine if we can activate the integrated chat widget or if Zendesk needs to be a part
- Need to have script sent to Freeform team to do timed update
- Need to set up launch time/bridge with all relevant parties (us, Freeform and possibly Zendesk?) **pre-Sept 30
Action Items:
- @Pigeon, Kevin in the meeting we discussed this prior to deciding we are moving the launch date to later in September, does this impact the decision of keeping/removing the SFTP work by Friday Sept 12? If it needs to be removed to launch 1.1 in September, need to create task for @Dekok, Darryl
- @Dodd, Curtis do we need Zendesk to be a part of the launch because of integrated chat widget work?
- @Oliveira, Darlesson need to have script sent to Freeform team to update on their sites
- @Ross, Katie coordinate launch date with relevant participants
2.0
- WebSockets
- Waiting official confirmation about using WebSockets in the flexible Google Apps environment
- Server LOE looks like 2 sprints worth – doable with the targeted Nov launch timeline
Action Items:
- @Dekok, Darryl keep team in loop when there is official word on if it’s sec approved to use the flex env
- @Oliveira, Darlesson @Dodd, Curtis continue to update story points as tasks are filled out
Next Products onboarding to 1.1
- ABC
- Client work – adding theme, Zendesk integration
- Need to provide script for CTO tracking on feedback page/non-Help Site pages that host the Chatbot
- Agent/Intent creation
- Movies Anywhere
- need to create user stories to assist with code design and research as there will be webhook work to integrate with the Title APIs
- Client work – adding theme, Zendesk integration
- Agent/Intent creation
Action Items:
- @Ross, Katie meet with ABC to kick off onboarding - get theme, icon etc.
- @Ross, Katie create user stories with user stories re: title searching using the MA title API
Other things
- @Oliveira, Darlesson update calendar with vacation time
- Does this affect 2.0 launch or are we able to steal Curtis for some time/tasks? 😃
- Process to call out dependencies for launches (e.g. 1.1)
- @Ross, Katie sync with Dave/AJ – is there something already used by GXtext that can be used for the Chatbot project for tracking dependencies?
Set up path from server to redux with action type in client
Protocol message list (*may not be comprehensive list):
- Initialize - on page open
- Initiate connect (hello) - when bot opened
- Reconnect
- session tracking to see if session token expired
- Talking
- Disconnect
- Escalate and other actions/triggers
Configuration can be separate
Darryl and Darlesson will work on:
- creating sequence diagram
- decide on a schema for the messages
Curtis and Darlesson will test Zopim with SDK to test trigger system
Can triggers be designed in a way that in the future they can be part of the admin service? (e.g. sD currently has triggers hard coded)
Attended by: Darryl, Darlesson, Curtis, Katie
Agenda: discuss LOE for web sockets work on server side for Chatbot 2.0
Summary:
- Heck yes the Chatbot rebuild should move to web sockets vs polling (unanimous)
- Admin Service - is the second priority to the SDK build
- ideally we could launch them at the same time or quickly launch Admin Service shortly after 2.0 is complete
- Timelines for server work:
- Websockets to SDK - 1 sprint
- Admin server - 3 sprints
Next Steps:
- Meeting between Darryl and Darlesson (Curtis tentative) to define protocol messaging between Client and Server
- Create Server tasks tickets in Jira
- Start Client work - first focus on Zendesk SDK
Concerns:
- Re: Darryl's time - outstanding work on Pipeline
- Work towards potential date to have bot in Production
- need clearer breakdown of time req, blockers etc for a live CB agent
- DEB doesn't have LC however, agent hand off to email support (same flow as for products that have LC that is offline)
- Plans to sync with DEB prod for beta?
reporting for charge back - did the bot provide an answer:
Yes if:
- bot gave answer that was not the default answer
Also: internal improvements - track guest feedback/CSAT/thumbs up/down
There are also analytic features in Dialogflow that can be used for Agent improvements. Other list of 'nice to have' reporting features can be found here: Chatbot - Reporting
- Darryl working on extending the POC (currently slated to end September 13)
- Looking to have date for demo POC - possibly in 3 weeks
- Darryl looking into reducing delays (could be impacted by location of project)
- Katie cleaned up intents in DialogFlow, will sync with Google team with next set of questions
Attendees: Dave, Curtis, Darlesson, Darryl, Katie
- Security meeting postponed
- Entering tasks to POC Jira board by EOW
- Meeting with Google team May 14 - 1:00 pm
- Darryl having meeting with security next week re: obfuscating PII text entered by Guests
- Katie will have initial Twine flow ready to review next scrum
Attendees: Darryl, Kevin, Katie
- idea to have bot "typing" indicator when a message is being populated (in design example)
- Gives more conversation flow rather than bombarding an answer as soon as the guest submits text
- option to back out of support if it is chosen
- example is to ask for name/age in pre-chat form, add something like a "never mind" option that goes back to the bot convo
- Karina suggestion: when the general issue is determined, populate the solution for the most pain point for that topic and have the opportunity for the guest to agree or say that wasn't the issue (or they tried that)
- Have a queueing system if the guest starts a new topic before one is resolved, bot could then circle back to make sure each is reviewed before ending the chat
- eg Guest starts with wanting to know how to connect to FB, then mid-support types in a general game play question, bot should be able to address the current question, but circle back to complete the support for how to connect to FaceBook
- how many open issues could a bot flag?
- If there are many how can they be surfaced in a low friction way to the guest (eg, you had questions about *provides button list, do you still need help with them?)
- eg Guest starts with wanting to know how to connect to FB, then mid-support types in a general game play question, bot should be able to address the current question, but circle back to complete the support for how to connect to FaceBook
- If putting on website, how to design so any links/buttons/details of the website are not covered
Attendees: Karina, Curtis, Darryl, Kevin, Katie, Dave
reviewed Gliffys
- bot should initially be freeform only
- Add in buttons if no clear path to next step
- use button to surface option for live support without directing Guest out of experience (bot directs to try using different words to describe issue to encourage use of free form but allow Guest to move on if they want to contact live support)
- try to find way for troubleshooting to be surfaced in more conversational way
- avoid large blocks of text instructions
- need way to close conversation - did bot solve the problem?
- use of thumbs up/down to assist in training bot (though the answer may have been correct.. the guest just may not have liked it...)
- possibly look at filter to prevent blatant curse words from going through?
Need to re-sync with Google peeps- Katie will reach out once there is a more detailed flow in place
April 5th meeting had: Pranav Chaudhari <pranavc@google.com>; Benedict Glynn <benedictg@google.com>; Breno Araújo <braraujo@google.com>; Chandni Shah <chanshah@google.com>; Uri First <ufirst@google.com>; JJ Jian <jjian@google.com>
Attendees: Darryl, Kevin, Katie, Dave