ALLY

.

Alliance Water Stewardship - Survey Chatbot

CASE STUDY

Overview

The AWS identified that there was very little research in how households use and dispose of water resources. This prompted the creation of Ally, a chatbot designed to survey water use in Australia.

Role

My role as product designer included building a list of requirements, researching chatbot platforms, building the chatbot dialogue, developing the survey scoring system and testing functionality.

Project Scope

Project Length: 5 Months
Team Size: 2

Skills

  • Chatbot Development
  • User Requirements
  • User Acceptance Testing
  • UX Design

Tools

  • Tars Chatbot

View Life Project

We launched 2020

CLICK TO VIEW

PROCESS

The process for this project included: 

DISCOVERY

User Persona Workshop

Chatbot Platform Research

IDEATION

Initial Build

Feedback

DESIGN

Full Build

User Testing

DEPLOY

Pre-Release

Soft Release

Full Release

DISCOVERY

User Persona Workshop

To build empathy and understand the target market I ran a User Persona Workshop with the CEO of AWS-Asia. Together we developed the target demographic.

AWS Chatbot - Target Persona

Chatbot Platform Research

In my research, I found that most Chatbots are designed to be assistants (e.g., Siri) or made for simple tasks, such as ordering pizza. Very few could accommodate back end calculations needed for the survey scoring and multi-pathway nodes that allowed multiple responses in the same question.

Ultimately we chose to use Tars Chatbot. The system provided integration HTML code, chat exports, analysis of chat data and an internal scoring system that could be used for the survey. This covered most of the list of requirements generated from the client brief.

AWS Chatbot - List of Requirements

IDEATION

Initial Build

AWS provided a brief containing 13 questions with scoring metrics to develop a Minimal Viable Product (MVP). Users could qualify for platinum, gold or silver labels if they reached a certain score. The goal of the initial build was to construct the dialogue flow for the 13 questions and test the scoring system worked.

AWS Charbot - Initial Build

Feedback

The MVP had a major issue with the scoring system as multiple choice answers were not registering. To resolve this I worked directly with the Tars development team to generate custom code that compiled the scores.

DESIGN

Full Build

The full build expanded to 21 questions and included conditional responses to receive a “net-zero” rating. I redesigned the chatbot flow to encompass these new questions and built a conversational experience that was more personal and friendly.

AWS Chatbot - Full Build

DEVELOP

User Testing

To test the build we ran user tests with a small team at AWS. Using the feedback we developed a few variations of Ally to test different features.

These tests resulted in the following features:

  • To increase the goal conversion rate we introduced a skip question option to reduce users abandoning the survey.
  • For questions with multiple answers, we discovered users would make mistakes and wouldn't be able to correct them. I developed a repeat function to allow users to redo the question if necessary.
  • The scoring system was too hard with most participants not reaching the Silver level. I adjusted the system to balance the scoring.
  • I adjusted the timing of conversational nodes to enhance the user experience.

AWS Chatbot - User Testing

DEVELOP

Soft Release

A soft release was conducted with 100 users. Where possible users were interviewed for feedback.

The data resulted in the following changes:

  • Some questions were repetitive and annoyed users. Where possible these questions were consolidated to shorten the length of the survey.
  • The scoring system had become too easy from the last adjustment. A large number of users were scoring platinum. We adjusted the scores again to push users towards the silver range.

Once we completed these changes we proceeded to a full release.

Full Release

The full release occurred on the 22nd March (World Water Day). To preview the campaign click here

AWS Chatbot - Full Release

Complete

Results

Ally currently sits at a conversion rating of 80% and has been accessed all accross Australia. The AWS is now pitching the system to water authority bodies for a wider release.

Reflection

Having very limited experience with chatbots I wasn’t sure what to expect or if the end goal was even possible. Thankfully all the functionality came together and the AWS team was very pleased with the result. 

If I could do anything differently I would explore surveying techniques to optimise the interaction and length of the chat flow.

Want to chat about a project?