Tone Detector

Whispir

Help marketing managers craft meaningful messages for customers by analysing tone of voice

Role

Design lead, ideation, testing, refinement and build

OUTCOME

A core feature of Whispir’s Message builder

background

Whispir’s first Intelligent Message builder feature

The product vision at Whispir is to send 10x more effective than the competition. As the design lead on the Messaging Team, my goal was to deliver an intelligent feature to help users design content that is engaging and well received.

Our Product Manager, carried out the initial discovery work and talked to marketing managers to find out JTBDs before I started on the team. The job to Increase Audience Engagement was selected as it was an important task, although there was low satisfaction of completing the job at the time among competitors. This was an opportunity to address an underserved need in the market.

Approach

discovery

To understand how other competitors Increase Audience Engagement I conducted an in depth analysis of direct but mostly indirect competitors. I found a number of possible features, such as messages scheduling at the right time, best channel prediction, auto completion of text in emails, A/B testing and evaluation, sentiment analysis and analysing tones on copy.

Given the platform was undergoing some infrastructure upgrades, we strategised to use an off-the-shelf AI model that didn’t depend on the upgrades. We prioritised the feature to analyse the tone on message copy.

Competitor analysis

future of tone dectector

I explored more ideas to unlock value in analysing the tone of voice of messages. In reading up on Natural Language Processing, if we combined the tone analysis score on message copy (input) to the success of the messages (output), we could identify:

Recommendations to change a brand’s tone of voice or
New segments that respond better to similar tone groupings

Design & Concept test

2 concepts x 5 user interviews

To understand user needs and test out Tone Detector flow in setting up and utilising the feature. All participants were Marketing Managers working in a range of industries, who use a number of different messaging channels to communicate with their customers.

The main takeaway was

  • People wanted to see precisely where to alter their messages at a sentence level, to better align the overall score with their brand voice.

  • They also felt overwhelmed with the bracket and % breakdown. They wanted a simpler way to understand what was successful and what was not.

Blockers

We faced cost constraints in using the desired off the shelf product tone analyzer recommended by the AI team, as each call via the platform would be charged. So it was decided to go with an open source AI language BERT base model due to the costs. This meant that we needed to work with the tones offered.

How I solved it

In light of the intended AI model cost constraint, findings from the interviews and concept tests, I had to go back to the drawing board, but this time with the design priciple to easily analyse copy.

From Usability Tests of the new design, it was found that marketing managers wanted flexibility in setting desired tone types that may differ from the brand tone. For instance messages around billing that should be more serious.

Although the tones are not the most desired tones to analyse messages, in using the Bert model, the AI team was able to start training their own model. This meant that the tones can be altered in future iterations.

In addition to this insight, the design validated that the feature was now easily understood and would be valuable in message creation.

Setting up success measures

Once the design was finalised, we needed to track the success of the new feature. I set up a few hypotheses with the team to test on Segment. We had two hypotheses:

steps

  • Competitor Analysis

  • Design and Prototype

  • Concept Test and findings

  • Feasibility

  • Iteration

  • Usability Test

The feature had launched a few weeks before I left, more data is needed to validate the hypotheses.

HMW capture desired tones for further iterations?

Solution

Tone detector

HMW help marketing managers create messages that align with their brand?

HMW improve discoverability of the new feature upfront?

project learnings

I transitioned to my role at Whispir during the pandemic with the intention to challenge myself to develop new skills in interaction design amd prototyping. There was a time constraint in picking up the tools although I have achieved these new skills.

In addition to the UI skills I’ve developed, I have learnt a lot about myself in working in a fast-paced environment such as Whispir. I thrive in a collaborative environment when aligned with the product manager.

What I would have done differently

  • Have a stronger voice for the customer by conducting more in-depth user interviews at the beginning.

  • Test early and often, this goes for hypothesis or concepts.

  • Request prioritisation and avoid having 4-5 internal projects on-going simultaneously, this was a large stress on the team.

  • Consider telemetry as part of the design process in ensuring all interactions are trackable

  • Question the validity of prior research and try to have more say in the discovery process. This meant having the difficult conversations about the value of the Engagement Score and it’s promise to the board.