Tone Detector
Whispir
Help marketing managers craft meaningful messages for customers by analysing tone of voice
Design lead, ideation, testing, refinement and build
Role
Outcome
A core feature of Whispir’s Message builder
Background
Whispir’s first Intelligent message builder feature
The product vision at Whispir was to send 10x more effective than the competition. As the design lead on the Messaging Team, my goal was to deliver an intelligent feature to help users design content that is engaging and well received.
The research team uncovered the insight that the task of increasing audience engagement is a priority for marketing managers, yet competitors offered low satisfaction in helping them achieve it. This was an opportunity to address an underserved need in the market.
Approach
Discovery
Competitor analysis
To understand how other Saas platforms Increase Audience Engagement. I conducted an in depth competitor analysis including direct and indirect competitors. I found potential features, such as
messages scheduling at the right time,
best channel prediction,
auto completion of text in emails,
A/B testing and evaluation,
sentiment analysis and
analysing tones on copy.
Given the platform was undergoing some infrastructure upgrades, we strategised to use an off-the-shelf AI model that didn’t depend on the upgrades. We prioritised the feature to analyse the tone on message copy.
Future of tone dectector
I explored more ideas to unlock value in analysing the tone of voice of messages. In reading up on Natural Language Processing, if we combined the tone analysis score on message copy (input) to the success of the messages (output), we could identify:
Recommendations to change a brand’s tone of voice or
New segments that respond better to similar tone groupings
Design and concept testing
2 concepts x 5 user interviews
I conducted a research piece understand user needs and test out Tone Detector flow in setting up and utilising the feature. All participants were Marketing Managers working in a range of industries, who use a number of different messaging channels to communicate with their customers.
The main takeaway was
People wanted to see precisely where to alter their messages at a sentence level, to better align the overall score with their brand voice.
They also felt overwhelmed with the bracket and % breakdown. They wanted a simpler way to understand what was successful and what was not.
Blockers
We faced cost constraints in using the desired off the shelf product tone analyzer recommended by the AI team, as each call via the platform would be charged. So it was decided to go with an open source AI language BERT base model due to the costs. This meant that we needed to work with the tones offered.
How I solved for it
In light of the intended AI model cost constraint, findings from the interviews and concept tests, I had to go back to the drawing board, but this time with the design priciple to easily analyse copy.
From Usability Tests of the new design, it was found that marketing managers wanted flexibility in setting desired tone types that may differ from the brand tone. For instance messages around billing that should be more serious.
Although the tones are not the most desired tones to analyse messages, in using the Bert model, the AI team was able to start training their own model. This meant that the tones can be altered in future iterations.
In addition to this insight, the design validated that the feature was now easily understood and would be valuable in message creation.
Setting up success measures
Once the design was finalised, we needed to track the success of the new feature. I set up a few hypotheses with the team to test on Segment. We had two hypotheses:
The feature had launched a few weeks before I left. Although, it has now become a core feature of the Whispir message builder.
Solution
Tone detector
HMW help marketing managers create messages that align with their brand?
HMW improve discoverability of the new feature upfront?
HMW capture desired tones for further iterations?
