'More Stories Like This, Fewer Stories Like That'. The only perception metric CX Professionals need

Sensemaker is an approach to the collation of customer and employee/ stakeholder survey data via narrative (free text).


The basic principles are:


That narratives (known as micro-narratives or stories) are the way people communicate to themselves and others. If we want to understand ‘the experience the customer has’, understanding and measuring narrative is therefore the best way.


I agree with this after all ‘no-one ever walked out of a store saying that was a great 8.5 out of 10 experience’. And forcing them to do so in a 20-item survey, creates survey bias since:

  • you answer what the survey tells you to answer in a standardised way rather than what is important to you.

  • statistics are used to manipulate the data (to a certain extent) in order to justify this approach: challenge: take any column of NPS data and ask the standardised r squared against say spend, it will be about 5% and sometimes even negative with a confusion between correlation and causation and an 80%+ relationship to CSAT. (this does not make NPS bad, it is a cultural metric, I use it, but it does mean it is not objectively the one number to grow).

  • we use falsifiable assumptions that customers measure everything therefore everything is important

Stories are the way we communicate to ourselves and others. So, to be valid we must measure the story.


Stories are also stated frequently in a less concrete way than surveys would assume. Which means we miss how people communicate information if we take the view that customer talk is the same as the functional way we frequently position things in surveys i.e., a survey might say 'download speed' , a customer might say 'couldn't use this when I was on holiday in hunstanton'


We must also clearly ring-fence its limitations i.e., its not all about narratives. Customer experience is also about:


  • what customer’s do (not aware)

  • what customer’s could do (not known)

  • what customer’s are aware of but underreport (such as the extent I do X and how it impacts buying behaviour).

Also customer experience is not a mono-maniac approach, sometimes it is not the answer (which is why I believe it should be part of CI not a separate function but that’s for another debate).


That we must quantify and the best person to quantify a story is the person who wrote it.


I agree with this up to a point.


In many instances where processes and impacts are linear, we can use an algorithm i.e., 4 dropped calls predict churn; or everyone says the problem is ants in the room, so create an algorithm to pick this out


However, such a linear approach is myopic.


At the end of the day, NO algorithm can replicate: subjective understanding (subtext, metaphor, genuine feelings about a statement e.g., ‘I hate queuing outside this Apple store’ could be read by an algorithm as ‘hate’ Apple. In fact, the story could be titled ‘Love Apple’ and the sentiment through self-quantification as positive – yes I am investing my time but that’s because I love Apple.


By asking the customer to self-quantify, we get a100% valid response, guaranteed. By asking an algorithm we end up with weak validity on sentiment scores and emotion (not no validity). In addition, I note other key benefits of self-quantification:


  • We see how narrative impacts other variables such as: how queueing impacts my propensity to spend or my brand satisfaction. This relationship is hidden to an algorithm.

  • The importance of identifying outliers removed by algorithms and traditional statistical processing (i.e., a few people are doing X, if we amplified this message we would get a better result). This last one is important read: https://narrate.co.uk/2016/03/nudging-people-towards-latrines/. This also picks up what is known as exaptation: how experiences are being repurposed, The example here is from Singapore where a growing number of people faced with a monsoon were driving cars into large bags. They were exapting what they had for a new purpose. This data sees how things are moving


Sensemaker takes account of the psychology of the customer or employee in responding to the survey with special techniques


Too many people look at surveys as methodologically correct. Sensemaker is the only survey approach I see that takes better account of survey bias and tries to reduce it. It does this in the following ways:


Use of triads and stones: These are graphical approaches where the options presented are all either negative or positive i.e., to encourage a correct response, unbiased by what the consumer or employee ‘thinks’ the answer should be. With Stones we try to encourage more thinking, more consideration rather than a tick in the box response.


Use of extreme values: This is a scale approach that puts extreme values at either pole to encourage customers or employees to not ‘gift’ responses i.e., in a normal scale I see what 9 or 10 means so even though you were just OK I gift you 9.


Use of Titles: the method asks customers to reflect on their story and title it. Titles are a great way to highlight the key essence of what is important in a story.


Focus on the event: survey questions are put more concretely to the customer - less in the abstract i.e., not ‘do you recommend the brand’ but ‘if you were in a situation where you were talking to a friend…’ or ‘thinking of your last interaction’…


Obliqueness: asking a direct question tends to lead to an answer based on what you would like to hear. Survey questions tend to be asked obliquely i.e., rather than saying ‘rate quality of mobile signal’ it might be ‘thinking of the last time you phoned a friend, tell us a story’ (or words to that effect)… not I might or might not recall signal


Avoid Gaming of Data: One key aspect of using stories as a measure is that they are difficult to game? A customer literally has to change the story they write to change the scoring.


Enables better prioritization


Since customers quantify their own stories, executives don’t need to use analytics to make assumptions about what is important or not.


In other words, the analyst who brings their own prejudice to bear on statements and their own delay in reporting is removed. There is a direct link, data to alert if deemed important.

Maintains humans in the loop


The approach tends to highlight quantitatively (the customer/ or other quantifying what is important to them), the narratives of interest. This means that key statements can be quantified and prioritized i.e., in triage we can say this week these 50 stories are critical.


Critically, since narratives are usually quite fuzzy, we maintain human interpretation : what could this mean for our business on review of these 50 stories. This maintenance of humans in the loop is importance since it enable contextualization to the business, triage and an ability to leverage artisan knowledge.


Scans the horizon


As a free text approach, SenseMaker enables us to see where things might be going i.e., we scan the horizon for items needing to be amplified or reduced typically ‘on the fly’

Many items in a Bharti Survey were uncovered not on the list of survey response. This enabled us to scan for developing or outlier experiences that would have been simply missed by a static survey (or) see how these change over time (or) identify new items that if we only invested in them might become more influential.

An example of this is how ‘early billing’ and ‘rude staff on the phone’ were identified as key narratives: influential even though we were most concerned with mobile performance i.e., signal.


More stories like this, fewer stories like that


The key metric aims to amplify positive stories dampen negative.


This is a nice easy metric and focuses on what we are doing rather than in over statistical game playing ( as you get typically with NPS, CES i.e., did we raise it +10pts/ what are the abstracted drivers).


It is also difficult to game a narrative i.e., its easy to game the data in a matrix, but hard to change the story someone writes.


Stories still relate to what is of value to the respondent


Of course, stories must be related to the value of an experience important to customers


What does this mean?


Well if I go to the shops I might ask for stories against the key values of interest to the customer – is it convenient, is the quality of food good, is it value for money, was it easy to shop there.


Notice, how these values are not over abstracted (CES, NPS) but relate to the experience in question.


SenseMaker works well to reveal complex situations


Human situations are not bound by root-cause approaches alone: as you would assume in root cause analytics. Emerging causes are critical.


So, corporate culture is the continuously emerging outcome of our interactions with one another. You say something to me, I change my view of the culture and so forth. This interactional view, seeing customer experience as emergent, applies to Brand, Trust and so forth. So Brand is not a pre-set equation, it is emergent from the stories we tell each other of the brand in an ongoing and changeable fashion. This notion of managing to change is critical to Sensemaker.


Note: this does not deny there are aspects to experience that are more ‘set’ in stone. Just that you cannot and should not assume that things such as Trust, Brand, Culture can be delivered in a way akin to a construction site when so much of it is influenced by everyday interaction and how things change.


  • Heading in a Top-down only route to a new normalcy could cause inadvertent harm since it is clueless on how work really gets done in the organization. http://gswong.com/tag/narrative/. We can see this with the way HR tries to impose a general ‘trust one another’ culture, only to see that not reflected in behaviours and ruthlessly manipulated.

So, in a contact centre, to assume that employees can be manipulated top down to be customer-centric can do as much harm as good. It takes on board an approach divorced from the reality of an everyday contact centre i.e., sometimes the customer is wrong; sometimes our daily interactions do create antagonism; etc…. In other words, the context for customer centricity can be nurtured but it is never a dictat in these environments as in how you might build a bridge.


SenseMaker is all about getting close to the reality as far as you can on the ground.

This is a complex point: please review the Cynefin framework and Cognitive Edge blogs on complex adaptive systems.


Sensemaker is more aligned to customer experience


To engage customer sensitivity requires the application of metrics that are valid to the ‘experience the customer has’ and can swiftly inform us as to what that experience is and its directionality. Or at least get us as close as we can.


It is not the role of the CX professional to dictate ease, 6 pillars, NPS, CSAT or any other generic metric on to the experience the customer has; rather it is the role to listen and adapt. This is why in its openness I believe for perception measures SenseMaker works best along with Cynefin.


SenseMaker allows for multiple measures


It enables multiple departments to measure the same thing since stories around a value such as say ease are reflected within different contexts.


So, marketing may measure aspects of ease; finance also, and with the use of stories with numbers we understand the context i.e., not just 5/10 in marketing and 5/10 in finance but the contextual narratives that go along with it.


SenseMaker allows for multiple responses


It enables people to give mixed answers i.e., it does not force one answer or the other (which could be wrong). The use of triads enables us to give more accurate reflections:

Is my boss authoritarian, aggressive, a micromanager or N/A. I might answer a mix of all 3.


People understand it


Although it looks different, in fact from work with Bharti Airtel (n =1,300) responses were very quick and accurate


Narrative surveys give more colour to the quality of the experience


Its not web download speed, but download speed when I am in a rural location: all other times I don’t care.. narratives give colour which can also be used in brand marketing and as clues for how people use experience, what they care about and how we might innovate.


Flow charts are better illustrations for executives


The charts below are sourced to studies with Boeing and the NHS. They show two dimensions, for instance, stories that tell us we can have to break the rules to achieve positive outcomes.


If we take a close look at another client:


This is a story-generated 2D contour map from Safety Pulse. Each dot is a story that can be read by clicking on it. The red X marks a large cluster of undesirable stories – rules are being bent to get a low amount of work done. In our military analogy, we have soldiers on a hill but it’s the wrong high ground.

For illustrative purposes, the higher ground or new normalcy is the green checkmark where quality work is being completed on-time, on-budget, and within the safety rules. Thanks to the map, we now have our compass heading. The question is how do we make it easy to head to the higher ground? In other words, how might we get fewer stories at red X and more stories like green checkmark?

Highlights adjacent possible and looks for weak signals


From the chart above we can see that, the position we want to aim for is the top right. The stories in the top right are identified and could be interrogated (actions we could perform), the stories closest to the mass of stories but a step closer to moving he majority of stories to the top right are the adjacent possible stories that may be more replicable.


The continuous flow of stories is required for the dashboard and maps to maintain their near real-time value to manage the PM portfolio.


Establishing a human sensor network would also fuel the Early Detection capability [C]. Imagine being able to respond to “I’ve got a bad feeling about this” attitudinal stories well before they turn into “We wouldn’t be in this mess if someone had listened to me

Other links

https://narrate.co.uk/about/

https://www.youtube.com/watch?v=VKLw52K-zkc

https://www.ttec.com/articles/finding-strength-weak-signals#.WT1sCZDytPY

https://www.youtube.com/watch?v=dxnkjRrDLEQ

The Negatives


· Can be over-blown: by seeking to enable a scan for any aspect it can have too many triads and stones


· Looks a bit too different: initial responses are that it might put off consumers


· Difficult to get especially when talk about flow and that this is not about just defining root causes but looking for where things are heading ie. Complex adaptive systems. Execs like simple dashboards that over-promise, (we raised NPS +10pts). This is less the case with SenseMaker which is more likely to say : look at this flow chart, we need to move in this direction, this is what we are focused on doing next.


· Depends on Qual: the input and creation of triads and stones is dependent on good quality, Qual data


· Can be the same as normal narrative: many times narrative reflects a linear experience so existing forms of data work as well


· How do we prioritise something that may be important: the fact that something is growing in importance but does not adversely affect value or drive value now says nothing about if we should or should not do something about it. It therefore is ‘effortful’ in requiring humans to make decisions.


· It is academic: not the case but can appear so. There are good practical cases from Tony Quinlan but it is true that there is a strong degree of theory behind the approach that can detract from its value.


Results


Using a leading-edge tool from complexity, specialist’s Cognitive Edge, Ericsson sampled 1,378 customers in the Delhi region in India. The aim was to capture their perceptions of the mobile experience based on the stories they would tell a friend or colleague.

In unpacking the Networks relationship to NPS we found the following:

Finding 1: High market share leads to high NPS




A high NPS score of +33 driven by an unusually high number of promoters. This conforms to market theory that a high brand share, will lead to a high NPS score.

Figure 1 NPS in Delhi


At first sight this may feel like a market where, since most customers are already promoters, the job is done! However, we wanted to look behind the number and see whether the categories – Promoter, Detractor and Passive - related to the expected behaviour.

Finding 2: Promoters-Passives-Detractors do not always behave as people believe


When we unpacked these categories what we found was they did not relate to the expected behaviour.


Not all promoters promoted


Not all detractors detracted


And Passives frequently behaved in an undifferentiated way from promoters.

This is shown in the triad in figure 2.


In this analysis customers are given 3 negative or 3 positive options with a not applicable option. This makes them feel that they can answer in an open way. Each dot represents an individual’s personal and memorable story and how they rated that story; the positioning of the dot gives it a score out of 100 against the three options so a dot in the middle would score 33 since all three options are equally desirable.

Figure 2 Promoters don’t promote



Figure 2 shows how 35% of promoters stated that they would either ‘never, ever recommend the network’ or ‘never, ever recommend’ would form part of their opinion of the CSP.


Since, ‘never, ever’ recommend is a detractor option, we can assume that the Promoter score 9 or 10 out of 10 is soft. Particularly as the n/a option did not attract the expected high response.


Figure 3 Detractors don’t detract


A similar result was uncovered for Detractors. Although this time the result was that 62% did not record a ‘never, ever recommend’ answer either in part or in whole. We can assume that the Detractor score 0-6 is soft. Particularly as the never, ever recommend option did not attract the expected high response.


The reason for this finding is that we have picked up how customers will when faced with a scaled survey tend to just put a tick in the box based on general brand dominance. But when they are enabled to think about their behaviour in an unbiased way, we get the real story!


The management implications of this are that the business needed to move away from a simple focus on pre-set categories. For instance, throwing away the response of passives is fundamentally flawed when their behaviour is similar to promoters.


Likewise, it is not a question of constantly focusing on raising the NPS number when promoters act in such an unexpected way: After all more of a hygienic service may mean nothing at all!


In this market, the focus should be therefore less on the number and more on targeting actions that change the dynamic. Hence, we move to using the stories to uncover what practical KPIs and interventions we should put in place.


And critically test these out. This may mean a technology intervention; it may mean a marketing intervention or a cultural change or a bit of all 3.


Finding 3: More stories like this, fewer stories like that KPIs


So what of the 1001 things should we focus our actions on?


Fortunately we have answered this in the dataset, because we have uniquely got the customers themselves to ‘self-score’ (self-signify) how their stories correlate to NPS and a series of other dimensions such as coverage, reliability, ease of use, value for money and so forth. In other words we come back to the concept of having ‘mass qualitative data quantified’.


This is important not just because the quantification enables us to focus on the ‘as is’ stories of importance. But also because stories indicate possibilities (the law of hunches) and give detail to what customers mean when they something like ‘reliability is important’.


Hence in unpacking NPS we use the triads to understand how important their stories are in terms of the dimensions most correlated to NPS.

For instance, in figure 4 the story rating is against reliability, value for money and ease of use.


Reliability is most highly correlated to NPS. Hence, we unpack all the stories that relate to a high score here (note this does not deny the power of internal innovation, we are just following the data clues given by customers).


We use the stories to provide actionable content around what this means. Our metrics now go beyond NPS.


Figure 4: Network perception targets



The triad is shown to customers.


1. It gives 4 options (inc N/A) that do not create bias (we do not say one is negative and another positive)

2. It allows customers or employees (etc) to rate how they feel even if it is a mix of things

3. It allows easy eyeballing of where the stories are placed

4. Each dot is a narrative and can be interrogated

5. The positioning of the dot gives a score

6. We can see which stories are related where (see also Flow Charts are Better Illustrations for Executives)

Figure 5. Customer experience perception of reliability

For executives:


Executives are used to traditional style move the dial linear reports. We saw earlier how we could use flow charts but the same data can also be shown in a more traditional format.


Example:


From this data, we set a ‘more stories like this, fewer stories like that’ metric for reliability. This is shown by boundaries i.e., any reliability score must fall within thresholds, we look for those that show movement towards the edges. If we are falling towards the lower boundary (more negative stories, dampen them) or towards the upper boundary (more positive stories, amplify them).


The example above shows the format of the KPIs reported.


The aim is to move the mean reliability to a higher level marked by the arrow and set by the experience KPI ‘more stories like this, fewer stories like that’ i.e., more customers report reliability as an important experience in their story.


Once we have established a metric, we still have to determine interventions to change it. But here we also have a rich set of data since we have captured all the stories that score highly on reliability! This means we can mine them for actionable content.

It should be remembered that some of this content may not be about network reliability directly. For instance, customer service may be a key talking point within the stories. This may influence how reliability is perceived: bad customer service, I’ll mark reliability down as well. This is one of the key values of SenseMaker – we scan the horizon for indirect influences.


Setting perception metrics up like this also gives a platform for pre and post-tests to see if an innovation in the customer experience has moved the dial or not.

Finding 4: We can measure S-KPIs more accurately


So far we have spoken about customer stories. But we can also use this approach to ‘unpack’ the customer experience in a more detailed way.


For instance, we also looked at customer perception of more detailed aspects of the mobile experience such as video and web download speed and file transfer. Things people normally wouldn’t talk about unless it goes wrong.


The problem with these types of experiences is that just asking a scaled question will usually lead customers to tune out.


They will answer the question but really be responding based on brand.

They will also reinterpret the question based on their understanding of the application as a whole rather than specifics such as speed.


Example: I like your brand, I give the same score to this other stuff like download speed. Hence, a CSP may find itself getting decent 8 out of 10 score but this is a false positive if the respondent is really answering on the basis of brand.


We found that it was possible to get a better understanding of not just the score but the importance of that score, if we ensured customers thought a little more about their answer. This is why we used something called Stones. This is shown in figure 6 and displays in this case customer’s opinion towards an application – here it is web browsing.


Figure 6. Web browsing

What we can see from this is that customers tend to score a mid-range 5 out of 10 with a coalescing group of consumers in the strongly recommend area. These we would call Network Sensitive’s.


This notion of coalescence is important since it dynamically shows where the market is heading; a slightly different concept from segmentation which tends to show post-hoc where it has arrived!


This method also provides for high level of data quality, since we are more assured that customers are thinking about the specific application of interest; this has been achieved since they are made to think of 2 dimensions rather than autonomically just tick a box.

Management Implications


1. Net Promoter like all attitudinal scores tends to be quite general and makes broad assumptions of consumer behaviour. Customer experience executives must unpack the score to get to the actionable content and use this content as the basis for a perception metric. They need to get more concrete to the situation.


2. NPS and other attitudinal scores assume that more of NPS is good. This is not necessarily true. Innovations can shift the line so you get more value for the same score or even less: think about how 10 years ago checkout rep attitude may have been 10% of satisfaction with shopping at Morrisons. Now it is 0% with self-serve. Is that bad? Nope. We have scanned the horizon, found innovations that mean we can get more value and customer satisfaction with less of something.


3. Since the currency of customer experience is customer stories, these must be captured in a way that minimises cognitive bias.


4. Since the unit of measurement is the human mind which is highly subject to bias, this is a critical feature of any dataset. Don’t just accept scaled responses as fact.


5. Use this method to understand Network Sensitive groups, some of which you may not have noticed before. Particularly if you have bought into a pre-canned segmentation


6. Always drive action out of the dataset. This doesn’t necessarily mean something such as reliability or coverage always relates one to one to these aspects of an experience. For instance a bad customer service experience may also lead to a customer reducing their reliability or coverage scores. We can understand these effects by understanding the stories.

0 views

© 2023 Website created by cophus.com