About

Populus
Northburgh House
10 Northburgh Street
London EC1V 0AT

Tel: +44 [0]20 7253 9900
Fax: +44 [0]20 7253 9911

Connect with delayed passengers in real time using social media and app-based research

By: Andy Barker (Head of Qualitative Research, Populus) and Keith Bailey (Senior Insight Advisor, Transport Focus)

Imagine…

You finish the remains of your latte and put down the paper cup. The caffeine is starting to have its effect and you become conscious that the train is slowing to a stop. It is actually quite a nice morning out there now that you are aware of it; sun shining, rabbits hopping in the fields. Was that a deer disappearing into the woods…?

 

Delayed-Disrupted-01

The challenge

Transport Focus is the independent watchdog for rail, bus and tram passengers and users of the strategic road network. It is an evidence-based organisation looking to influence government and transport operators to do the best for passengers. It interviews 60,000 rail passengers each year to monitor their satisfaction and highlight where improvement is most needed.

Punctuality and reliability are key requirements, but when things do go wrong, the provision of information is critical. Keep passengers informed; provide them with information so they can rearrange their day; use a genuinely apologetic tone — this all helps to ameliorate the situation. Say nothing, and passengers think the train company does not care; they begin to lose any trust that they might have had in the company.

The rail industry has a Code of Practice (published by the Association of Train Operating Companies) for what information should be available and how it should be provided when passengers’ journeys are disrupted. In 2014, the rail industry regulator, the Office of Rail and Road (ORR), was planning a review of the code and how it was working in practice for passengers. It asked Transport Focus to undertake a programme of research to look at passengers’ actual experiences of disruption and how this was being handled. In turn, Transport Focus commissioned Populus, one of its research agency partners, to conduct research among passengers to provide the evidence to inform the ORR’s review.

The research

The research had at its core an online survey of 1,020 passengers who had reported disruption to their journey in the previous week. This indicated, among other things, that just 22 per cent of passengers recalled hearing an announcement within two minutes of their train coming to an unscheduled stop — as stipulated. It also showed how frustrated — and above all, powerless — passengers feel in such circumstances.

While the quantitative survey provided useful hard data on what was happening, it could never adequately portray passengers’ full experience of the disruption nor explore what might have been done to improve the situation. Populus undertook further qualitative research to bring additional colour to the hard data. The obvious approach was to conduct focus groups where six to eight passengers could discuss their experiences and where the experiences of one passenger could be used to explore others’ attitudes and spark cross-fertilisation of ideas for what would make a constructive difference for passengers as a whole.

But even with focus groups, there was a risk that passengers would be too relaxed, sitting in comfy surroundings, cup of coffee to hand, with all the aggravation of the disruption starting to fade with each passing day. Much better to get to them ‘in the moment’, while thoughts and passions were fresh in their minds — but how? Cue a small social media experiment.

Using social media

The methodological challenge was to capture an experience that was unpredictable and thus hard to sample in the usual way (ie approaching people who might have already had an experience and asking them to talk about it). Transport Focus therefore turned to social media (a) as inspiration for a method that would enable people to share their ‘real-life’ experience and (b) to sample people.

To capture authentic experiences, thoughts and feelings, a mobile web-app based qualitative feedback mechanism was created to encourage passengers to share their feelings about the disruption they were experiencing, using the bite-sized posts, mobility and in-the-moment immediacy of social media. Transport Focus and Populus then scanned social media for a month (in winter, when disruption was more likely) and approached people who were actively posting about the disruption they were experiencing (eg tweeting) and sent them the web app link to invite them to share further details. As is customary in in-depth qualitative research, an incentive was offered (in this case a £10 payment for taking part). Over 50 participants were successfully secured using this method.

The link to the web app was also shared with a wider panel of rail users, who were asked to share their experiences of any disruption to their train journey. Over a one-month period there were 175 responses from this group of rail users — a substantial sample by qualitative standards — providing a rich data set for analysis.

The questions asked were generally open-ended, for example, ‘What is happening now?’ and ‘Can you describe how you feel right now and why?’ Participants were also asked what their information needs were at that point in time, following up the question ‘Please describe in your own words what you have been told’ with ‘What else do you need/would you like to know that you haven’t been told so far?’ and ‘What could the train company have done differently, in terms of information, to improve your experience?’ Participants were also asked to share what was happening over time, so a proportion of the contributors reposted as the disruption progressed.

Showing a difference

The research was unusual in capturing passengers’ unfolding stories and mindsets in real time — most research is conducted post hoc and captures people’s sometimes unreliable recollections of what happened, how they felt and what they needed. More ‘traditional’ post hoc focus groups with travellers who had recently experienced disruption was also conducted. A comparison of the ‘tone of voice’ of their responses versus those from the social media approach showed that the emotional intensity of the in-the-moment disruption experience was captured more authentically in the latter case. This was more useful for research purposes as it indicated how people ‘really feel’ and thus what they ‘really need’ as they experience an event such as a disrupted journey. As an illustration, compare the following quotes — the first three captured via the social media inspired method; the second three from traditional focus groups:

In the moment

‘I’m annoyed and p***ed off that we’ve not been told why we’re delayed. It’s usually due to a late running train on the main line but this time it’s longer than that.’ (App comment)
‘I can’t tell you how much it annoys and ruins your day. Plus, we are paying a huge amount of money for this sh*t.’ (App comment)
‘I feel violated. Because I am powerless.’ (App comment)
‘Are connecting trains being held? What platforms will they be on? And what is the reason for the delay?’ (Focus group comment)
‘They should have provided up to date information as soon as they had it. It is very frustrating to have no information.’ (Focus group comment)
‘If [a poster] told me that they were doing something about it, then that would make me happier I suppose.’ (Focus group comment)

From a research innovation perspective, the qualitative social media inspired experiment was deemed to be a success, but what contribution did it make to the project as a whole?

The social media element was a small but valuable contribution to the overall project. This has shown that as a first step, the industry needs to adapt its model for information delivery to reflect what passengers need to know. The current model is one of problem > impact > advice, whereas passengers indicate that their first concern is the impact — will they get to work today? Will they be waiting ten minutes or two hours? With the impact clear, the next requirement is for advice — particularly if this is not a regular journey for the passenger. Is there a bus I can take? Can I perhaps drive to another station and get a train from there? Can I get my money back if I choose not to travel? Only when these immediate needs are satisfied does the passenger start to wonder ‘why?’ It may be useful to be able to explain to the passenger’s boss — or wife — but it is of no immediate value to the passenger him or herself.

Nor is there any reason why individual, tech-savvy passengers should have more detail through Twitter and the like than staff have. Whatever the medium, the information needs to be consistent, accurate and delivered with an appropriate tone. On train companies’ websites, if there is major disruption on a given day, this should be the first information passengers see — not an enticing promotion of cheap tickets for those unable to travel… Furthermore, staff are crucial in relaying information and it is to be hoped that with more training coming through, more staff will be prepared to pick up a microphone and explain the situation in their own words rather than using stock phrases or, worse still, just pressing the button for those stock phrases to be enunciated by a robotic voice incapable of sounding remotely apologetic.

Role of social media research

The findings of the research programme have enabled Transport Focus to prepare a document containing various recommendations for improvements that train companies and Network Rail should make. These address not only the information provided on trains during disruption, but also information screens and announcements at stations, as well as online and through social media where there is a chance to reach people in advance of their journey. The code has now been revised and reissued. In addition, a 40-point industry action plan has been agreed, and there are positive indications that train companies are starting to implement this.

Clearly, social media can provide rich insights to both researchers and their clients. However, the question of exactly how to address and approach individuals ‘in the moment’ when they are experiencing whatever it is we want to research remains a challenge. As a case in point, the mobile network operators can see when they have a couple of hundred customers stationary in the middle of a field for 40 minutes, with calls and other communications increasing by the second, but without advance permission, they cannot connect us to them. There are apps aplenty and any number of startups offering mobile ‘solutions’. Beacons and other clever technologies may enable us to reach out to users who have agreed to talk to us, but so often, we do not know who we want to talk to until they engage in some particular transaction or activity — and we may well not be able to ask permission at that point in time. There is still a way to go before mobile research becomes the preferred means of collecting customer feedback.

 


Back to previous page
Webflow to WordPress theme development by whois: Andy White