What Works Fund: results
24 Nov 2017 | Written by Laurence Piercy
People learn more about money if they spend their own money online.
The project was designed to test this research question:
Are individuals receiving financial capability support better able to transact online if they are also supported to undertake a live transaction online?
We now know that the answer is:
This means that, if you help someone to transact online using their own money, then they are more likely to transact again on their own. This is a very simple technique, but it can change the way in which people embed financial capability.
Transacting online tangibly saves money. Saving money increases a person’s disposable income, which makes them more financially resilient. Online tools also give people more choice, which makes them less vulnerable to expensive goods in their local shops. This is one of the ways in which online transacting helps to address the poverty premium.
Creating the best evidence
Transacting online is a big step for some people, and it is hard to teach it as part of financial capability. For these reasons, it was important for us to prove that it is worth it, for learners and the people who support them.
To create the best evidence we could, we ran the project as a Randomised Control Trial (RCT). An RCT compares two sets of data, the test and control groups.
- Test: the group of people who received the intervention
- Control: the group of people who did not receive the intervention.
An RCT is the gold standard for data comparison, because it randomises which participants will be in the test group, and which will be in the control. By randomising, the model tries to remove bias. These are the factors which are not contained in the intervention, but which might have an effect on how effective it is.
An RCT can say whether an intervention works. Removing bias can help us to say whether an intervention will still work if we run it in the same way but in different places.
The intervention was designed to test our research question. It worked like this:
- Learners attend financial capability classes for 8 weeks at an Online Centre
- After 4 weeks, participants choose a transaction which they would like to complete. The transaction should save them money on something which they would have bought anyway
- Someone in the Online Centre helps them to go through the transaction, one-to-one
The test group did this supported transaction in week 5 of an 8-week financial capability course. The control group did not do a transaction, but they did do an 8-week financial capability course.
Participants filled in surveys at the start and at the end of the project. Through these surveys, participants rated themselves against a range of skills and attitudes.
We looked at five key measures. For each of these measures, the intervention created a bigger effect than the control. The measures were:
- self-assessment of capability
- consumer behaviour
- financial outlook
- digital inclusion.
In the control group, participants reported scores which had improved by 12–22%. In the test group, participants reported scores which had improved by 18–38%. The difference in improvement between test and control groups was significant across all these measures.
We also looked at whether different groups of people experienced the intervention in different ways. We found out that there was very little difference between groups. The change that the test intervention produced was the same, whether or not the participants were male, female, BME, or White British. This is a good sign. The supported transaction is generally effective. It’s effectiveness is not limited to a specific demographic.
Likelihood of transacting in the future
We looked at the likelihood of participants transacting again after the supported transaction. This effect was modelled against those participants who had not transacted online before, at the point of the pre-survey.
Our statistician then analysed these results using something called an odds ratio. If something has a high odds ratio, this means that the presence of one property is very strongly associated with the presence of another property.
In this case, we are looking at the association between the intervention and the likelihood of further transactions. This allows us to say that:
People who were part of the intervention were 6.5 times more likely to transact again if they were supported through the test intervention.
We now know that, if you help someone to transact online, then they are more likely to transact on their own in the future. This is a simple technique which changes people’s online behaviour.
If a person doesn’t know how to save money online, then they are more subject to the poverty premium. The most up to date research confirms that digitally excluded people pay more for essential goods.
A supported transaction embeds the skill to buy online. By embedding skill, it empowers people to save money by changing the way in which they buy things. It is more effective than giving someone information alone. We think this is big news, and now we’re telling everyone about it.