In this post, we will focus on the experience users have with passive metering and surveys. We asked our respondents a variety of questions about different aspects of both data collection methods in order to get an idea of how their experience can be improved and how we can prevent a further decline in response rates.
Research results
We started the survey with a very broad question in order to get a sense of people's general motivations for participating in market research.
Why do you participate in market research?
Note: Multiple answers were possible
What we found in the answers is that incentives are still the overwhelming reason for people to participate. However, many people also indicate that they think their opinions and behaviors can have an impact on society, by giving them a voice.
The panelist were also asked to rate their experience with both surveys and passive metering, giving them a grade from 1 to 5.
How would you rate your experience with passive metering/surveys?
Although the results were close, more than twice as many people rated passive metering higher than surveys.
The participants were then asked follow up questions about their preferences to specify why they prefer one method over the other.
Why do you prefer surveys/passive metering?
For people who prefer surveys, the answers were mostly related to being able to share their opinion directly, as well as having the chance to choose the things they want to share. In the 'other' open answer category, they also indicated that they felt there were more offers of surveys than opportunities to be passively tracked, which is why they prefer surveys.
In terms of passive metering, people chose it as their preferred method because it is low effort/high reward. They indicated they feel they get better rewards with passive metering for the amount of effort and time they spend on it, and in the open answer they mentioned it was preferable because they could do it while working/continuing with their daily life. They also indicated that they can’t always remember everything they do, and so when they are asked to recount specifics in surveys, they aren’t able to answer correctly. With passive metering, they don’t have to remember their own behavior, because we can see their activity instead of having to ask for it.
The next questions aimed to understand what the participants think can be improved for both data collection methods.
How can surveys be improved?
Mostly, our panelists indicated that they wanted surveys to have better rewards and be more efficient. They also mentioned their frustration at not qualifying for enough surveys and therefore not being offered any, and being screened out during a survey.
How can passive metering be improved?
Mostly, participants felt that passive metering could have better and more rewards. Generally, they also wanted more control and clarity, such as making the tracker easier to install. In the open 'other' answer, many people suggested adding a pause or stop button to the tracker, and making it possible to turn it off for a short while if the participant wants to. However, this is already possible, suggesting there could be some improvement in how passive metering is communicated to the panelists.
Since privacy appears to be an important topic with participants who use passive metering, we decided to also ask how protected they feel in the general online environment as well as being passively measured for research purposes.
Do you feel that your privacy is protected in the general online environment?
Their answer was mostly ‘unsure’, and 23% of the people definitely did not feel safe.
Do you feel that your privacy is protected with passive metering?
When we asked the same question about passive metering, almost 2.5 times more panelists said they felt passive metering does protect their privacy compared to the ones who indicated they don't feel their privacy is protected.
When we asked the 28% who didn't feel protected why this was the case, there were some concerns generally about sharing information and private activities. Some people even said that although they didn’t feel their privacy was being protected, they also didn’t mind.
For those who felt like their privacy was protected, the most frequently chosen reason was that they felt the data was anonymous. Many participants (25%) also said they forgot the tracker was even installed, and that they understood about what the data was being used for, so they weren’t concerned about breaches in their privacy.
Interestingly, some participants indicated that they felt protected because they could turn the tracker on and off whenever they wanted. This is in contrast to those who indicated, while being asked what they would want to change about passive metering, that they would like the option to turn the tracker off.
Conclusion
From the early planning stage of selecting the data collection methods to the actual experience that people have during the execution, there are a lot of interesting take-aways from this study that can have an impact on the research results.
It's crucial to carefully choose the appropriate method for collecting data to get a better understanding of today's consumers and to reveal the insights you are looking for. Since people can’t correctly report their online behavior, passive metering will deliver more accurate data than survey data if the purpose of the study is to find out WHAT people are doing online.
A combination of both methods can also help to assess WHY people behave the way they do. The fact that people have a better experience with passive metering than with surveys, but still want to have the option to express their full opinion, can also be addressed by combining both methods.
Moreover, it's important to respect and appreciate the efforts of research participants by improving their experience. The main reason for people to participate in market research is still rewards, which might lead to the conclusion that the more attractive the incentives are, the happier the participants are with their participation. However, there are also other factors that affect the experience consumers have when participating in market research e.g. the way the data is collected in terms of usability and privacy or the effort it takes to receive an incentive.
To avoid misconceptions regarding the data protection, we see that the communication about the privacy of passive metering needs to be improved to ensure research participants are aware what data is collected for which purpose.
The participants' demand for more efficient surveys as well as their frustration when being screened out, can be addressed by using passive metering for targeting a specific survey sample. This way of sampling can improve the relevance of the survey, so the experience they have with it and prevents them from dropping out. Recent research has also indicated better targeting increases participation rates, reduces dropout rates, improves survey experience and yields the same or better survey data quality.
To conclude, we think there are two primary factors to always keep in mind when conducting a research in order to succeed in the future: Data quality and user experience.
