Our method

The large general surveys about what Swedes do on the Internet do not manage to reach enough people for the relevant diagnoses, impairments and difficulties to be represented.

In order to be able to say something about people with different impairments, it is a prerequisite that these people are represented in the collected data. Of course, a possible solution is to increase the number of participants so that the sample is large enough, but such large surveys are difficult to finance.

Shadow survey

Instead, we have developed a methodology that we call "shadowing" the broad population surveys. The idea is to use other selection methods to reach the groups that are not represented in the broad surveys and ask the corresponding questions to these groups to see if there is a gap between these groups and the entire population.

We also want to investigate if there are differences between different groups and within each group.

Construction of available questions and answer alternatives

Before we constructed our own survey, we analyzed and tested the questions and answer alternatives found in Internetstiftelsen (The Swedish Internet Foundation) and SCB's (Statistics Sweden) surveys.

We discovered that some of the questions used were too difficult. Even the answer alternatives are experienced many times to be difficult. We also noticed that the amount of questions asked created an accessibility problem.

We have therefore repeatedly processed questions and answer alternatives to produce questions and answer alternatives that can be handled by as many people as possible. We have learned that it is important for people with mental and cognitive impairments to be able to participate in surveys. Although we are relatively satisfied with the design of our survey, we know that it can of course be improved. We also do it step by step.

We have been generous with the opportunity to leave your own answers by having prompts to describe things in your own words. For certain people, it is easier to express yourself more freely or to supplement an answer via a fixed answer alternative with a free text answer. We have learned that it is good that there are several ways to share your opinions.

Some examples of what can be difficult:

• Question about how often you do something: Estimating how many hours a day, per week or year you do something specific is difficult for many.
• Questions about when you last did something: For example, to remember if you shopped online in the last quarter.
• Scales with many steps: Many people find it difficult to place their own opinion on a 5, 7 or 10-point scale.
• Questions that are illogical: For example, having two questions in one or having something that is implied or not connected creates great difficulties.
• Questions or answer options that contain difficult words: For example, we have discovered that some people have difficulty with the concept of the Internet. Then examples and explanations are important. Or in cases where it is possible, we replace difficult words with ones that are easier to understand.
• Questions or answer options with negations or double negations: The little word "not" completely changes the meaning, but many people have a hard time understanding when this happens.

Methods for data collection in an accessibility perspective

When we carried out tests on how we could collect data about the Internet among people with impairments, a consistent comment was that both the The Swedish Internet Foundation and Statistics Sweden's surveys were too extensive.

These surveys were previously conducted as telephone interviews and consist of several hundred questions. Nowadays, participants can choose between answering a web survey or being interviewed over the phone.

Many people in the groups we want to investigate are unable to answer so many questions. Others may find it difficult to answer over the phone. We investigated about 20 different possible ways to collect data in the Begripsam group (which consists of people with cognitive impairments).

Participants were asked to rank each collection method in the categories

• Yes, please
• Yes, it works
• Rather not
• No
• Do not know

We then found that by combining 3-5 different ways of collecting data, the chance is quite high that one of these ways will end up in the category "Yes, please" or "Yes, it works". As soon as the person who wants to do a survey only chooses one collection method, a large number of people with cognitive impairments fall away.

In our survey SMFOI, participants were able to answer via

• Survey, online
• Questionnaire, paper
• Telephone interview
• Visits by data collectors

We have sought out certain groups, for example people in nursing homes, daily activity center and at association meetings. We have then been able to provide personal assistance to those who need help filling in the survey. This outreach activity is resource-intensive and results in slow data collection, but is necessary for certain groups to be represented in the material.

Selection and representativeness

The broad surveys of what the population does on the Internet try to achieve representative samples through so-called stratified random samples from registers of the population. This sample consists of approximately 3,000 people. With those methods, relatively small samples can be used to make statements about quite a lot and with a fairly high degree of certainty about things that affect large groups in the population.

But those methods are worse when it comes to people with impairments. Of the people selected, Statistics Sweden and the Internet Foundation have asked the question whether the selected individuals happen to have an impairment, and in that way got a hold of a number of people who then happen to represent different types of impairments, several of whom do not really create any difficulties specifically with using the Internet .

A consequence of this way of getting hold of people is that several types of diagnoses and impairments are not represented at all, while others are represented in very small numbers. The impairments that are likely to be represented are the groups with impairments that are relatively large. One such example is people with reduced mobility. It's a large group but the only mobility imapirment that is relevant when we talk about using the Internet is perhaps difficulties with fine motor skills. Other mobility impairments probably do not affect how well or poorly it goes.

We do not have access to registers where we can make representative selections of people with various impairments. For certain diagnoses, such registers may exist within the healthcare system, but as information about people's possible diagnoses and or functional limitations are considered particularly sensitive information about health, worthy of protection. They are surrounded by strong privacy protections and restrictions on who may use them.

For many groups there are no records at all. If you want to reliably reach people with certain diagnoses or impairments, you either have to scale up in very large samples of the entire population or find other recruitment methods. In this study, we have applied other recruitment methods.

Recruitment methods

In order to reach people with as many different diagnoses, impairments and difficulties as possible, extensive recruitment work is required. Begripsam has extensive and close cooperation with a number of different organizations that represent people with various impairments. We have partnered with these to reach out to their members and then encourage members to refer other people they think might be interested in participating in the survey. We also have extensive collaborations with networks, groups and individuals. We collaborate with organizations and companies that, for example, organize daily activities or offer various forms of group and support accommodation. In the end, we have used social media and mainly Facebook to reach out.

Many have helped in passing on the information about the survey and in this way we have reached out to the people who chose to participate in the survey. People who took part in the survey have also tipped other people. With such recruitment methods, we do not get representative samples with certainty.

A possible shortcoming is that we have found it easier to reach users of the Internet compared to non-users. Our survey focuses on whether using the Internet is difficult. This makes it difficult for us to comment on the people with various diagnoses and impairments who do not use the Internet at all. And we cannot say anything about the distribution between users and non-users within the investigated groups.

For example, a survey conducted by The Swedish Association of the Visually Impaired shows that 25 percent of their members have never used the Internet. In our survey, we can comment on how blind and severely visually impaired people use the internet, but the non-users are too few to make an analysis.

It is conceivable that there are individuals who would like to participate in our survey but who, despite the fact that we have taken several different measures to make the survey available to everyone, are unable to participate. This apply, for example, to people with aphasia, where we know that there are some people who need a lot of support to be able to answer the questions.

We have sought out a number of such people, but in one day's work we receive 3-5 answers, which makes it a very resource-demanding collection.

We see that people who find it difficult to participate in the survey also report greater difficulties in using the Internet. It may be that there are individuals within each group who similarly find it difficult to participate in the survey and who have greater difficulties with the Internet than the people within the group who have participated have reported.

Overall, this makes us believe that, through our way of recruiting participants, we have obtained a sample of people with less difficulties in relation to the Internet compared to all people with impairments.

In statistical contexts, this is stated as a so-called "bias", that is to say that in our material we obtained a systematic deviation from the entire population.

In that case, the difficulties in using the internet are greater than our survey shows.

Continued development of methods and continued development of channels for recruiting participants can mean that in future surveys it will be possible to reach even several of the people who can be assumed to have great difficulties in participating.

But we have already reached many people who otherwise never seem to be asked to participate in surveys.