Skip to main content
. 2022 Sep 12;11:376. Originally published 2022 Mar 31. [Version 2] doi: 10.12688/f1000research.109662.2

Table 1. Questionnaire.

Section No. Question Response options of original study Response options final (adapted/translated if necessary) Comments Ref.
I) Screening 1.1 How old are you? /
  • -
    under 16 years
  • -
    16 to 19 years
  • -
    20 to 24 years
  • -
    25 to 29 years
  • -
    30 to 34 years
  • -
    35 to 39 years
  • -
    40 to 44 years
  • -
    45 to 49 years
  • -
    50 to 54 years
  • -
    55 to 59 years
  • -
    60 to 64 years
  • -
    65 to 69 years
  • -
    70 years and older
For quotation purposes;
exclusion of subject if under 16 years of age or 70 years and older.
9
1.2 You are … /
  • -
    Female
  • -
    Male
For quotation purposes 9
1.3 Which state do you live in? /
  • -
    Baden-Württemberg
  • -
    Bayern
  • -
    Berlin
  • -
    Brandenburg
  • -
    Bremen
  • -
    Hamburg
  • -
    Hessen
  • -
    Mecklenburg-Vorpommern
  • -
    Niedersachsen
  • -
    Nordrhein-Westfalen
  • -
    Rheinland-Pfalz
  • -
    Saarland
  • -
    Sachsen
  • -
    Sachsen-Anhalt
  • -
    Schleswig-Holstein
  • -
    Thüringen
For quotation purposes 9
II) Usage behavior 2.1 What do you use the internet for? / Please mark all applicable answers:
  • -
    Browsing the web, e.g., for entertainment, to pass the time
  • -
    Search for something
  • -
    Read news, reports, articles
  • -
    Use social networks, communities, e.g., Instagram, Facebook
  • -
    Communicate via e-mail, messenger
  • -
    Online shopping/ordering/booking
  • -
    Online banking/broking
  • -
    Watch movies, videos
  • -
    Listen to and download music
  • -
    Gaming
  • -
    Other
9
2.2 If you are searching for something online:
Which search engine(s) do you usually use?
Please mark all applicable answers:
  • -
    Bing
  • -
    Ecosia
  • -
    DuckDuckGo
  • -
    Google
  • -
    Web.de
  • -
    Yahoo!
  • -
    Others, namely … (free input)
  • -
    None
Exclusion of respondent if no search engines are used 8, adjust-ments by: 9
2.3 Which search engine do you use most often?
  • -
    Google
  • -
    Yahoo Search
  • -
    Bing
  • -
    AOL
  • -
    Ask
  • -
    Lycos
  • -
    MyWebSearch
  • -
    Dogpile
  • -
    WebCrawler
  • -
    Other (SPECIFY)
  • -
    None/Don’t use any regularly
- Don’t know
- Refused
  • -
    Bing
  • -
    Ecosia
  • -
    DuckDuckGo
  • -
    Google
  • -
    Web.de
  • -
    Yahoo!
  • -
    Another
  • -
    I don’t know/not specified
Only used search engines (according to previous question) are displayed.
Question omitted if only one search engine is used.
6, adjustments by: 9
2.4 Which devices do you use search engines with? Multiple Choice:
  • -
    Desktop PC/Laptop
  • -
    Smartphone
  • -
    Tablet
Please mark the appropriate answer in each case:
  • -
    via desktop computer, PC
  • -
    via laptop
  • -
    via tablet
  • -
    via smartphone
  • -
    via smart speaker (e.g., Amazon Echo, Alexa, Google Home)
  • -
    frequently
  • -
    occasionally
  • -
    rarely
  • -
    never
  • -
    I don’t know
8, adjustments by: 9
2.5 Why is [search engine] the search engine you use most often? Please mark up to 5 answers. I use [search engine] most because …
  • -
    it is easy to use
  • -
    it is fast
  • -
    the results list is clearly arranged
  • -
    it seems objective to me
  • -
    the most important results are always at the top of the results list
  • -
    I always find what I’m looking for
  • -
    I’ve always used it
  • -
    I think it covers most of the internet
  • -
    it provides helpful information on the individual results
  • -
    it does not show any dubious results
  • -
    I know exactly how it works
  • -
    it sometimes shows surprising results
  • -
    it offers the possibility to fade out dubious results
  • -
    my friends and colleagues also use it
  • -
    I like the layout and colors of the search engine page
  • -
    Scale from 1= not applicable to 4= fully applicable
I use [search engine] most because …
  • -
    the results list is clearly arranged
  • -
    I like the layout and colors of the search engine result page
  • -
    it is easy to use
  • -
    it is fast
  • -
    it seems objective to me
  • -
    I always find what I’m looking for
  • -
    I know exactly how it works
  • -
    I think it covers most of the internet
  • -
    the most important results are always at the top of the results list
  • -
    it provides helpful information on the individual results
  • -
    it sometimes shows surprising results
  • -
    it does not show dubious hits or these can be hidden
  • -
    I’ve always used it
  • -
    my friends and colleagues also use it
  • -
    I do not know any other search engines
  • -
    it is the default setting in the browser
  • -
    no particular reason
  • -
    Other reason, namely… (free input)
The name of the most frequently used search engine is shown 5, adjustments by: 9
2.6 Can you estimate how many queries you submit to search engines in a regular week?
  • -
    several times a day
  • -
    about once a day
  • -
    3 to 5 days a week
  • -
    1 to 2 days a week
  • -
    once every few weeks
  • -
    less often
  • -
    never
  • -
    don’t know
  • -
    refused
  • -
    more than 100 per week
  • -
    over 50 to 100 per week
  • -
    over 20 to 50 per week
  • -
    over 10 to 20 per week
  • -
    6 to 10 per week
  • -
    1 to 5 per week
  • -
    less than 1 per week
  • -
    I don’t know
6, adjustments by: 9
III) Self-assessment 3.1 When it comes to finding something on the internet using search engines: How do you assess your own abilities in this respect?
  • -
    German school grades (1-6)
My skills in search engine usage are…
  • -
    perfect
  • -
    excellent
  • -
    good
  • -
    fair
  • -
    bad
  • -
    I don’t know
Check for correlation between self-assessment and actual knowledge 3, adjustments by: 9
3.2 And how often do you think you find what you are looking for using search engines?
  • -
    always
  • -
    most of the time
  • -
    only some of the time
  • -
    hardly ever
  • -
    don’t know
  • -
    refused
I find what I’m looking for…
  • -
    always
  • -
    most of the time
  • -
    sometimes
  • -
    rarely
  • -
    never
  • -
    I don’t know
6, adjustments by: 9
IV) Trust 4.1 If you think of search engines in general: To what extent do you think the following statements apply to search engines? a) “In general, do you think internet search engines are a fair and unbiased source of information, or do you think search engines are NOT a fair and unbiased source?”:
- Yes, they are a fair and unbiased source of information
- No, they are NOT a fair and unbiased source of information
- Depends
- Don’t know
- Refused
b) “In general, how much of the information you find using search engines do you think is accurate or trustworthy? Would you say…”:
- All or almost all
- Most
- Some
- Very little
- None at all
- Don’t know
- Refused
Please mark the appropriate answer in each case:
  • -
    Search engines are fair and unbiased sources of information
  • -
    The information I find through search engines is correct and trustworthy
  • -
    absolutely correct
  • -
    correct
  • -
    neutral
  • -
    rather not true
  • -
    doesn’t apply at all
  • -
    I don’t know
6, major adjustments regarding the question structure and responses by: 9
4.2 And if you think especially of Google:
To what extent do you think the following statements apply to Google?
a) “In general, do you think internet search engines are a fair and unbiased source of information, or do you think search engines are NOT a fair and unbiased source?”:
- Yes, they are a fair and unbiased source of information
- No, they are NOT a fair and unbiased source of information
- Depends
- Don’t know
- Refused
b) “In general, how much of the information you find using search engines do you think is accurate or trustworthy? Would you say…”:
- All or almost all
- Most
- Some
- Very little
- None at all
- Don’t know
- Refused
Please mark the appropriate answer in each case:
  • -
    Google is a fair and unbiased source of information
  • -
    The information I find through Google is correct and trustworthy
  • -
    absolutely correct
  • -
    correct
  • -
    neutral
  • -
    rather not true
  • -
    doesn’t apply at all
  • -
    I don’t know
6, major adjustments regarding the question structure and responses by: 9
V) Query match 5.1 If you think of search engines in general: To what extent do you think the following statement applies to search engines?
  • -
    The results displayed in search engines match my queries perfectly
  • -
    absolutely correct
  • -
    correct
  • -
    neutral
  • -
    rather not true
  • -
    doesn’t apply at all
  • -
    I don’t know
Questions 5.1 and 5.2 follow on from the previous questions on trust and were added to the questionnaire in consultation with F&M. 9
5.2 To what extent do you think the following statement applies to Google?
  • -
    The results displayed in Google match my queries perfectly
  • -
    absolutely correct
  • -
    correct
  • -
    neutral
  • -
    rather not true
  • -
    doesn’t apply at all
  • -
    I don’t know
9
VI) Knowledge of search result influences 6.1 When it comes to the search results displayed on Google:
What do you think influences the ranking of search results on Google?
  • -
    The Google search results and their ranking depend on… (free input)
  • -
    I don’t know
9
VII) Knowledge of ads 7.1 What do you think: Where does Google generate most of its revenue from?
  • -
    Google generates revenue primarily through… (free input)
  • -
    I don’t know
3
7.2 Do website operators or companies have the opportunity to pay for their results to appear high up on Google’s search results page?
  • -
    Yes, this is possible
  • -
    No, that possibility does not exist
  • -
    I don’t know
3
7.3 Do such paid search results differ from the other search results?
  • -
    Yes, you can recognize them and distinguish them from the other search results
  • -
    No, they cannot be identified
  • -
    I don’t know
[If “Yes” on previous question] 3
7.4 And how do the paid search results on Google differ from the other results that have not been paid for?
  • -
    The paid search results on Google can be recognized by… (free input)
  • -
    I don’t know
[If “Yes” on previous question] 3
VIII) Knowledge of SEO 8.1 Do website operators or companies have the ability or influence to appear higher in the Google results list for certain queries without paying any money to Google?
  • -
    Yes, this is possible
  • -
    No, that possibility does not exist
  • -
    I don’t know
1
8.2 Do you know what term is used to describe these measures to improve the ranking in the Google search results list (without payment to Google)?
  • -
    Yes, this is called… (free input)
  • -
    I don’t know
[If “Yes” on previous question] 1
8.3 And by what means can a website be designed or programmed so that it is ranked higher in the Google search results lists? Please enter all possibilities/measures that you know:
  • -
    With the help of the following measures: … (free input)
  • -
    I don’t know
[If “Yes” on question 8.1]
Serves for further differentiation of SEO knowledge levels
1
Information part “SEO/PSM“: Website operators have several ways to ensure that their web pages appear at the top of the Google result page for a specific query, namely I) Payment: They pay money to Google*, or II) Search engine optimization: They design their websites accordingly, e.g., by using certain keywords, quick page speed, and appropriate image titles and descriptions. Next, we will show you two different Google result pages and would like to ask you whether or which results can be influenced by payment to Google and/or search engine optimization. 10, adjustments by: 9
IX) Ability to distinguish ads from organic results 9.1 You will now see a Google results page.
Are there any search results on this page that can be influenced by the website operator paying Google?
  • -
    No, there are no search results on this page that can be influenced by payments to Google
  • -
    Yes, the following search results can be influenced by paying money to Google:
Please click on the corresponding search results
SERP screenshot from block I (A or B) to mark all ads 3
9.2 One more question about this search results page:
Are there any search results on this page that can be influenced by search engine optimization?
  • -
    No, there are no search results on this site that can be influenced by search engine optimization
  • -
    Yes, the following search results can be influenced by search engine optimization:
Please click on the corresponding search results
SERP screenshot from block I (A or B) to mark all organic results 1
9.3 You will now see another Google results page.
Are there any search results on this page that can be influenced by the website operator paying Google?
  • -
    No, there are no search results on this page that can be influenced by payments to Google
  • -
    Yes, the following search results can be influenced by paying money to Google:
Please click on the corresponding search results
SERP screenshot from block II (C or D) to mark all ads 3
9.4 One more question about this search results page:
Are there any search results on this page that can be influenced by search engine optimization?
  • -
    No, there are no search results on this site that can be influenced by search engine optimization
  • -
    Yes, the following search results can be influenced by search engine optimization:
Please click on the corresponding search results
SERP screenshot from block II (C or D) to mark all organic results 1
X) Assessments and opinions regarding SEO 10.1 Now please think again about search engine optimization.
In your opinion, how strong is the influence of search engine optimization on the ranking of the search results in Google?
Influence of search engine optimization on the order of search results in Google:
  • -
    very strong influence
  • -
    major influence
  • -
    medium influence
  • -
    little influence
  • -
    no influence
  • -
    I don’t know
1
10.2 How big are the positive and negative effects of search engine optimization on the Google search results from your perspective? Please mark the appropriate answer in each case:
  • -
    I perceive the positive effects of search engine optimization as …
  • -
    I perceive the negative effects of search engine optimization as …
  • -
    very large
  • -
    large
  • -
    medium
  • -
    low
  • -
    non-existent
  • -
    I don’t know
1
10.3 Which positive effects does search engine optimization have in your opinion?
  • -
    I assess the following effects of search engine optimization as positive: … (free input)
  • -
    I can’t say
Question to internet users who see high or very high positive SEO effects 9
10.4 Which negative effects does search engine optimization have in your opinion?
  • -
    I assess the following effects of search engine optimization as negative: … (free input)
  • -
    I can’t say
Question to internet users who see high or very high negative SEO effects 9
XI) Personalization 11.1 If a search engine records your search queries and uses this information to customize search results for you in the future: What do you think about that? - It’s a bad thing if a search engine collected information about your searches and then used it to rank your future search results,
A: because it may limit the information you get online and what search results you see
B: because you feel it is an invasion of privacy
- It’s a good thing if a search engine collected information about your searches and then used it to rank your future search results,
A: because it gives you results that are more relevant to you
B: even if it means they are gathering information about you
- Neither of these
- Don’t know
- Refused
  • -
    I think that's a positive thing
  • -
    neutral
  • -
    I think that's a negative thing
  • -
    I don’t know/not specified
6, adjustments by: 9
11.2 And have you ever taken measures to limit the amount of data that search engines collect about you?
If so, which ones?
- Changed your browser settings
- Deleted your web history
- Used the privacy settings of websites
- Yes
- No
- Don’t know
- Refused
Please mark all applicable answers:
  • -
    Deleted past activities (for example, search history)
  • -
    Disabled storage of future activities (e.g., search queries)
  • -
    Disabled determination of my location
  • -
    Deactivated delivery of personalized advertising
  • -
    Other measures
  • -
    No, not yet - but I was aware that it is possible
  • -
    No - I was not aware that this was possible
6, adjustments by: 9
XII) User profile 12.1 In what way do you use search engines? Please mark the appropriate answer in each case:
  • -
    By typing in my search query
  • -
    By submitting my search query by voice
  • -
    frequently
  • -
    occasionally
  • -
    rarely
  • -
    never
  • -
    I don’t know
10
12.2 In a regular week, for how long do you use the internet approximately? Scale from 1-7 (days per week) Please indicate the average number of hours per week:
  • -
    less than 3 hours per week
  • -
    3 to under 6 hours per week
  • -
    6 to under 10 hours per week
  • -
    10 to under 20 hours per week
  • -
    20 to under 30 hours per week
  • -
    30 to under 40 hours per week
  • -
    40 and more hours per week
  • -
    I don’t know
4, adjustments by: 9
12.3 Which of the following activities do you mainly pursue?
  • -
    in training or studies
  • -
    working
  • -
    unemployed or no longer employed
  • -
    employee or public official
  • -
    self-employed person, freelancer, entrepreneur
  • -
    student
  • -
    trainee, apprentice
  • -
    pupil
  • -
    housewife/houseman
  • -
    occasionally employed
  • -
    not or no longer employed
  • -
    other
7, adjustments by: 9
12.4 Which of the following topics play a role in your professional activity? Please mark all applicable answers:
  • -
    purchasing, procurement, logistics
  • -
    finance, controlling
  • -
    marketing, sales, distribution
  • -
    IT
  • -
    digitalization, internet
  • -
    e-commerce, online trading
  • -
    online marketing, social media
  • -
    production, manufacturing
  • -
    law
  • -
    none of them
Question for employed internet users.
Examine whether people with “SEO-related” professions (e.g., e-commerce) have a different perspective on SEO.
2
12.5 Which of the following topics belong to your training/studies? Please mark all applicable answers:
  • -
    business studies or economics
  • -
    informatics, business informatics
  • -
    engineering, electrical engineering
  • -
    digitalization, internet
  • -
    e-commerce, online trading
  • -
    online marketing, social media
  • -
    law
  • -
    pedagogy
  • -
    social sciences
  • -
    none of them
Question to internet users who are still in training.
Check whether people with “SEO-related” topics in training/studies (e.g., e-commerce) have a different perspective on SEO.
2
12.6 What is your highest educational level?
  • -
    None
  • -
    Certificate of Secondary Education
  • -
    General Certificate of Secondary Education
  • -
    university entrance exams
  • -
    University degree
  • -
    Certificate of Secondary Education without completed apprenticeship
  • -
    Certificate of Secondary Education with completed apprenticeship
  • -
    General Certificate of Secondary Education
  • -
    university entrance exams
  • -
    University degree
  • -
    None
  • -
    (Still) without school-leaving certificate (e.g., student)
  • -
    Other
7, adjustments by: 9

References of the questions: 1: Project goals “SEO Effect”, 2: Expert interviews ( Schultheiß & Lewandowski, 2021d), 3: ( Lewandowski, 2017), 4: ( Stark et al., 2014), 5: ( Schweiger, 2003), 6: ( Purcell et al., 2012), 7: ( Lewandowski & Sünkler, 2013), 8: ( Schultheiß & Lewandowski, 2021b) 9: Market research company “Fittkau & Maaß”, 10: Project staff, *This is a simplified explanation. The fact that a payment to Google is only made after an ad is selected was left unmentioned for the sake of comprehensibility.

SEO: Search engine optimization, PSM: Paid search marketing, SERP: Search engine results page