Skip to main content
. 2022 Sep 12;11:376. Originally published 2022 Mar 31. [Version 2] doi: 10.12688/f1000research.109662.2
Step 1: Calculation of correct and incorrect markings
Calculation (addition of variables and dividing by 2, as each marked result is recorded with "2".) Description (Task, SEO or PSM, device, results type) New Variable (from the calculation shown in the left column, a new variable is created for step 2 explained below)
(ad201+ad202+ad213+ad214)/2
Example: (2+0+2+2)/2=3 ➔ 3 text ads were marked for task A on large screen when ads were to be marked (results type „PSM“)
Task A, PSM, large screen, text ads ad2t01
➔ Thus, „ ad2t01” indicates the number of text ads that have been marked in “task A, PSM results, large screen” (in our example, 3 text ads)
(ad203+ad204+ad205+ad206+ad207+ad208+ad209+ad210+ad211+ad212)/2 Task A, PSM, large screen, organic results ad2o01 ➔ Number of organic results that have been marked in “task A, PSM results, large screen”
(ad401+ad402+ad413+ad414)/2 Task A, SEO, large screen, text ads ad4t01
(ad403+ad404+ad405+ad406+ad407+ad408+ad409+ad410+ad411+ad412)/2 Task A, SEO, large screen, organic results ad4o01
(bd201+bd202+bd213+bd214)/2 Task B, PSM, large screen, text ads bd2t01
(bd203+bd204+bd205+bd206+bd207+bd208+bd209+bd210+bd211+bd212)/2 Task B, PSM, large screen, organic results bd2o01
(bd401+bd402+bd413+bd414)/2 Task B, SEO, large screen, text ads bd4t01
(bd403+bd404+bd405+bd406+bd407+bd408+bd409+bd410+bd411+bd412)/2 Task B, SEO, large screen, organic results bd4o01
(am201+am202+am213+am214)/2 Task A, PSM, small screen, text ads am2t01
(am203+am204+am205+am206+am207+am208+am209+am210+am211+am212)/2 Task A, PSM, small screen, organic results am2o01
(am401+am402+am413+am414)/2 Task A, SEO, small screen, text ads am4t01
(am403+am404+am405+am406+am407+am408+am409+am410+am411+am412)/2 Task A, SEO, small screen, organic results am4o01
(bm201+bm202+bm213+bm214)/2 Task B, PSM, small screen, text ads bm2t01
(bm203+bm204+bm205+bm206+bm207+bm208+bm209+bm210+bm211+bm212)/2 Task B, PSM, small screen, organic results bm2o01
(bm401+bm402+bm413+bm414)/2 Task B, SEO, small screen, text ads bm4t01
(bm403+bm404+bm405+bm406+bm407+bm408+bm409+bm410+bm411+bm412)/2 Task B, SEO, small screen, organic results bm4o01
(cd201+cd202)/2 Task C, PSM, large screen, text ads cd2t01
(cd203+cd204+cd205)/2 Task C, PSM, large screen, news results cd2n01
(cd206+cd207+cd208+cd209+cd210+cd211)/2 Task C, PSM, large screen, organic results cd2o01
(cd212+cd213+cd214+cd215+cd216+cd217+cd218+cd219)/2 Task C, PSM, large screen, shopping ads cd2s01
(cd220)/2 Task C, PSM, large screen, knowledge graph cd2w01
(cd401+cd402)/2 Task C, SEO, large screen, text ads cd4t01
(cd403+cd404+cd405)/2 Task C, SEO, large screen, news results cd4n01
(cd406+cd407+cd408+cd409+cd410+cd411)/2 Task C, SEO, large screen, organic results cd4o01
(cd412+cd413+cd414+cd415+cd416+cd417+cd418+cd419)/2 Task C, SEO, large screen, shopping ads cd4s01
(cd420)/2 Task C, SEO, large screen, knowledge graph cd4w01
(dd201+dd202)/2 Task D, PSM, large screen, text ads dd2t01
(dd203+dd204+dd205)/2 Task D, PSM, large screen, news results dd2n01
(dd206+dd207+dd208+dd209+dd210+dd211)/2 Task D, PSM, large screen, organic results dd2o01
(dd212+dd213+dd214+dd215+dd216+dd217+dd218+dd219)/2 Task D, PSM, large screen, shopping ads dd2s01
(dd220)/2 Task D, PSM, large screen, knowledge graph dd2w01
(dd401+dd402)/2 Task D, SEO, large screen, text ads dd4t01
(dd403+dd404+dd405)/2 Task D, SEO, large screen, news results dd4n01
(dd406+dd407+dd408+dd409+dd410+dd411)/2 Task D, SEO, large screen, organic results dd4o01
(dd412+dd413+dd414+dd415+dd416+dd417+dd418+dd419)/2 Task D, SEO, large screen, shopping ads dd4s01
(dd420)/2 Task D, SEO, large screen, knowledge graph dd4w01
(cm201+cm202)/2 Task C, PSM, small screen, shopping ads cm2s01
(cm203+cm204)/2 Task C, PSM, small screen, text ads cm2t01
(cm205+cm206+cm211+cm212+cm213+cm214)/2 Task C, PSM, small screen, organic results cm2o01
(cm208+cm209)/2 Task C, PSM, small screen, news results cm2n01
(cm210)/2 Task C, PSM, small screen, knowledge graph cm2w01
(cm401+cm402)/2 Task C, SEO, small screen, shopping ads cm4s01
(cm403+cm404)/2 Task C, SEO, small screen, text ads cm4t01
(cm405+cm406+cm411+cm412+cm413+cm414)/2 Task C, SEO, small screen, organic results cm4o01
(cm408+cm409)/2 Task C, SEO, small screen, news results cm4n01
(cm410)/2 Task C SEO, small screen, knowledge graph cm4w01
(dm201+dm202)/2 Task D PSM, small screen, shopping ads dm2s01
(dm203+dm204)/2 Task D, PSM, small screen, text ads dm2t01
(dm205+dm206+dm211+dm212+dm213+dm214)/2 Task D, PSM, small screen, organic results dm2o01
(dm208+dm209)/2 Task D, PSM, small screen, news results dm2n01
(dm210)/2 Task D, PSM, small screen, knowledge graph dm2w01
(dm401+dm402)/2 Task D, SEO, small screen, shopping ads dm4s01
(dm403+dm404)/2 Task D, SEO, small screen, text ads dm4t01
(dm405+dm406+dm411+dm412+dm413+dm414)/2 Task D, SEO, small screen, organic results dm4o01
(dm408+dm409)/2 Task D, SEO, small screen, news results dm4n01
(dm410)/2 Task D, SEO, small screen, knowledge graph dm4w01