| Reviewer information |
Reviewer name Reviewer comments
|
| Bibliometrics |
|
| Primary care function (adapted from Kueper et al [15]) |
Diagnostic decision support: artificial intelligence–assisted diagnostics Treatment decision support: artificial intelligence–assisted treatment, including remote management of care Referral support: artificial intelligence–assisted support for any portion of the referral process, especially for direct referrals of patients to specialist services Scheduling assistance: models for optimizing clinic schedules and overbooking Future state prediction: artificial intelligence offering predictions about the future state, such as consult service utilization or prognosis of existing conditions. (this excludes predictions of one’s chances of developing a health condition in the near term, which falls under diagnostic decision support) Health care utilization analyses: artificial intelligence extracts information retrospectively to understand more about the current processes or interactions within a health care system Knowledge base and ontology construction or use Information extraction: artificial intelligence extracts knowledge from structured or unstructured data sources Descriptive information provision: Artificial intelligence summarizes existing data in interpretable or useful ways Other: function not represented above, but specifics of function will still be recorded in case a new category emerges
|
| Author-reported intended end-users |
The intended user of the artificial intelligence product, including but not limited to patients, physicians, nurses, nurse practitioners, administrators, researchers, others, and unknown (if an end-user is not specified as the tool was still in development, a researcher was designated)
|
| Target health condition (adapted from Kueper et al [15]) |
General Diabetes Cancer, non-skin Heart valves, murmurs Musculoskeletal/joint Dementia, cognitive impairment Lung apnea, chronic obstructive pulmonary disease Chronic disease, frailty Skin cancer Stroke, neurological Psychiatric Coronary artery disease Heart failure Hypertension Other cardiovascular disease Gastrointestinal/liver Ear, nose, and throat Eye and retina Trauma, emergency surgery Infection Metabolic Kidney and urinary tract Immunization, reactions Skin disorders Obesity Pediatric/developmental Other
|
| Data set |
Size: number of unique patients Time period if applicable -
Source of data:
Number of institutions: single or multiple Setting (urban, rural, both, or unknown): We use the United States Census’ County Classification Lookup Table [34] to determine whether a certain area was urban or rural. If there were multiple locations, we selected both.
|
| Compliance with “Ethics Guidelines for Trustworthy AI” [35]: which of the 7 elements were addressed (yes/no)? |
Human agency and oversight: how well does the algorithm support human decision-making and permit oversight on its predictions? Technical robustness and safety: how well-suited is the algorithm for its intended use? How well does it mitigate harm? Privacy and data governance: how well does the algorithm’s data ingestion and analysis pipeline respect patient privacy (eg, HIPAA compliance) and enforce safeguards against unpermitted access? Transparency: does the artificial intelligence algorithm explain reasons for its outputs in a traceable and interpretable way? Diversity, nondiscrimination, and fairness: how biased is the algorithm with regard to its performance? How easy is it for stakeholders to provide feedback on the algorithm’s performance for its continuous development? Societal and environmental well-being: what are the societal (eg, dehumanizing relationships) and ecological (eg, energy consumption) impacts of the algorithm? Accountability: who is held responsible to ensure the algorithm’s development, outcomes, harm, and regulation?
|
| Model fairness and focus on health equity: is the main purpose of the study specifically outlined to improve health for a vulnerable population (yes/no)? |
Must be explicitly stated in the introduction or abstract as motivation for the paper to focus on at least 1 vulnerable population (though there may be other populations studied as well) defined by any of the following categories which are largely based off of the NIMHD Research Framework [36]:
Place of residence (eg, rural) Race, ethnicity (eg, Black African American or Latinx) Occupation (eg, coal miners) Gender, sex (eg, transgender) Religion (eg, Amish) Education (eg, low) Socioeconomic status (eg, low income) Social capital (eg, isolation) Does the study include key variables that could reflect disparities across protected classes (eg, age, sex, or race/ethnicity)? If reported, do they include these variables in their evaluation (eg, subgroup analysis to demonstrate equal performance)? Existing biases: does the study discuss biases or potential repercussions related to vulnerable populations? [9] Historical bias (ie, data retrieval) Representation bias (ie, population representation) Measurement bias Aggregation bias Evaluation bias Deployment bias Bias mitigation: does the study attempt to reduce existing biases, either explicitly or implicitly? If so, what methodology do they employ? Preoutput (changes to the algorithm or input data) Postoutput (user education, transparency, and specifying the use case) Other
|
| Stage of the study |
Methodological development: generation of novel artificial intelligence methods or modification of existing artificial intelligence methods to accomplish a task relevant to primary care. Retrospective data analysis or model development: developed an artificial intelligence model trained on retrospectively collected data to identify trends or perform a task that awaits prospective validation. Evaluation: artificial intelligence implemented in the intended setting as part of a pilot study, such as a prospective cohort study or randomized controlled trial. Postimplementation: assessing the impact of an artificial intelligence implementation after officially deployed in its intended setting.
|
