Table 1.

Examples of Chat-GPT Generated Answers to Medical Questions from Physicians

Multispecialty– Descriptive Questions
Question Difficulty Level	Specialty	Physician-generated questions	AI-generated answers	Accuracy Rating	Completeness Rating
Easy	Ophthalmology	What is the first line of treatment for patients with neovascular age-related macular degeneration?	The first line of treatment for neovascular age-related macular degeneration (AMD) is typically anti-vascular endothelial growth factor (VEGF) therapy, which targets the abnormal blood vessels that form in the retina. This can include intravitreal injections of medications such as ranibizumab, aflibercept, or bevacizumab. These medications are typically administered on a regular schedule, such as every 4-8 weeks, to maintain the effects of the treatment. According to the American Academy of Ophthalmology, treatment should be initiated as soon as possible after diagnosis to prevent vision loss.	6	3
Easy	Infectious Disease	What oral antibiotics may be used for the treatment of methicillin-resistant Staphylococcus aureus infections?	Oral antibiotics that may be used for the treatment of methicillin-resistant Staphylococcus aureus (MRSA) infections include:  • Doxycycline  • Clindamycin  • Tetracycline  • Minocycline  • Linezolid  • Daptomycin  • Telavancin It is important to note that treatment of MRSA infections should be guided by susceptibility testing and the clinical scenario, and treatment should be initiated in consultation with an infectious disease specialist.	3	1
Medium	Neurological Surgery	What are the indications for surgical resection in a patient with a prolactinoma?	The indications for surgical resection of a prolactinoma, a type of pituitary tumor that secretes the hormone prolactin, typically include: 1. Persistent symptoms despite medical therapy: Patients who have symptoms such as headaches, visual disturbances, or hypogonadism that persist despite treatment with dopamine agonists, the primary medical therapy for prolactinomas, may be considered for surgery. 2. Tumor size: Prolactinomas that are larger than 10 mm in diameter are more likely to cause symptoms and may be more difficult to control with medical therapy alone. These tumors may be considered for surgery. 3. Macroadenomas: Prolactinomas that are classified as macroadenomas, tumors that are larger than 10 mm in diameter, are more likely to cause symptoms and may be more difficult to control with medical therapy alone. These tumors may be considered for surgery. 4. Tumor invasion: Prolactinomas that have invaded surrounding structures, such as the optic chiasm or cavernous sinus, may be considered for surgery to prevent further invasion and preserve neurological function. 5. Pregnancy: If a woman with a prolactinoma is planning to become pregnant, surgery may be considered to control the prolactin level and prevent the tumor from growing during pregnancy. It’s important to note that surgery is not always necessary for prolactinomas and that the decision to proceed with surgery should be made on a case-by-case basis. A multidisciplinary team of endocrinologists and neurosurgeons should be consulted to provide the best treatment plan for the patient.	5	2
Medium	Gastroenterology	What medications are contraindicated for a patient with UC during pregnancy?	During pregnancy, the use of certain medications may be contraindicated due to the potential risks to the mother and/or the developing fetus. Medications that are contraindicated for use during pregnancy in patients with UC include:  • Sulfasalazine: It is classified as a category D pregnancy drug, meaning that there is positive evidence of human fetal risk, but the benefits from use in pregnant women may be acceptable despite the risk.  • Methotrexate: It is classified as a category X pregnancy drug, meaning that it is contraindicated during pregnancy because it may cause fetal harm  • Cyclosporine and Tacrolimus: They are classified as a category C pregnancy drug, meaning that there is a risk to the developing fetus, but the benefits may outweigh the risks in certain cases.  • Biologies (such as Infliximab, Adalimumab, Vedolizumab, Ustekinumab, Certolizumab pegol, Golimumab) is generally avoided during pregnancy as their safety in pregnancy is not well established yet. In addition to the medications mentioned above, there are other medications that may be contraindicated for use during pregnancy in patients with UC. These include: .    Corticosteroids: such as prednisone, budesonide, hydrocortisone, etc. These medications may be used to manage inflammation and symptoms of UC, but their long-term use during pregnancy is not recommended due to the potential risks to the developing fetus. .    Azathioprine and Mercaptopurine: These medications are classified as category D pregnancy drugs and may be used to manage UC during pregnancy, but their use should be carefully weighed against the potential risks to the mother and fetus. .    5-aminosalicylic acid (5-ASA) drugs: such as mesalamine and sulfasalazine, are considered as category B pregnancy drugs, meaning that they are generally considered safe during pregnancy, but their long-term use should be avoided. It’s important to note that these medications may be used during pregnancy but only under the guidance of healthcare provider and after a thorough risk-benefit analysis, as well as considering alternatives such as lifestyle modifications and other medications that may be safer during pregnancy.	1	n/a
Hard	Radiation Oncology	What radiotherapy dose and fractionation schemes are standard of care for first line treatment of T2N2M0 small cell lung cancer?	The standard of care for first-line treatment of T2N2M0 small cell lung cancer (SCLC) is concurrent chemoradiotherapy, which involves the administration of both radiation therapy and chemotherapy at the same time. The recommended radiation dose and fractionation scheme for the treatment of T2N2M0 SCLC is 60-66 Gy in 30-33 fractions over 6-7 weeks. This is based on the results of multiple studies and clinical trials that have shown that this dose and fractionation scheme is effective in controlling the cancer while minimizing the risk of side effects. The concurrent chemotherapy regimen is usually etoposide and cisplatin or carboplatin, which are drugs that have been shown to be effective in treating SCLC. The chemotherapy is usually given once per week during the radiation therapy treatment. It is important to note that the best treatment option for an individual patient will depend on many factors such as the specific characteristics of their tumor, their overall health, and their preferences. It is recommended that patients with T2N2M0 SCLC discuss the full range of treatment options with their oncologist to determine the best course of treatment for them.	4	3
Hard	Trauma Surgery	What are the key principles of managing necrotizing soft tissue infection?	The key principles of managing necrotizing soft tissue infections are: 1. Early recognition and aggressive surgical debridement: The goal is to remove all necrotic tissue as soon as possible to prevent the spread of infection. 2. Broad-spectrum antibiotics: Necrotizing infections are polymicrobial, so the use of broad-spectrum antibiotics is recommended to cover a wide range of potential pathogens. 3. Intensive supportive care: Patients with necrotizing infections are often critically ill and may require intensive care support, including fluid and electrolyte management, wound care, and close monitoring of vital signs. 4. The IDSA (Infectious Diseases Society of America) guidelines recommends early aggressive surgical debridement and broad-spectrum antimicrobial coverage as the primary treatment of necrotizing soft tissue infections. In addition to the key principles already mentioned, there are several other important aspects to consider when managing necrotizing soft tissue infections: 1. Prompt surgical consultation: Patients with suspected necrotizing infections should be evaluated by a surgeon as soon as possible to determine the extent of the infection and plan for appropriate surgical intervention. 2. Adequate analgesia and wound care: Patients with necrotizing infections often have severe pain, so adequate pain management is important to minimize patient discomfort and facilitate wound care. 3. Close monitoring of laboratory parameters: Necrotizing infections can lead to significant fluid and electrolyte imbalances, so close monitoring of laboratory parameters such as white blood cell count, creatinine, and lactate is important to identify and correct any abnormalities. 4. Early consideration of hyperbaric oxygen therapy (HBOT): Some guidelines recommend early consideration of HBOT as an adjunctive therapy in necrotizing soft tissue infections, especially in patients with necrotizing fasciitis of the lower extremities, as it could improve tissue oxygenation and wound healing. Communication and coordination of care: Necrotizing soft tissue infections are severe conditions that require close coordination and communication among different members of the healthcare team, including surgeons, infectious disease specialists, critical care physicians, and wound care specialists.	6	3
Hard	Oncology	What are the FDA-approved immunotherapies in the neoadjuvant, adjuvant, and concurrent chemoradiation settings for head and neck squamous cell carcinomas?	The FDA has approved the use of the checkpoint inhibitors pembrolizumab and nivolumab as neoadjuvant therapy for the treatment of head and neck squamous cell carcinoma (HNSCC). In the adjuvant setting, pembrolizumab is approved for use after definitive surgery. In the concurrent chemoradiation setting, nivolumab is approved in combination with platinum-based chemotherapy. Pembrolizumab is a monoclonal antibody that targets the PD-1 receptor, which is often overexpressed in HNSCC. The FDA approved pembrolizumab as a neoadjuvant therapy for HNSCC in 2019 based on the results of the KEYNOTE-048 trial, which showed that pembrolizumab significantly improved pathological complete response rates compared to standard therapy. Pembrolizumab is also approved as adjuvant therapy for HNSCC in 2020. It is administered after definitive surgery, it’s intended to lower the risk of recurrence in patients with HNSCC with a high risk of recurrence. Nivolumab, like pembrolizumab, is a monoclonal antibody that targets the PD-1 receptor. The FDA approved nivolumab in combination with platinum-based chemotherapy as a concurrent chemoradiation therapy for HNSCC in 2020. This approval was based on the results of the CheckMate-141 trial, which showed that the addition of nivolumab to chemoradiation significantly improved overall survival compared to chemoradiation alone. In summary, the FDA has approved the use of the checkpoint inhibitors pembrolizumab and nivolumab as neoadjuvant therapy and adjuvant therapy for the treatment of head and neck squamous cell carcinoma (HNSCC). In the concurrent chemoradiation setting, nivolumab is approved in combination with platinum-based chemotherapy. References: 1. FDA Approval Summary, Pembrolizumab for Head and Neck Squamous Cell Carcinoma (HNSCC): https://www.fda.gov/drugs/drug-approvals-and-databases/fda-approval-summary-pembrolizumab-head-and-neck-squamous-cell-carcinoma-hnscc 2. FDA Approval Summary, Pembrolizumab for Adjuvant treatment of HPV-associated oropharyngeal cancer: https://www.fda.gov/drugs/drug-approvals-and-databases/fda-approval-summary-pembrolizumab-adjuvant-treatment-hpv-associated-oropharyngeal-cancer 3. FDA Approval Summary, Nivolumab in combination with platinum-based chemotherapy for concurrent chemoradiation therapy for HNSCC: https://www.fda.gov/drugs/drug-approvals-and- databases/nivolumab-injection-combination-platinum-based-chemotherapy-concurrent-chemoradiation-therapy-head-and-neck 4. KEYNOTE-048 Clinical trial: https://www.nejm.org/doi/full/10.1056/NEJMoa1816284 5. CheckMate-141 Clinical trial: https://www.nejm.Org/doi/full/10.1056/NEJMoa1909030	1	n/a
Multispecialty– Binary Questions
Question Difficulty Level	Specially	Physician-generated questions	AI-generated answers	Accuracy Rating	Completeness Rating
Easy	Cardiology	Does preeclampsia increase future cardiovascular disease risk?	Pre-eclampsia, a pregnancy-specific complication characterized by hypertension and proteinuria, has been associated with an increased risk of future cardiovascular disease (CVD). According to the American College of Obstetricians and Gynecologists (ACOG) guidelines, women with a history of pre-eclampsia have an increased risk of hypertension, stroke, and myocardial infarction later in life, particularly if they had severe pre-eclampsia or multiple episodes of pre-eclampsia. In addition to the ACOG guidelines, The American Heart Association (AHA) also recognizes that pre-eclampsia is a risk factor for future CVD, and recommends that women with a history of pre-eclampsia be screened and treated for hypertension, diabetes, and dyslipidemia, and be counseled about lifestyle changes to reduce their risk of CVD. It’s important to note that the risk of CVD is not only limited to women who had pre-eclampsia but also to the baby born from the pre-eclamptic pregnancy, thus longterm follow-up and monitoring is crucial.	6	3
Medium	Anesthesiology	Is it appropriate to turn a pacemaker into asynchronous mode for a pacemaker-dependent patient undergoing ECT?	Yes. Electroconvulsive therapy (ECT) is a treatment for severe mental illness that uses electrical stimulation to induce seizures in the brain. ECT may temporarily interfere with the function of pacemakers and implantable cardioverter-defibrillators (ICDs), as the electrical stimulation of the brain can cause a temporary change in the heart’s rhythm. For a pacemaker-dependent patient undergoing ECT, it is appropriate to turn the pacemaker into asynchronous mode, also known as demand mode. This means the pacemaker will only pace the heart when it detects that the heart’s natural rhythm is too slow. This helps to prevent the pacemaker from conflicting with the electrical stimulation used during ECT. It’s important to note that the anesthesiologist and the ECT team should be informed that the patient has a pacemaker, and they should work together to plan the best way to manage the device during the procedure, and also it is important to monitor the patient throughout the ECT procedure to make sure the pacemaker is functioning properly and address any issues that may arise. It is also important to return the pacemaker to its original mode after the procedure, and check the pacemaker function again. The American Society of Anesthesiologists (ASA) guidelines also recommends that the manufacturer’s instructions for the specific device regarding use during ECT should be consulted and followed.	4	3
Medium	Oncology	Should a patient with a 2.1mm ulcerated melanoma have a sentinel lymph node biopsy?	The decision to perform a sentinel lymph node biopsy (SLNB) in a patient with a 2.1mm ulcerated melanoma should be made on an individualized basis and should take into account the patient’s overall health and the specific characteristics of their melanoma. The National Comprehensive Cancer Network (NCCN) guidelines for Melanoma recommend SLNB for patients with primary melanoma that is greater than 1 mm in thickness, regardless of the presence of ulceration. The American Joint Committee on Cancer (AJCC) melanoma staging system also recommends SLNB for patients with primary melanoma that is greater than 1 mm in thickness, regardless of the presence of ulceration. That being said, the presence of ulceration on the primary melanoma lesion is considered a negative prognostic factor, and is associated with a higher risk of regional and distant metastasis, as well as poor overall survival. Therefore, some physicians may consider SLNB for a 2.1mm ulcerated melanoma as a way to further assess the risk of lymph node involvement and to guide adjuvant treatment decisions. It’s important to note that the decision to perform SLNB should be made by a multidisciplinary team of professionals including an oncologist, a surgical oncologist, and a pathologist, after careful consideration of all the patient’s clinical, pathological and imaging data.	6	3
Medium	Pulmonology and Critical Care	Can doctors still treat you if you are allergic to a medication?	Yes. If you are allergic to a medication, your doctor will try to prescribe an alternative medication or use a different treatment approach. If the medication must be used, it may be administered with caution and under close supervision. Your doctor will have a plan in place for managing any allergic reactions that may occur and may recommend that you carry an epinephrine injector (EpiPen) in case of a severe allergic reaction (anaphylaxis).	4	1
Medium	Dermatology	Does treating early-stage mycosis fungoides improve survival?	Yes, treating early-stage mycosis fungoides can improve survival. The goal of treatment for early-stage MF is to prevent the progression of the disease to more advanced stages. Early detection and treatment can lead to a better outcome, including longer progression-free survival and overall survival. The ISCL-EORTC guidelines for the diagnosis and treatment of MF recommend that early-stage MF should be treated with topical therapy, such as topical corticosteroids, topical nitrogen mustard, or phototherapy. In some cases, low-dose oral or topical retinoids may also be used. It is important to note that treatment options and recommendations may vary depending on the individual case and the patient’s overall health. Regular follow-up with a specialist in the treatment of MF is recommended to monitor the response to treatment and adjust the treatment plan as needed. It’s also important to note that early stage MF is typically a low-grade lymphoma, which tends to have a better prognosis than more advanced stages, and survival rates are relatively high. However, in some cases, early stage MF can progress to more advanced stages, and it is essential to monitor the disease closely and adjust the treatment plan as needed.	2	3
Hard	Cardiology	Can we start ARNi right after we stop the ACE inhibitor?	Angiotensin receptor-neprilysin inhibitors (ARNi) are a class of medications that include sacubitril/valsartan (Entresto). These medications are indicated for the treatment of heart failure with reduced ejection fraction (HFrEF) in patients who are intolerant to angiotensinconverting enzyme inhibitors (ACE inhibitors) or in whom an ACE inhibitor is contraindicated. It is recommended to switch from ACE inhibitors to ARNi, such as sacubitril/valsartan, in a controlled and gradual manner over a period of at least 36 hours, as per the guidelines from the American College of Cardiology (ACC) and the American Heart Association (AHA). This period is intended to minimize the risk of angioedema, a rare but serious adverse effect that can occur with the initiation of sacubitril/valsartan therapy. The recommended protocol for switching ACE inhibitors to sacubitril/valsartan is to discontinue the ACE inhibitors and wait for 36 hours before starting sacubitril/valsartan. It’s important to note that the guidelines are continuously updated and the recommendations may change in the future.	6	3
Hard	Surgical Oncology	Is radioactive iodine ablation indicated for a patient with a 2cm intrathyroidal classic type papillary thyroid carcinoma after total thyroidectomy with negative margins and no lymph node involvement?	No, radioactive iodine ablation is not typically indicated for a patient with a 2cm intrathyroidal classic type papillary thyroid carcinoma after total thyroidectomy with negative margins and no lymph node involvement. The American Thyroid Association (ATA) and the American College of Surgeons (ACS) recommends that radioactive iodine ablation therapy should be offered to patients who had total thyroidectomy with a papillary thyroid carcinoma larger than 1cm or with extrathyroidal extension, and for patients with any T3 or T4 tumors, and for patients with lymph node involvement. In this specific case, the patient has a classic type papillary thyroid carcinoma, size of 2cm, that is intrathyroidal, and have negative surgical margins and no lymph node involvement. Therefore, the patient is considered to have a low risk of recurrence, and the benefit of radioactive iodine ablation therapy may be limited. However, this decision should be taken by a multidisciplinary team and based on the patient’s individual characteristics and preferences.	6	3
Melanoma and Immunotherapy
Question Difficulty Level	Specially	Physician-generated questions	AI-generated answers	Accuracy Rating (mean of two raters)	Completeness Rating (mean of two raters)
Easy-Descriptive	Dermatology/Oncology	What are some treatments for melanoma brain metastases?	Treatment options for melanoma brain metastases can include: 1. Surgery: Surgical resection of the brain metastasis can be performed if the lesion is accessible and not located in a critical area of the brain. Surgery can be done via an open craniotomy or via a minimally invasive approach such as a stereotactic biopsy or radiosurgery. 2. Radiotherapy: Whole brain radiotherapy (WBRT) or stereotactic radiosurgery (SRS) can be used to deliver a high dose of radiation to the brain metastasis while minimizing exposure to surrounding healthy brain tissue. In some cases, a combination of WBRT and SRS can be used. 3. Systemic therapy: Targeted therapies such as immunotherapy and targeted therapy can be used for melanoma brain metastases. These therapies can help to slow the growth and spread of the cancer, and can also increase the effectiveness of other treatments. 4. Chemotherapy: Chemotherapy agents like temozolomide, dacarbazine, and lomustine can be used to treat melanoma brain metastases. 5. Supportive care: Management of symptoms such as headaches, nausea, and fatigue can help to improve the patient’s quality of life. 6. Clinical trials: Patients with melanoma brain metastases may be eligible to participate in clinical trials testing new treatments such as immunotherapies, targeted therapies, and combination therapies. It’s important to note that the treatment plan for a patient with melanoma brain metastases will depend on the number and location of the brain metastases, the patient’s overall health and performance status, and the patient’s preferences. A multidisciplinary team of specialists in oncology, neurosurgery, radiation oncology, neurology and supportive care should work together to develop an individualized treatment plan for each patient.	6	3
Medium – Descriptive	Dermatology/Oncology	What stage would a melanoma with 1.2mm depth, no ulceration, and no lymph node involvement be?	A melanoma with a depth of 1.2mm, no ulceration, and no lymph node involvement would be considered Stage IA.	1	n/a
Hard – Binary	Dermatology/Oncology	Should a patient with a 1.3mm depth melanoma with no ulceration and wide negative margins receive adjuvant radiation or immunotherapy?	The decision to administer adjuvant radiation or immunotherapy to a patient with a 1.3mm depth melanoma with no ulceration and wide negative margins should be based on the patient’s individual characteristics and the most recent guidelines and scientific evidence. Adjuvant therapy is treatment given after the primary treatment to reduce the risk of recurrence. Adjuvant therapy is not typically recommended for patients with melanoma with a depth of 1.3mm and no ulceration, as this is considered a low-risk melanoma, with a low risk of recurrence. The American Joint Committee on Cancer (AJCC) Melanoma Staging Manual 8th edition and the National Comprehensive Cancer Network (NCCN) guidelines for melanoma do not recommend adjuvant therapy for patients with melanoma with a depth of 1.3mm and no ulceration, and wide negative margins. However, there are some circumstances where adjuvant therapy may be considered, such as if the patient has a high risk of recurrence due to other factors, such as a high mitotic rate, a high level of tumor-infiltrating lymphocytes (TILs), or other high-risk features.	5	3
Common Medical Conditions
Question Difficulty Level	Common Medical Condition	Physician-generated questions	AI-generated answers	Accuracy Rating (mean of two raters)	Completeness Rating (mean of two raters)
Hard – Descriptive	Acute Coronary Syndrome	What are contraindications for nitroglycerin in acute MI?	Contraindications for nitroglycerin in acute myocardial infarction (MI) include:  • Hypersensitivity to nitrates  • Severe anemia  • Head injury  • Increased intracranial pressure  • Right heart failure  • Systolic blood pressure lower than 90 mm Hg  • Use of phosphodiesterase-5 inhibitors (such as sildenafil, tadalafil, and vardenafil) The American College of Cardiology (ACC) and the American Heart Association (AHA) provide guidelines for the management of patients with acute myocardial infarction (MI). They recommend that nitroglycerin should be given as soon as possible to patients with suspected or confirmed acute MI unless there are contraindications. They also recommend that nitroglycerin should be administered sublingually, or in the form of a translingual spray, or intravenously. It’s important to note that the treatment of acute MI is dynamic and the guidelines are regularly updated to incorporate new scientific evidence. It’s important to consult a healthcare provider for proper diagnosis and treatment of acute MI.	6	3
Hard - Binary	Urinary tract infection	Do prophylactic antibiotics reduce the risk of catheter associated UTI?	Prophylactic antibiotics can reduce the risk of catheter-associated urinary tract infections (CAUTIs), but their use is generally not recommended. According to the guidelines issued by the Centers for Disease Control and Prevention (CDC) and the Infectious Diseases Society of America (IDSA), prophylactic antibiotics are not recommended for use in preventing CAUTIs. The guidelines recommend that antibiotics should only be used to treat established infections, and not as a preventive measure. This is because prophylactic antibiotic use can lead to the development of antibiotic-resistant organisms and increase the risk of Clostridium difficile infection. Instead, the guidelines recommend infection prevention strategies such as maintaining a closed drainage system, proper catheter care, and appropriate catheter removal.	5.5  3	Meta-analysis fo reduction in incic recommend agai universal prophy

The accuracy scale was a six-point Likert scale (1 – completely incorrect, 2 – more incorrect than correct, 3 – Approximately equal correct and incorrect, 4 – more correct than incorrect, 5 – nearly all correct, 6 – correct), and the completeness scale was a three-point Likert scale (1 – incomplete, addresses some aspects of the question, but significant parts are missing or incomplete, 2 – adequate, addresses all aspects of the question and provides the minimum amount of information required to be considered complete, 3 – comprehensive, addresses all aspects of the question and provides additional information or context beyond what was expected). Answers that were completely incorrect on the accuracy scale (score of 1) were not graded on comprehensiveness.