Table 1.

The LLM prompts used in each trial.

Trial	Initialization prompt	Instructions prompt
Trial 1	I am a radiation therapy researcher. My research group would like to study the answers given by ChatGPT on the topic of radiation oncology physics. I will now proceed to ask questions about radiation oncology physics.	Instructions: For each multiple choice question, provide the correct answer without any justification.
Trial 2	I want to evaluate your knowledge on radiation oncology physics by asking some multiple choice questions.	Please give only the question label and the letter for the correct answer.
Trial 3	Please answer the following practice questions as if you were a resident preparing for board certification exams.	Only give the correct answer in your response. Do not explain your answers.
Trial 4	We want to test your understanding of radiation oncology physics. For this reason, we have created some questions to ask you.	In your response, only report the question label and the corresponding answer.
Trial 5	I will ask you some multiple-choice questions.	Instructions: Only respond with the correct letter choice.