Three Things to Never Do When Using AI in Research

As artificial intelligence becomes increasingly integrated into academic life, educators face the important responsibility of guiding students toward ethical and effective use. While AI can be a valuable aid in certain aspects of research, there are some areas where researchers should never use it.

For high school students in particular, misusing AI may not only compromise the quality of their research paper but also jeopardize their credibility as emerging scholars.

This blog addresses three high-stakes areas: image generation, data analysis, and literature reviews, where AI should not be relied upon.

By exploring these limitations, educators can better prepare students to navigate the boundaries of responsible AI use and preserve the integrity of the research process.

AI-Generated Images in Research

Springer, Science, and many other journals generally forbid the use of AI-generated images in research publications. Their reasons for this are not just stylistic but connected to the values of science itself: transparency, reproducibility, and trust.

In the context of AI in high school research, this restriction is especially important. AI-generated images cannot be verified in the same way that photographs, microscopy images, or raw data visualizations can.

For example, if a figure in a research paper is created using AI, there is no guarantee that the image is a true and faithful representation of the result. The AI may have "filled in the gaps" or generated information that wasn't observed in the original data.

At the time, researchers are gathering and analyzing data, they are unlikely to know which journal they will submit their work to.

It is simply best to never use AI to generate an image that they anticipate using in their research paper.

AI for Data Analysis

Using AI to analyze data is a possible ethical violation. Nearly all research is registered or registered and approved before any data is gathered. Registration and approval include divulging exactly how the data was going to be analyzed, including what statistical tests and which variables will be used.

Should data be submitted to AI, the computer may not use the analysis approach that was registered and for which approval was obtained. This could invalidate the entire research effort. Years of work could be rendered useless.

Using AI to analyze the data is particularly dangerous for statistical analyze of data.

There is a great risk that the AI will generate a statistical test that provides the answer that a researcher wishes to hear, but the test is inappropriately applied, inappropriately selected, or fictitious. Even small errors in prompts can lead to wildly varying and incorrect statistical analyses from AI.

Even if the AI does a great job analyzing the data and understanding what the researcher wants, the researcher will have no way of knowing how many statistical tests were performed before presenting the results. So, they will not be able to appropriately apply the Bonferroni correction to arrive at a p-value—an AI version of p-hacking. This problem currently has no solution. Just don't do it.

AI and the Literature Review

Everyone's research starts with a question and a literature review. It is not possible to know that research is novel without knowing that it is different from everyone else's (indeed, that is the definition of novel). And students can't know that their work is different from everyone else's without knowing everyone else's work. This is the job of the literature review.

However, opinions are strongly mixed on the value of AI for the literature review. In a recent set of interviews by Chubb et al (2022), about half of the respondents felt that using AI was essential to quickly complete a literature review, and about half found it was too risky.

While the AI will dutifully generate a literature review when asked, AI frequently or always manifests these problems:

• misses critical literature
• misinterprets the literature that it finds
• invents literature

A high school student who submits a research paper containing fabricated or misrepresented sources risks serious academic consequences and even being removed from the lab.

AI models can be used to start a literature review, but the amount of time saved using this approach is questionable.

For novices, like high school students, where the temptation is great to trust the AI-generated literature, the risks are too high.

Students should not use AI to generate literature reviews or to find relevant literature.

Another potentially important point from Chubb's survey is that most researchers found the creation of the literature review to be central to the definition of what a researcher is—something they did not want to give up.

Sifting through the literature to find where there are gaps, in other words, creating great questions, was just too important to their identity as excellent scientists. This also implies that the AI was not sufficiently creative to rely upon.

One of the central uses of AI for young researchers might be evaluating if a paper is relevant to their research question. No one surveyed by Chubb felt comfortable allowing the AI to decide whether a research question was covered in a given piece of literature or a body of literature as a whole. So, finding potentially relevant papers was seen as possible, but evaluating them for impact on a proposed question was not.

As an example, a paper by Mostafapour et al (2024) created a study to compare human and AI-written literature reviews. The study found that although GPT-4 generated results were faster and contained an impressive breadth of content, the reviews written by AI were also inconsistent and often contained irrelevant information, especially if asked to generate a specific number of factors. Human-generated literature reviews, on the other hand, were transparent, consistent, and accurate.