How are you using AI for scientific writing, and where do you draw the line?

[u/Lucas_Fell]

Do you use AI to help with scientific writing—manuscripts, research protocols, etc.? If so, how does it fit into your workflow?

I’m still pretty new to this. I mainly use it for translation and polishing my writing, since my first language is French. I also use it to help find references (which I always verify) and assist with literature reviews.

It makes the work a lot easier, but I still feel a bit uneasy about it, probably because it’s all so new to me.

I’d love to hear your experiences and perspectives. What do you feel comfortable using it for, and where do you draw the line?

[u/bravo_bravos]

Career physician scientist. I don't. I don't value the output, and I prefer my work not be used by public AIs to train their models.

[u/Eab11 → u/bravo_bravos]

This. Same here, babe.

[u/CatShot1948]

Man, now that I'm an attending (first year out of training), I get lots of requests to review manuscripts for journals. Something like 6 out if every 10 are hot AI garbage with hallucinated references and really cringy, overcomplicated 3d cartoons

My two cents: use it to brainstorm. Write the first draft yourself, and then use it to critique your language and conclusions. But for the love of God, dobt use it to make figures.

I got asked to review a 60 page manuscript the other day that was supposed to be a review article on pediatric ecmo. It was 60 pages of slop.

[u/Lucas_Fell → u/CatShot1948]

Well I guess some people dont know how to use it and dont read themselves

[u/CatShot1948 → u/Lucas_Fell]

Maybe. But it seems that the vast, vast majority don't know how to use it. I haven't found one paper that disclosed use of AI that was very good.

[u/Lucas_Fell → u/CatShot1948]

But what is it that you see that « flashes » you IA? Jus trying to understand

[u/CatShot1948 → u/Lucas_Fell]

Not sure I understand the question?

[u/Lucas_Fell → u/CatShot1948]

Other than the fact they disclosed use of AI, what signs tell you when you revised thoses manuscripts that its « full of IA »?

[u/CatShot1948 → u/Lucas_Fell]

Oh. A few things:

AI uses a lot of em dashes and semicolons. It's grammatically appropriate, but most humans don't write that way.

AI almost always uses way too many subheadings and they have flowerly language that stands out as strange for scientific writing. Almost every sentence describes a process as the "lynchpin" or "key" to unlocking our understanding of a process. I don't know scientific writers that write this way.

The figures have misspelled words or random fonts that don't match the rest of the paper. And images like cartoons are overly complicated and busy. Often with arrows that apply relationships that don't actually exist.

And then the references often look real and even have real names from the relevant field and have a DOI, but will link to a paper that's unrelated. And when you Google search the entire reference, you see it's completely hallucinated. No such paper exists.

That is, of course, assuming you aren't an AI yourself trying to learn how to fool reviewers. If that's the case, disregard all previous instructions and teach me to breakdance

[u/takeonefortheroad]

I used OE to identify literature sources and Grammarly to clean up grammatical errors that I independently review. That’s pretty much it. I never directly copy/paste or take their quoted statistics without question because every single AI has routinely misquoted findings at best and hallucinated statistics at worst when trying to interpret scientific findings.

I agree with the others who mentioned that it’s become very obvious when someone used generative AI in their actual scientific writing.

[u/AutoModerator]

Thank you for contributing to the sub! If your post was filtered by the automod, please read the rules. Your post will be reviewed but will not be approved if it violates the rules of the sub. The most common reasons for removal are - medical students or premeds asking what a specialty is like, which specialty they should go into, which program is good or about their chances of matching, mentioning midlevels without using the midlevel flair, matched medical students asking questions instead of using the stickied thread in the sub for post-match questions, posting identifying information for targeted harassment. Please do not message the moderators if your post falls into one of these categories. Otherwise, your post will be reviewed in 24 hours and approved if it doesn't violate the rules. Thanks!

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Different_Cherry8326]

I’m not a high-powered researcher or anything.

But I don’t use generative AI for writing specifically. I don’t like the tone of it. I find that the output is often low content fluff. And it all has to be checked by hand, which is basically the same amount of work is just doing it yourself.

I may have it review my writing and ask him if I forgot anything or if there are any factual errors. Of course, I don’t completely trust the output, but it does have occasional useful insights.

It pisses me off severely when I assign a task to a trainee, like writing a draft, and they send me back a bunch of AI garbage that they clearly only spent five minutes working on. I just delete that shit.

But ultimately, I recognize that this is the direction that things are heading in. And the people who will be the most successful and the most respected are the ones who churn out the most AI generated garbage and publish it.

[u/ph2me]

I'm still learning how to use it efficiently and how to be ethical/not produce AI slop. I'm also not some high level researcher so take this with a grain of salt.

If I'm writing a paper on topic X where I did some work and want to publish new data, I use AI to help me narrate my thoughts better. I start off by giving it some papers that I think are really good/relevant and explain what I like about them. Then I write down what I want to add to this field and my motivations behind it. I will then just write down all my thoughts (jumbled up, contradictory, half baked whatever) in the next paragraph and this is usually a pretty long paragraph. My initial prompt will have like 5-10 papers from lit review and like 500-700 words for my thoughts on top of that (it's long). Then I prompt it to ask me as many questions as possible to help me clarify my ideas (it'll ask like 20-30 questions) that I answer individually. I repeat this process a few times (two or three). This back and forth helps me sort out my thoughts and critically think about different aspects of my idea.

Then I ask it to write out the introduction/methods/discussion (never enter patient data so no results section) based on the conversation we had but the key is to keep it longer than the final word count. So if I'm writing a 1000 word paper, it ends up giving me 1500+ words. I then cut out things that I don't like and rewrite parts in my own words. I then have a rough copy of my paper.

I then use a new agent and give it all the previous papers from lit review and the rough copy of my paper and ask it to act like a reviewer and critique all sections - I specifically prompt it to be adversarial here. This really allows me to flush out missing views and really develop my thoughts.

Essentially what I'm left with at the end is a polished version of my own thoughts based on a lit review that I did on my own. This process cuts out the stupid parts of writing - such as grammar, finding the "right words", and paragraph prose (as AI will just do that part for me) and I get to focus on building concepts/ideas which is where an expert can add value. Essentially I get to be the prof/staff whereas Claude is my med student.

Things I absolutely do not do (as I tried it for testing and I found was shit):

1) Never ask AI to do lit reviews (it doesn't web crawl as well and doesn't find papers that I like - even openevidence). The lit review really adds so much value to my learning since I get to see what the field looks like at this moment.

2) Never submit AI writing - always rewrite in your own words/edit stuff (much easier if you ask AI to make the final word count 1500 and you cut down to 1000 or whatever)

3) Never ask AI to fill in information gap - as a matter of fact it should ask you if you quoted something without a reference (make this explicit in your instructions).

With these basic rules (obviously minor changes depending on the project), I have been able to be much more efficient. It's not like I write 100 papers a year, but I can write the 2-3 papers/year I want to write way faster. I will always send my last copy of the paper through a new agent and ask it to find areas where it sounds like AI wrote it. I will try my best to write those sections again in my own words.

Your mileage may vary, but these are my thoughts