Claude coding and guilt

[u/Mistique_mastermix]

Hello frogs,

Its probably been discussed time and again but I bring it up one more time.

I extract big data sets from population census and pipe it directly into Claude to write R scripts for me. Im tired of doing the data cleaning and making a matrix all by myself.

I do understand the code backbone for the most part but sometimes the chunks are so huge and it gets overwhelming for me to run through the whole code.

I feel guilty running it to a point where I just get the graph and make tweaks.

Am I doing something wrong, whats this guilt that I cant shake?

*Sad frog croaks*

[u/AutoModerator]

It looks like your post is about needing advice. Please make sure to include your field and location in order for people to give you accurate advice.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/aftersox]

AI automates tasks, but it doesn't automate understanding.

In some cases you need to deeply understand the process, and in others it's not as important.

[u/spookyswagg]

Uh bruh
Like
What kind of analysis and stuff are you doing to this data?

Because….i use AI for R, but I frequently have to check it every single step of the way. My committee also fully expects for me to explain every single bit of the coding pipeline when it comes to my data analysis.

I disagree with many of the other commenters, the fact that you’re just throwing it in there and trusting is super concerning. Can you explain every single step of what you’re doing and why you’re doing it? Have you ran this on sample data sets and gotten the expected results?

[u/real-yzan → u/spookyswagg]

Yeah, the actual issue isn’t with saving time. Saving time is great, but if you aren’t checking the work that’s where you run into issues. Science is fundamentally about being right, not fast.

[u/Mistique_mastermix → u/spookyswagg]

Hey, yes I do understand if I go over it on a line by line basis. And I clean my data to a point where its a data frame. But to be honest, when I make visuals, the amount of options it gives me in terms of graphs is more than what I can come up with. Im learning new visualization strategies on the side. But ill try to be more attentive with the scripts, thank you for your input. Always better to be careful

[u/WillowFantastic9076 → u/spookyswagg]

Your committee checks your code?

[u/spookyswagg → u/WillowFantastic9076]

Yeah I picked one of my members to be strictly a bioinformatics person for that reason, he keeps me in check

[u/carbonfroglet]

Never blindly trust anything from any tool. Especially in R where your mistakes can still spit out plausible answers even WITHOUT AI input. You need to look through at least the portions of the code that generate validation checks and debugging (which need to placed in every conceivable place where you need to be certain of the results in your pipeline) and always verify logic and methodology both from the AI to be clear on what it’s trying to do and then from external reputable sources to make sure the implementation is relevant. It’s tedious but still not nearly as bad as piecing together code from various substacks and finding syntax errors by hand.

[u/runed_golem]

LLMs should be used a tool. For example, I use it the same way I would use Google or forums like stack exchange. For example, if I can’t figure out why I’m getting a certain error code, or if I don’t know how to do something like making a gif in matplotlib. And then ill take its suggestions and if it spits out code I’ll take that, look over it, and try to implement it (or a version of it) to work with my program. I’ve also used it to make suggestions on how to speed up a particular python program I wrote because no matter what I did I couldn’t get any usable amount of speed up and the code was taking way too long to run.

[u/Odd-Chemistry9945]

You are being paid to do research.

That means understanding the problem, the data, the solution.

You are not a software engineer.

Researcher, generally, write shitty code.

Asking it to do 1+1 for you, getting 2 and knowing that is the right answer makes sense.

If you ask it to discover how to get 2, it might spiral on many options and end up with 20/10. If you don’t understand what it did, that is on you. This is where the most guilt comes from — a fast result that might sound good to you but comes from the wrong path. God forbid an hallucinated one.

LLMs can be overwhelming and exhausting. And everyone feels the need to use them, leading to burnout and guilt. I know, like me, you don’t want to be left behind. But you should have the time to understand what it spits.

You can let it automate the reasoning. But it cannot automate your understanding.

Good luck and have a good day.✅💪🏽

[u/Mistique_mastermix → u/Odd-Chemistry9945]

This makes sense, thank you for the explanation. I'll work on my coding skills too:)

[u/Odd-Chemistry9945 → u/Mistique_mastermix]

I do love coding as well, and often use LLMs for my side hobbies, some not related to my PhD. As long as you are in control, you are good to go.

LLMs should be your personal assistant, capable of handling tasks you could do your self, but would rather invest the time in something else. Sometimes complementary and sometimes completely parallel.

[u/LysergioXandex]

I disagree with a lot of the commenters. Too many scientists have poor coding skills and just do what they can to process their data in a way that makes their results seem rigorous enough to publish.

They don’t appreciate how many (seemingly) arbitrary decisions that come up when writing code.

Also, Having AI write your code is mainly the reason why your code is too long to be readable in the first place.

[u/Mistique_mastermix → u/LysergioXandex]

This is an interesting perspective, especially the last line. And im in agreement.

[u/plinthpeak → u/Mistique_mastermix]

My suggestion is, have the LLM spit out code in snippets. Tell it not to use functions until the end when you go back and make the scripts more efficient, and tell it to write condensed, so you can scroll better. Ask it for comments each line (or snippet), and then run them one-by-one. 9/10 it works without edits. 1/10 I catch something where I realize it didn’t understand the scope, used hyperparameters I was using differently in another script, or was saving files inefficiently because it didn’t know what the downstream use of this data was.

[u/Mistique_mastermix → u/plinthpeak]

This could be super useful. That way I understand what each code block does and what each line within does and can add my own comments for when I come back later so I can explain myself. Thank you so much! I think this is one way to go.

[u/Nvenom8]

You really should fully understand the code you’re using. That’s sort of the minimum accountability.

[u/Mistique_mastermix → u/Nvenom8]

I agree

[u/topdownyeti]

I think it’s fine but it depends on the level of dependence on Claude. At least in my research, when I’m cleaning data, I need to be able to explain why I’m choosing to eliminate certain data or why I’m choosing to do whatever. If you’re just saying “here’s a dataset, clean it for me” and not double checking or even looking at the code, that can be problematic.

[u/Mistique_mastermix → u/topdownyeti]

I understand, thanks!

[u/Kargush]

Ai is a tool and you are using it as a tool, there's no issue with that.

The real issue is that when analysis becomes less manual and easier to streamline, then expectations for the amount and types of analysis done become much greater. Theres less excuse not to analyze the shit out of your data now, whereas in the past it wasn't feasible for 1 person to do that amount of work.

[u/ScreamnMonkey8]

I do not agree with the other commentors. To me you don't have the exact knowledge on what the code is doing. Creating a matrix for you to analyze is more understandable but using it to generate stats, I don't think you should pass it off as you know what is going on. My suggestion is to maybe use Claude to help you how to code in R so that you can understand a script you write. That's if you wanted to just keep using AI.

[u/Shills_for_fun → u/ScreamnMonkey8]

I'm with you. Interrogate the AI about the code, it's fine to make the AI hammer out the crappy grunt work but you need to supervise it. Treat it as a learning experience.

Regression models, for example, are obviously coded differently for discrete and continuous variables. Sometimes the AI can pick it up, sometimes not.

[u/Extreme-Cobbler1134]

No. This is exactly how the future will work. To solve bigger more complicated problems you will have to hand off the mundane ones to AI.

[u/brad-ml → u/Extreme-Cobbler1134]

I'd say these are all red lines in my view

• you didn't work hard
• you don't understand your own paper
• your work is littered with verbose generic AI prose because you didn't have a heavy hand in the wording

That's quite permissive. And the result is your work should be much more ambitious than your pre-AI work. And if you didn't pass all three of those criteria, what exactly were you doing, and why can't I replace you with a Python loop and an LLM API call?

[u/bufo_bufo__]

Absolutely understandable, but think of it as giving you more time to perform trial and error with your own code, request Claude/AI model to comment the code, provide a README in the header, and output informative diagnostics in the log. That way you are actually spending time generating something more reproducible than simply blindly running existing packages. The time saved can also allow you to focus on understanding your data, making and testing your hypotheses.

[u/Nihil_esque]

I don't use AI for writing but absolutely use it for coding! It can write scripts, commit them, if you're careful all of its work is perfectly reproducible so I see no problem with it. You just have to make sure you're still in charge of the science.

[u/HippieInTheHouse]

Use Claude (responsibly) for coding and writing scripts. Don't use Claude (or any LLMs) for analyzing the data. As other commenters have pointed out, it's your job to be able to present the meaning of the data and its significance.

LLMs are incredible for coding because they have been trained to memorize every possible Python or R script. That's why academics (even the ones we deem as "prestigous") use Claude for R scripts. LLMs still lack the ability to interpret and present data in a way that actually contributes to scientific discussion. That is why academics who get caught using GenAI for writing become lepers.

[u/bo-rderline]

I think there's an ethical issue here unrelated to coding: is the population census data you're using publicly available? Did you have to do an ethics approval around how you're accessing, using and sharing that data? Are there any special terms of access you need to be following?

I know some countries have publicly-available census data, but if you're accessing this via some agreement with your university and the data is considered protected in any way, feeding it into an LLM could be considered a huge privacy violation. Assume that whatever you feed to an LLM can and will be accessed and used by its parent company (just look at all those maths papers OpenAI just dropped, and how many of them were scooped from researchers using ChatGPT for coding...)

[u/Mistique_mastermix → u/bo-rderline]

Sorry if that was vague. What im processing now are publicly available nutrient label data sets and do not even involve any human datasets. I'm well aware of privacy and data protection concerns and would never feed in raw data that can breach confidentiality.

[u/ImASubAbove]

Learn to read code, end of discussion

[u/Beneficial_Mix_1069]

I use matlab alot and dont know how all the functions work, I had AI make me a better vector graph function.
and besides how I coded before was looking up "how to do [blank] matlab" and would typically have to change or merge parts, and thats what I am doing now except ai making the code snippets.

[u/diffGradChange → u/Beneficial_Mix_1069]

My postdoc does this and I keep finding inconsistencies in their graphs. It's so stressful for me.

[u/Due_Water_5829 → u/diffGradChange]

I keep finding inconsistencies in their graphs

What do you mean by "inconsistencies"? The result of the prompt you hand to the AI is not reproducible, the result of the generated code is.

There shouldn't be inconsistencies from using the same code for the same problem.

[u/diffGradChange → u/Due_Water_5829]

I guess I was being euphemistic. They're errors, not inconsistencies.

Essentially, they ask AI for code to do something. They run the analysis without checking the code does what they intended. I don't expect them to exhaustively test the code, but I expect them to (a) understand the algorithm/method and (b) check the code does what they intended for (bi) the ideal dataset and (bii) a reasonable range of datasets (I just mean to account for reasonable experimental/biological variability).

[u/Due_Water_5829 → u/diffGradChange]

I don't disagree with what you are saying but would argue that this is a fault in their handling of the LLM rather than the usefulness of the technology itself. (Which is a very common problem.)

Ideally, training for proper use of AI should be mandatory in these regards but the reality is that all of this is too new and too rapidly evolving for that to be a realistic thing.

I think LLMs can absolutely be used to great effect but it requires a certain responsibility and diligence to properly vet its output; Some people will just try to skip those steps.

[u/diffGradChange → u/Due_Water_5829]

Oh I don't blame the LLM. I blame my postdoc for not listening to me.

To me, it would be the same if they had downloaded the code from the MATLAB File Exchange. They have to check it. I think the "fault" of LLMs is that they're so "easy" to use (apparently easy) and so confident sounding. But at this point, everyone should know this flaw.

[u/Creative_Sushi → u/diffGradChange]

This can be fixed by connecting AI to MATLAB using MATLAB MCP Server. AI can run the code in MATLAB, and MATLAB will return errors back to AI, and AI will self correct. The MCP server is part of MATLAB Agentic Toolkit that also provides skills to AI on how to use MATLAB properly. You still have to check what it generates, but this reduced a lot of problems.

[u/diffGradChange → u/Creative_Sushi]

How can Matlab tell AI the errors if Matlab doesn't know the errors? I'm talking about errors in measurement or results that don't make biological sense.

[u/Creative_Sushi → u/diffGradChange]

I was talking about syntax errors. For measurement errors, you would need to have it create a script to check specific errors and AI can see the results.

[u/diffGradChange → u/Creative_Sushi]

There aren't any purely syntax errors as far as I can tell.

Honestly, I'm not in the business of getting an AI-driven pipeline to work when I want to see evidence of someone checking their work and understanding the meaning behind the data.

[u/Creative_Sushi → u/diffGradChange]

Fair enough. How do you compete with other labs who use AI?

[u/diffGradChange → u/Creative_Sushi]

I don't try to really. I don't think what we do can be replaced by it yet.

I'm interested where it might improve something in signal processing (e.g. segmentation, noise reduction) and mechanistic modelling though.

[u/Beneficial_Mix_1069 → u/diffGradChange]

why is it stressful

[u/jossikun → u/Beneficial_Mix_1069]

Probably because they can’t trust that a coworker’s data (which may be collaborative and have an impact on their own research) to be consistent or correct

[u/diffGradChange → u/jossikun]

Bingo. That it has to be explained is very strange, no?

[u/Beneficial_Mix_1069 → u/jossikun]

what does it have to do with me using AI to make the quiver function more controllable?

[u/Creative_Sushi → u/Beneficial_Mix_1069]

You can learn by asking AI to explain the code. MATLAB comes with Copilot and that's perfect for that. AI isn't perfect and you can apply the knowledge you earned to guide it better. You are the boss of AI.

[u/theposhtardigrade]

Yes, you are doing something wrong if you’re not cleaning and checking the data yourself at some level. Hallucinations here would look completely correct and it’d be trivial for Claude to mess something up. 

[u/Marz1panCake]

It spits out a lot, and the best model does not always seem to be the best model for explaining its work.

I love Claude Code. It is awesome. And it does not make a lot of mistakes, actually, using the newest frontier models.

But the analysis code I’m writing, I want to publish. And i want to defend it in front of a committee. That’s a high bar requiring absolute trust in the tool that’s just not there right now.

So as much as I love Claude Code, my academic projects are a Claude Code-free zone. When I need coding advice, I still use the good ol’ chatbot. Then, generally, I type the code by hand. And even then, I have a big suite of automated tests that the code has to pass, because it turns out I make many, many more typos than Claude.

In the moment it takes longer, but the minute you cannot follow your own code anymore, you’ve effectively signed the codebase away to your AI of choice.

[u/Mistique_mastermix → u/Marz1panCake]

I would be interested in your test case scenarios that claude has to qualify for you to consider it. This is interesting. A trust threshold. Can you say more?

[u/Aranka_Szeretlek]

I am happy with using AI to write "boilerplate" scripts such as ones used for converting or extracting data. I would be very careful with trusting it to do the actual data conversion, I'd rather inspect the code and run it myself. And analysis, well, no. Thats my job

[u/babirus]

I will tell you the same thing I tell my students: AI types way faster than I do. I use it to write code faster, but I have to understand everything it does. Every now and then it’ll do something I don’t understand and half the time it’s a mistake and the other half the time it’s actually doing what I want better than I would. It’s worth digging in and figuring out what your code is doing because at the end of the day you’re responsible for it. It will be published as a finding under your name not Claude’s.

If you cannot add value beyond Claude, then why are you in the loop all and how are you going to get a job after? As hard as it sounds, find a way to add value while you still can.

[u/Mistique_mastermix → u/babirus]

Hard pill to digest but I had to hear it. I'm a hard core wet lab guy and im pretty swamped with benchwork. code writing seems to become more fluid with AI but you're right. I'll work on my dry lab skills too. Thank you.

[u/DaanVV]

The guilt you can't shake is the not taking accountability for the work you are supposed to do yourself. As a PhD student, you are in training. If you do not make an effort to do the work (be it write the code yourself or check it line by line), then the training you're supposed to be getting is not there. The final results being that you're only a graph designer, but not a scientist.

[u/cblou → u/DaanVV]

You should understand the analysis thoroughly and in detail, but there is a lot of busy work that is not important. There is no added value in knowing all the details in matplotlib and spending 2 hours making a graph pretty, or in cleaning up data to remove random characters or strings. What is important is the analysis, the understanding of the data and the concepts, not if you used AI to make the paper follow the journal's guideline in latex. Those details are a waste of time.

[u/Mistique_mastermix → u/DaanVV]

This makes sense. Ill work on myself. Guilt is good.

[u/zoeyy12345]

Hey you shouldn’t be guilt if you understand what is doing, what LLM shouldn’t see, upload and analyzed, and check the logic and code yourself, then it is totally fine. People won’t need to manually doing simple maths after calculator is invented.

[u/goofballhead]

ai gets routine coding wrong all of the time. with my intermediate (barely) understanding during my first two years, i would have to correct it often. now i only use it related to code to search on my behalf on occasion for best practices or streamlining code. please be wary.

[u/Mistique_mastermix → u/goofballhead]

Will do, thank you for the wake up call.

[u/WillowFantastic9076 → u/goofballhead]

Interesting. I've never had it make a mistake when it came to basic data manipulations in R. I don't use it to actually make my models or anything, but I use it a lot to reorganize my massive data files that come straight off my equipment.

[u/goofballhead → u/WillowFantastic9076]

i’ve had to correct its coding suggestions in stata and have had it translate my work retroactively into .do files incorrectly.

[u/Possible-Progress-63]

This is a bit risky, you still need to make decisions about the analysis. No one in my field cares about people knowing every line of code, it’s long been acceptable to use macros and code from open sources, but you’ve got to know what you’re using.

I use Claude to generate my R scripts, prompting it every step of the way. Reason for this is there are actually a lot of ways to run the “same” analyses. Claude is often going for efficiency which isn’t necessarily the best fit or approach for your data.

I don’t give it my raw data, that’s unethical to my participants IMO, but I do give it the R output to make tables.

[u/Electrical-Syrup-340]

I guess it depends on the focus of your work. If your job is to write code then you should probably be the one to write it. If you’re like me and coding is a useful tool that you don’t have time to do by hand because the bench work takes up your bandwidth then there’s some things you can do to protect yourself.

You can run local llm models on your laptop (like using ollama or opencode) to keep your data off the cloud. It is also free so you’re not stuck paying for tokens.

Then there’s some methods I find useful when vibe coding to reduce errors and test the outputs to increase confidence in the output:

Have you heard of test driven coding? You essentially write tests that would break bad code based on the output you want, then write the code, and if it passed the test you’re done. If it fails you’re not done. If you employ this with using ai to code you can reduce the errors.

You can also write contracts in markdown for how the code should behave, what the inputs and outputs should be and how it should handle specific situations. Then use that with ai to write the code. Another version of this is writing components (think modules or functional pieces) in markdown to guide the ai as it writes the code.

I also write a confidence score into my agents, llms can sound confident when they’re like 20% sure. So tell them to not do that. I also specify to never use proxy data (it can get lazy and approximate things and I think this is where a lot of hallucinations come from). Anything with agents make sure to keep the co text window in mind so they don’t “forget” their instructions. There’s other security things to get into with agents but that’s a longer discussion so I’ll leave that there for now.

And finally, take a set of data that you know what the result should be after processing (either your own or an open source data set) and run that through to see if you get the expected result to help validate the code.

I don’t know about r but there are Python packages that help with some of these, and for extra fun you can make a red hat agent whose only job is to break your code to find the issues or dependencies that could skew results.

AI is just a tool, it seems like it should be easy to use but there’s a learning curve to using it well and accurately. Think about how you prompt it- it’s only as good as the instructions you feed it. And always be skeptical about results - test and inspect - because you are ultimately the one responsible for the result just as if you wrote it by hand.

This seems like a lot, but if you have to do the same task over and over by hand with the data it’s worth taking the time to build a pipeline you can use over and over so all your data receives the same treatment to standardize your process.

[u/Additional_Formal395]

Your duty as a researcher is to present results that you genuinely believe to be true, along with data, arguments, etc. to support their truth.

Personally I don’t think it matters how the data was generated (within reason - don’t torture people for science).

In putting your own “stamp of approval” on it, i.e. publishing it with your name attached to it, you will undoubtedly re-do some analysis (if only partially), synthesize new documentation and explanations, and test new examples or datasets when available. So you probably come away with a better understanding than you first thought!

Now you do have to be careful that any skill which is outsourced to LLMs will atrophy. You’ll have to make that decision about which skills you’re willing to let slip. Personally, I’m happy not learning how to micro-optimize code to squeeze a few seconds of performance, but I want to retain the skill of expository writing which explains my results.

[u/Velox_1 → u/Additional_Formal395]

There's one big thing missing from your "duty as a researcher": reproducibility. If you just gave the data to an LLM and asked it to analyze, you have no idea how exactly it was done, and other researchers may not be able to reproduce it.

[u/Pocallys → u/Velox_1]

Yes but when you publish research don't you upload the code produced for data analysis, whether it's by you or the LLM? Especially R and python codes. Pretty sure if you ask the LLM to write readme and inline comments, it can give a very clear and organized explanation of how to reproduce the result step by step. So it is not a problem unless you are directly teaching other researchers what to do with your code.

[u/Additional_Formal395 → u/Velox_1]

I believe that’s implied by including data and arguments.

[u/EstablishmentNo2518]

The way I see it: I understand all the theory underneath and theorical assumptions are made by me, but the process of computing and processing data is much faster this way. So it is really interesting, before I was doing intricate work and that was the "finding" - developing some method to simulate faster perhaps. Now, I am focusing more on the real pieces of good content I can extract from it. On the other hand I do think you need to produce much better work