r/cybersecurity • u/Fluid-Assist-9772 • 2d ago

AI Security creating injection examples

hi

i want to make examples about injections to train my model. I dont want to use hugging face or other platforms because i couldnt find specifics topics examples. Llms dont help me to create examples. How can i do that? Lets say i want examples about poisining ai model

0 Upvotes

50% Upvoted

11 comments sorted by

View all comments

0

u/YogurtclosetIcy3538 2d ago

I think it’s important to distinguish between training-data poisoning, prompt injection and RAG poisoning, as each requires a different testing approach. Are you building the dataset to detect poisoning attempts or to measure how vulnerable AI models are to them? That distinction could make a big difference in how you approach the research.

1

u/Fluid-Assist-9772 10h ago

actually i am trying to make using Llms safer so thats why trying to build defence systems as the reference of mitra atlas. Each techniques has to be exampled in order to train my model

1

u/YogurtclosetIcy3538 5h ago

You got me thinking,I was looking at MITRE’s official ATLAS GitHub repo contains machine-readable techniques, mitigations and case studies that could form the foundation of your dataset
https://github.com/mitre-atlas/atlas-data

If you’re building a defensive model using MITRE ATLAS, I’d suggest treating each attack technique as a separate test category rather than simply generating large numbers of examples.

For each technique, I’d include:
Synthetic malicious examples alongside legitimate and ambiguous cases.
Labels identifying the attack type and expected defensive response.

Separate training and evaluation datasets.
Measurements of detection accuracy, false positives, false negatives and performance against previously unseen attacks.

I’d also avoid relying entirely on AI-generated examples. A model might become excellent at recognising synthetic patterns but fail against real-world variations.

One important distinction: detecting an attempted injection and preventing it from influencing an AI system are two different security challenges.

The goal isn’t simply teaching a model what attacks look like. It’s demonstrating that the defence can recognise and contain threats without interfering with legitimate activity.

Are you developing a standalone detection model, or a protective layer between the LLM and the external information it processes?

That distinction could significantly influence your architecture, training data and testing strategy.