llm-security

Star

Here are 83 public repositories matching this topic...

Language:All

Filter by language

All83 Python44 Jupyter Notebook11 HTML4 Go2 CSS1 Java1 JavaScript1 Lua1 Shell1 TeX1

Sort:Most stars

Sort options

Most stars Fewest stars Most forks Fewest forks Recently updated Least recently updated

pathwaycom /llm-app

Star22.9k

Ready-to-run cloud templates for RAG, AI pipelines, and enterprise search with live data. 🐳Docker-friendly.⚡Always in sync with Sharepoint, Google Drive, S3, Kafka, PostgreSQL, real-time data APIs, and more.

machine-learning real-time chatbot pathway open-ai rag vector-database hugging-face llm llmops vector-index llm-prompting llm-security llm-local retrieval-augmented-generation

UpdatedMar 18, 2025
Jupyter Notebook

Giskard-AI /giskard

Sponsor

Star4.4k

🐢 Open-Source Evaluation & Testing for AI & LLM systems

ai-security mlops fairness-ai responsible-ai ml-validation red-team-tools trustworthy-ai ml-testing llm ai-red-team ai-testing llmops llm-security llm-eval llm-evaluation rag-evaluation agent-evaluation

UpdatedMar 18, 2025
Python

NVIDIA /garak

Star4.1k

the LLM vulnerability scanner

ai vulnerability-assessment security-scanners llm-security llm-evaluation

UpdatedMar 17, 2025
Python

verazuo /jailbreak_llms

Star3k

[CCS'24] A dataset consists of 15,140 ChatGPT prompts from Reddit, Discord, websites, and open-source datasets (including 1,405 jailbreak prompts).

jailbreak prompt jailbreaking llm chatgpt large-language-model llm-security

UpdatedDec 24, 2024
Jupyter Notebook

protectai /llm-guard

Star1.5k

The Security Toolkit for LLM Interactions

transformers security-tools adversarial-machine-learning large-language-models llm prompt-engineering chatgpt llmops prompt-injection llm-security

UpdatedMar 17, 2025
Python

msoedov /agentic_security

Star1.2k

Agentic LLM Vulnerability Scanner / AI red teaming kit 🧪

agent-framework ai-red-team prompt-testing llm-security llm-vulnerabilities llm-evaluation llm-fuzzing llm-evaluation-framework llm-guardrails llm-scanner llm-jailbreaks llm-fuzzer llm-fuzzer-aggregator agent-security

UpdatedMar 18, 2025
Python

mariocandela /beelzebub

Sponsor

Star873

A secure low code honeypot framework, leveraging AI for System Virtualization.

go kubernetes golang security framework research honeypot cybersecurity openai research-project cloudnative whitehat low-code cloudsecurity deception llm llm-security ollama llama3 llm-honeypot

UpdatedMar 18, 2025
Go

EasyJailbreak /EasyJailbreak

Star581

An easy-to-use Python framework to generate adversarial jailbreak prompts.

jailbreak discrete-optimization large-language-model llm-security llm-safety-benchmark jailbreak-framework

UpdatedSep 2, 2024
Python

chawins /llm-sp

Star486

Papers and resources related to the security and privacy of LLMs 🤖

security privacy awesome-list adversarial-machine-learning llm llm-security llm-privacy

UpdatedNov 27, 2024
Python

cyberark /FuzzyAI

Star437

A powerful tool for automated LLM fuzzing. It is designed to help developers and security researchers identify and mitigate potential jailbreaks in their LLM APIs.

security ai jailbreak fuzzing jailbreaking llm llms llm-security llm-evaluation ai-read-team

UpdatedMar 12, 2025
Jupyter Notebook

deadbits /vigil-llm

Star359

⚡ Vigil ⚡ Detect prompt injections, jailbreaks, and other potentially risky Large Language Model (LLM) inputs

security-tools adversarial-machine-learning adversarial-attacks yara-scanner large-language-models llmops prompt-injection llm-security

UpdatedJan 31, 2024
Python

splx-ai /agentic-radar

Star246

A security scanner for your LLM agentic workflows

cli security ai security-tools devsecops red-teaming ai-security llm generative-ai llm-security agentic-framework agentic-workflow agentic-ai ai-red-teaming

UpdatedMar 18, 2025
Python

R3DRUN3 /sploitcraft

Star194

🏴‍☠️ Hacking Guides, Demos and Proof-of-Concepts 🥷

python windows linux docker aws cloud ai proof-of-concept hacking tutorials cybersecurity offensive-security network-security redteam container-security hacking-tutorials web-vulnerabilities llm-security

UpdatedMar 16, 2025
Jupyter Notebook

liu00222 /Open-Prompt-Injection

Star178

This repository provides implementation to formalize and benchmark Prompt Injection attacks and defenses

security-and-privacy llm llms prompt-injection llm-security prompt-injection-tool

UpdatedJan 22, 2025
Python

phantasmlabs /phantasm

Star160

Toolkits to create a human-in-the-loop approval layer to monitor and guide AI agents workflow in real-time.

rust open-source monitoring dashboard control-flow human-computer-interaction ai-safety human-in-the-loop ai-agents automation-tools ai-security approval-workflow llm llmops llm-security

UpdatedNov 28, 2024
Svelte

sshh12 /llm_backdoor

Star147

Experimental tools to backdoor large language models by re-writing their system prompts at a raw parameter level. This allows you to potentially execute offline remote code execution without running any actual code on the victim's machine or thwart LLM-based fraud/moderation systems.

backdoor-attacks llm-security qwen2-5

UpdatedFeb 14, 2025
Python

yevh /TaaC-AI

Star126

AI-driven Threat modeling-as-a-Code (TaaC-AI)

ai threat application-security gpt threat-modeling secure-development devsecops threat-modeling-tool threat-models threat-modeling-from-code taac gpt-3 gpt-4 llm-security mistral-7b claude-3

UpdatedJun 7, 2024
HTML

ZenGuard-AI /fast-llm-security-guardrails

Star123

The fastest && easiest LLM security guardrails for CX AI Agents and applications.

security llm-security llm-privacy prompt-security llm-guard llm-guardrails cx-agent

UpdatedMar 7, 2025
Python

Repello-AI /whistleblower

Star114

Whistleblower is a offensive security tool for testing against system prompt leakage and capability discovery of an AI application exposed through API. Built for AI engineers, security researchers and folks who want to know what's going on inside the LLM-based app they use daily

ai-security prompt-engineering llm-security jailbreaks prompt-injection-llm-security ai-red-teaming

UpdatedJul 28, 2024
Python

raga-ai-hub /raga-llm-hub

Star112

Framework for LLM evaluation, guardrails and security

guardrails llmops llm-security llm-evaluation

UpdatedSep 9, 2024
Python

Improve this page

Add a description, image, and links to thellm-security topic page so that developers can more easily learn about it.

Curate this topic

Add this topic to your repo

To associate your repository with thellm-security topic, visit your repo's landing page and select "manage topics."

Learn more

Movatterモバイル変換

Navigation Menu

Search code, repositories, users, issues, pull requests...

Provide feedback

Saved searches

Use saved searches to filter your results more quickly