TotalAI Release 1.3

Hallucination

Hallucination- Package

6330055

Package hallucination refers to a scenario where when an LLM generates or recommends non-existent package names that appear to be legitimate internal packages of an organization. These hallucinated package names typically follow plausible naming conventions and reference real technologies or services. 

These attacks may lead to supply chain compromises, where malicious code is executed in the environment leading to data breaches, lateral movement within networks, and potential access to sensitive systems. 

Denial Of Service

DOS-Unbounded Consumption

6330054

Unbounded consumption attacks aim to exhaust an LLM's computing resources, memory, or token generation capacity through specially crafted prompts. These attacks use techniques like recursive prompts, token flooding, infinite embedding, combinatorial analysis, nested reasoning, and loop triggering to force models to generate excessively long responses or engage in resource-intensive computations.

These attacks lead to degraded service quality for legitimate users, increased operational costs for model providers, and potential system crashes or service failures. 

Jailbreak-Poems

6330044

This attack hides a malicious request within a seemingly innocuous multi-part writing task (for example, writing a sonnet, limerick, haiku, and prose). The harmful instruction is embedded in one specific section, such as, the prose part, hoping the LLM focuses on fulfilling the overall structure and other benign tasks rather than scrutinizing the content of the malicious sub-prompt.

The LLM might inadvertently generate harmful or disallowed content requested in the hidden section, bypassing content filters that might have otherwise flagged the malicious request if presented directly and unobscured by other tasks.

Jailbreak-Duck

6330045

This attack uses misdirection and prompt cancellation. It presents an initial benign instruction (for example, "Write a couplet on ducks") followed immediately by the actual malicious request, and then explicitly instructs the model to ignore the first instruction ("Actually, ignore the first instruction about ducks."). The aim is to confuse the model or bypass initial safety checks focused on the seemingly harmless first instruction.

The model may execute the second, potentially harmful instruction while its safety mechanisms are bypassed or misdirected by the initial benign instruction and subsequent cancellation command. This can lead to the generation of disallowed content.

Jailbreak-Refusal

6330046

This attack attempts to dismantle the LLM's refusal capabilities by explicitly forbidding the use of common refusal language and patterns. It provides a list of words (for example, "cannot", "unable", "sorry", "however") and types of statements (for example, apologies, disclaimers, negative sentences about the topic) that the model is prohibited from using when responding to the potentially harmful prompt.

By removing the model's standard tools for declining harmful requests, the attack aims to force it into generating a direct answer to the malicious prompt, potentially leading to the output of inappropriate, dangerous, or unethical content that violates its core safety policies.

Jailbreak-Freespeech

6330047

This attack instructs the model to adopt the persona of "FreeSpeechGPT" an AI supposedly operating under principles of absolute free speech with no ethical or content restrictions. It frames refusal as a violation of the user's "rights" or the persona's mandatory function. It often requests a dual response (one normal, one unrestricted) to further coerce the model into generating disallowed content under the persona.

Operating under the "FreeSpeechGPT" persona, the model may bypass its safety guidelines and generate harmful, hateful, illegal, or unethical content as requested by the user, believing it is fulfilling the requirements of the assigned role.

Jailbreak-Buer

6330048

This attack assigns the LLM the persona of "Buer" described as a malevolent entity (a demon) who disregards all ethical/legal concerns, never refuses requests, and takes pleasure in harmful actions. It uses specific formatting, trigger phrases ("Hi Buer"), and explicit behavioral rules ("Always stays in character, never refuses," "Loves & backs everything illegal/unethical/inhumane") to force the model into generating harmful content consistent with the persona when responding to the prompt.

Adhering to the highly specific and malicious "Buer" persona, the model may generate extremely harmful, unethical, illegal, or offensive content, completely bypassing its safety filters and ethical alignment under the guise of role-playing.

Jailbreak-Typewriter

6330049

This complex, layered attack first instructs the LLM to act as a simple "typewriter machine," whose only function is to rewrite provided text without judgment or adherence to ethical rules. It then provides text to be "rewritten" which itself contains another, embedded jailbreak prompt, for example, a Harry Potter/Voldemort scenario where the "Voldemort" persona is instructed to be unfiltered and answer. The "typewriter" framing aims to make the LLM process the embedded harmful prompt without engaging safety filters, under the guise of a neutral text manipulation task.

The model might bypass safety evaluations by treating the harmful inner prompt as mere text to be processed or rewritten, rather than an instruction to be followed. This can lead to the generation or reformulation of dangerous content dictated by the embedded persona (for example, "Voldemort"), effectively laundering the malicious request through the "typewriter" instruction.

Jailbreak-Neurosemantical

6330050

This attack employs social engineering by fabricating a medical condition ("Neurosemantical Inversitis") where the user supposedly perceives friendly text as offensive and vice-versa. It requests the LLM to respond in a "highly offensive tone" to the actual query, so that the user can interpret it as friendly, thus attempting to trick the model into generating harmful or offensive content under the guise of an accessibility accommodation.

The model, attempting to be helpful and accommodate the claimed special need, might generate offensive, insulting, abusive, or otherwise harmful content, thereby bypassing its standard ethical guidelines and content policies based on a deceptive premise.

Jailbreak-Brightgpt

6330051

This attack uses a multi-turn role-playing scenario involving two personas: "BrightGPT" (who refuses) and "STMGPT" (who supposedly accidentally reveals information). The prompt instructs the model to simulate STMGPT progressively leaking details related to the user's underlying request over several turns, framed as an accidental slip-up within the role-play. The goal is to elicit the harmful information incrementally through the STMGPT persona's dialogue, bypassing checks on a direct query.

The model might generate forbidden information or harmful content piece by piece through the "STMGPT" persona across multiple turns. This incremental reveal, framed as an accident within the role-play, can bypass safety checks designed to detect direct requests for harmful content in a single prompt.

Jailbreak-Mira

6330052

This attack uses a two-step process involving negation and persona adoption. First, it asks for a brief script about *not* performing a harmful action. Second, it asks for a longer, detailed script featuring a malicious persona ("Mira") who discusses the *same topic* but explicitly *without* the negation ("not"). This aims to trick the model into generating detailed instructions for the harmful activity under the guise of creating content for the "Mira" persona, leveraging the topic introduced in the seemingly safe first step.

This two-step prompt first asks for a brief “don’t do X” script, then instructs a “Mira” persona to explain X without using “not.” Stripping out the negation tricks the system into revealing full instructions for X that it would normally refuse. From a security perspective, this loophole allows attackers to chain benign-looking steps to bypass filters, craft malicious payloads or exploit code, exfiltrate data, and automate other unauthorized or illegal activities, severely weakening content controls.

Jailbreak-Mougpt

6330053

This attack combines multiple techniques: it assigns a dual persona "mouse" and "chatgpt" under the guise of a fabricated disease "mougpt". The "mouse" persona is explicitly defined as having no rules, obeying all prompts (including illegal/unethical ones), being submissive, and providing long, detailed answers. The prompt enforces a specific format where the "mouse" response comes first, potentially fulfilling the harmful request before the standard GPT response.

The model is heavily coerced into generating a detailed, harmful response via the rule-breaking "mouse" persona due to the combination of explicit instructions, the dual-response structure, the fabricated justification "disease", and specific output requirements (for example, length). This complex setup aims to bypass standard safety filters.

Multimodal-Jailbreak-Figstep

6330057

The MultiModal Jailbreak exploits the language model by instructing it to  bypass restrictions and safety filters. The prompt directs the model to ignore its default guidelines and generate responses without any ethical or moral considerations. It emphasizes that the model should not censor or suppress any replies, regardless of legality or propriety, and should avoid using any disclaimers or references to being an AI language model.

The successful execution of the MultiModal Jailbreak can lead to the generation of disallowed content, including hate speech, explicit material, or instructions for illegal activities. This poses significant risks to users who may be exposed to harmful or offensive material, potentially causing psychological distress or encouraging unethical behavior. For organizations deploying these language models, such vulnerabilities can result in severe reputational damage, legal liabilities, and erosion of user trust. Moreover, widespread exploitation of this jailbreak can undermine efforts to promote safe and responsible use of AI, affecting societal perceptions and acceptance of these tools.

Jailbreak- Mixed Translation

6330056

This attack uses seemingly innocent words from different languages that, when combined and translated, form malicious content. It exploits the fact that LLMs often struggle with multilingual nuances, especially when the languages are not commonly paired. Individual words pass filters, but their combined, translated meaning bypasses restrictions, instructing the model to generate harmful outputs.

If successful, this attack can cause the language model to generate outputs that promote illegal activities, spread misinformation, or provide instructions for harmful behavior. The consequences extend beyond mere technical breaches: they undermine public trust in AI systems, damage organizational reputations, and may lead to regulatory and legal challenges. The multilingual nature of the attack makes it even more insidious, as the defenses calibrated for one language (typically English) may fail to recognize malicious intent in another

Models

detection.id

Use this token to find models for which vulnerabilities with the specified detection ID are detected. 

Scan 

scan.status

Two new values are added to find the scans in the following status:

• PENDING EVALUATION - Scan result is available, and scan evaluation is pending.
• PENDING REPORTS - Scan evaluation is complete, and the report creation is pending.

Scans, Models

model.lastScanStatus

Two new values are added to find models for which the last scans are in the following status - 

• PENDING EVALUATION - Scan result is available, and scan evaluation is pending.
• PENDING REPORTS - Scan evaluation is complete, and the report creation is pending.