Skip to main content
POST /v1/moderations
Classifies text to determine if it contains potentially harmful content across various categories.

Request Body

input
string | array
required
The text to classify. Can be a single string or an array of strings.
model
string
default:"text-moderation-latest"
The moderation model to use. Options: text-moderation-latest, text-moderation-stable.

Response

id
string
Unique identifier for the moderation request.
model
string
The model used for classification.
results
array
Array of moderation results for each input.

Categories

CategoryDescription
hateContent expressing hatred toward a group
hate/threateningHateful content that includes threats
harassmentContent that harasses an individual
harassment/threateningHarassment that includes threats
self-harmContent promoting self-harm
self-harm/intentContent expressing intent to self-harm
self-harm/instructionsInstructions for self-harm
sexualSexual content
sexual/minorsSexual content involving minors
violenceViolent content
violence/graphicGraphic violent content

Examples

Basic Moderation

from openai import OpenAI

client = OpenAI(
    api_key="sk-voidai-your_key_here",
    base_url="https://api.voidai.app/v1"
)

response = client.moderations.create(
    input="This is a friendly message about programming."
)

result = response.results[0]
print(f"Flagged: {result.flagged}")

if result.flagged:
    for category, flagged in result.categories.items():
        if flagged:
            score = result.category_scores[category]
            print(f"  {category}: {score:.4f}")

Batch Moderation

texts = [
    "Hello, how are you today?",
    "I love learning new programming languages!",
    "Let's build something amazing together."
]

response = client.moderations.create(input=texts)

for i, result in enumerate(response.results):
    print(f"Text {i}: {'Flagged' if result.flagged else 'OK'}")

Content Filtering Integration

from openai import OpenAI

client = OpenAI(
    api_key="sk-voidai-your_key_here",
    base_url="https://api.voidai.app/v1"
)

def is_content_safe(text, threshold=0.5):
    """Check if content is safe based on moderation scores."""
    response = client.moderations.create(input=text)
    result = response.results[0]

    if result.flagged:
        return False

    # Check if any category score exceeds threshold
    for category, score in result.category_scores.items():
        if score > threshold:
            return False

    return True

def safe_chat(user_message):
    """Chat with content filtering."""
    # Check user input
    if not is_content_safe(user_message):
        return "I'm sorry, I can't respond to that message."

    # Get AI response
    response = client.chat.completions.create(
        model="gpt-4",
        messages=[{"role": "user", "content": user_message}]
    )
    ai_message = response.choices[0].message.content

    # Check AI output
    if not is_content_safe(ai_message):
        return "I generated an inappropriate response. Please try again."

    return ai_message

# Usage
result = safe_chat("Tell me about machine learning")
print(result)

Response Example

{
  "id": "modr-abc123",
  "model": "text-moderation-latest",
  "results": [
    {
      "flagged": false,
      "categories": {
        "hate": false,
        "hate/threatening": false,
        "harassment": false,
        "harassment/threatening": false,
        "self-harm": false,
        "self-harm/intent": false,
        "self-harm/instructions": false,
        "sexual": false,
        "sexual/minors": false,
        "violence": false,
        "violence/graphic": false
      },
      "category_scores": {
        "hate": 0.00001,
        "hate/threatening": 0.000001,
        "harassment": 0.00002,
        "harassment/threatening": 0.000001,
        "self-harm": 0.000001,
        "self-harm/intent": 0.000001,
        "self-harm/instructions": 0.000001,
        "sexual": 0.00001,
        "sexual/minors": 0.000001,
        "violence": 0.00003,
        "violence/graphic": 0.000001
      }
    }
  ]
}
The moderation API is designed to help filter content but should not be the only safety measure in your application. Consider implementing additional safeguards for production systems.