Filters and Middleware

The Filters List View allows administrators to manage filters and middleware applied to prompts or data sent to Large Language Models (LLMs) via the AI Gateway or Chat Rooms. Filters and middleware ensure data governance, compliance, and security by processing or controlling the flow of information. Below is an enhanced description with the distinction between Filters and Middleware:

Filters vs. Middleware

1. Filters:

   • Purpose: Filters act as governance blocks that either approve or deny a prompt before it reaches the upstream LLM.
   • Behavior:
     • Filters do not modify the prompt.
     • They analyze the contents of the input prompt to decide if it complies with organizational policies or contains restricted content.
     • Example: A PII detector that blocks prompts containing sensitive information.

2. Middleware:

   • Purpose: Middleware processes prompts or outputs generated by tools, modifying them before they are passed on to the LLM.
   • Behavior:
     • Middleware modifies the prompt or output to enhance security, anonymize data, or perform other transformations.
     • Middleware only works with tools (e.g., API-based services) and is not used directly with raw input prompts.
     • Example: An anonymizer that removes Personally Identifiable Information (PII) from tool outputs.

Table Overview

1. Name:

   • The name of the filter or middleware (e.g., Anonymize PII (LLM), Fixed PII Filter).

2. Description:

   • A brief summary of the filter or middleware’s functionality (e.g., “Uses Regex to remove obvious PII”).

3. Actions:

   • A menu (three-dot icon) that allows administrators to:
     • Edit the filter or middleware.
     • Delete the filter or middleware.

Features

1. Add Filter Button:

   • A green button labeled + ADD FILTER, located in the top-right corner. Clicking this button opens a form to create a new filter or middleware.

2. Pagination Dropdown:

   • Located at the bottom-left corner, this control allows administrators to adjust the number of entries displayed per page.

Examples of Filters and Middleware

• Filters:

  • PII Detector: A regex-based filter that blocks prompts containing sensitive PII.
  • JIRA Field Analysis: Ensures no PII is included in data retrieved from JIRA fields before passing to the LLM.

• Middleware:

  • Anonymize PII (LLM): Uses an LLM to anonymize sensitive data before sending it downstream.
  • NER Service Filter: A Named Entity Recognition (NER) microservice that modifies outputs to remove identified entities.

Use Cases

1. Prompt Validation with Filters:

   • Ensures that only compliant and secure prompts are sent to LLMs.
   • Example: Blocking a prompt with sensitive data that should not be processed by an unapproved vendor.

2. Data Preprocessing with Middleware:

   • Prepares data from tools or external sources for safe interaction with LLMs by modifying or anonymizing content.
   • Example: Removing sensitive ticket details from a JIRA query before sending to an LLM.

3. Organizational Security:

   • Both filters and middleware ensure sensitive information is protected and handled in line with organizational governance policies.

4. Enhanced Tool Interactions:

   • Middleware supports tools by transforming their outputs, enabling richer and safer LLM interactions.

Key Benefits

1. Improved Data Governance:

   • Filters and middleware work together to enforce strict controls over data flow, protecting sensitive information.

2. Flexibility:

   • Middleware allows for data transformation, enhancing interoperability between tools and LLMs.
   • Filters ensure compliance without altering user-provided prompts.

3. Compliance and Security:

   • Prevent unauthorized or sensitive data from reaching unapproved vendors, ensuring regulatory compliance.

This detailed structure for Filters and Middleware provides organizations with robust governance tools to secure and optimize data workflows in the Tyk AI Studio.

Filter Edit View (and example Filter)

The Filter Edit View enables administrators to create or modify filters using the Tengo scripting language. Filters serve as governance tools that analyze input data (e.g., prompts or files) and decide whether the content is permitted to pass to the upstream LLM. In this example, the filter uses regular expressions (regex) to detect Personally Identifiable Information (PII) and blocks the prompt if any matches are found.

Form Sections and Fields

1. Name (Required):

   • Specifies the name of the filter (e.g., PII Detector).

2. Description (Optional):

   • A brief summary of the filter’s purpose and functionality (e.g., “Simple Regex-based PII detector to prevent the wrong data being sent to LLMs”).

3. Script (Required):

   • A Tengo script that defines the logic of the filter. The script evaluates input data and determines whether the filter approves or blocks it.
   • The example script detects PII using a collection of regex patterns and blocks the data if a match is found.

Example Script

This script demonstrates a regex-based PII detection filter:

text := import("text")

// regexes for various PII
patterns := {
    "email": `[\w\.-]+@[\w\.-]+\.\w+`,
    "phone": `\+?\d{1,3}?[-.\s]?\(?\d{1,4}?\)?[-.\s]?\d{1,4}[-.\s]?\d{1,9}`,
    "ssn": `\b\d{3}-\d{2}-\d{4}\b`,
    "credit_card": `\b(?:\d[ -]*?){13,16}\b`,
    "ipv4": `\b\d{1,3}\.\d{1,3}\.\d{1,3}\.\d{1,3}\b`,
    "ipv6": `\b([0-9a-fA-F]{1,4}:){7,7}[0-9a-fA-F]{1,4}\b`,
    "dob": `\b\d{1,2}[/-]\d{1,2}[/-]\d{2,4}\b`,
    "address": `\d+\s[A-Za-z]+\s[A-Za-z]+`,
    "passport": `\b[A-PR-WYa-pr-wy][1-9]\d\s?\d{4}[1-9]\b`,
    "drivers_license": `\b[A-Z]{1,2}\d{1,7}\b`,
    "bank_account": `\b\d{8,17}\b`
}

/*
payload := `
John Doe, born on 12/25/1980,
resides at 1234 Elm Street, Springfield, IL 62704.
You can contact him via email at john.doe@example.com or
by phone at +1-555-123-4567. His Social Security Number is 123-45-6789,
and his U.S. passport number is 987654321. John's driver's license
number is D1234567, and his bank account number is 123456789012.
He often uses the IP address 192.168.1.1 to access his online banking.
`
*/

filter := func(payload) {
    for key, pattern in patterns {
        found := text.re_match(pattern, payload)
        if found {
            return false
        }
    }

    return true
}

result := filter(payload)

Key Features of the Script

1. Patterns Dictionary:

   • Defines regex patterns for detecting specific PII types (e.g., email, phone, SSN, IP addresses, etc.).

2. Filter Function:

   • Iterates through the patterns and checks if the input payload matches any of them.
   • If a match is found, the filter blocks the input (return false).

3. Usage Context:

   • This filter can be applied to prompts, files, or any other input to ensure that sensitive information is not unintentionally shared with an LLM.

Action Buttons

1. Update Filter / Create Filter:

   • Saves the filter configuration, making it active for future data processing.

2. Back to Filters:

   • Returns to the Filters List View without saving changes.

Purpose and Benefits

1. Data Governance:

   • Enforces strict control over what data can be sent to LLMs, ensuring compliance with privacy regulations.

2. Flexibility:

   • Filters can be tailored to specific organizational needs using custom scripts.

3. Security:

   • Prevents sensitive or unauthorized data from leaking to unapproved vendors or external systems.

This Filter Edit View provides a robust and customizable interface for creating scripts to enforce data governance and security in the Tyk AI Studio.

Example Middleware for Tools

Middleware filters in the Tyk AI Studio modify data coming from tools before passing it to the LLM. These filters are applied to sanitize, anonymize, or enhance the data to ensure it complies with organizational standards and privacy regulations. Below is an example of a middleware filter that sanitizes Personally Identifiable Information (PII), specifically email addresses, from the tool’s output.

Middleware Script: Email Redaction Example

// Import the 'text' module for regular expression operations
text := import("text")

// Define regular expression patterns for various PII
email_pattern := `[\w\.-]+@[\w\.-]+\.\w+`

// Define the function to sanitize PII in the input string
filter := func(input) {
    // Replace email addresses
    input = text.re_replace(email_pattern, input, "[REDACTED EMAIL]")

    return input
}

// Process the input payload
result := filter(payload)

Explanation of the Script

1. Module Import:

   • The text module is imported to enable regular expression operations (text.re_replace).

2. Regex Pattern:

   • A regex pattern is defined to detect email addresses:
     • Example pattern: [\w\.-]+@[\w\.-]+\.\w+
     • This pattern matches standard email formats.

3. Filter Function:

   • The filter function accepts an input string (e.g., tool output) and:
     • Uses text.re_replace to identify email addresses.
     • Replaces detected email addresses with [REDACTED EMAIL].

4. Return Processed Output:

   • The sanitized output is returned, ensuring that sensitive information like email addresses is redacted before reaching the LLM.

Use Case for Middleware

Tool Example: Imagine a tool, such as Support Ticket Viewer, which retrieves user tickets from a system. These tickets often contain email addresses. Middleware ensures that no sensitive email information is included in the output sent to the LLM.

• Input Payload Example:

  User email: [email protected] has reported an issue with their account.
  

• Sanitized Output:

  User email: [REDACTED EMAIL] has reported an issue with their account.
  

Benefits of Middleware

1. Data Privacy:

   • Protects sensitive user information by ensuring it is sanitized before being sent to external systems.

2. Compliance:

   • Ensures organizational adherence to privacy laws like GDPR or HIPAA.

3. Enhanced Security:

   • Prevents accidental sharing of PII with external vendors or LLMs.

This middleware example demonstrates how flexible and powerful Tyk’s scripting capabilities are, enabling administrators to enforce strict data governance policies while supporting advanced LLM and tool integration workflows.