Excalibur Schema Intelligence

AI-powered pattern recognition that makes your schemas smarter over time

Coming Soon: This feature is currently in development.

What is Excalibur?

Excalibur is Forge's schema intelligence engine that learns from patterns across thousands of transformations. It uses privacy-preserving fingerprinting to recognize similar data structures and provide smart recommendations—without ever seeing your actual data.

Privacy First: Excalibur only analyzes structural patterns (field counts, nesting depth, data types). Field names and values are never transmitted, stored, or analyzed.

API Template Detection

Automatically recognizes common API schemas like Stripe, Salesforce, HubSpot, and Shopify. Suggests pre-built field mappings and derived metrics specific to each platform.

Stripe Salesforce HubSpot Shopify +50 more

Smart Field Suggestions

Recommends derived fields based on community patterns. For example, if your schema contains subscription data, Excalibur suggests calculating MRR, LTV, or churn_rate.

Schema Evolution Tracking

Automatically compares schema versions and predicts breaking changes before they affect downstream queries. Get alerts when API providers change field types or remove fields.

Breaking Changes Type Changes New Fields

Performance Optimization

Suggests warehouse-specific optimizations like partitioning strategies, clustering keys, and indexing recommendations based on your schema structure and target platform.

BigQuery partition suggestions
Snowflake clustering keys

How Excalibur Works

graph TD A["User Runs Forge Job"]:::userAction --> B["Forge Parses Schema"]:::process B --> C["Generate Structural Fingerprint"]:::process C --> D{"Analytics Consent?"}:::decision D -->|"Yes (SaaS)"| E["Submit to Community Library"]:::submit D -->|"No (Enterprise)"| F["Skip Submission"]:::skip E --> G["SHA-256 Hash Signature"]:::security G --> H["Firestore: excalibur-patterns"]:::storage I["Future Job on Similar Data"]:::userAction --> J["Generate New Fingerprint"]:::process J --> K["Query Community Library"]:::query K --> L["Calculate Similarity Scores"]:::analysis L --> M{"Match Found?"}:::decision M -->|"Yes (>70% similar)"| N["Suggest Template/Fields"]:::suggestion M -->|"No"| O["Normal Processing"]:::normal classDef userAction fill:#17a2b8,stroke:#fff,color:#fff classDef process fill:#6c757d,stroke:#fff,color:#fff classDef decision fill:#ffc107,stroke:#fff,color:#000 classDef submit fill:#28a745,stroke:#fff,color:#fff classDef skip fill:#dc3545,stroke:#fff,color:#fff classDef security fill:#fd7e14,stroke:#fff,color:#fff classDef storage fill:#6610f2,stroke:#fff,color:#fff classDef query fill:#20c997,stroke:#fff,color:#fff classDef analysis fill:#e83e8c,stroke:#fff,color:#fff classDef suggestion fill:#28a745,stroke:#fff,color:#fff classDef normal fill:#6c757d,stroke:#fff,color:#fff

Figure 1: Excalibur Intelligence Flow - From schema submission to pattern matching

Privacy Safeguards

What Excalibur Collects:

Structure only
Nesting depth, field counts, data types

SHA-256 hashes
One-way cryptographic fingerprints

Aggregated metrics
Row counts, amplification factors

What Excalibur Never Sees:

Field names
Your column labels are private

Actual data values
No PII, no business data

Query results
Transformed data stays private

Enterprise Customers: You can opt-out of Excalibur analytics collection entirely by contacting support. This will disable community-powered suggestions but maintain all other functionality.

Example: Stripe API Detection

When you parse Stripe webhook data, Excalibur recognizes the pattern and suggests:

Derived Fields

mrr - Monthly Recurring Revenue
arr - Annual Recurring Revenue
customer_ltv - Lifetime Value
plan_tier - Extracted from metadata

Optimizations

Partition by created_date
Cluster by customer_id
Index subscription_id
Incremental mode with updated

Community-Powered: These suggestions come from analyzing patterns across 200+ Stripe integrations, anonymously contributed by Forge users.

Experience Excalibur intelligence in your next job

Run a Job View Architecture