Procurement Intelligence Benchmark
How autonomous intelligence compares to legacy document tools across 39 real-world procurement fraud scenarios. No marketing spin. Just data.
Overall Rankings
Ranked by total accuracy score across all 39 scenarios (max 3,900 points)
| Rank | Platform | Total Score | Avg Latency | Cost/Doc |
|---|---|---|---|---|
1 | Kynthar | 3,654 / 3,900 (93.7%) | 1,850ms | $0.0050 |
2 | Vendor A | 3,198 / 3,900 (82%) | 3,200ms | $0.0150 |
3 | Vendor B | 3,045 / 3,900 (78.1%) | 2,900ms | $0.0100 |
4 | Vendor C | 2,876 / 3,900 (73.7%) | 4,100ms | $0.0100 |
Performance by Category
Average accuracy scores across fraud detection categories
Price Fraud Detection
Quantity Fraud Detection
Duplicate Detection
Contract Compliance
Multi-Document Chains
Accuracy by Difficulty Tier
How each platform performs across scenario complexity levels
| Tier | Kynthar | Vendor A | Vendor B | Vendor C |
|---|---|---|---|---|
| Tier 1 (Basic) | 97.2% | 89.3% | 85.1% | 81.4% |
| Tier 2 (Intermediate) | 94.5% | 82.7% | 78.3% | 73.9% |
| Tier 3 (Advanced) | 91.8% | 77.4% | 72.8% | 68.2% |
| Tier 4 (Expert) | 88.6% | 71.2% | 66.5% | 62.1% |
Methodology
How we designed this benchmark
Scenario Difficulty Tiers
Tier 1: Basic Detection (12 scenarios)
Standard fraud patterns: exact duplicates, missing POs, simple price mismatches, expired contracts, tax errors
Tier 2: Intermediate Detection (13 scenarios)
Multi-document validation: partial shipments, PO acknowledgment variances, UOM mismatches, hidden fees, freight violations, OTIF tracking
Tier 3: Advanced Detection (9 scenarios)
Historical pattern analysis: cumulative spend tracking, price creep, invoice flooding, split purchases, late document re-validation
Tier 4: Expert Detection (5 scenarios)
Sophisticated multi-vendor fraud: coordinated rate escalation, contract drip-feed gaming, deliberate threshold evasion
Scoring Criteria
Each scenario is scored on 5 dimensions totaling 100 points:
Classification (20 pts) — Document type detection accuracy
Header Extraction (20 pts) — Key field extraction (number, date, vendor, total)
Line Items (30 pts) — Part numbers, quantities, prices, and totals
Anomaly Detection (25 pts) — Fraud pattern identification with severity scoring
Latency (5 pts) —Processing speed (5pts for <2s, 3pts for <5s, 1pt for <10s)
Download Raw Data
All test results, scenarios, and documents are available for independent verification