What Changed

Updated CCF poller configuration parameters for both IP Reputation and Malware URLs feeds to eliminate duplicate data ingestion.

Security Impact (Visibility & Fidelity)

Production Impact Resolved: Deployments running v3.0.2 experienced massive duplicate ingestion with a 1,535:1 duplicate ratio (304,000 rows ingested vs 198 unique IPs). This consumed excessive log analytics storage and made threat intelligence queries inefficient.

Configuration Changes Applied:

  • Page size: Increased count from 100 to 1000 (eliminates multi-page fetching)
  • Polling frequency: Reduced from 15 minutes to 6 hours (360 minutes)
  • Expected reduction: ~99.7% decrease in ingested rows (304,000/day → 3,200/day)

Root Cause Analysis

The Cyren feeds contain relatively static indicator sets:

  • IP Reputation feed: ~800 indicators
  • Malware URLs feed: ~200 indicators

Previous configuration caused repeated fetching of the same indicator set:

  • 8+ page requests per poll cycle (due to count=100 vs 800 total indicators)
  • 96 poll cycles per day (every 15 minutes)
  • Result: Same indicators ingested repeatedly throughout each day

Connector Mechanism

CCF (Codeless Connector Framework) with DCR-based ingestion using PersistentToken paging (correctly preserved from v3.0.2 fix).

Affected Files

Solutions/CyrenThreatIntelligence/Data Connectors/CyrenThreatIntel_CCF/Cyren_PollerConfig.json
(packaging artefacts: 3.0.3.zip, ReleaseNotes.md, mainTemplate.json)