What Changed

Three test infrastructure files were modified:

ingestASimSampleData.py (major refactor) The script previously fetched parser YAML and sample data files from raw.githubusercontent.com using the current commit SHA. At workflow runtime, newly added files in the PR are not yet pushed to the GitHub raw URL, causing all file fetches to return 404 and the ingestion loop to silently continue without testing anything. The fix replaces all remote URL reads with local filesystem reads using the checked-out repo path.

Additional improvements:

  • convert_data_csv_to_json now returns datetime_keys to enable correct type inference
  • A new infer_schema_from_data fallback generates a schema when no Schema CSV exists
  • Error handling added throughout (FileNotFoundError, KeyError, empty-data guards)
  • senddtosentinel now returns a boolean success/failure status
  • Removed get_current_commit_number() and SENTINEL_REPO_RAW_URL constants that drove the broken URL pattern

runAsimTesters.ps1 (block uncommented) The git remote add upstream, git fetch upstream, and git diff –name-status upstream/master logic was commented out and replaced with a hardcoded single test file (Cynerio authentication parser). This caused the workflow to test only that one parser regardless of what was actually modified in the PR. This PR reverts that to the correct dynamic diff-based file detection.

VerifyASimParserTemplate.py (minor fix) Error message for missing sample data file now includes the expected filename rather than a partially-constructed template string.

Sample Data (non-functional update) Cynerio authentication and network session ingested log CSV files were regenerated (line count unchanged at 847 and 955 respectively).

Security Impact

The ASIM parser test pipeline was effectively non-functional: any PR modifying ASIM parsers would pass CI without actually running schema or data ingestion tests. New ASIM parsers with broken KQL, incorrect field mappings, or missing required columns could have been merged without automated validation. This fix restores the intended test gate for all ASIM parser PRs.

Affected Files

.script/tests/asimParsersTest/VerifyASimParserTemplate.py
.script/tests/asimParsersTest/ingestASimSampleData.py
.script/tests/asimParsersTest/runAsimTesters.ps1
Sample Data/ASIM/Cynerio_Cynerio_Authentication_IngestedLogs.csv
Sample Data/ASIM/Cynerio_Cynerio_NetworkSession_IngestedLogs.csv