Digital Shadows Connector: Logs Ingestion API Migration Required Before September 2026 Ingestion Cutoff

What Changed

The Digital Shadows (ReliaQuest SearchLight) connector has been migrated from the HTTP Data Collector API (workspace shared-key auth, retiring 2026-09-14) to the Logs Ingestion API (DCE + DCR + system-assigned Managed Identity). This is a breaking infrastructure change requiring customers to re-deploy the ARM template.

Security Impact (Visibility and Fidelity)

Deadline-driven ingestion gap risk: Microsoft is retiring the HTTP Data Collector API on 2026-09-14. Customers running the pre-3.1.0 connector will experience a complete ingestion failure for Digital Shadows data after that date. No alerts, no incidents, no external attack surface visibility from this source.

New table, no historical backfill: Data now flows to DigitalShadows_V2_CL. The legacy DigitalShadows_CL table receives no new data after upgrade but is retained per workspace retention policy. There is no re-pull of historical events.

Detection continuity preserved: Both Analytic Rules have been updated to query DigitalShadows_V2_CL. The existing detection logic is preserved unchanged via KQL project-rename aliases that map the new PascalCase V2 columns back to the legacy _s/_t/_d/_g column name references.

Ingestion Mechanism Changes

• Auth: Workspace shared key (WorkspaceID + WorkspaceKey) removed. Function Apps now authenticate via system-assigned Managed Identity, granted Monitoring Metrics Publisher on the DCR.
• New ARM parameters: DcrWorkspaceResourceId (full Log Analytics workspace resource ID) is now required. Legacy WorkspaceID and WorkspaceKey parameters are removed.
• Python runtime: Both Function Apps upgraded from inconsistent runtimes (include on 3.11, exclude on 3.8) to Python 3.11 uniformly.
• Connector JSON: DigitalShadowsSearchlight_API_functionApp.json updated to reflect new DCE/DCR-based ingestion configuration.

Customer Upgrade Path

Per the added UPGRADE.md guide: re-run the ARM template against the existing resource group, supplying the new DcrWorkspaceResourceId parameter. Playbooks are unaffected (they operate on Sentinel incidents, not raw log tables). Detection logic requires no changes.

Affected Files

.script/tests/KqlvalidationsTests/CustomTables/DigitalShadows_V2_CL.json
Solutions/Digital Shadows/Analytic Rules/Digital_Shadows_incident_creation_exclude.yaml
Solutions/Digital Shadows/Analytic Rules/Digital_Shadows_incident_creation_include.yaml
Solutions/Digital Shadows/Data Connectors/DigitalShadowsConnectorAzureFunction/AS_api.py
Solutions/Digital Shadows/Data Connectors/DigitalShadowsConnectorAzureFunction/DS_poller.py
Solutions/Digital Shadows/Data Connectors/DigitalShadowsConnectorAzureFunction/__init__.py
Solutions/Digital Shadows/Data Connectors/DigitalShadowsSearchlight_API_functionApp.json
Solutions/Digital Shadows/Data Connectors/digitalshadowsARM.json
Solutions/Digital Shadows/Data Connectors/readme.md
Solutions/Digital Shadows/Data Connectors/requirements.txt
Solutions/Digital Shadows/Data Connectors/smoke_test.py
Solutions/Digital Shadows/Data Connectors/test_field_mapping.py
Solutions/Digital Shadows/Package/testParameters.json
Solutions/Digital Shadows/UPGRADE.md
Solutions/Digital Shadows/Workbooks/DigitalShadows.json
Workbooks/WorkbooksMetadata.json
(packaging artefacts: 3.1.0.zip, ReleaseNotes.md, Solution_DigitalShadowsSearchlight.json, createUiDefinition.json, digitalshadowsConnector.zip, mainTemplate.json)