Skip to content

[FUAM] Gateway Monitoring From Files: refresh fails on nested DataSource objects in QueryExecutionAggregationReport #671

Open

Description

@selimovd

Component: FUAM_Gateway_Monitoring_From_Files_SM.SemanticModel → table QueryExecutionAggregationReport

Symptom: Scheduled refresh fails with
DM_GWPipeline_Gateway_MashupDataAccessError – The parameter is expected to be of type Text.Type or Binary.Type.
Error details contain ValueError.ConnectorType=HTTP and ValueError.FullPath=https://<tenant>.sharepoint.com/.../_api/SP.APIHubConnector.GetTables.

Root cause: In QueryExecutionAggregationReport_*.log the DataSource column can contain a list of nested objects, e.g.
[{"ConnectorType": "HTTP", "FullPath": "https://<tenant>.sharepoint.com/sites/<site>/_api/SP.APIHubConnector.GetTables"}]
instead of the usual list of JSON-encoded strings ["{\"kind\":\"SQL\",\"path\":\"...\"}"].
This happens as soon as the gateway serves SharePoint/Teams-backed sources. The step
#"Parsed JSON1" = Table.TransformColumns(#"Expanded DataSource",{{"DataSource", Json.Document}})
is not guarded and throws on the record. The outer try Json.Document([DataSource]) does not protect this inner call.

Proposed fix (I tested it, for me the refresh worked then):

#"Normalized List" = Table.TransformColumns(#"Removed Columns2", {{"DataSource", each if _ is list then _ else {_}}}),
#"Expanded DataSource" = Table.ExpandListColumn(#"Normalized List", "DataSource"),
#"Parsed JSON1" = Table.TransformColumns(#"Expanded DataSource", {{"DataSource", each if _ is text then Json.Document(_) else _}}),
#"Expanded DataSource1 - kind" = Table.AddColumn(#"Parsed JSON1", "DataSource.kind", each
    if [DataSource] is record then Record.FieldOrDefault([DataSource], "kind") ?? Record.FieldOrDefault([DataSource], "ConnectorType") else null),
#"Expanded DataSource1 - path" = Table.AddColumn(#"Expanded DataSource1 - kind", "DataSource.path", each
    if [DataSource] is record then
        Record.FieldOrDefault([DataSource], "path") ?? Record.FieldOrDefault([DataSource], "FullPath")
        ?? Text.Combine(List.Transform(List.RemoveFirstN(Record.FieldValues([DataSource]), 1), each if _ is text then _ else Text.FromBinary(Json.FromValue(_))), ";")
    else null),

The same if _ is text then Json.Document(_) else _ guard should be applied to #"Parsed JSON1" in QueryExecutionReport and #"Inserted Parsed JSON1" in QueryStartReport_DataSources for robustness.

Environment: On-premises data gateway (standard mode), gateway log files from 2026-02 to 2026-09, Power BI Service Import refresh.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions