run_7deb60dd09e7a963

input

{
  "request": "Triage Jira ticket MZCLD-3657: Investigate why fx_non_ci_daily_actual_view stopped updating 2025-12-08 \u2014 Firefox Non-CI DevTest export is stale",
  "references": [
    "https://mozilla-hub.atlassian.net/browse/MZCLD-3657"
  ],
  "sourceContext": {
    "issue": "MZCLD-3657"
  }
}

summary

MZCLD-3657 tracks investigation of a stalled data pipeline: the fx_non_ci_daily_actual_view (Firefox Non-CI DevTest export) stopped ingesting new rows after 2025-12-08, even though the underlying Azure subscription is still active and billing (~$2,106.16 in spend since that date visible in mozilla_all_actual_view but not reflected in the dedicated view). The ticket is in Backlog with no comments yet. There is currently no cost-reporting undercount because the MZCLD-3399 fallback (EA pull) is covering the gap, but the dedicated export itself is broken, and the ticket correctly flags a latent risk: if this view is wired into unblended_cost_estimated_v1 as previously considered, the post-Dec-2025 spend would be silently dropped (excluded from the fallback due to subscription presence in the dedicated CTE's scope, yet missing from the stale CTE itself). Root cause is suspected to be either (a) the Azure cost export/transfer for the fx_non_ci container stopping after 2025-12-08, or (b) the fx-non-ci-daily-actual-function Cloud Function failing silently (e.g., auth/permission expiry, source bucket path change, or an unhandled exception with no alerting). Investigation requires checking Cloud Function logs (fx-non-ci-daily-actual-function) and the source GCS/Azure export bucket for the fx_non_ci container to confirm whether new export objects are landing after 2025-12-08.

findings

• Ticket is unassigned/untouched: status Backlog, 0 comments, no investigation logged yet despite being filed for a specific dated data gap.
• Data gap is precisely bounded: fx_non_ci_daily_actual_view has 21,696 rows spanning 2024-09-01 to 2025-12-08, then stops abruptly, while billing continues.
• Live billing evidence confirms the subscription is still active and incurring spend that the dedicated view is missing ($2,106.16 through 2026-08), meaning this is an ingestion failure, not a subscription cancellation.
• There is currently no user-facing cost undercount because a separate fallback mechanism (from MZCLD-3399, using an EA pull) is covering the 2026 spend gap in the aggregate view.
• There is a latent/future risk clearly called out in the ticket: if fx_non_ci_daily_actual_view is later wired into unblended_cost_estimated_v1 (a change that was 'considered'), it would cause a silent undercount of $2,106+ because the subscription would be excluded from the fallback logic (since it appears to have a dedicated view) but that dedicated view is stale/dead.
• Two concrete, plausible root causes are proposed by the reporter, pointing investigation toward two specific systems: the fx-non-ci-daily-actual-function Cloud Function and the underlying Azure cost export/transfer job for the fx_non_ci container.
• The pipeline is identified as a dedicated infra resource (Terraform config variables_all_the_vars.tf, PR/change #6171, app_code devsandbox), which should be checked for recent changes (e.g., permission/IAM changes, deploy that may correlate with the 2025-12-08 stop date).
• Attempted to pull related ticket MZCLD-3399 (the fallback mechanism referenced as currently masking the gap) for additional context, but access was blocked by policy since it is outside the current run's scope.

proposed actions

tool calls

[
  {
    "capability": "jira.issue.read",
    "args": {
      "key": "MZCLD-3657"
    },
    "result": {
      "key": "MZCLD-3657",
      "summary": "Investigate why fx_non_ci_daily_actual_view stopped updating 2025-12-08 \u2014 Firefox Non-CI DevTest export is stale",
      "status": "Backlog",
      "description_text": "The Firefox Non-CI DevTest dedicated pipeline (variables_all_the_vars.tf, #6171, app_code: devsandbox) has data but it stops at 2025-12-08: fx_non_ci_daily_actual_view: 21,696 rows, 2024-09-01 \u2192 2025-12-08 The subscription is still active \u2014 mozilla_all_actual_view shows $2,106.16 of Firefox Non-CI DevTest Subscription spend after 2025-12-08 (through 2026-08) that never reaches the dedicated view. The daily Cloud Function (fx-non-ci-daily-actual-function) appears to have stopped ingesting new data around that date. Impact: No current undercount \u2014 the MZCLD-3399 fallback covers the 2026 spend from the EA pull. But the dedicated export is dead, and wiring this view into unblended_cost_estimated_v1 (as was considered) would drop the $2,106+ of post-Dec-2025 spend, since the subscription would be excluded from the fallback but absent from the stale dedicated CTE. Likely root cause: Azure export/transfer for the fx_non_ci container stopped after 2025-12-08, or the load/lookup Cloud Function is failing silently. Check function logs and the source bucket for objects after that date.",
      "comment_count": 0,
      "comments": []
    }
  },
  {
    "capability": "jira.issue.read",
    "args": {
      "key": "MZCLD-3399"
    },
    "result": {
      "error": "read blocked by policy: resource not in run subject",
      "args": {
        "key": "MZCLD-3399"
      }
    },
    "blocked": true
  }
]

log