agentplat▸control runs deployments agents metering + agent
← runs

run_452703005975d10f

SUCCEEDED retrigger →
jira-triage v1 · jira-triage-k8s · owner ingest:jira-new-ticket (api) · via jira-new-ticket · cloud-engineering

input

{
  "request": "Triage Jira ticket MZCLD-3686: MozCloud Helm charts: fix redundant pod-level volume when mounting a StatefulSet volumeClaimTemplate",
  "references": [
    "https://mozilla-hub.atlassian.net/browse/MZCLD-3686"
  ],
  "sourceContext": {
    "issue": "MZCLD-3686"
  }
}

summary

MZCLD-3686 is a well-specified bug fix for the `mozcloud` Helm chart (mozilla/helm-charts, chart mozcloud/application, v3.7.0): when a container volume name collides with a key in that workload's `statefulSet.volumeClaimTemplates`, the chart currently renders a redundant/incorrect pod-level `volumes:` entry pointing at a non-existent standalone PVC, relying on undocumented StatefulSet controller behavior (`updateStorage` name-collision substitution) to mask the error. Ticket status is "In Progress" with zero comments, a fully detailed problem statement, reproduction, proposed fix, acceptance criteria, and precise implementation pointers (file/line ranges in `_workloadTemplate.yaml` and `_helpers.tpl`), plus explicit test-gap and versioning guidance. This is an actionable, already-scoped engineering ticket rather than something needing triage clarification — the main value-add is confirming scope/ownership and flagging a couple of verification/test items before implementation proceeds or is reviewed.

findings

  • Root cause and impact are clearly and rigorously documented, including a concrete repro (lando's 5 StatefulSets, 200Gi files-volume) discovered during the MZCLD-3089 dev migration.
  • The current behavior depends on unverified StatefulSet controller semantics (updateStorage substituting colliding pod-template volumes), which the reporter flags as a real risk if their reading is wrong.
  • Proposed fix is scoped and justified: filter out colliding pod-level volumes rather than introducing a new explicit type, chosen specifically because it requires no values changes and also fixes the persistentVolume/VCT name-collision case.
  • Ticket includes precise implementation guidance (files, line ranges, helper reuse pattern) and a defensive-coding note about `.statefulSet` always being present via the default merge, reducing ambiguity for whoever picks this up.
  • A documented test gap exists: the current upstream fixture declares a VCT but never mounts it, so the exact collision scenario being fixed is currently untested. The ticket explicitly calls this out as needing a new/updated fixture.
  • Ticket is status 'In Progress' with no comments yet, so no blockers or review feedback have been logged; work appears to be actively underway per the detailed notes already present.
  • Ticket correctly distinguishes itself from a related but separate issue (maxUnavailable not expressible) already tracked in MZCLD-2754, avoiding scope creep.

proposed actions

none

tool calls

[
  {
    "capability": "jira.issue.read",
    "args": {
      "key": "MZCLD-3686"
    },
    "result": {
      "key": "MZCLD-3686",
      "summary": "MozCloud Helm charts: fix redundant pod-level volume when mounting a StatefulSet volumeClaimTemplate",
      "status": "In Progress",
      "description_text": "Problem When a workload container mounts a volume whose name matches a key under that workload's  statefulSet.volumeClaimTemplates , the  mozcloud  chart renders  both  the volumeClaimTemplate  and  a redundant pod-level  volumes:  entry claiming a standalone PVC of the same name. That PVC does not exist:  type: persistentVolume  is meant to reference a claim declared under top-level  persistentVolumes: , which is not how a per-pod VCT is declared. There is currently no way to express \"mount this StatefulSet's volumeClaimTemplate\" \u2014 the container  volumes  list always produces a matching pod-level volume. Reproduction (chart 3.7.0) workloads:\n  test-db:\n    type: statefulset\n    statefulSet:\n      volumeClaimTemplates:\n        data:\n          accessMode: ReadWriteOnce\n          storageClassName: standard-rwo\n          storage: 10Gi\n    containers:\n      app:\n        volumes:\n          - name: data\n            type: persistentVolume\n            path: /data Renders: volumeClaimTemplates:\n    - metadata:\n        name: data          # correct\n...\n      volumes:\n        - name: data        # WRONG - no such standalone PVC\n          persistentVolumeClaim:\n            claimName: data Impact This only works today because the StatefulSet controller's  updateStorage  replaces pod-template volumes whose name collides with a volumeClaimTemplate, substituting the correct per-pod claim ( data-test-db-0 ). The chart is emitting a manifest that is wrong on its face and depends on undocumented controller behaviour that has  not been verified on a live cluster . If that reading is wrong, pods fail to schedule with a missing-PVC error. Observed on all 5 lando StatefulSets during the MZCLD-3089 dev migration (200Gi  files-volume  per worker). No standalone PVC is rendered for them, confirming the pod-level claim dangles. Proposed fix Omit the pod-level volume when its name matches one of the workload's  statefulSet.volumeClaimTemplates  keys, mirroring what the controller already does. Keep the container  volumeMount  \u2014 a mount referencing a VCT name is valid with no pod-level volume. Chosen over adding an explicit  type: volumeClaimTemplate  because it fixes existing affected charts with no values changes, and because a  persistentVolume /VCT name collision would remain broken under the explicit-type approach. Acceptance criteria A container volume whose name matches a VCT key renders no pod-level  volumes  entry. The container  volumeMounts  entry is unchanged. When the colliding volume is the  only  pod-level volume and the nginx sidecar is disabled, the  volumes:  key is omitted entirely rather than rendered empty. (This is the lando case \u2014 it is the main implementation subtlety.) Deployments and Rollouts are unaffected (no volumeClaimTemplates). Non-colliding volumes of every existing type still render. lando's 5 StatefulSets render with the VCT and no  claimName: files-volume  pod volume. Implementation notes Repo  mozilla/helm-charts , chart path  mozcloud/application , currently 3.7.0. templates/workload/_workloadTemplate.yaml  \u2014  $volumes  is assembled from container and initContainer volume lists (~L37-48); the render guard and helper call are at ~L243-250. Filter colliding names out here, gated on  eq (default \"deployment\" $config.type) \"statefulset\" , and base the  if or $nginxEnabled (gt (keys ... | len) 0)  guard on the  filtered  dict so the empty case drops the  volumes:  key. Keep the unfiltered  $volumes  for the existing configMap-checksum and gcsFuse annotation logic above it. templates/workload/_helpers.tpl  \u2014  mozcloud.workload.volumes  (L36-92) already has a  $skip  mechanism for the nginx configmap volume (L44-49); reuse that pattern if the skip is better placed in the helper. workloads.default.statefulSet  exists in  values.yaml  (~L2537) and is merged into every workload by  mozcloud.formatter.workloads  ( templates/_formatter.yaml  L249), so  .statefulSet  is always present \u2014 access  volumeClaimTemplates  defensively. Test gap:   tests/values/statefulset-configuration.yaml  declares a VCT named  data  and never mounts it, so this collision is completely uncovered upstream. Add the mount to a fixture (or a new one) plus assertions;  tests/values/workload-volumes.yaml  is the model for volume-type coverage. Per  CONTRIBUTING.md , do  not  bump the chart version by hand \u2014 CI infers the release label from the PR title, so a  fix:  title yields the correct patch bump. Run  make update-dependencies && make unit-tests ; expect snapshot updates for the statefulset suites. Source Item 2 of  lando/k8s/lando/.migration/DIFF_ANALYSIS_dev.md  in  webservices-infra , produced during the MZCLD-3089 lando dev migration. Related but separate: item 3 of the same document ( strategy.rollingUpdate.maxUnavailable  not expressible) is already tracked as MZCLD-2754.",
      "comment_count": 0,
      "comments": []
    }
  }
]

usage

{
  "model": "claude-sonnet-5",
  "input_tokens": 4916,
  "output_tokens": 2106,
  "estimated_cost_usd": 0.046338000000000004
}

approvals

[]

log

launched k8s job run-run-452703005975d10f in ns/mzcld-demo-dev image=us-docker.pkg.dev/moz-fx-mzcld-demo-prod/mzcld-demo-prod/agentplat:latest
runner started
live reasoning model=claude-sonnet-5