run_0ff38e433f37c55b

input

{
  "request": "Triage Jira ticket MZCLD-3663: design deterministic rubric for denoting whether an incident report is complete",
  "references": [
    "https://mozilla-hub.atlassian.net/browse/MZCLD-3663"
  ],
  "sourceContext": {
    "issue": "MZCLD-3663"
  }
}

summary

MZCLD-3663 is a backlog design task asking the team to replace the current Claude-based (LLM) classifier—used to judge whether an incident report satisfies the "incident completion checklist"—with a deterministic, rule-based rubric. The motivation is operational simplicity: running an LLM agent in infrastructure adds deployment/maintenance overhead that a deterministic check could avoid. The ticket has no comments and no explicit sub-tasks yet, so it is really a design/spec task that needs scoping before implementation.

findings

• Ticket is in Backlog status with zero comments, meaning no design discussion or scoping has started yet.
• The core ask is to design a deterministic 'good enough' rubric to replace an existing Claude-based classifier that determines incident report completeness against an 'incident completion checklist'.
• The description references an 'incident completion checklist' that is presumably the authoritative list of required fields/sections an incident report must contain, but no link/ID to that checklist doc was resolvable from the plain-text description, so the actual checklist content (needed to design the rubric) is not yet confirmed in this ticket.
• No acceptance criteria, examples of incomplete vs. complete reports, or list of checklist fields are attached to the ticket yet, which are prerequisites for a deterministic rubric (e.g., regex/field-presence checks, word counts, required-section checks).

proposed actions

Triage notes: This is a well-scoped but not-yet-detailed design task. To move forward, we should:

1. Locate and link the canonical 'incident completion checklist' doc (the description references it but the link didn't carry over into plain text — please attach/confirm the Confluence page).
2. Enumerate the checklist's required fields/sections (e.g., timeline, root cause, impact, action items, sign-off) so we can map each to a deterministic check (field non-empty, minimum length, presence of specific headers, regex pattern match, etc.).
3. Pull a sample set of past incident reports already labeled 'complete'/'incomplete' by the Claude classifier to use as a validation set for the new rubric—this lets us measure false positive/negative rates before cutover.
4. Define what 'good enough' means quantitatively (e.g., rubric must match Claude's judgment on >=95% of historical samples) so we know when the design is done.
5. Once the rubric is drafted, propose it as a checklist/spec in this ticket or a linked doc, then split implementation (e.g., a lint-style validator) into a follow-up ticket.

Recommend keeping this ticket scoped to the *design* of the rubric (steps 1-4) and filing a separate implementation ticket once the rubric spec is approved.

tool calls

[
  {
    "capability": "jira.issue.read",
    "args": {
      "key": "MZCLD-3663"
    },
    "result": {
      "key": "MZCLD-3663",
      "summary": "design deterministic rubric for denoting whether an incident report is complete",
      "status": "Backlog",
      "description_text": "Currently, we\u2019re using Claude to classify incidents as \u201ccomplete\u201d per the  incident completion checklist . However, that requires us to run an agent in infrastructure. It would be easier if we could avoid that. This covers designing a deterministic \u201cgood enough\u201d rubric for denoting whether an incident report is complete.",
      "comment_count": 0,
      "comments": []
    }
  }
]

log