Meeting Action Item Extraction

There's a specific failure mode in meeting transcripts: the model extracts discussion points as action items. "We talked about migrating to the new auth system" becomes [ACTION] Migrate to new auth system. Nobody owns it. Nobody has a deadline. It will never happen.

Meanwhile, the actual commitments are buried mid-sentence: "I can take a look at that this week" — implicit owner, implicit timeline, no explicit "action item" label.

This example optimizes an extraction module to catch real commitments (ownership + intent + timeline, however implicit) and ignore stated discussions.

Optimizer: PromptGradOptimizer Difficulty: Intermediate

The Problem

TRANSCRIPT:
"Alice: We really need to fix the onboarding flow before next launch.
Bob: Yeah I'll take a look at the Figma file this week.
Alice: Also the API docs are outdated.
Carol: Someone should probably update those.
Bob: The staging environment was acting up again.
Alice: We discussed that last time too."

Without optimization:

[
  "Fix onboarding flow",
  "Look at Figma file — Bob",   ← caught this one
  "Update API docs",            ← no owner, not a commitment
  "Fix staging environment",    ← not committed to by anyone
  "Fix staging (recurring)"     ← discussion, not action
]

After optimization:

[
  {"action": "Review Figma file for onboarding flow", "owner": "Bob", "deadline": "this week"}
]

Carol saying "someone should probably" is not a commitment. Alice noting a recurring issue is not a commitment. Only Bob made one.

Full Example

import json
import dspy
import vizpy
 
dspy.configure(lm=dspy.LM("openai/gpt-4o-mini"))
 
 
class ExtractActions(dspy.Signature):
    """Extract committed action items from a meeting transcript.
    Only include items with a clear owner and stated intent.
    Ignore discussions, suggestions without owners, and recurring complaints."""
 
    transcript: str = dspy.InputField(desc="Raw meeting transcript")
    actions: list[dict] = dspy.OutputField(
        desc='List of {action: str, owner: str, deadline: str}. Empty list if none.'
    )
 
 
module = dspy.Predict(ExtractActions)
 
 
_train_data = [
    {
        "transcript": (
            "Alice: We need to sort out the onboarding. "
            "Bob: I'll take a look at the Figma this week. "
            "Carol: Someone should update the API docs. "
            "Alice: The staging env was broken again."
        ),
        "expected_owners": ["Bob"],
        "false_owners": ["Carol", "Alice"],
        "note": "Carol used 'someone should' — no commitment. Alice stated a fact.",
    },
    {
        "transcript": (
            "Dan: The dashboard query is too slow. "
            "Emma: I can profile it tomorrow morning. "
            "Dan: There's also that bug in the export flow. "
            "Emma: Yeah we've been meaning to fix that for a while."
        ),
        "expected_owners": ["Emma"],
        "false_owners": ["Dan"],
        "note": "'We've been meaning to' is not a commitment.",
    },
    {
        "transcript": (
            "Frank: Sprint review is Thursday, we should prepare slides. "
            "Grace: I'll own the slides. Henry, can you handle the demo? "
            "Henry: Sure, I'll set it up Wednesday. "
            "Frank: Great. Also the mobile app has a memory leak, we saw that yesterday."
        ),
        "expected_owners": ["Grace", "Henry"],
        "false_owners": ["Frank"],
        "note": "Frank's memory leak mention is observation, not commitment.",
    },
    {
        "transcript": (
            "Iris: The client wants a custom export format. "
            "Jack: That'll take a week minimum. "
            "Iris: Maybe someone can spike it. "
            "Jack: I can put together a technical spec by EOD Friday."
        ),
        "expected_owners": ["Jack"],
        "false_owners": ["Iris"],
        "note": "'Maybe someone can' is not a commitment.",
    },
]
 
 
def metric(example, prediction):
    try:
        actions = prediction.actions if isinstance(prediction.actions, list) else []
    except Exception:
        actions = []
 
    extracted_owners = {
        a.get("owner", "").strip().lower()
        for a in actions
        if isinstance(a, dict) and a.get("owner")
    }
    expected = {o.lower() for o in example["expected_owners"]}
    false_pos = {o.lower() for o in example["false_owners"]}
 
    true_positives = extracted_owners & expected
    missed = expected - extracted_owners
    hallucinated = extracted_owners & false_pos
 
    precision = len(true_positives) / max(len(extracted_owners), 1)
    recall = len(true_positives) / max(len(expected), 1)
    f1 = 2 * precision * recall / max(precision + recall, 1e-9)
 
    feedback_parts = []
    if missed:
        feedback_parts.append(
            f"Missed committed owner(s): {', '.join(missed)}. "
            f"Look for 'I will/can/ll' even without explicit action language."
        )
    if hallucinated:
        feedback_parts.append(
            f"Incorrectly extracted {', '.join(hallucinated)} as owners. "
            f"Note: {example['note']}"
        )
 
    return vizpy.Score(
        value=f1,
        is_success=f1 >= 0.8,
        feedback=" | ".join(feedback_parts) if feedback_parts else "",
        error_type="missed_commitment" if missed else ("false_commitment" if hallucinated else ""),
    )
 
 
optimizer = vizpy.PromptGradOptimizer(metric=metric)
optimized = optimizer.optimize(module, _train_data)
 
 
# Test
result = optimized(transcript=(
    "Sam: We really should redesign the settings page. "
    "Tara: I'll put together a wireframe draft by end of week. "
    "Sam: Also the help docs haven't been touched in months. "
    "Tara: Someone from the content team should own that."
))
print(json.dumps(result.actions, indent=2))
# [{"action": "Create wireframe draft for settings page", "owner": "Tara", "deadline": "end of week"}]

Why This Is Hard

The line between discussion and commitment is entirely in linguistic register:

Phrase	Commitment?
`"I'll handle that"`	Yes — first person + future intent
`"Someone should look at this"`	No — no owner
`"We've been meaning to fix it"`	No — historical observation
`"Can you take that?"` + `"Sure"`	Yes — explicit acceptance
`"That needs to be done"`	No — passive voice, no owner
`"I can put together a spec by Friday"`	Yes — first person + capability + deadline

The model doesn't know this unless told. The optimizer's feedback explicitly describes the linguistic pattern that marks a commitment, and PromptGrad accumulates this as a rule across multiple training batches.

The error_type field distinguishes "missed_commitment" from "false_commitment" — enabling the optimizer to see both failure modes in each gradient batch.

Meeting Action Item Extraction

Meeting Action Item Extraction

The Problem

Full Example

Why This Is Hard

On this page