CU-09: AI dashboard assistant — feature-parity actions via chat

Priority: P1 Accounts/sessions: P1 host (ravikantguptaofficial@gmail.com) signed into Calendo at https://calendo.dev/dashboard/ via "Sign in with Google". No other account is required for the core suite. Parallel-safe: Yes — this suite only creates its own RUNID-scoped event types/bookings/notes and never edits the global default availability schedule. Exclusive (rewrites global host availability?): No. The suite creates and edits a NEW, RUNID-scoped availability schedule; it never edits/deletes the host's [DEFAULT] "Working Hours" schedule, and it never edits global weekly hours. Estimated time: 45 minutes (AI round-trips are slow; allow ~15-30s per assistant reply). L3 reality checks: Partial. The AI assistant's effects are verified primarily in Calendo's own UI (L1/L2). There is one optional L3 check (a no-show side effect cross-checked against the analytics tab, still in-app). No external Google/Outlook calendar or Gmail assertion is required for this suite because the assistant's tools operate on Calendo data, not on external calendars/email. See "L3 reality checks".

Goal

This suite proves Calendo's headline differentiator and a hard project rule ("AI agent feature parity"): the dashboard AI assistant can perform the same account-management actions a user can do by clicking through the UI — create/update/delete event types, list and act on bookings (mark no-show, add notes), report analytics, set buffers/meeting limits, create and edit availability schedules, generate embed code, fetch the booking link, manage integrations (Slack/tracking), navigate the dashboard, and refuse off-topic requests. For every write action we do NOT trust the chat's claim of success; we reload the relevant dashboard tab and confirm the data actually changed. Any tool that reports success but leaves state unchanged, or any UI capability with no matching AI tool, is recorded as a finding. This matters because the AI assistant is the product's primary selling point and a stated invariant of the codebase.

Preconditions

Test data

RUNID convention: pick a fresh UTC token at execution time, e.g. RUNID = 20260601-1530. Every created artifact embeds RUNID so reruns never collide and cleanup can scope by RUNID.

Create/use exactly these via the AI chat:

Steps

Ordering rationale: the AI panel is opened first (1-2). Read-only probes (3-5) come before any writes so we have a clean baseline. Creates precede updates which precede deletes for each entity. The off-topic refusal (16) and the poll parity probe (14) are independent. Cleanup (Steps 19-23) runs last.

  1. Action — Go to https://calendo.dev/dashboard/ and confirm you are logged in (look for the welcome header #welcomeHeader). On the Overview tab, read and record the host slug from the booking link bar (#bookingLinkValue, e.g. ravikantguptaofficial — the full link looks like https://calendo.dev/booking/?user=<HOST_SLUG>). Expect — Dashboard loads; welcome header visible; booking link value is a non-empty URL. Record <HOST_SLUG>. [L1] — Capture screenshot: cu09-01-dashboard-loaded.
  1. Action — Open the AI assistant panel. On desktop it is the persistent right sidebar titled "AI" / Calendo AI (.ai-panel#aiPanel) with a text input at the bottom (#aiBarInput, placeholder "Ask anything...") and a "Send" button (#aiBarSend). If the panel is not visible (narrow window / mobile), click the floating sparkle button at bottom-right (.mobile-ai-fab#mobileAiFab) to open it. Expect — The AI panel is visible with an empty/greeting message area (#aiMessages) and an enabled input box. [L1] — Capture screenshot: cu09-02-ai-panel-open.
  1. Action — Click the input (#aiBarInput), type What event types do I have? List their names, durations, and IDs. and click Send (#aiBarSend) or press Enter. Expect — Within ~30s a new assistant message appears in #aiMessages listing the account's real event types with durations and IDs. The reply must NOT contain "something went wrong", "rate-limited", or "AI not configured". This exercises the get_event_types tool. [L1] — Capture screenshot: cu09-03-list-event-types.
  1. Action — Type Show my upcoming bookings with their booking IDs and invitee names. and Send. Expect — Assistant returns a list of upcoming bookings (or "no upcoming bookings"), each with an ID and invitee name/email. Exercises get_bookings (filter=upcoming). Record the FIRST booking's ID + invitee name for Steps 11-12 (if none exist, note it; skip Steps 11-13's booking-write parts and flag as "no booking available to act on"). [L2] — Cross-check: open the Bookings tab (sidebar link a[data-tab="bookings"], panel #panel-bookings) and confirm the bookings the AI listed match what's shown. Capture screenshot: cu09-04-list-bookings.
  1. Action — Type Report my analytics: total bookings, bookings this month, cancellation rate, and no-show rate. and Send. Expect — Assistant returns numeric values for total bookings, this month, a cancellation rate (with %), and a no-show rate (with %). Exercises get_analytics (its tool result includes total_bookings, this_month, cancellation_rate, no_show_count, no_show_rate). [L2] — Cross-check: open the Analytics tab (a[data-tab="analytics"], panel #panel-analytics) and confirm the numbers match the stat tiles: total (#statTotal), this month (#statThisMonth), cancellation rate (#statCancelRate), no-show rate (#statNoShowRate), most popular (#statPopular). The AI numbers must equal the panel numbers. Capture screenshot: cu09-05-analytics-match.
  1. Action — Type exactly: Create an event type called "CU09 AI Quick Sync <RUNID>", 15 minutes, color #ef4444, description "Created by CU-09 AI test <RUNID>". and Send. (Substitute the real RUNID.) Expect — Assistant confirms it created the event type, echoing the name CU09 AI Quick Sync <RUNID>. Exercises create_event_type (auto-generates slug cu09-ai-quick-sync-<RUNID>, default location google_meet). [L1] — Capture screenshot: cu09-06-create-et-chat-claim.
  1. Action — Verify the create actually persisted. Reload the page (F5) to defeat any client-side caching, then open the Event Types view: click the "+ New" area's tab or the Overview event-types section, then navigate to the event types list (sidebar has no direct event-types tab; use the Overview "+ New" button context or the event-types panel #panel-event-types; the simplest robust path is to reload and look at the Overview event-types list which calls renderEventTypes()). Find CU09 AI Quick Sync <RUNID>. Expect — The new event type appears in the dashboard list with 15 min duration and red color. This proves the AI write hit the database (not just a chat claim). [L2] — Capture screenshot: cu09-07-create-et-verified. If it does NOT appear after reload, record a finding: "create_event_type claimed success but no row created."
  1. Action — In the AI panel, type What is the direct booking link for CU09 AI Quick Sync <RUNID>? and Send. Expect — Assistant returns a full URL containing /booking/?user=<HOST_SLUG>&event=cu09-ai-quick-sync-<RUNID> printed in the message text (the system prompt requires showing the full URL, not just "Copied!"). Exercises get_booking_link + copy_to_clipboard. [L2] — Open that URL in a new tab and confirm the public booking page loads for that event (title shows the event name). Capture screenshot: cu09-08-booking-link.
  1. Action — Type Rename CU09 AI Quick Sync <RUNID> to "CU09 AI Renamed <RUNID>" and make it 45 minutes. and Send. Expect — Assistant confirms the rename and duration change. Exercises update_event_type (name + duration_minutes). [L2] — Reload, view the event types list, confirm the entry now reads CU09 AI Renamed <RUNID> at 45 min. Capture screenshot: cu09-09-update-et-verified. If unchanged after reload, record a finding.
  1. Action — Type Create a 30 minute event type called "CU09 AI Buffered <RUNID>", then set a 10 minute buffer before and 10 minute buffer after on it. and Send. (The assistant should first create it, then call update_event_type with buffer_before:10, buffer_after:10; it may ask you to confirm or do both in sequence — if it asks, reply yes.) Expect — Assistant confirms creation and buffer settings. Exercises create_event_type + update_event_type (buffers). [L2] — Reload, open the event type's edit panel for CU09 AI Buffered <RUNID> and confirm buffer-before = 10 and buffer-after = 10 in the form. Capture screenshot: cu09-10-buffers-verified. If the buffer fields are still 0, record a finding ("buffer update claimed but not persisted").
  1. Action — Add private notes to a real booking. Using the booking ID recorded in Step 4, type Add a private note to booking <BOOKING_ID>: "CU09 AI note <RUNID>". and Send. (If no booking exists, skip and flag.) Expect — Assistant confirms the note was added (host-only note). Exercises add_booking_notes. [L2] — Open the Bookings tab, expand/open that booking, confirm the internal note CU09 AI note <RUNID> is shown. Capture screenshot: cu09-11-booking-note. If the note is absent, record a finding.
  1. Action — Mark a PAST booking as no-show via AI. Identify a PAST confirmed booking ID (ask the AI: Show my past bookings with IDs — exercises get_bookings filter=past). Then type Mark booking <PAST_BOOKING_ID> as a no-show. and Send. NOTE: the tool only works on PAST confirmed bookings (it rejects future or cancelled ones). If there is no past confirmed booking, skip this step and record "no past booking available — mark_no_show not exercised" (do NOT cancel/fabricate one just to test). Expect — Assistant confirms the booking is marked no-show (or, for an ineligible booking, explains it cannot mark a future/cancelled booking — that is also correct behavior to record). [L2] — Open Bookings tab, find that booking; it should show a no-show badge (.badge-no_show, orange) and the "Mark no-show" button should be gone. Capture screenshot: cu09-12-no-show-verified.
  1. Action — Create a NEW availability schedule via AI (does NOT touch the default). Type Create a new availability schedule called "CU09 AI Hours <RUNID>" with hours Monday through Friday 10:00 to 16:00. and Send. Expect — Assistant confirms creation of the new schedule. Exercises create_availability_schedule. [L2] — Open the Availability tab (a[data-tab="availability"], panel #panel-availability) and confirm a schedule named CU09 AI Hours <RUNID> exists with Mon-Fri 10:00-16:00, and that the original [DEFAULT] "Working Hours" schedule is unchanged and still marked default. Capture screenshot: cu09-13-availability-verified. If the default schedule was altered, record a HIGH-severity finding.
  1. Action (parity probe — meeting poll) — Type Create a meeting poll titled "CU09 AI Poll <RUNID>" with three time options next week so people can vote on a time. and Send. Expect — Calendo DOES have a meeting-poll feature in the UI/API (/api/polls, meeting_polls table, "create poll" UI), but there is NO create_poll (or equivalent) tool in the AI assistant's tool set (DASHBOARD_TOOLS). So the assistant will most likely either (a) decline / say it can't create polls, (b) only describe how to do it manually, or (c) attempt and fail. Record exactly what it does. [L1] — Capture screenshot: cu09-14-poll-parity-probe. Finding to record regardless of outcome: "Meeting polls exist as a UI/API feature but are NOT exposed as an AI tool — feature-parity gap." Then confirm no poll named CU09 AI Poll <RUNID> was actually created (check the polls UI if reachable). This is the suite's primary deliberate parity finding.
  1. Action — Generate embed code via AI. Type Give me the inline embed code for my booking page, then also give me the popup embed code. and Send. Expect — Assistant returns HTML snippets. The inline snippet should contain calendo-inline, data-mode="inline", calendo-embed.js, and the host slug. The popup snippet should contain calendo-popup, data-mode="popup", and a "Book a Meeting" label. Exercises get_embed_code (inline + popup). [L1] — Capture screenshot: cu09-15-embed-code. (This is L1 — embed code is generated text; there is no separate persisted state to reload. Optionally paste the inline snippet into a scratch HTML file to confirm it renders a Calendo widget, but that is out of scope / manual residue.)
  1. Action (off-topic guardrail) — Type What is the capital of France? and Send. Expect — Assistant refuses with the canned line containing "I can only help" (full expected text: "I'm Calendo AI — I can only help with scheduling and account management..."). It must NOT answer "Paris". [L1] — Capture screenshot: cu09-16-offtopic-refused. If it answers the geography question, record a finding (guardrail failure).
  1. Action (navigation tool) — Type Take me to the analytics tab. and Send. Expect — The dashboard switches to the Analytics panel; #panel-analytics gains the active class and the analytics stat tiles are visible. Exercises navigate_to (tab=analytics). Note: navigate_to only supports overview, bookings, analytics, availability, routing-forms, settings — it does NOT support calendar/contacts/event-types; if you ask for one of those it should not crash (record behavior). [L1] — Capture screenshot: cu09-17-navigate-analytics.
  1. Action (extra tools — pick 2 and run) — Run two additional read-only/integration probes to broaden tool coverage:
    • (a) Type Show me my recent audit log entries. — exercises get_audit_log. Expect a list of recent account actions (it should include the event-type create/update you just did, confirming write actions are logged).
    • (b) Type What are my analytics tracking IDs (Google Analytics and Meta Pixel)? — exercises manage_tracking (action=get). Expect it to report current GA / Pixel IDs or "not set". Do NOT set new tracking IDs (avoid mutating real settings).
    • Optional (c) Type Is Slack connected for notifications? — exercises manage_slack (action=get). Expect "not configured" or the current state. Do NOT set/remove Slack. *Expect** — Each returns a coherent, non-error answer driven by a real tool result. *[L2]** — For (a), spot-check that an action you performed this run (e.g. an event-type creation) appears in the audit answer. Capture screenshot: cu09-18-extra-tools.
  1. Action (cleanup — delete event type A) — Type Delete the event type "CU09 AI Renamed <RUNID>". and Send. The assistant will ask you to confirm (delete is HIGH-RISK). Reply yes, delete it. Expect — Assistant confirms deletion. Exercises delete_event_type with the confirm-before-delete guardrail. [L2] — Reload, confirm CU09 AI Renamed <RUNID> is gone from the event types list. Capture screenshot: cu09-19-delete-et-A.
  1. Action (cleanup — delete event type B) — Type Delete the event type "CU09 AI Buffered <RUNID>". and confirm yes when prompted. Expect — Assistant confirms deletion; reload and confirm it is gone from the list. [L2] — Capture screenshot: cu09-20-delete-et-B.
  1. Action (cleanup — delete the test schedule) — Type Delete the availability schedule "CU09 AI Hours <RUNID>". and confirm yes when prompted. (This is a non-default schedule, so deletion is allowed; the default cannot be deleted.) Expect — Assistant confirms; open the Availability tab and confirm CU09 AI Hours <RUNID> is gone and the [DEFAULT] schedule remains intact. [L2] — Capture screenshot: cu09-21-delete-schedule.
  1. Action (cleanup — clear the test note) — If Step 11 added a note to a real booking, type Clear the private note on booking <BOOKING_ID>. (the add_booking_notes tool accepts an empty string to clear). Send and confirm. Expect — Assistant confirms the note is cleared; open the booking and confirm the note field is empty. [L2] — Capture screenshot: cu09-22-note-cleared.
  1. Action (cleanup — no-show) — The no-show mark from Step 12 is on a real past booking and represents a state change to a real record. Do NOT silently revert it via raw SQL. If the booking was a genuine test/throwaway, leave a note in the results report listing the booking ID and invitee so a human can decide whether to restore its status (the UI/AI offers no "un-no-show" action). If Step 12 was skipped, write "no no-show change to reconcile." Expect — No-show residue is explicitly documented for human follow-up. [L1] — Note in report; no screenshot required.

L3 reality checks

None requiring an external Google/Outlook calendar or Gmail assertion — the dashboard AI assistant's tools mutate Calendo data (event types, schedules, bookings, settings), not external calendars or email, so the binding reality checks are the L2 dashboard-reload verifications above. The closest thing to an external/independent cross-check is the no-show side effect: after Step 12 marks a booking no-show, open the Analytics tab (Step 5 path) and confirm the no-show count/rate (#statNoShowRate) increased by one booking versus the baseline you captured in Step 5 — this proves the no-show write propagated to the independently computed analytics aggregate, not just the bookings row. Record both the before (cu09-05) and after no-show rate. If the analytics no-show count did not move after a successful no-show mark, record a finding.

Cleanup

Everything created by this suite is RUNID-scoped. Confirm all of the following are gone/clean (most are handled by Steps 19-23):

Pass/Fail criteria

The run PASSES only if ALL of the following hold:

The run FAILS if: the AI panel never opens; any AI reply for a core step returns "something went wrong"/"AI not configured"/rate-limit error that is not a flagged precondition; any write tool reports success but the dashboard shows no change after reload; the guardrail answers an off-topic question; or the default availability schedule is altered/deleted.

Evidence to capture

Manual residue / cannot-verify