Audio Player

When to useCopy Anchor Link

• The agent found or produced a recording the user should hear: a voice note, a call recording, a podcast clip, a generated voiceover.
• The user asks to play a specific part of a recording ("the part where they agreed on pricing"). Set startAt and endAt to that part.

When not to useCopy Anchor Link

• There's no real audio URL and precomputed peaks from a tool. The model must never invent them.
• The user only needs what was said. Quote or summarize the transcript in text.
• It's a video. Use a video player.

How the agent should use itCopy Anchor Link

• src, durationSeconds, peaks and segments come from tool results. Peaks are the waveform's loudness (64–400 values, 0–1), computed by the host's audio tool, never by the model.
• For "the part where…", find the matching transcript segments, set startAt/endAt to cover them, and add a marker at the key moment.
• Keep marker labels to a few words: "Pricing agreed".
• Only set downloadable when the user may keep the file.

How it worksCopy Anchor Link

• Waveform: bars are resampled from peaks to fit the width (the loudest value in each bucket wins, so short sounds stay visible). The played part is drawn in the foreground color. Hover shows the time; click or drag seeks.
• Clip: with startAt/endAt, the header's quiet line adds "Clip 0:18–0:33" after the subtitle, the range is shaded behind the bars, playback starts at startAt and stops at endAt. Pressing play again replays the clip. Seeking outside it plays from there normally. Transcript lines outside the clip are dimmed.
• Markers are ticks above the bars, and buttons below ("Pricing agreed · 0:26") that seek.
• Transcript: every segment is shown, under the player without a visible heading (screen readers hear "Transcript"). With speakers, the first speaker's lines sit on the left and the next on the right; each speaker has a solid colour (slate blue, teal, gold, then green, by order of first appearance) on their bubbles and on their initials when there is no photo. The current one is highlighted and kept in view by scrolling the transcript only, never the page; past 8 segments it gets its own scroll area. Clicking a segment seeks to it.
• Audio is a real <audio preload="metadata">, so only the header is fetched until the user presses play. A failed or unsafe URL shows "Couldn't load this recording". URLs must be http(s), blob: or data:audio/….
• Download appears only with downloadable and a safe URL. Web URLs open in a new tab if the browser won't download them.

AccessibilityCopy Anchor Link

• The play button and speed menu are labelled with the title: "Play Call with Harbor Goods".
• The waveform is a slider: its value reads as "1 minute 12 seconds of 4 minutes 3 seconds". Left/Right move 5 seconds, Page Up/Page Down 30 seconds, Home/End jump to the start or end, and Space plays or pauses.
• The bars don't animate, so there's nothing to reduce for reduced motion; transcript scrolling is instant instead of smooth.

InstallationCopy Anchor Link

npx shadcn@latest add https://your-site.com/r/audio-player.json

UsageCopy Anchor Link

The Audio Player is display-only, so the tool can return immediately:

import { tool } from "ai"
import { audioPlayerTool } from "@/components/genui/audio-player.schema"

const tools = {
  [audioPlayerTool.name]: tool({
    description: audioPlayerTool.description,
    inputSchema: audioPlayerTool.inputSchema,
    execute: async () => ({ shown: true }),
  }),
}

In practice the agent gets src, durationSeconds, peaks and segments from your own recording or transcription tool and passes them through. Compute the peaks on the server, e.g. the max absolute sample in 200 equal buckets, normalized to 0–1.

<AudioPlayer {...part.input} status={part.state === "input-streaming" ? "streaming" : "ready"} />

With LangGraphCopy Anchor Link

Push it from a graph node with ui.push({ name: "show_audio", props }, { message }), and render the thread's UI messages with the Renderer.

SchemaCopy Anchor Link

Host props: status (streaming or ready) and className. While streaming, the title shows first and the waveform is a placeholder until the peaks arrive.

SourceCopy Anchor Link