Back to Projects

Kettlebell Form

Started Page updated
On-device fitness prototype

A browser-based two-hand swing coach with adaptive full-frame capture, private 4–10 second clip analysis, and opt-in British command-style AI framing cues.

ProductTechnique-awareness coach
StatusLive engineering prototype
StackVite, React 19, TypeScript, MediaPipe, Web Workers, Three.js, OpenAI Realtime, WebRTC
KB FORM clip editor showing the full source, ten-second trim controls, and a lime analysis frame
The full source stays visible while the user selects only the clearest 4–10 seconds and movement region for local analysis.
Local 10-second clipsAdaptive full-frame cameraTrusted AI voice cues
Adaptive camera setup

Full-frame optics replace the step-back guesswork

Room and selfie modes preserve the complete 4:3 scene, prefer an environment-facing camera when available, expose exact post-permission device selection, and keep mirroring user-correctable.

Desktop KB FORM camera setup with room and selfie views, two disclosed British AI voice profiles, and a no-microphone privacy boundary
The setup targets a complete side view instead of digitally cropping away the athlete's head, hands, bell, feet, or floor.
Mobile framing

The same camera and voice choices remain legible on a phone

The full setup, two AI-generated voice profiles, explicit opt-in control, local-video boundary, and safety checklist collapse into one touch-friendly column.

Mobile KB FORM camera setup with full-frame guide, British male and female AI voice options, and local-processing disclosure
Verified at a 390 pixel viewport with no horizontal overflow.
Private clip entry

Video review starts with a clear privacy boundary

The upload state explains the supported formats, 120-second source and 200 MB limits, three-swing requirement, and local-only processing before a file is chosen.

Desktop KB FORM clip upload screen with privacy, framing, format, duration, and size guidance
No clip is uploaded, stored, transcoded, or added to the camera roll.
Branded processing

A kettlebell swing carries the waiting state

A motion-led brand animation, named stages, determinate progress, and a cancel control make the bounded one-frame-at-a-time analysis legible without blocking the interface.

Desktop KB FORM local analysis screen with animated kettlebell, progress bar, processing stages, and cancel action
The Worker samples one frame at a time while the interface stays responsive.
Guarded result

Pointers appear only when the visible evidence supports them

The result separates repetition evidence, view quality, supported movement signals, the current cue, and the observations a monocular clip cannot establish.

Desktop KB FORM clip result with pose overlay, current cue, repetition evidence, view quality, and model limitations
Insufficient reps, visibility, continuity, or coverage produce an explicit unassessed result instead of invented confidence.
Mobile selection

Trim and framing controls stay usable on a phone

The source preview, exact ten-second window, start and end controls, spatial frame, and primary analysis action collapse into a touch-friendly single column.

Mobile KB FORM clip editor with source preview, spatial frame, ten-second timeline, and analysis action
Verified at a 390 pixel viewport with no horizontal overflow.
Mobile pointers

The evidence hierarchy survives the smallest tested viewport

The selected frame and cue lead, followed by evidence counts, visible signals, limitations, and a direct route back to the editor.

Mobile KB FORM clip result with pose overlay, coaching cue, evidence summary, and visible signals
The same fail-closed result structure remains readable and actionable on mobile.

Browser first coaching

Kettlebell Form is now KB FORM: a technique-awareness coach for one deliberately narrow movement, the two-hand, shoulder-height, hip-hinge kettlebell swing. Alongside the live camera and no-camera preview, a user can choose an existing video, inspect the complete source, select the clearest 4–10 seconds, and move or resize a spatial frame around the useful movement without modifying the original clip.

The clip path is deliberately non-destructive: inspect the full source, select just the 4–10 seconds and region worth analyzing, and keep the original untouched.

The camera setup now treats optics as a first-class product problem. Room view asks for a full 4:3 environment-facing scene when the browser can provide it; Selfie view is explicit; granted devices can be selected exactly after permission; minimum zoom is attempted only when the camera exposes that capability; and mirroring follows the granted track with a manual correction. The preview and overlay preserve the full source frame so users no longer have to stand implausibly far away just to keep their head, hands, kettlebell, feet, and floor visible.

The runtime is built for browser responsiveness. Fresh camera frames or bounded clip samples are transferred to a dedicated MediaPipe module Worker with one-frame backpressure, while playback, trimming, cropping, progress, and the overlay remain responsive on the main thread. Clip inference is capped at 15 frames per second and a 640-pixel analysis edge; GPU inference falls back to CPU, and the pinned WASM runtime and checksum-verified pose model are served from the application's own origin.

What the model can support

The clip analyzer requires one visible athlete, at least three continuous backswing, drive, and float or finish sequences, and enough supported frames before it can produce pointers. Malformed landmarks, additional people, insufficient visibility, interrupted timing, unstable calibration, missing phase evidence, or inadequate coverage reset the in-flight movement and return an explicit unassessed result.

Unable to assess reliably is a first-class result, not an error hidden behind a flattering score.

That fail-closed state shapes both the interface and its lifecycle. The source is opened through a temporary local object URL; no clip, frame, landmark, filename, pointer, or analytics event is uploaded or retained by KB FORM. Exact Worker, job, and frame ownership prevents stale asynchronous work from mutating a newer session, while cancel, replace, close, and failure paths dispose transferred frames, media state, timers, Workers, and object URLs deterministically.

Shipping the prototype honestly

Framing guidance is visual first, with two optional, clearly disclosed AI-generated British command-style deliveries: a masculine presentation rendered with OpenAI's built-in cedar voice and a feminine presentation rendered with marin. They are not cloned people or Custom Voices. The browser creates one receive-only audio transceiver—never a microphone or Realtime data channel—and sends only an allowlisted cue ID plus a short-lived signed capability to the same-origin server. Trusted server code maps that ID to fixed text and controls OpenAI over a sideband channel; camera frames, clips, images, and landmarks never join the request. Hide, disable, session end, profile switch, and unmount pause locally, send bounded cancel/clear work, and close owned media resources, with an available local English device voice or visual-only state as the fallback. Device-voice availability, sound, timing, and OS/browser privacy behaviour vary by platform.

The app improves the engineering evidence. It does not turn a browser pose model into a biomechanics laboratory.

The claims boundary is equally deliberate. Monocular pose can support observations about visible joint relationships and timing; it cannot measure pain, breathing, bracing, muscle activation, spinal load, tissue capacity, kettlebell force, or injury risk. The optional body, region, skeleton, trail, and 3D layers are illustrative.

The release path now has real engineering gates: linting, strict TypeScript, 392 deterministic Vitest tests, 4 Playwright browser tests, coverage thresholds, bundle and model-integrity budgets, zero known dependency vulnerabilities, GitHub Actions, CodeQL, Dependabot, restrictive production headers, deep-link verification, and Vercel-native function builds. Public H.264, VP8, VP9, variable-frame-rate, exact-EOF, damaged-file, and immediate-retry clips exercise the real browser transport; a live voice smoke test verifies both profiles, mid-speech replacement, receive-only RTP, no microphone or data channel, and peer cleanup. A final diff-scoped security review reconciled all 38 changed source-like files and left no surviving code vulnerability. Those checks establish engineering behavior, not coaching accuracy.

This is still an engineering prototype, not a validated coach. Held-out athlete clips with qualified coach labels, repetition-level benchmark reporting, target-device latency and thermal tests, physical front/rear/ultra-wide camera sessions, assistive-technology checks, independent privacy-egress inspection, and motion-capture comparison remain the evidence needed before stronger performance or safety claims. The session firewall quota is live, while a durable shared cue quota remains an explicit deployment hardening item because the current Vercel plan rejected that second rate-limit rule.