Quick start with a camera
- Open Gesture Synth and press Start with camera. The browser will request camera permission only after this click.
- Hold one hand about 30–60 cm from the camera. Keep the wrist and all fingertips visible. Face a window or lamp if the Lighting status reports that the hand is hard to see.
- Match the ghost hand with one, three, and five fingers. The short calibration confirms that the tracker can read your working position. It does not save a biometric profile.
- Press Start challenge, listen to the four-beat count-in, and show the target finger count before the next bar.
Finger and movement controls
| Input | Result | Practical tip |
|---|---|---|
| 1 finger | Chord degree I | Use the index finger and keep the others clearly folded. |
| 2 fingers | Chord degree II | A clear V shape is easier to classify than close fingers. |
| 3 fingers | Chord degree III | Keep the thumb folded if the count changes unexpectedly. |
| 4 fingers | Chord degree IV | Fold the thumb across the palm. |
| 5 fingers | Chord degree V | Spread the fingers enough for the tips to remain distinct. |
| Hand higher/lower | More/less intensity | Move the whole hand instead of stretching the fingers. |
| Fist | Release | Hold it briefly so a single folded finger is not mistaken. |
What the three chord labels mean
Target is the chord requested by the challenge. Playing is the chord currently sounding. Next is the stable gesture waiting for Beat Lock. When the next beat arrives, Next becomes Playing. Together, the three labels show why the picture can update before the sound.
Beat Lock and Instant mode
Beat Lock is the default because camera input is naturally uneven. The tracker first waits for three similar frames, then keeps a new gesture in the Next position until a musical boundary. This makes a small correction less likely to become an accidental extra chord. Challenge mode always uses Beat Lock so timing scores have a stable reference.
Instant mode is available in Free Play. It applies a stable gesture as soon as classification finishes. Use it for direct sound design or slow movement where groove is less important. Switch back to Beat Lock when rapid finger changes sound untidy.
Mouse, touch, and keyboard controls
You can play without a webcam. Choose Mouse Mode and click one of the five chord pads, or drag across the stage: left to right chooses I through V, while vertical position changes intensity. On a phone or tablet, tap the same pads. A physical keyboard uses number keys 1–5 and Space to release. Every input reaches the same audio engine, challenge score, and MIDI export.
Camera troubleshooting
Camera blocked
Choose Mouse Mode immediately, or open the browser’s site permissions and allow camera access for gesturesynth.online. Reload only if the browser requires it. Gesture Synth never needs microphone permission.
Hand not found
Move back until the wrist and all fingertips fit inside the stage. Avoid placing the hand directly against a bright window. A plain background can help, but clear separation and steady lighting matter more than background color.
Wrong finger count
Rotate the palm toward the camera, separate extended fingers, and fold the others fully. Make one deliberate pose instead of changing finger count while moving the whole arm. The Stability status changes once the pose has survived the confirmation window.
No audio
Press any chord pad so the browser can start audio. Confirm that Sound is on, then check the tab and system volume. If the page was in the background, return and interact once to resume the audio context.
Video export unavailable
Browser codec support varies. Complete the challenge and use MIDI as the fallback. MIDI preserves the beat-locked chord performance but never includes camera frames, audio, or hand landmarks.