Skip to content
Clexerl
All roles

Frontier AI · Multimodal

Research Engineer, Audio & Speech

Build speech understanding and generation that holds up in a noisy, real room.

  • San Francisco
  • Hybrid
  • Full-time
  • $300K – $480K

About the role

The audio team builds speech understanding and generation that holds up in a noisy, real room. That means recognition that copes with overlapping speakers, accents, reverberation and cheap microphones, and synthesis that is natural, controllable and fast enough to hold a conversation. The challenge is as much systems work as modelling.

Voice is a natural interface for the agents in Clexerl Go: dictating a task, joining a meeting, or asking what changed overnight. It is also an input stream for the wider multimodal effort, so the audio models feed the same shared model, not a separate stack. Latency, interruption handling and privacy are product requirements from day one.

Research engineers here move between modelling and deployment. A change is not done until it runs within a latency budget on realistic hardware and has been measured on recordings that sound like real offices, cars and kitchens, not studio data.

What you’ll do

  • Train and fine-tune speech recognition, diarisation and speech-to-speech models on large, messy audio corpora.
  • Build streaming inference with low first-token latency, including chunked encoders, endpointing and barge-in handling.
  • Develop expressive text-to-speech and voice synthesis with controllable prosody, plus safeguards against misuse such as unauthorised voice cloning.
  • Create evaluation harnesses covering word error rate, speaker attribution, naturalness and robustness to noise and codec compression.
  • Optimise models for serving using quantisation, caching and batching strategies suited to real-time audio.
  • Work with the multimodal research team to integrate audio tokens into the shared model.

What you’ll bring

  • Experience building speech or audio machine learning systems, such as ASR, TTS, speaker modelling or audio codecs.
  • Solid signal processing foundations: spectrograms, filterbanks, resampling, beamforming and what compression does to a signal.
  • Working knowledge of modern architectures for audio, including conformers, neural codecs with residual vector quantisation, and autoregressive or diffusion decoders.
  • Strong Python and PyTorch skills, and comfort profiling latency in a streaming system.
  • Habit of testing on out-of-distribution recordings, and honesty about how performance drops on them.
  • Awareness of consent, privacy and misuse risks specific to voice data.

Nice to have

  • Experience with on-device or low-power speech inference.
  • Research on full-duplex dialogue and turn-taking models.
  • Background in acoustics, room simulation or microphone array processing.
  • Fluency in several languages or experience with low-resource speech.

Work you might take on

  1. 01

    Build a streaming recognition stack that keeps first-word latency under a fixed budget while preserving accuracy on overlapping speech.

  2. 02

    Create a noisy-room test set by simulating reverberation, background chatter and phone-call codecs, and track models against it.

  3. 03

    Add speaker diarisation to meeting transcripts so Clexerl Go agents can attribute decisions and action items correctly.

  4. 04

    Train an audio tokeniser and measure how well downstream language models reason over its output.

Compensation

$300K – $480K

The US base salary range for this role. Every offer also includes equity in Clexerl and, when it spins out, in the venture you help build. We tell you the reasoning behind your number.

Benefits

  • Health, fully covered

    Medical, dental and vision for you and your dependants, premiums paid in full.

  • Two kinds of equity

    Equity in Clexerl, plus a stake in the venture you help build when it spins out.

  • 22 weeks parental leave

    Fully paid, for every parent, however your family arrives.

  • Rest that sticks

    Flexible time off with a four-week minimum we actually check on, plus a company-wide week in December.

  • Learning and compute

    A yearly budget for courses, conferences and books, and compute credits for side projects.

  • 401(k) with a 4% match

    Matched from your first paycheque and vested immediately.

How we hire

  1. Apply 5 minutes

    Send links to what you have built and a few lines on why this role. No cover letter needed.

  2. Intro call 30 minutes

    A conversation with the hiring manager about the work, the team and what you want next.

  3. Work sample 2–4 hours

    A paid take-home or a pairing session on a real-shaped problem. You pick the format.

  4. Studio day One day

    Meet the team, go deep on your work and ours. Remote or in person.

  5. Offer Within 5 days

    A written offer with the full range, the equity and the reasoning behind both.

Apply for this role

Opens an email with a short template. Links beat résumés.

All Frontier AI roles