Most of what I build listens before it decides.

Emitrr runs live voice conversations for other businesses, which means the pipeline gets exactly one chance to get a sentence right before the moment it was said has already passed. I spent my internship inside that pipeline, mostly in the few hundred milliseconds between someone talking and the system understanding them.

Endpointing

The transcription layer was the first target. I migrated the real-time pipeline off GCP Speech-to-Text and onto Deepgram, then spent most of the tuning time on endpoint detection and audio buffering — the unglamorous knobs that decide whether a pause means “the sentence is over” or “they’re still thinking.” Set the threshold too short and the system talks over people mid-thought; too long and every reply arrives late. Together with the rest of the pipeline work, it took end-to-end latency on live calls from 500 ms to 350 ms — a 30% cut.

The voice on the other end

On the output side, I integrated ElevenLabs for text-to-speech and optimized the streaming path, so the voice on the other end sounds like it’s keeping up rather than catching up.

When calls fall apart

I built a multi-threaded state manager that correlates five concurrent call states in real time, which made isolating pipeline failures far faster. Then I replaced the hard-coded fallback rules for incomplete calls with a reward-driven agent that learns what the next best action should be.

Sub-patch: stutter detection

Before Emitrr, I built a stutter detector that fuses two views of the same speech: wav2vec embeddings of the audio and BERT embeddings of its transcript, concatenated into one vector for a binary classifier. When it flags stuttering, it reports how much, which signs, and the exact seconds where they occur.

Finger-prick tests work, but nobody enjoys doing them four times a day. Photoplethysmography — the optical technique your phone’s camera can approximate for a heart-rate reading — offers a non-invasive alternative, if you can get a model to pull blood glucose out of what’s essentially a light-absorption waveform. I worked on this as part of a research team collecting PPG recordings from 611 participants, paired with reference readings from a standard glucometer.

Cleaning the signal

Most of the work happened before any model saw the data. PPG signals are noisy and drift, so the pipeline runs Kalman smoothing to cut high-frequency noise, a wavelet-based algorithm to correct baseline wander, and the AMPD algorithm to find the trough of every pulse so individual heartbeats can be isolated and measured.

Features that carry the signal

From those isolated pulses we extracted 392 candidate features — polynomial fits, spectral and heart-rate-variability measures, and a set of novel area-based features I helped design, which measure how much of a pulse’s area sits above or below given amplitude and time thresholds. Feature selection cut that to 26, then 16, then as few as 5, with almost no loss in accuracy; the area-based features carried most of the signal.

Accurate, or readable

A Voting Regressor ensemble reached R² 0.921, the strongest result in the study. But a black box can’t tell a clinician why it made a prediction, so we also trained a white-box model with PySR — symbolic regression that searches for an actual mathematical expression. It landed at R² 0.815: a real drop, but the whole model is one equation a person can read. On the Clarke Error Grid, 90% of the ensemble’s predictions and over 70% of the white-box model’s landed in Zone A.

A single Wireless Capsule Endoscopy procedure produces somewhere between 50,000 and 60,000 frames, and someone has to look at nearly all of them. That review takes about eight days per patient. This work asks a narrower question: how much of those eight days is necessary, and how much is the bottleneck of human eyes on frames that are mostly unremarkable.

The pipeline

The pipeline uses YOLOv11, trained on WCEBleedGen — 2,618 frames split evenly between bleeding and non-bleeding — to classify, detect and segment bleeding regions in a single pass. A C3k2 backbone extracts features efficiently, SPPF pools across scales, and a C2PSA attention block learns to look where bleeding actually is. Preprocessing matters almost as much: adaptive histogram equalization for contrast, Gaussian filtering for the noise WCE images are known for.

The number that matters

97.4% average precision with 98.6% recall on classification, 0.678 box mAP@0.5, and 0.798 mask mAP@0.5 — the one I care about most, since a box says “something’s here” while a mask says where to look first. Paired with triage that surfaces the clinically significant frames, the goal is to take an eight-day review down to hours: not by replacing the clinician, but by pointing them at the right 2%.

I keep building the same kind of system in different clothes: take a pile of material nobody wants to read end to end, and let someone ask it a question.

Papers, audio, careers

PWC-RAG indexes recent ML research from Papers with Code into a serverless vector database with BGE embeddings, so you can ask about a technique instead of skimming fifty abstracts. RAG-over-Audio transcribes recordings, chunks the transcripts, and answers questions with the source passages attached. Career Compass, a team project, grounds multi-agent career advice in real documents through a history-aware retriever over FAISS.

Grounded, not confident

The part I care about is the same each time: an answer should trace back to something real rather than something merely plausible. This module is that idea pointed at the page you’re reading. Your question is embedded, compared against every passage on this site, and the closest passages are shown as-is, with links. Nothing is generated, so nothing can be made up.

This started as a question: what does it look like when a language model is trying not to tell you something? Most “adversarial” demos are keyword filters wearing a costume. I wanted a narrator that genuinely withholds — answering in metaphor and misdirection, the way a person guarding a secret would rather change the subject than say “no comment.”

Eight ranks of evasion

The world of Aethermoor holds seven secrets. Each one you extract raises your rank, from Novice through Initiate, Apprentice, Journeyman, Adept, Expert and Sage to Master, and each rank swaps in a more evasive system prompt: early answers are clear stories; late answers dissolve into paradox.

Judging understanding, not keywords

Keyword matching breaks the moment someone paraphrases, so a second model acts as judge: it reads your extraction against the real secret and decides whether the idea landed. Between turns, a closeness score — inverted response entropy, keyword presence and token overlap — tells you whether you’re getting warmer, without handing over the answer.