Negotiation Task Reader
Six-condition human-AI experiment platform with citation-grounded assistance, schema validation, parallel evidence scans, SSE streaming, model fallbacks, and revision-safe behavioral logging.
I build and evaluate production LLM systems.
AI researcher and founding engineer. I work on what happens after the base model: evaluation, data pipelines, and human-AI systems.
Postdoctoral Fellow · Harvard Business School AI Institute
Ph.D. CS · Harvard
Cambridge, MA
Current systemProblem taxonomy
01Recover web evidence120 snapshots
02Extract structured problems88,336 firms
03Build human gold sets6 coders
04Optimize & evaluateF1 0.70–0.88
05Learn interpretable themes265 clusters
AWS · Python · DSPy · PyTorchversioned prompts + artifacts
88,336startups in the final LLM measurement pipeline
120Common Crawl snapshots processed from 2013–2026
331founder interviews in an ongoing validation program
10,000+participants across controlled online experiments
01 / FLAGSHIP
From historical web pages to evaluated measurements.
The Problem Taxonomy project turns licensed company data and historical web archives into structured measurements. Every stage is recoverable and testable, from WARC retrieval through human-label evaluation.
From 30.1M archived pages to an evaluated problem taxonomy
I built the data recovery, LLM measurement, annotation, evaluation, orchestration, and analysis layers. The pipeline uses Athena to reduce the Common Crawl index before fetching WARC byte ranges, then applies structured LLM extraction, human-gold evaluation, and interpretable representation learning.
Constraint: 883 GiB of archived HTML, probabilistic model outputs, a licensed dataset that cannot be exposed, and a paper that must reproduce every number.
Python · AWS Athena/S3 · WARC · OpenAI APIs · DSPy/GEPA · PyTorch · scikit-learn · statsmodels
Read the engineering case study
01IndexDomain-year targets against Common Crawl
02RecoverRanked WARC pointers, byte-range fetches, checkpoints
03MeasureSchema-constrained problem statements at temperature 0
04EvaluateGold sets, IRR, held-out F1, GEPA search
05OrganizeTop-K sparse features, stability checks, audit trail
02 / SELECTED SYSTEMS
Systems people use and studies depend on.
CareCorgi
Memory-enabled caregiving agent and realtime phone service spanning ADK function calling, Vertex AI Memory Bank, Firestore, Twilio Media Streams, audio transcoding, bounded queues, and VAD-gated barge-in.
Interview Workbench
An HBS research team uses this tool for diarized transcription, LLM paragraph segmentation, source search, and structured evidence review; it supports an ongoing 331-interview program localized for Chinese startups.
Human Annotation Tools
Research infrastructure for balanced assignment, resumable coding, and gold-set construction. More than 800 annotations across psychological frameworks feed agreement metrics and DSPy evaluation inputs.
03 / HOW I WORK
Engineering choices I repeat.
eval_first()Evaluation is a subsystem
I build codebooks, independent human labels, inter-rater reliability, held-out metrics, ablations, and error analysis into the pipeline from the start.
resume_from(checkpoint)Long pipelines must recover
Pointer tables, bounded concurrency, content-addressed outputs, checkpoint rows, manifests, and versioned prompts turn expensive runs into inspectable state.
preserve(provenance)Human-AI systems need contracts
Evidence scope, model fallbacks, interaction revisions, completion semantics, and audit logs sit alongside structured-output schemas.
04 / RESEARCH
2025
2024
2022
Complete publication list
Selected first-author work.
“ChatGPT, Don’t Tell Me What to Do”
Designing AI for context analysis in humanitarian frontline negotiations.
LLM chatbots for LGBTQ+ mental health support
Human evaluation of large language model based chatbots in a high-stakes support context.
Not Just a Preference
A controlled experiment on reducing biased decision-making on dating websites.