Eric Modesitt

ML Researcher — LLM reliability, uncertainty quantification, sequential inference

Builder–researcher focused on reliability, calibration, and robustness in large language models. I design sequential inference systems that provide statistical guarantees on model outputs—work motivated by ensuring AI systems remain trustworthy as they scale.

Arlington, Virginia

Research interests

Sequential inference
Systems with statistical guarantees, early stopping, and optimal cost–accuracy tradeoffs.
Uncertainty & calibration
Reliable, well-calibrated model outputs with principled uncertainty estimation.
Robustness testing
Stress-testing models via perturbations, distribution shift, and adversarial inputs.
Efficient ensembles
Adaptive computation using KV caching and low-cost branching strategies.

Selected research

2025

Sequential Multi-Persona Direct Logit Inference (SMP-DLI)

A sequential LLM inference system that adaptively allocates compute based on confidence. Demonstrates calibrated uncertainty estimates, statistical early stopping, and methods for knowing when models should abstain. Evaluated across MMLU, ARC-Challenge, HellaSwag, and CommonsenseQA.

2024

ORBIT: Cost-Effective Dataset Curation for LLM Domain Adaptation with an Astronomy Case Study

A scalable data filtering pipeline over 11T tokens, showing that data quality dominates scale. Consistent multi-point MMLU gains on smaller, cleaner corpora.


Experience

2025 –

Software Engineer, ML Infrastructure

Reliability checks and validation systems for production ML pipelines. Built automated testing frameworks reducing manual verification by 99%. Working on fault-tolerant distributed systems, observability, and CI/CD.

2025

Graduate Research Assistant — Information Retrieval & LLMs

Built a 135M-parameter Transformer reranker competitive with multi-billion-parameter baselines. Ran controlled ablations on scaling, regularization, and representation quality. Contributed to work awarded SIGIR 2025 Spotlight; first-author publications in ACL Findings and SIGIRD.

2023 – 24

Student Researcher

Trained domain-specific LLMs using selective pretraining and noise-filtered corpora. HPC-scale training workflows and parameter-efficient tuning.


Education

2025

M.C.S. Computer Science

Advanced NLP, deep learning, computational neuroscience.

2024

B.S. Computer Science

National Merit Finalist · James Scholar · ISUR Scholar.


Skills

ML & AI
Transformers, LLMs, ViTs, PyTorch, W&B, RAG, parameter-efficient tuning
Reliability
Sequential inference, uncertainty estimation, calibration, robustness analysis
Systems
Python, C++, Docker, Kubernetes, AWS, Linux, Git, CI/CD, distributed systems

Contact

Always interested in conversations about LLM reliability, calibration, and uncertainty quantification — research collaborations, new opportunities, or just a good thread.