Research

Research

I work in three main areas of machine learning research: Hessian analysis, entropy, and attention heads. Some earlier research topics have come and gone as the work shifted; the old pages redirect here. The notes below describe each at a high level only — what the area is, and why I’m interested in it.

Hessian analysis

The Hessian of a model’s loss surface carries information about the local geometry of training. I’m interested in what it can tell us about generalization — the structure of the curvature, and how it relates to the model’s behavior on out-of-distribution data.

Entropy

How confident a model is in its predictions, and how that confidence is distributed across the input. Useful as a diagnostic — a way to ask whether a model is uncertain in principled ways or just guessing.

Attention heads

The individual heads inside a transformer each learn to attend to different things. I’m interested in what those patterns reveal about how the model organizes information, and how the patterns change across training.

The page is intentionally short. Detailed write-ups belong elsewhere — this is a topic-level summary.