Areas of interest
Across medicine, neuroscience, media and security, my question is whether the way an AI system is measured matches the way it is used. Each area below gives a short abstract of the current project and the goals for what comes next.
Medical imaging
How scanner settings change AI tumor measurements
Lung cancer screening follows nodules by growth, so volume matters, yet AI segmentation is usually validated with overlap scores. Using public screening CT scans in which the same acquisition was reconstructed with two filters, I compared both measures for two models. Overlap did not change (Dice difference −0.009), but measured volume moved by more than 25% in 24% of participants. A model guided by a radiologist’s lesion box was far more repeatable. The protocol was fixed before training and the test set was scored once.
Goals
- Publish the study with code and result files that reproduce every number.
- Extend to dose, slice thickness and growth across repeat scans.
- Understand why small tumors are affected most.
Computational neuroscience
Steering a game with a fly-brain wiring diagram
How much behavior can a measured wiring diagram carry on its own? A frozen spiking model of the fruit-fly brain controls a player in hide and seek, with only the inputs, outputs and game policies tuned around it. Shuffling the wiring cuts exploration from 8.2 rooms to 2.1, and silencing 24 neurons of one steering cell type drops the seeker from 13 wins in 20 games to 1. A hand-written policy still outperforms the brain, which marks what the diagram is missing.
Goals
- Identify the missing signal behind the steering errors that make the model hit walls.
- Use shuffling and silencing as a general test of which circuits a behavior needs.
- Close the gap with the scripted baseline, and report it either way.
Neural signal analysis
Measuring cognitive effort from EEG
Cognitive effort is important for assessing neurological disease and engagement but hard to measure. We analyzed the multifractal structure of EEG from 55 subjects during mental rotation tasks and rest. Generalized Hurst exponents were higher at rest than during tasks and lower for correct than incorrect answers, and machine learning models separated task from rest well. These exponents are candidate markers of effort.
Goals
- See the paper through to publication.
- Test whether the markers hold across tasks, sessions and new subjects.
- Check what the classifiers actually use.
Deepfake detection
Face-swap deepfake detection that generalizes
A detector combining a frame-level image model with a temporal video model scores an AUC of 0.99 on its own benchmark, but that describes one generator: on a different face-swap method it falls to 0.66. Training with synthetic blend-boundary fakes lifts the unseen-method score to 0.77 and keeps 0.96 on the benchmark, and performance holds down to 240p call-quality video. It detects face replacement only, and would miss a fully synthetic clip.
Goals
- Raise the worst-case score on unseen methods.
- Cover fully synthetic video, which has no blend boundary.
- Stay accurate under real call compression, and say so when unsure.
Cybersecurity
Prompt injection and tool use in AI agents
AI assistants that can read images and files and call tools can be steered by instructions hidden in the content they process. We tested twelve multimodal prompt injection attacks against three leading models under five conditions, from a plain chat to a setting where the model has tools and an attached file. Two independent language-model judges labeled each run by what the model said and by what its tool calls actually did, and we adjudicated the cases where they disagreed. The question is how much the ability to act turns a bad answer into a bad action.
Goals
- Publish the study with the data and scripts that reproduce every figure.
- Test more models and defenses against the same attacks.
- Keep measuring what the model did, not only what it said.