Independent research on how AI systems are evaluated.
I study whether the standard ways of measuring AI systems reflect how those systems behave in use. Current projects cover medical imaging, neuroscience, deepfake detection and cybersecurity.
A routine scanner setting can move AI tumor measurements past a clinical threshold.
Using public lung screening CT scans, AI tumor segmentation was compared on the same acquisition reconstructed with two different filters. Overlap scores did not change. Measured tumor volume did, by more than 25% in 24% of patients with one model and 15% with another, and guiding the model with a radiologist box cut that to 3%.
Steering a game with a fly-brain wiring diagram
A frozen spiking model of the fruit-fly brain plays hide and seek. Silencing 24 neurons in its compass pathway leaves the seeker almost unable to win.
Measuring cognitive effort from EEG
Multifractal analysis of EEG from 55 subjects during mental rotation, tested as a marker of how hard the brain is working.
Face-swap deepfake detection that generalizes
AUC is 0.99 on the detector’s own benchmark and 0.66 on a method it has not seen. Training on blend-boundary fakes lifts the worst case to 0.77.
The Rankings Board: college football ratings on a conspiracy wall
An alternative to the AP poll for the Power Four and Notre Dame. Opponent-adjusted ratings, a strength of schedule that re-grades itself every week, and what-if schedules you can play out on a pin board of red string.
QCube: a gadget for building quantum circuits
Flick, turn and bop a cube to compose a circuit. Try it with your mouse; a real plug-in version would send the same events.
I am James Irwin, a computer science and mathematics graduate of Texas Tech.
I have led imaging and machine learning research in a lung cancer research lab, and have worked on cybersecurity and EEG projects. Golden Toad Research is where I do my own work.
Read more