What I work on

Many systems are naturally graphs — parts or measurements, and the relations between them. A graph neural network learns from that structure directly rather than from a flat table of readings, and what it learns ends up spread across the graph, which is what makes it hard to say what any one prediction rests on.

A prediction is only worth acting on if the person acting can see why it was made. My thesis is about explaining these models: what an explanation of a graph model can be asked to show, and what it takes for one to be worth believing rather than simply believed. The figure below is a small graph network running in your browser, not a model from my work.

Alongside the thesis, I implement explainable-AI methods inside analysis tooling, so that people who are not machine-learning specialists can see why a model decided what it did. It is the practical half of the same question. An explanation has two audiences — the evaluation, which wants a true account of the model, and the person, who wants a reason they can act on — and a method can satisfy one while being useless to the other.

Where someone looks is a remarkably strong signal about what they are doing, and it is available before they act. A scanpath carries far less information than a video frame, but it sits much closer to intent — which is what makes classifying activity from gaze a good problem.

I designed and ran the study in the DFKI smart factory, with volunteers performing ordinary industrial tasks while a HoloLens 2 tracked their eyes. The published toolchain I started from turned out to sample too slowly to be useful, so I moved the study onto a recorder built on ARETT.

Tree ensembles separated the activities cleanly. The more instructive result was the one that failed: the recurrent model I added to the inherited methods lost, and kept losing — the dataset is small, which is not what such a model wants. A manuscript on the work is in preparation. The paths in the figure below are drawn to show the kind of difference the study’s nineteen features measure, not recorded from it.

A two-person project and a full perception-to-manipulation loop on real hardware: find an object placed anywhere on the bench, recover its pose, and have a seven-axis arm pick it up and set it down elsewhere without touching the obstacles it already knows about. My part included the object detector: I recorded and annotated its dataset and trained it.

The instructive part of a project like this is never the detector. It is everything between a bounding box and a gripper that closes in the right place.

FUME is a lower-limb exoskeleton for paraplegic users, built in the lab. The inertia of a machine like that you can write down. The Coriolis and gravity terms, with a person strapped into it, you cannot — so the controller I designed for the swing phase learns that part online, with a generalised fuzzy hyperbolic model cheap enough to run in real time. It ran on the robot itself rather than in simulation, and outperformed a tuned PID.

This is the work that made me a researcher: a control problem where the plant is a person, the failure modes are not abstract, and the specification is written in terms of what a human body can tolerate. It became my first publication. The figure below is simulated rather than measured: both controllers are given the machine’s inertia, and neither is told what the wearer weighs.

The classical hard case in control: two links balanced upright with a motor at only the second joint — more degrees of freedom than actuators, and an unstable equilibrium. I built and compared three controllers on it — linear-quadratic, classical PID and fuzzy — which is the most efficient way I know to learn what each family of controller actually buys you. The figure below runs a close relative live in your browser: a double inverted pendulum on a cart, the full nonlinear model held upright by an LQR controller, which you can take hold of and push.