What computer vision sees at the bench — and what it doesn’t

Putting a camera in a lab raises fair questions. What does it see? What happens to the footage? Who can watch it? Here’s how Augmentiv’s computer vision works, and the lines we’ve drawn from day one.

Why watch the work at all?

Much of what makes an experiment succeed never makes it into the written protocol: how a sample was mixed, how long a plate really sat, the order in which reagents went in. When the Reproducibility Project: Cancer Biology tried to repeat published experiments, two-thirds of its carefully reviewed protocols still had to be modified once work began. The written method and the work at the bench drift apart — and nobody can see where.

Video closes that gap. It captures what words leave out, at the moment it happens.

Seeing the work, not the worker

Our cameras are positioned to frame the bench: hands, tools, samples and instruments. They’re not pointed at faces or the wider lab, and they’re only active while a team is running Augmentiv-guided work.

From pixels to protocol steps

Lab work is a sequence of physical actions — aspirate, dispense, mix, seal, spin, incubate. Our models recognize these actions as they happen and align each one to the protocol step it belongs to. Over a run, that becomes a time-stamped account of what was done, in what order, and how long each step took.

That account is recorded as the work happens, not reconstructed afterwards — which is exactly what the FDA’s data integrity guidance means by records being “contemporaneously recorded.”

Catching deviations

Because the model knows what should come next, it notices when something doesn’t. If a plate goes into a thermal cycler without being sealed, the model flags it. The deviation, and how it was handled, goes into the run record — so it’s visible, not hidden.

A deviation on the record is a fact you can act on. A deviation hidden in the data is a mystery.

What it looks like from the bench

For the operator, very little changes. The protocol appears as clear steps on a tablet beside the bench. As each step is completed, the camera follows along and the next step comes up. If something is missed — a tip not changed, a plate not sealed — it’s flagged in the run record. When the run is processed, the record is complete.

Most of the time, the system is quiet. It’s designed to stay out of the way when the work is going well, and to speak up only when it can save a run.

People stay in the loop

Computer vision flags; people decide. Operators and scientists review every flag, and scientists on our team review records where it matters. The goal is to support skilled hands, not second-guess them.

What we don’t do

• We don’t use footage to monitor, score or rank individual operators.
• We never train on or share a lab’s quantitative experimental results.
• We don’t capture work outside Augmentiv-guided runs.

With a lab’s consent, de-identified video of movements — paired with protocols, never with results — can help improve step-by-step guidance for everyone.

Why off-the-shelf hardware

We use standard cameras and tablets, connected. There’s nothing to install on your instruments, setup is quick, and labs can get started without capital spend.

Read about security & privacy →