How the measurement works

The optical principle

Blood absorbs light. Not uniformly, and not entirely, but enough.

With every heartbeat, the volume of blood in the capillary bed just under the skin rises and then falls. Because that blood absorbs part of the light falling on the skin, the amount of light reflected back changes in step with it. The skin gets very slightly darker as the pulse wave arrives, and very slightly lighter as it passes.

The change is far too small for a person to see. It is not too small for a camera sensor to record, provided you know where to look and you average over enough pixels and enough frames.

This is photoplethysmography — “writing the volume with light”. Done with a contact sensor, it is the technique behind the clip on a hospital patient’s finger. Done at a distance, with an ordinary camera and ambient light, it is remote photoplethysmography, or rPPG. That is what Saphere does.

From frames to a waveform

Four things happen, in order.

A face-landmark model locates the face. Not to identify anyone — to find the geometry. The model runs in the browser, on every frame, and reports where the face is and how it is oriented.

Stable skin regions are selected. Not the whole image, and not even the whole face. The measurement samples regions where the skin is thin, well perfused and relatively still — the forehead above all. Eyes, mouth and hair are excluded: they move, and movement is noise.

Those regions are reduced to numbers, frame by frame. Each frame contributes a small set of averaged values rather than an image. This is why a measurement transmits far less than a video would, and why the capture is bounded in size regardless of how close the person sits.

The accumulated series becomes a pulse waveform. Over roughly thirty seconds, the per-frame values trace a periodic curve. Its fundamental frequency is the heart rate. Its beat-to-beat spacing gives variability. A slower modulation riding on top of it gives the respiratory rate. Morphological features of the waveform, combined with what you told us about the person, feed the remaining indicators.

Why the constraints are what they are

Every requirement in the overview follows from the paragraph above.

Light must fall on the face, not behind it. The signal is reflected light. With a bright window behind the subject, the camera exposes for the window; the face falls into shadow and the reflected component the measurement needs is compressed into a handful of dark values. There is nothing to recover.

Light must not change during the capture. A cloud passing, a screen changing colour, someone turning a lamp on — each writes a variation into the signal that is larger than the pulse, and indistinguishable from it by amplitude alone.

The person must stay still. When the face moves, the sampled regions move across different parts of the skin, different angles to the light source, and different parts of the sensor. The resulting variation is enormous compared with the cardiac one.

The face must be uncovered and in frame. Skin the camera cannot see contributes nothing. A face partly out of frame gives fewer usable regions and a noisier average.

The pipeline, end to end

A live measurement through the widget goes through five stages.

1 — The widget obtains a token. At the start of every measurement it asks your application for a short-lived access token. Your server mints it against your API key. The widget never sees the key.

2 — The camera opens and the person is guided into frame. The face-landmark model runs locally. Nothing has been transmitted yet.

3 — Capture runs for about thirty seconds. Frames are reduced in the browser and streamed to the measurement server over a websocket, acknowledged as they arrive. A degraded network slows this down but does not lose data; the widget reports it.

4 — The server computes. When the last frame is acknowledged the widget sends an end-of-measurement marker and waits. Processing happens server-side.

5 — The result comes back over the same connection, is validated, and is handed to your onEvent handler as measure:result. It is also persisted, so you can read the same measurement back through the API afterwards.

For a recorded file the first four stages collapse into a single HTTP call: you POST the video or the images, the server does the frame reduction itself, and the response is the completed measurement.

Where the data goes

Frames are reduced in the browser before transmission — what leaves the device is a numeric signal derived from the sampled regions, not a video recording of the user. The completed measurement, its signal and any userData you supplied are stored against your client account and are retrievable through the API until you delete them.