Further Out — Event Cameras, Time of Flight, Hidden Markers and the Phone

The ideas research turned up beyond the siblings' hand-offs: what each is, what has been published or sold, what it could and could not do for a Pico gun with a PixArt camera, and why none of them is the next experiment

Figure 1 — A Prophesee event-camera evaluation kit on a small tripod: an industrial camera body with a C-mount lens. Its sensor does not take frames. Each pixel reports, with a timestamp, the moment its brigh… — Figure 1 — A Prophesee event-camera evaluation kit on a small tripod: an industrial camera body with a C-mount lens. Its sensor does not take frames. Each pixel reports, with a timestamp, the moment its brightness changes by more than a set fraction. Photo: Auledas, Prophesee Event Camera Evaluation Kit, CC BY 4.0, via Wikimedia Commons.

7.1 Event Cameras

What they are. An event camera, or dynamic vision sensor, has no frames. Each pixel compares its brightness with the level at its last event, and when the change passes a threshold it emits an event: its address, the sign of the change and a timestamp. A static scene produces almost nothing; a moving edge or a blinking light produces a dense stream with microsecond timing.

Figure 2 — The same scene seen by a conventional camera, as a strip of separate frames, and by an event camera, as a continuous cloud of events in space and time. Image: Timo Stoffregen, Event camera comparis… — Figure 2 — The same scene seen by a conventional camera, as a strip of separate frames, and by an event camera, as a continuous cloud of events in space and time. Image: Timo Stoffregen, Event camera comparison, CC BY-SA 4.0, via Wikimedia Commons.

The published prototype. In 2013 Andrea Censi, Jonas Strubel, Christian Brandli, Tobi Delbruck and Davide Scaramuzza tracked a robot’s pose from LEDs blinking at over 1 kHz, each at its own frequency, with an event camera. Their abstract makes the two points that matter to a light gun: the sensor’s time resolution distinguishes the frequencies, so each marker is identified by its frequency and no “data association” (deciding which blob is which) is needed; and “the DVS performance is not affected by fast motion, unlike the CMOS camera, which suffers from motion blur.” That is the arcade frequency jumper (Volume 2, §2.4) taken to its limit: every emitter carries its own code.

The product. Prophesee’s GenX320 is the small, low-power sensor of the kind, on sale with evaluation kits. Its product page lists 320 × 320 pixels of 6.3 µm, pixel latency under 150 µs at 1,000 lux, dynamic range over 140 dB, a nominal contrast threshold of 25 %, an on-chip anti-flicker filter, and under “Active markers”, “high-speed LED frequency detection in the 10s of kHz” for “constellation tracking”. The Raspberry Pi 5 starter kit connects over MIPI CSI-2, with 76° or 104° lenses (0.24° or 0.33° a pixel), under 50 mW, and is sold on request for a quotation; no price is published.

What it would buy this gun. Very little that the gun lacks. The latency it would save is the wait for the next camera frame, on average 2.4 ms of a gun’s share of about 5.6 ms, in a chain dominated by the game’s frame and the television (Aiming Maths and Calibration, Volume 7, §7.5). Motion blur is already small with a 300 µs exposure (IR Point Tracking, Volume 1, §1.7). Frequency-coded emitters would identify points robustly, but the PAJ7025 with a modulated bar (Volume 2) can approach that. Against it: a MIPI interface that neither the RP2040 nor the ESP32-S3 can read, so a Raspberry Pi 5-class computer in or tethered to the gun; a quotation-only kit; and an entire tracking pipeline to write. Label: prototype exists (robotics), product exists (sensor); no event-camera light gun was found. Verdict: not this build.

7.2 Time of Flight to the Screen

Figure 3 — Pololu's carrier board for ST's VL53L1X time-of-flight sensor: the black sensor package with its two windows, one for the 940 nm laser emitter and one for the receiver, with a regulator and level s… — Figure 3 — Pololu's carrier board for ST's VL53L1X time-of-flight sensor: the black sensor package with its two windows, one for the 940 nm laser emitter and one for the receiver, with a regulator and level shifters. Photo: Pololu, product 3415, used with credit.

What it is. A single-chip laser rangefinder. ST’s VL53L1X has a “940 nm invisible laser emitter (Class 1)”, a receiver field of view “programmable from 15 to 27 degrees”, and ranges “up to 50 Hz”. Pololu sells it on a carrier for US$22.95 (19 October 2026).

What a gun would want from it. Distance. The one quantity the aiming maths needs and the original OpenFIRE firmware does not compute is the player’s distance from the screen, which sets parallax (Aiming Maths and Calibration, Volume 5). A rangefinder in the muzzle would report it directly.

Why it does not help. The homography already contains the distance: that volume shows how it is recovered from the four points once the emitters’ real spacing is known, which a kit bar fixes. And the datasheet’s own limits bite. Its Table 5 gives maximum distances in the dark and under strong ambient light: 1.36 and 1.35 m in short mode, 2.90 and 0.76 m in medium, 3.60 and 0.73 m in long. A living room with a window is not dark, and a player is usually more than 0.76 m away. The minimum timing budget for medium and long modes is 33 ms, about 30 readings a second against the camera’s 209. Its 27° cone is about a metre across at 2 m, so it measures to the television, the wall and the cabinet at once. One small point in its favour: its 940 nm light is outside the PixArt camera’s filter (IR Point Tracking, Volume 2), so it would not disturb the kit camera, though it would be inside the 940 nm stopgap camera’s band. Label: product exists (sensor). Verdict: not this build; use the homography.

7.3 The Phone as the Gun

A phone has a camera, a processor, a radio and a battery, and turning one into a border-tracking gun has been done: AndroidLightGun (MIT, version 1.0.0, August 2026) binarises the camera image, finds the border, and sends the aim to a PC as a Bluetooth device, with the trigger detected through the microphone input (Border Tracking, Volume 2, §2.8). Platform questions belong to Mobile and Handhelds. For this build it offers nothing: the gun is a purpose-built Pico device with a solenoid, and a phone’s rolling-shutter camera and wireless stack add latency and variability the kit does not need. Label: prototype exists. Verdict: not this build.

7.4 Markers Hidden in the Picture

A border is a visible marker the game must draw. The question is whether markers could be drawn so the player does not see them and a camera does.

The patent. Interphase Corporation’s US 8,982,050, “Motion compensation in an interactive display system” (Yoram Solomon and Branislav Kisacanin, granted 17 March 2015), describes a wireless pointing device whose camera captures “human-imperceptible positioning targets”, each “presented as patterned modulation of the intensity … in a display frame of the visual payload, followed by the opposite modulation in a successive frame”. The eye averages the two frames and sees the picture; the camera differences them and sees the target.

The research prototype. Luiz Sampaio and colleagues at the Nara Institute of Science and Technology (2015) showed markers that alternate between two colours faster than the eye’s flicker fusion, detected by ordinary rolling-shutter phone cameras, used to identify displays and estimate the camera’s pose against them; a companion paper reports that choosing complementary colours keeps the markers imperceptible, that tracking is robust on static content, and that large movements lose it. A Dartmouth group’s HiLight (MobiSys 2015) hid data, rather than markers, in small changes of pixel translucency on arbitrary content, a cousin of the same idea.

For a light gun. This needs a camera that sees visible light, so not the PixArt; a display fast enough that the complementary pair fuses; host software that modulates every frame on every platform, with every video path passing it untouched; and tolerance of a moving gun, which Sampaio’s group found to be the weak point. It is border tracking’s cost list with an invisible border. Label: prototype exists; patent. Verdict: a cheap curiosity as a host-side experiment with a webcam, alongside the border trial (BT-2), and no more.

7.5 A Laser and a Camera at the Screen

The presentation-pointer approach puts the camera at the screen and a laser in the hand: the camera watches the screen, finds the laser’s dot and reports its position. Dan Olsen and Travis Nielsen’s “Laser pointer interaction” (CHI 2001) is a published example, and its abstract names the problems: hand jitter, detection error, slow sampling and latency. For a television rather than a projection screen it is worse: a glossy panel reflects most of a laser specularly away from a camera beside it, so the dot is faint where it lands; a visible laser aimed at a screen in a room with people is an eye-safety matter; and the gun becomes a laser rather than a camera, with the camera to be mounted and calibrated at the set. Label: prototype exists (projection screens). Verdict: not this build.

7.6 Handed to Positional and Inertial

The category’s plan listed ultra-wideband ranging, ultrasonic and magnetic positioning, optical flow, and inertial sensors among the experimental ideas. All of them measure position or orientation rather than imaging the screen, and all are treated in Positional and Inertial, including their fusion with an optical fix. They are not repeated here.

7.7 Why Not These

Table 1 — 7.7 Why Not These

Idea	Label	What it would add	Why not now
Event camera with coded LEDs	Prototype (robotics); product (sensor)	~2 ms less latency; robust point identity	MIPI sensor, Raspberry Pi-class host, quotation-only kit, whole new pipeline
Time of flight	Product (sensor)	Distance	The homography already has it; ambient light cuts range to 0.73–0.76 m
Phone as the gun	Prototype	A gun that is a phone	Not the gun being built
Hidden on-screen markers	Prototype; patent	A gun with no bar and no visible border	Visible-light camera, host software, fast display, weak under motion
Laser and camera at the screen	Prototype	No camera in the gun	Glossy screens, eye safety, latency

None of these attacks the build’s two measured weaknesses, the R3’s light grasp and room light, as the synchronised bar does; none removes hardware as the retroreflector does. They are recorded so that the next person who has the idea can see what is known.

7.8 What Could Not Be Verified

The GenX320 kit’s price, which is by quotation, and any event-camera light gun.
The VL53L1X’s behaviour against a television screen.
Whether Interphase’s patent became a product.
The accuracy of hidden-marker pose estimation for a hand-held moving camera at living-room distance.

Sources

A. Censi, J. Strubel, C. Brandli, T. Delbruck, D. Scaramuzza, “Low-latency localization by Active LED Markers tracking using a Dynamic Vision Sensor”, IROS 2013, pp. 891–898, DOI 10.1109/IROS.2013.6696456; abstract via the Caltech and EPFL repositories, read 2026-10-19.
Prophesee, GENX320 sensor page, https://www.prophesee.ai/event-based-sensor-genx320/, and Starter Kit GenX320 for Raspberry Pi 5, https://www.prophesee.ai/event-based-starter-kit-genx320-raspberry-pi-5/, read 2026-10-19.
STMicroelectronics, VL53L1X datasheet DS12385 Rev 8, August 2024, features, Table 5 and §3.5.2, https://www.st.com/resource/en/datasheet/vl53l1x.pdf, read 2026-10-19.
Pololu, VL53L1X Time-of-Flight Distance Sensor Carrier with Voltage Regulator, 400cm Max, item 3415, US$22.95, https://www.pololu.com/product/3415, read 2026-10-19.
Y. Solomon and B. Kisacanin (Interphase Corporation), US Patent 8,982,050 B2, “Motion compensation in an interactive display system”, 17 March 2015, abstract (PDF via patentimages.storage.googleapis.com), read 2026-10-19.
L. Sampaio, Y. Yamada, G. Yamamoto, T. Taketomi, C. Sandor, H. Kato, “Detection of Imperceptible On-Screen Markers with Unsynchronized Cameras”, 45th meeting of the VRSJ Mixed Reality research group, NAIST, January 2015 (bibliographic record at scholars.cityu.edu.hk, read 2026-10-19); the companion NAIST paper on complementary colours was seen only as a search summary.
T. Li, C. An, X. Xiao, A. T. Campbell, X. Zhou, “Real-Time Screen-Camera Communication Behind Any Scene” (HiLight), MobiSys 2015.
D. R. Olsen Jr. and S. T. Nielsen, “Laser pointer interaction”, CHI 2001, pp. 17–22, DOI 10.1145/365024.365030.
Border Tracking, Volume 2 for AndroidLightGun; Aiming Maths and Calibration, Volumes 5 and 7.