1: You could absolutely use gyro/accel for detecting taps, but for proper pressure sensitivity (i.e., changing pressure in the middle of a stroke), there's not much you can do except have a pressure sensor. It's theoretically possible with a sufficiently accurate pose estimate and a springy pen tip, but not feasible at the level of accuracy I got.

2: I paid about $20 AUD for the pressure sensor, but they can be had for quite a bit cheaper (~$5 USD) in the US (https://www.arrow.com/en/products/hsfpar003a/alps-electric). Only problem is they're quite specialized, so not many places sell them. The custom PCB was another $10AUD, and the Arduino was about $20. There's a full parts list at https://github.com/Jcparkyn/dpoint/blob/main/setup-guide.md.

another question: how do you estimate _depth_ - i.e. the dimension _away_ from the camera?

I'm guessing it's the size of the ArUco markers, combined with accelerometer for smoother tracking - but that seems quite imprecise?

That's done by the PnP solver, but yes it's essentially looking at the size of the aruco markers on the screen. This is implicit in the equation the PnP solver is trying to solve: "find the pose (position and orientation) that results in the minimum re-projection error for all the marker corners". Re-projection error is the difference between the observed 2D location of each marker corner, and the theoretical 2D location for a given pose (using standard camera projection equations).

You're right that this is the least precise dimension for PnP, but it turns out to be good enough if the corner positions are decently accurate. Using sub-pixel refinement for the corner locations helps a lot (this is built into OpenCV — it looks at the brightness values for multiple pixels around the corner to get an estimate more accurate than one pixel). Having corner positions further apart in 3D space helps as well, which is part of the reason I used two "rings" of markers.

In my case I was getting somewhere in the ballpark of 1-2mm of depth error at ~3cm away from the camera, so ~0.5% relative error. Smoothing from the accelerometer stops this from being noticeable most of the time, but it will sometimes crop up in bad conditions (e.g. if one of the markers is being missed on some frames due to lighting problems).