I think the latency is non existent, and is incorectly percieved, becuase the latency is only in the visualisers rendering, but non in the Leap tracking. You can check that by using "h" in the visualiser, I think, to see the frame-rate of either.
Btw. If Leap should want to test their raw data API, count me in. I think I'd have a couple ideas, but probably none that would remain additionally valuable for long. Great work.