The question is quite descriptive. We are using LeapTrainer (https://github.com/roboleary/LeapTrainer.js) in my project. We ran this js app in the browser (Online demo Here Edit: Online demo might not work, have to download the trainer to test.), and recorded still sign language gestures (Like this, Sign language) and then copied the JSON data for each gesture into a json file of our own. When the gesture is re-did in this browser app, it correctly identifies it from its previous recordings.
However, we imported this library into our project, a desktop NodeJS app. We imported the json file which has the data of the recorded gestures. Then iterate over it to find out which currently made gesture matches to one currently made Here is the code:
const {ipcMain} = require('electron')
const Leap = require('leapjs')
var jsonfile = require('jsonfile');
let controller = new Leap.Controller();
const LeapTrainer = require('./leaptrainer.js')
var trainer = new LeapTrainer.Controller({controller: controller});
const low = require('lowdb')
const db = low('gestures.json')
db.defaults({gestures : []})
const gestures = db.get('gestures').value();
trainer.fromArray(gestures)
for(var i=0; i<gestures.length; i++){
trainer.on(gestures[i].name, function(eventName){
console.log(eventName + " detected");
})
}
This is what the Json data in the json file looks like for a gesture, say, "3":
{
"gestures" : [
{"name":"3","pose":true,"data":[[{"x":0.08514791794676319,"y":-0.059559120670074954,"z":0.23407734879587827,"stroke":1},{"x":-0.005716489907424349,"y":-0.06423104502589716,"z":0.20567955813264638,"stroke":1},{"x":-0.09658089776161186,"y":-0.06890296938171937,"z":0.17728176746941426,"stroke":1},{"x":-0.18744530561579936,"y":-0.07357489373754159,"z":0.14888397680618226,"stroke":1},{"x":0.2045947753380724,"y":0.26626802881523304,"z":-0.7659226512041217,"stroke":1}]]},
]
}
However the gestures do not get recognized accurately from the desktop app. When I ran both the desktop app we made, and the browser app from git, the browser was able to detect the gestures accurately from the same json recorded, while the desktop app failed miserably at detecting gestures that look like, (Eg., both "3" and "6" involves three fingers, so the app kept recognizing both as "3"). I raised this issue with the author of the library and he said he wouldn't be able to answer as he is not aware of my app.
To check if the Leap input here in desktop app is as accurate as the json recorded, I did a
console.log(controller.frame(0).dump());
I noticed that the data has much lower floating point precision:
Example data:
{"bases":[[[0.997866,-0.0281459,-0.0589186],[0.0318279,0.997536,0.0625182],....}
I suspect that this is the reason for the reduction of accuracy.
Can you confirm if this could be the possible case? If so, how can I increase the floating point precision?
Is there any other factor which would cause a library to behave differently in a browser and in a desktop app?
I am using Leap Motion Controller with Orion.
Software Version: 3.1.3+41910
Firmware Revision: 1.7.0