I'm currently working on a serious game for a school project. In short, this game is supposed to help people with differtent types of hand arthritis (and related diseases). The general idea is that the game will help the patiënt with moving their hands and fingers to keep them as healthy as possible.
The fact is, that every patiënt is different, so each patiënt has different "exercises" they have to perform. We have thought about this problem and we think the best way to handle this, is to record exercises for each patiënt.
We came up with the following idea. We created a web portal for doctors and physiotherapists. In here, they are able to "create" patiënt files. Each patiënt file has different information about that particular patiënt. Next to that, the doctor is able te record different exercises using the leap motion. I have used the jestPlay project to achieve this. So far, the doctor/therapist is able to record an exercise for a particular patiënt and store this exercise as a .json file on the server.
In addition to this, another part of the time is building a game in the Unity3D engine (C#). In this game, the player uses spells to clear the game. These spells are supposed to be the particular exercises the doctor recorded in the web application. When the patiënt logs in on the game, the .json file of the exercises will be send to the Unity3D engine. So far, so good.
Once the .json file is in Unity, I use a json parser to put it into an object(array). The data should be translated to a movement in leap motion (a replay so to speak). Once the particular patiënt does the movement/exercise the doctor initially recorded, then the spell is should complete.
Now, I have the following issues and I don't really know where to start to tackle this. I really hope there is someone that has dealed with something like this before and is able/willing to help me out with it.
- How do I translate the data from the webapplication (see http://www.yorickkeijzer.nl/cotf/application/exercise.json for an example) to a leap motion replay / exercise / movement in Unity?
- How do I make Unity recognize the diffent movements/exercises (so compare the .json data from the web application with the real time leap motion data in unity)?
I hope, I have explained it clear enough. If you have any questions, please don't hesitate to ask them.
Thanks a lot!
I would not recommend sending every piece of information in the frame to the web player; if you know what information you want -- finger tips, palm location, gestures -- send that information to Unity in a custom message JSON block.
If you are sending in JSON keep in mind you'll be sending strings of JSON; to pick apart that data you would use the JSONObject code found in the wiki and referenced in that article.
There are Gists of the basic files used by me in my implementation.
Senior Web Developer