Update FaceTrackNoIR

[Rough Knight 9]

EDIT: Disregard questions below. I found out you have to redefine all the translations in settings because they get blanked out due to the new changes in curve definitions you have made. Looks nice so far..

Gday sthalik,

I grabbed 325 dev version but I get a problem. I overwrite the files in my folder of 1.6 with all the files in the BIN directory I downloaded from the repository. FacetrackNoir fires up as version 1.7 and will start tracking using accela filter, but the widget does not show and signs of life even though the raw input seems to be working correctly. Any tips?

I am using freetrack protocol but translations don't seem to be written to the widget or outputed by facetrackNoir.

Thanks for you help with this project, I hope I am just doing something wrong here, Iam keen to help you out with some testing.

Cheers.

Edited May 3, 2012 by Rough Knight

[sturmfalkerda 8]

Hello!! I have a request.. would you be able to rebuild headtracknoirs installation msi or whatever to be installed without visual studios..??

I have a corrupted visual studio, it works fine, but sometimes programs "side by side" installation script fails or some other random garbage.. GARRRBAAAGEEEE.. another issue was a wierd token error, anyways, I would be jumping for joy if you could do this!

Thanks!

Falke

**EDIT** to clear this up a bit... I cant install headtracknoir because of a visual studio error, and also I cant launch it because of a "side by side installation script has failed" error..

**EDIT** again.. can you COMPLETELY REMOVE the installation of visual studios from this special build of facetracknoir? it might be best to also upload this one to your site if you can do this at all.. a version without prequisits and assumes you have installed them already..

also, because of some random failure in my eyesight, I did not check "allow pms from other members".. so if you do this.. you would have to post it haha

for anyone who can help.. heres the error..

The application has failed to start because its side-by-side configuration is incorrect.

Microsoft.VC80.CRT,processorArchitecture="x86",publicKeyToken="1fc8b3b9a1e18e3b",type="win32",version="8.0.50727.6195"

C:\Users\****\Desktop\FaceTrackNoIR_Release_V160\Test\FaceTrackNoIR.exe

thats whats in the viewer..

Edited May 3, 2012 by SturmFalkeRDA

[sthalik 10]

@RK,

Glad you got it to work :) I have removed old profiles so there isn't any confusion anymore.

@SturmFalkeRDA,

> The application has failed to start because its side-by-side configuration is incorrect.

You need to install the Visual C++ 2005 Redistributable, as well as run Windows Update later and install SP1 update for it (nothing major, no reboot required). It was like this since forever. I dare not remove this requirement or something bad happens, probably.

If you use my tracker, there's no need for Intel IPP which tends to break in many ways.

MSVC runtime is an integral part of windows and in this case is required to be of certain newer version. Try looking for "vcredist_x86.exe" in the repo where you got the tracker from.

@Kremator,

Thanks for your help, sent PM :)

Guys, once you get it to work please tell me what needs to be improved, especially for the 1.70 version :)

== UPDATE ==

Now when using X-axis rotation, the tracker shouldn't zoom forward anymore! Took me a few days and was a big headache but now should work rather reliably.

Edited May 4, 2012 by sthalik

[sturmfalkerda 8]

@RK,

Glad you got it to work :) I have removed old profiles so there isn't any confusion anymore.

@SturmFalkeRDA,

> The application has failed to start because its side-by-side configuration is incorrect.

You need to install the Visual C++ 2005 Redistributable, as well as run Windows Update later and install SP1 update for it (nothing major, no reboot required). It was like this since forever. I dare not remove this requirement or something bad happens, probably.

If you use my tracker, there's no need for Intel IPP which tends to break in many ways.

MSVC runtime is an integral part of windows and in this case is required to be of certain newer version. Try looking for "vcredist_x86.exe" in the repo where you got the tracker from.

@Kremator,

Thanks for your help, sent PM :)

Guys, once you get it to work please tell me what needs to be improved, especially for the 1.70 version :)

== UPDATE ==

Now when using X-axis rotation, the tracker shouldn't zoom forward anymore! Took me a few days and was a big headache but now should work rather reliably.

lol. accually I have a wierd bug where my antivirus deletes trusted installer.. remade a registry key, now I can install everything, also I already had 2005.. it let it install finelly!

YAY ANTIVIRUS FAILS!!

also, everythings flawless with my vx 6000 30 fps or more camera on arma 2, the only thing is I cant get non flaily settings..

[sthalik 10]

err, what settings?

[sturmfalkerda 8]

lol uhh.. mine..

I just want a nice and slow when I turn almost completely left it turns almost completely left or a little bit more..

right now if I move my head just a bit it moves it very far.. and its hard to aim with my head bobbing around ingame..

can you recommend anything..??

[sthalik 10]

Yes. Remake the profile. Left-click to place and/or drag a point and right-click to remove one.

[doveman 7]

Hi

Just discovered FaceTrackNoIR and it looks great but I can't get it to work with my PS3 Eye at the moment.

I've got the CL-Eye-Driver-5.0.1.0218 installed and CL-Eye-Test activates the camera and shows me my face! I've also tested it with a few webcam and mouse control programs and they all work with it.

When I start FaceTrackNoIR, it shows PS3Eye Camera in the window, but when I click Start the camera doesn't activate and the upper-left window doesn't track. If I click on Settings, it shows Engine: Terminated and clicking on Camera does nothing.

I've tried installing the previous version of NPClient.dll and adding the cleye.config and running as Administrator (I'm using an Admin account anyway though). I'm running Win7 x64 Ultimate.

EDIT: Sorry, ignore me. I just needed to install the VCredist. It's working now :bounce3:

---------- Post added at 16:32 ---------- Previous post was at 15:34 ----------

OK, now it's working I'd appreciate some help configuring it ;)

I'm trying to use the Mouse Look profile, but the pointer is lagging and drifting (continuing to move after I stop moving my head) quite a bit. I also need to increase the sensitivity so that I don't need to move my head so much (up and down, left to right's not too bad) but now it's slipping sideways when I try and move the cursor up or down, so trying to go from the bottom to the top of the screen ends up making a sort of S-pattern. It seems to work better with the PS3 Eye's lens zoomed out, but even then with Mouse X/Yaw set to 20 and Mouse Y/Pitch set to 50 it still requires too much head movement to move the cursor to the top/bottom of the screen. Also, it keeps changing the profile by itself (this seems to be happening when I use the shortcut keys Home or End), which obviously makes it start moving funny and confuses me until I notice what's happened!

I dare say I just need to do something withe the curves but I don't understand how they work.

EDIT: Just some feedback on the cleye.config patch. Without it I get 30 FPS and 6-8% CPU and with it I get 50 FPS and 9-14% CPU. This is with a Phenom II X4 955 @3.8ghz.

Edited May 15, 2012 by doveman

[doveman 7]

I've now tried to use it in ArmaII with the ArmA profile but it was moving the view up or down a lot when I was trying to turn left or right and leaning all the time as well. If I monitor the input in FaceTrackNoIR I see there's a set of numbers above the camera view and a different set marked "Raw Input" below it.

If I turn my head to the left as far as I can whilst still being able to see the screen properly, Raw rotX goes to 28 and X up the top to 40. Turning to the right it's about -23 and -40.

Looking up Raw rotY is 44 and up the top 55 and looking down -15 and -35.

Tilting my head to the left Raw rotZ is 20 and up top -70 and to the right -28 and 100

[sthalik 10]

Which version? 1.6 or 1.7? In 1.6 you'll get some jitter no matter what. With 60 FPS in the new tracker you'd get awesome results though. Even 30 in 640x480 is enough.

Also need to center the camera in front of your face. Maybe it's not centered enough?

http://img1.uploadscreenshot.com/images/orig/5/13602590066-orig.jpg

This is the new curve configurator.

[doveman 7]

I'm using 1.6. I can't find a 1.7 either on http://facetracknoir.sourceforge.net/home/default.htm or at http://sourceforge.net/projects/facetracknoir/files/

I'd be glad to give it a go if you can provide a link.

[sthalik 10]

http://arma.misaki.pl

Navigate to the "ftnoir-devel" repository and grab it from there.

I should really make a tutorial on how filtering works, how to set up filtering in case of jitter, and so on. For now I do it over skype and TeamViewer for each person :)

[doveman 7]

http://arma.misaki.pl

Navigate to the "ftnoir-devel" repository and grab it from there.

I should really make a tutorial on how filtering works, how to set up filtering in case of jitter, and so on. For now I do it over skype and TeamViewer for each person :)

Thanks. The new interface looks great. The first time I tried it it seemed to be as bad in terms of jitter and skid but I've just tried it again and it seems a lot better now.

I'm having trouble testing it though as it keeps changing profile (or individual settings, like Smoothing) whilst I'm moving my head. I originally thought it was only doing this when I pressed Home or End but it's not. 1.6 had this problem as well, so it's not something that's been introduced by 1.6. I don't know if Comodo Firewall/Defense+ or Avira Antivir would be interfering with it.

EDIT: It's gone back to jittering and drifting again. I'll PM you my mouselook.ini if you could take a look to see if I'm doing something obviously wrong. I tried Smoothing on 9, 50 and 1 and messed around with the curves a bit but nothing seems to help with these problems.

Edited May 17, 2012 by doveman

[sthalik 10]

Does it still do it if you unfocus the program? Perhaps it's some stray keypress?

Don't set smoothing at all. I should remove it completely as it's just an arithmetic average of last frames.

Don't press "settings" while tracking. This might be the cause. While still there, zoom so face takes 1/2 to 1/3 of the screen and increase timer of redetection in tracker settings (but less than 1k) until the CPU usage isn't too much. Also, play with contrast. Lower is better.

Make a screenshot of your Accela filter settings.

Why do you bother with mouselook at all? Use freetrack in Arma.

I made some new videos of it. But no Arma, as I barely play Arma anymore. Now I bought a force-sensing stick and can't play TOH anymore :(

http://www.youtube.com/watch?v=-FT4seraqz8

Will update post with new video I made of Maverick action, but it has very low framerate and due to it poor flying.

Edit: I'll actually make a different video :-) That one's framerate is horrible, about 19 in-game.

[Rough Knight 9]

Hi sthalik,

It has taken me a while, but I am finally getting my settings tuned. I must say I am impressed with your work [Running v329].

The FaceDetect tracker seems pretty CPU intensive so I am still using Face API, but good to see another means of tracking in progress.

I am still working out my deadzones [with smoothing set to 1] and with my Curves set how I have them, response to head movements seems much better. I have not experienced any jitter. The only thing now in my opinion if the CPU overhead could be reduced my any margin then we have a winner in every respect.

Thanks for the improvements. Is there any specific feedback you are chasing, I will be happy to provide some if you can point out what you are specifically looking for.

Cheers to the FacetrackNoir team,

[sthalik 10]

Please try build 335 and see if CPU usage is acceptable now. It's 11% on my i7.

If you need more free CPU, try increasing the redetection timer to 667.

What I'm looking for is presently:

- misdetection, i.e. the tracker going bonkers all of the sudden

- precision, in this case less oscillation when tracking. Shouldn't be more than 1 deg. when struggling to keep head straight. FaceAPI 3.2.6 is 2 deg. in the avg case

- glasses support is problematic, but will work on it some time

[doveman 7]

Does it still do it if you unfocus the program? Perhaps it's some stray keypress?

If I press Home or End after clicking Start, these buttons switch the Filter so I've changed from the defaults to Numpad 0 and 1 for now. Home and End would be no good in Arma anyway (it's always tricking finding keys that aren't already used in Arma). Actually, even pressing Num 1 (Centre) is causing the Filter to change now. It also just changed from Mouse Look.ini to a FSX profile without my pressing anything. Perhaps that was because Z was mapped to the mouse wheel though, so I've disabled that now.

Don't set smoothing at all. I should remove it completely as it's just an arithmetic average of last frames.

Don't press "settings" while tracking. This might be the cause. While still there, zoom so face takes 1/2 to 1/3 of the screen and increase timer of redetection in tracker settings (but less than 1k) until the CPU usage isn't too much. Also, play with contrast. Lower is better.

OK. I'll set smoothing to 1. Not sure what you mean by the second bit. I don't see any zoom option, nor a way to "increase timer of redetection". The PS3 Eye only has two fixed zoom settings and with it on max, it looks like this image at the distance from the camera to my sitting position. http://img29.imageshack.us/img29/1154/ft17.png (114 kB)

Make a screenshot of your Accela filter settings.

I take it this is recommended over EWMA now then? There's nothing there at the moment anyway and I'm not sure what sort of line I should plot.

Why do you bother with mouselook at all? Use freetrack in Arma.

I've just been testing with mouselook in Windows. partly because I was hoping to be able to control the mouse cursor in Windows with FTNoIR and also to avoid having to keep alt-tabbing between Arma and FTNoIR whilst I tweak it, but apparently it's not really ready for Windows mouse control yet, so I'll stick to testing in Arma from now on. Having said that, with the Accela Filter (even with blank settings) rather than EWMA it's night and day and all jitter/erratic movement is gone. I think the reason I was having problems even with 1.7 is it was changing to the EWMA filter and changing back to Accela doesn't seem to take until I Stop and Restart. It's still a bit hard to get the cursor exactly where I want it though, as it tends to not move in response to small head movements and then move too far suddenly (not skidding like it was before though) but perhaps a bit of tweaking the settings will get me there.

---------- Post added at 15:32 ---------- Previous post was at 15:25 ----------

Has anyone got an Arma profile for 1.7 they can share to get me started, as the ones I found for 1.6 just give blank curves in 1.7?

[sthalik 10]

If I press Home or End after clicking Start, these buttons switch the Filter so I've changed from the defaults to Numpad 0 and 1 for now. Home and End would be no good in Arma anyway (it's always tricking finding keys that aren't already used in Arma). Actually, even pressing Num 1 (Centre) is causing the Filter to change now. It also just changed from Mouse Look.ini to a FSX profile without my pressing anything. Perhaps that was because Z was mapped to the mouse wheel though, so I've disabled that now.

OK. I'll set smoothing to 1. Not sure what you mean by the second bit. I don't see any zoom option, nor a way to "increase timer of redetection". The PS3 Eye only has two fixed zoom settings and with it on max, it looks like this image at the distance from the camera to my sitting position. http://img29.imageshack.us/img29/1154/ft17.png (114 kB)

I take it this is recommended over EWMA now then? There's nothing there at the moment anyway and I'm not sure what sort of line I should plot.

I've just been testing with mouselook in Windows. partly because I was hoping to be able to control the mouse cursor in Windows with FTNoIR and also to avoid having to keep alt-tabbing between Arma and FTNoIR whilst I tweak it, but apparently it's not really ready for Windows mouse control yet, so I'll stick to testing in Arma from now on. Having said that, with the Accela Filter (even with blank settings) rather than EWMA it's night and day and all jitter/erratic movement is gone. I think the reason I was having problems even with 1.7 is it was changing to the EWMA filter and changing back to Accela doesn't seem to take until I Stop and Restart. It's still a bit hard to get the cursor exactly where I want it though, as it tends to not move in response to small head movements and then move too far suddenly (not skidding like it was before though) but perhaps a bit of tweaking the settings will get me there.

---------- Post added at 15:32 ---------- Previous post was at 15:25 ----------

Has anyone got an Arma profile for 1.7 they can share to get me started, as the ones I found for 1.6 just give blank curves in 1.7?

redet in tracker src settings

accela falcon bms profile has curves

sorry posting from phone

[doveman 7]

Thanks. It seems only FaceDetect has a Haar Detection Timeout setting. Nothing like that for FaceAPI which I'm using at the moment as I think FaceDetect is a bit new and probably doesn't work as well yet (could be wrong on that though).

I'll use the Falcon BMS profile as a starting point then.

[sthalik 10]

I'd be very grateful for testing FaceDetect. Yes, it's new but simpler and doesn't have any show-stopping bugs unlike FaceAPI 3.2.6 which goes bonkers on high-degree angles with jumping all around. It's also open-source. :-)

FaceDetect shouldn't crash (never does for me at least), and has 1 degree accuracy unlike FaceAPI which has 2.

I'll be pushing some changes once I write it. Trying to get *both* accuracy and self-correction working at the same time. Just 2 numbers to change.

Can we get into IM contact for some fast tweaks and feedback?

As for Accela (I really should write it down once):

You get a screen position P_s. You get a tracker position P_t. So let x be the value | P_t - P_s |. Go into the graph. See that if you have a high X, change will take a low amount of time while a very small change will just slide there sluggishly. That's supposing you have a logarithm-like curve. You can set it up to the range it oscillates in, i.e. if you get 2 deg. precision, make up to 2 deg. low and then skyrocket :) Like I did for about 1-1.5 deg.

Edit: The present version uses 10% cpu on my i7. That's 40% of one core. If that's not acceptable nothing will be.

Edited May 17, 2012 by sthalik

[doveman 7]

OK, sounds like I was mistaken about FaceDetect not being as good as FaceAPI ;) I'll certainly be happy to test it.

I'll try and connect with you on IM sometime soon. Thanks for trying to explain Accela but I'm afraid most of that went over my head! It might make things a bit easier if the graph axis were labelled Tracker Position and Screen Position so it's clear which is which, but that's a minor point and obviously I need to get my head around how the two correlate before I'll be able to tweak it properly. I guess what I find hardest to grasp is how Accela will affect the Curves.

[Rough Knight 9]

Hi sthalik,

I am using accella filter with FaceAPI [Revision 336]. After re-defining and getting my curves right, I am pleased to announce I have had the best performance and operation of FaceTrackNoir since it's inception. I wound the filtering down to 1, and the response is amazing I must say and very precise.

I have not had time to use FaceDetect yet as I spent some time tweaking the curves with the above setup. Only drama is in starting and stopping the tracking, sometimes it takes a little while to recapture my face. I think this was partially to do with having a background light playing up with the camera though. I need to try this in the daylight hours.

Thanks for your efforts mate, this is getting really good now.

[sthalik 10]

FaceAPI sets up its classifiers for several seconds, all while using much CPU. This issue doesn't exist in FaceDetect.

> Thanks for trying to explain Accela but I'm afraid most of that went over my head!

Let's try explaining one more time. You have a position the tracker detects, as well as the position that's currently transmitted to the game/sim. The latter "catches up". The X is the absolute difference between these 2. More you set for Y for given X, faster it catches up.

Just try not to set any position to zero except for zero itself, it always *has to* go faster up than the previous position. This is in my understanding the tangent of a function. The monotonicity is always upward, too.

Thank you for your kind words.

Here are my curves used with FaceDetect:

http://img1.uploadscreenshot.com/images/orig/5/13806411049-orig.jpg

Also, if your "multiplier" is lower than mine (40 deg. means for me 180 deg. X, similar for Y upward in a flight sim), you can set more aggressive Accela curves, too.

Edited May 18, 2012 by sthalik

[Rough Knight 9]

FaceAPI sets up its classifiers for several seconds, all while using much CPU. This issue doesn't exist in FaceDetect.

I haven't had a lot of luck with faceDetect. I can get it to track, but interpolation is all over the place, and the widget shows it moving out of range. Did you find you need to change your curves much between facedetect and FaceAPI?

I have not had much time to try this but will give it another shot tonight.

[sthalik 10]

Half of the code was recently rewritten. Please give it a try now. Should be accurate to 1-2 degrees. Drifts with time, but drifts very slowly. There's autocorrection, too, and works up to 50 deg.

But as long as you find the use for the filter, i'm happy anyway that you're using it. It would be better if FaceDetect flourished, but guess can't have it all :)

I had to rewrite the code because of bad detection causing weird shifts in view. This no longer happens. Whenever a shift outside the good range, new detection is discarded. Unless there are no points left on a face-part.

Also, POSIT is now tracking only 4 points, which gives it correct depth information at all times.

If any programmer wants to contribute, now is the time, because the messy parts were rewritten and it's pretty readable by now. The offending part was `face_detect_cycle()'.

My new curves after a rewrite look like this:

http://img1.uploadscreenshot.com/images/orig/5/14408234052-orig.jpg