Recently I was playing with face detection demos done using ported from OpenCV Marilena Lib by Ohtsuka Masakazu and optimized one by Mario Klingemann. I have an idea of using it with my ASSURF project but as far as ASSURF uses lots of CPU resources I cant merge it with Face detection. So I decided to revisit the implementation with more simple and fast solution. Taking into account the speed of Linked Lists and also using TDSI memory optimizer I was able to produce a lot more faster version. I also decided not to hardcode haar cascades XML files into the lib but allow to load it. Currently library supports simple Stump based cascades and Tree based. I wouldn’t recommend using Tree based cascades cause they are significantly slower. This is because we have to use recursion while analyzing image with Tree based cascades.
Update: Added Canny Edge Detector to skip low edged areas of image that result in some nice performance boost. (this will be added to repository soon)
You will find some usage info at wiki page
Download Library SWC
Try demo application (recognizing faces, eyes and mouth) (make sure that u are not using debug player)
Works great but I am only getting 2-4 FPS at the most.
are u using debug player?
Great work, the performance is very impressive!
Nice work. What kind of speed up did using the alchemy opcodes give you?
I’ve found that getting the image data into the form (ByteArray) that the opcodes requires, ie using BitmapData.getPixels(), is a lot slower than BitmapData.getVector(), by a factor of around 6! Subsequent data access is about twice as fast though.
@Jake i’m not using Bitmapdata.getPixels at all. you dont need it cause there are several workarounds to send pixels data to Alchemy. but there is no Alchemy here. just Memory class provided by Joa Ebert and TDSI to speed up ByteArray read/write operations.
Joa Ebert’s TDSI uses the opcodes that were included in Flash10 specifically for Alchemy, so yes, although Alchemy itself is not used, the opcodes for speeding up the ByteArray read/write are.
So how are you extracting the pixels from the webcam feed into the required ByteArray?
I thought of doing it in pixelbender.
@Jake I’m simply getting Vector of colors from bitmapData and then write them to Byterray via Memory class.
I’m not using the debug player and i get max 5 fps with Safari and 1 fps with Firefox :S
Great works, anyway…
G.
@Giulio well i have tested in all browsers using release flash player and min FPS i had was 45 FPS
Hi Eugene,
i was on my laptop, and, as everybody know, laptop performance are limited.
I’m now on a intel 3.00 GHz 4GB Ram.
I get 7 fps…not bad…but still i think that if an average pc gets 3 fps, the app it’s still not usable.
Testing on super computer never helps.
Nice app though…
G.
@Giulio i still think u are using Debug version of flash player. I’m not on super computer, tested at MacBook and standard PC with release flash player i’m getting results around 40-45 FPS.
Eugene,
you are riht! Mea culpa…
I wonder how did it happen…and why…
hey guys,
check your player version here:
http://kb2.adobe.com/cps/155/tn_15507.html
G.
Hi Eugene, you are an amazing programmer/wizard.
I have learned more about computer vision programming with Flash reading your blog than anywhere else.
I have been trying out your HaarDetector.swc as part of a Flex CV ’sandbox’ using my MacBook Pro (running Flash 10). The problem I am encountering is that the detector will find my face for the first frame, then subsequently for each ‘handleEnterFrame’ throws an exception (here is the trace):
success, rects.length:1
RangeError: Error #1506: The specified range is invalid.
at flash.utils::ByteArray/clear()
at ru.inspirit.image.feature::HaarCascadesDetector/set image()[E:\_exp\HaarCascadesDetector\src\ru\inspirit\image\feature\HaarCascadesDetector.as:451]
at com.iwakari::HaarCascades/handleEnterFrame()[/Users/stevexm/Documents/Flex Builder 3/CVSandbox/src/com/iwakari/HaarCascades.as:39]
Will you be releasing the source to your swc at any time, or do you have any suggestions or advice as to why this exception is coming up (apologies for not posting all my code and then asking for help, in any case I would be happy to send it to you)?
Again, much appreciation for everything you have posted, and thanks in advance.
Steve Melnikoff (Melbourne, Australia)
@Steve make sure u correctly using detector. I’m then sure that smth wrong with your initialization or EnterFrame code. See wiki page for the example
Hi Eugene,
it’s smooth and stable, I love it.
Compiling I get the error ‘abc bytecode decoding failed.’ and I cannot figure out why.
I used your wiki example.
Cheers,
Henk
Hi Eugene, worked out the RangeError bug. As you suggested, the problem had to do with the bitmapData object from the webCam that I was setting to haarCascadesDetector.image when initializing. Face detection is now ‘go’.
Thanks, Steve
Hi Eugene, can you pls put your code on wonderfl (http://wonderfl.net/libraries) Thanks,
@mochioochie my code could be easily downloaded form google repo. it is already available as SWC Lib. so if u want it to be available at Wonderfl I think u can ask them to add it. (i’m not using it)
Thanks Eugene, I think that to put ASSURF on wonderfl, you have to have an agreement with them/give them permission to use the code? Like how Eric allowed them to use FLARmanager. I could be wrong. I have already sent them an email to ask them to ass the ASSURF library, they might contact you soon. Thanks for your cool code and for sharing.
Sorry for so many posts, by the way PLEASE PLEASE PLEASE work with Eric to marry FLARmanager and ASSURF (if it’s possible) — they are meant to be together! The kid would be beautiful
Thanks,
http://www.mukimuki.fr/flashblog/
а Вы не думали о возможности использования данного фрэймворка?
хотя бы в части определения области лица по цвету?
Hi Eugene, I’d like very much your work.
We are in the process of developing an application using a tool similar to yours.
I’d like to get in touch with you, with some questions. Could you send me an email, so we can discuss this further.
Thanks in advance.
Rodrigo.
Hi,
I have a question, hopefully not too dumb, because noone had this problem so far. (hm!)
2) When I try to construct the HaarCascade detector with
“private var detectorHaar:HaarCascadesDetector = new HaarCascadesDetector(‘haarcascade_frontalface_default.xml’, false);
it throws an error of implicit coercion of string to XML, so I looked at the constructor and found out, it really has to be a XML direct. So I tried
“private var detectorHaar:HaarCascadesDetector = new HaarCascadesDetector(new XML(‘haarcascade_frontalface_default.xml’), false); ”
but that also throws an error.
“TypeError: Error #1009: Cannot access a property or method of a null object reference.
at ru.inspirit.image.feature::HaarCascadesDetector/setupCascadeData()[E:\_exp\HaarCascadesDetector\src\ru\inspirit\image\feature\HaarCascadesDetector.as:606]”
And there I ended up squeezed by helplessness.
@brmčátko u have to load xml file first and only after that u will haev XML Object that u can pass to HaarCascadesDetector
a big ThankYou.
I wouldn’t get it alone.
I’m not the real programmer with pure logic thinkin’ like You all have.
well, what to do
Hi again.
I’m still experiencing difficulties.
Although following Your example code on wiki, my vector of rectangles stays empty.
Please, may You have any suggestions what could be wrong?
The quality of photo is relatively good, the xml file should be standard.
No errors are thrown.
I trace the vector’s length, which stays 0.
Hey Eugene,
Impressive work! Looks like you figured out some good workarounds to speed things up. I worked on a similar project a while back (http://flashfacedetection.com) and still need to go back and revisit it. I think I’m going to see if I can incorporate your optimizations into mine and check out the results. I bet it will be wayyyyyyyyyyy faster than Marilena or Klingemann’s stuff. If you’re interested in talking about this, shoot me an email. I published a paper about it so I can send it to you if you want.
-Theo
Hi – this code looks perfect for a project I’m doing, but i’m getting stuck on implementing it.
wher I try to call detector.detect I get the following –
at crazyface_fla::MainTimeline/updateStage()
RangeError: Error #1125: The index 76800 is out of range 76800.
at ru.inspirit.image.feature::HaarCascadesDetector/detect()[E:\_exp\HaarCascadesDetector\src\ru\inspirit\image\feature\HaarCascadesDetector.as:100]
at crazyface_fla::MainTimeline/updateStage()
I’m calling it in my update stage event. Help? I’m a bit new to AS3 and esepecially anything to do with camera input.
Hi.
I tried the demo, but no image is loaded and I can’t see any button for loading some.
So I see only some sliders.
What am I doing wrong?
@quincey You should see the image from you web camera. if u dont have one then it wont work for u.
@Cassie see the detailed example code in my code repository. I described all needed steps to init and run detector.
Thanks Eugene, I think I’m on the track to getting it sorted now – can I ask what cascade file you used to detect the eyes? Sorry for being such a bother.
This is really cool
I’m trying to recognise a custom object I have, but still haven’t figured out an easy way to generate an Haars cascade for it.
Sucks
Hi!
I’ve been trying to create a hand tracking app using your swc, but alas failed
… When i’m trying to compile with the example in your google code page, flash throws me a
error.
What am i missing ? …
I guess it’s something stupid but can’t quite put my finger on it … please help
All and all love your apps and web page. Haven’t found anything as grouped and opened as i did here.
Thanks again,
Gabriel
And the error :
1046: Type was not found or was not a compile-time constant: HaarCascadesDetector.
I wrote it in a “” and it removed it from the post.
@Gabriel you should use Flex SDK to compile your project with provided SWC lib.
I have been trying out your HaarDetector.swc as part of a Flex CV ’sandbox’ using my MacBook Pro (running Flash 10). The problem I am encountering is that the detector will find my face for the first frame, then subsequently for each ‘handleEnterFrame’ throws an exception (here is the trace):success, rects.length:1RangeError: Error #1506: The specified range is invalid. at flash.utils::ByteArray/clear() at ru.inspirit.image.feature::HaarCascadesDetector/set image()[E:\_exp\HaarCascadesDetector\src\ru\inspirit\image\feature\HaarCascadesDetector.as:451] at com.iwakari::HaarCascades/handleEnterFrame()[/Users/stevexm/Documents/Flex Builder 3/CVSandbox/src/com/iwakari/HaarCascades.as:39]
+1
Hi Eugene,
Good work. BTW you should use a binary format instead of loading the full XML file which is quite large > 1MB and slow. By using a binary representation it can come down to something much more manageable. See this example of how to do this.
http://coderepos.org/share/browser/lang/actionscript/ashaardetect/trunk?rev=17167
Eugene,
Awesome work !
I was wondering if you have the fully body recognition working ? I found haarcascade for the full body, but did not get it working, I guess I’m missing few libraries / source code…
Would you be able to give some insights on this ?
Hi
and for tracking hand motions?
tks
Hi Eugene,
Great work! I was wondering how I can extend it such that when given a pattern, it attempts to find the pattern in the video. i.e. If I give it a red ball, would the same code work to detect the red ball?
Thanks,
Teren
Hey Eugene,
I’m not sure if I implement the detector right. I get resulting rectangles but somehow it takes some seconds to find them. So my FPS is way under 1FPS.
Even if I use a single Bitmap, it takes some seconds to find a face.
Could you provide a working example (the one from this post eventually) on google code?
hey eugene,
which cascade xmls are you using for the eyes, seperate for each eye? and are those from opencv?
cheers, eva