Speaking and/or Creating

Just got back from the last conference I had this year. Getting back from holidays is always sad. You know. But this time its a different kind of sadness. After last years Flash on The Beach I was invited to several conferences in different countries and even different parts of the world. I guess the subject of my session was quite new for all that conferences and that’s why it got so much attention. No doubt I was very happy to travel around to meet lots of old friends and find new ones, but after couple of events I started to realize that something is wrong.

The problem is that my sessions don’t evolve – and me neither. Of course I was adding some new examples/projects and was trying to put some new ideas to make it look fresh but for me it was old. I look back and see that I simply was trying to find something more interesting to try and if it works add it to my talk. It is totally different from how I was working before: I always experimented in different fields trying to combine results at the end or just have it as separate, unrelated, project. Now I feel like I’m stuck with single subject. People associate my name with Augmented Reality in Flash and that’s it. Only few folks out there remember what I was doing a couple of years ago. :) Of course: It was always about messing with images and math but not how it looks at the moment. I start missing those days :)

No, I’m not going to drop ASFEAT/IN2AR development. It is not possible since it is a part of me already. I still have lots of ideas related to AR and image detection/reconstruction that I want to try out.
What I want/need is to stop doing everything for this particular project. As I mentioned during my last talk in Brighton it became boring to me. Someone tweeted that’s it is strange to say that u bored of your session subject, but hey did u ever try to listen to the same song everyday for couple of hours? How does it feel?
I did couple of different things like CaptureDevice Native Extension (that I plan to update with new features for mobiles soon) and the GPUImage library that allows to apply different Instagram-like effects to images and even video streams utilizing Stage3D rendering power. And it was fun! At that time I really felt that I have something fresh/new/interesting and it doesn’t matter if I (or someone else) can/will use it! :) Sounds selfish right? But its my spare time and I want to spend it the way I want without thinking about the next conference and what I would show there.

So, for my next session I will just gather my experiments and put it under the subject “evolving and creating”. I am looking forward to what it will be about and I hope you too! :)

Shifting social activity

Looks like my blog activity is somewhere around ZERO. That’s very disappointing I know…
In past year so many things changed in social networks, and for me it is impossible to update each resource I have with most recent info. I guess that’s what happened to my Blog, my social priority was shifted in direction of Twitter and Google+ so if u would like to get most recent information of my work, experiments, thoughts, ideas, etc. please add/follow these accounts.

I’m not going to remove the Blog but most likely it wont be updated in near future since it is more reasonable to provide complex post in Google+ and posting fast statuses/notes in Twitter. I will keep the Blog to make some of my old shared experiments available for audience, also it contains some useful discussions in comments.

Latest experiments & tryouts

Finally I got my hands into Molehill Flash Player. I must say that current version is pretty buggy but u can estimate the amount of future possibilities already.
I was working on Dynamic Light system for my ASFEAT project that will allow u to relight your 3D world using Augmented Reality (real world) lights. The process is straightforward: collects light clues to estimates initial light map and then update it continually with each marker-image detected/tracked.
After getting first world light map I decided to try Molehill to get use of it. So I build very simple 3D engine using new Flash Player API to be able to render all my data on GPU and also to apply custom shaders to it.

Check out this screencast (first 10 secs we do different calibrations):

Rendering is very smooth and fast cause all CPU resources goes only to ASFEAT routines while 3D rendered using GPU.

After getting so promising results I decided to port my old SuperShape3D experiment to new API and also connect ASFEAT to it with relighting option.
Sound enabled so be ready for some action! ;-)

Keep heads up! ;-)

Summing up

Looks like I completely forgot about my blog. But if u follow me on twitter u know what I was doing all this time. Writing blog posts require lots of time while posting to twitter is really fast ;-) Let me tell u whats going on around.
FITC SFO
I was in SFO doing a session at FITC, besides the fact that I was pretty nervous cause of my first serious session it went pretty well I guess. Lots of people were interested in my subject and I got lots of comments after it. But I know that next time I should prepare better.
I was very happy to meet lots of online friends in person. the first person I met was Mario and the first thing he said was “c’mon, lets have a dinner with us!” :-) skipping the fact that I’m not Asian food fan it was fun to see all this guys together talking about random things. Thank you for that! I enjoyed the whole event and what I like most of all it is the chance to meet all that people in person, to speak to them, to discuss and to share ideas. Special thanks to Bartek and Rob for making me a company when I didn’t know what to do or where to go. Greetings to Alan Shaw and Joa for hanging with me during lunch breaks.

FITC SFO
FITC SFO
FITC SFO
FITC SFO
FITC SFO



NO MORE SURF
After SFO I continued ASSURF development but focused on optimizing main algorithms. The heart of ASSURF was keypoints descriptor adopted from well known SURF algorithm. I always knew it wasn’t good idea to use it because it is patented with all consequences. My attempt was to create fast and strong descriptor that can be used in any project without caring of legal issues. I wont go in details here but I’ve ended up with random tests based descriptor that stores point information in bits. As the result it is very fast in computation and blazingly fast to compare using simple XOR (^) operator.




Next phase was to optimize after detection step aka tracking. My first attempts was to use different algos (Optical Flow, SSD) to recover previous frame points and to skip new frame computations. But that works very unstable. I chose different approach – template based tracking. It uses precomputed reference image matrix to find object bounds in provided ROI. This approach requires training process but this can be done offline and then loaded into the tracker. Tracking step is very fast it takes about 1-3ms to track single object and no point detection/description/matching needed.

And finally we need to estimate 3D pose of the detected/tracked object. This can be done using Homography matrix that we already estimated during tracking step (see green border around image). I shared the class that allows u to decompose Homography into Rotation matrix and Translation vector in 3D space (see Homography decomposition code) It uses SVD routines for matrix decomposition that can be found in my small Linear Algebra library (I will add Wiki page soon with usage summary)




Besides the ability to detect and track single reference library would be able to handle multiple reference objects. Of course it will be slower in detection/description step but still blazingly fast when all needed references found.





You can find more demos at my Vimeo Album page
As u see lots of changes were done and no more SURF descriptor used so that was the reason to change project name to different one, again. Next project reincarnation will be under ASFEAT name. Also I’m going to change license to GNU GENERAL PUBLIC LICENSE v.2 that means u wont be able to use it for commercial projects without contacting me and applying for different rights. But I’ll also provide open source version of library that will allow u to create open source or non commercial projects. For those who are interested in commercial developments new name will be created:

IN2AR



I’m currently finishing ASFEAT development and preparing public SDK. Expect it to be released in January 2011. Thats all I can share with u at the moment, it is very short summary of what I’m in and what we all can expect from it. ;-) If anyone have any questions/suggestions feel free to contact me.

FITC San Francisco 2010

FITC San Francisco 2010

Well this is going to be my first post on the subject without code ;-)
FITC events are really well known in the world and presented in many countries. Unfortunately not in Russia yet… :-) For me this is the first large event to speak at. (first was Flash2000 in Antwerp – greetz to Mr. Henry) But I’ve to mention that I’m still in Visa getting process and who knows how it turns next Monday during my interview in US embassy (solved) ;-)
In any case I’m more then sure that this particular event will be great! Just look at the speaker list. My personal flash hero is Yugo Nakamura who is inspiring thousands of people to try and learn Flash more then anyone/anything else! (still my personal thought)! I’ll be happy to see him in person.
Also there are several people like Mario Klingemann, Joa Ebert, Ralph Hauwert, Seb Lee-Delisle and others w/whom I speak often but never seen in person and this gonna be fun to talk to them in real life!
Fingers crossed – wish I can see you all soon!

Oh, and I almost forget about my session – “Recognize This! Seeing with Flash“! As you may guess this talk will be covering ASSURF project. I’m going to tell you a story of my journey from first working examples to smth completely new that is not presented yet!

ASLSD – Line Segment Detector

ASLSD example

I found this detector very useful for one of my projects and decided to share the code.
You can find original implementation here. It is not like simple edge detectors but designed specially for line detection. As the result you got an array of {x, y} coordinates that represent all available line segments and this opens very large usage opportunities.

Try demo application
Download Library SWC
WIKI Page

Viola-Jones object detection [updated]

Face detection example

Recently I was playing with face detection demos done using ported from OpenCV Marilena Lib by Ohtsuka Masakazu and optimized one by Mario Klingemann. I have an idea of using it with my ASSURF project but as far as ASSURF uses lots of CPU resources I cant merge it with Face detection. So I decided to revisit the implementation with more simple and fast solution. Taking into account the speed of Linked Lists and also using TDSI memory optimizer I was able to produce a lot more faster version. I also decided not to hardcode haar cascades XML files into the lib but allow to load it. Currently library supports simple Stump based cascades and Tree based. I wouldn’t recommend using Tree based cascades cause they are significantly slower. This is because we have to use recursion while analyzing image with Tree based cascades.

Update: Added Canny Edge Detector to skip low edged areas of image that result in some nice performance boost. (this will be added to repository soon)

You will find some usage info at wiki page
Download Library SWC
Try demo application (recognizing faces, eyes and mouth) (make sure that u are not using debug player)

ASFFT update

ASFFT project got some updates:

  • 1 dimensional FFT added
  • FFT Spectrum analyzer class added
  • updated methods that lets you work with FFT data via BitmapData object

Special thanx to Mario Klingemann for the help with the correction of setDataBitmapData and getDataBitmapData methods.

See Wiki page for more info on project

Demo of 1D FFT and FFTSpectrumAnalyzer usage
Pitch Shifter demo that uses FFT to transform sound data

Fast Fourier Transform for Flash

FFT simple usage example

There is smth new I would like to share with Flash community: ASFFT Project. Well project is too loud name as far as it is only small library that allows users to decompose images using Fast Fourier Transform in to Real and Imaginary parts. But if you dig more in to the title you will find lots of amazing possibilities it brings to you. The problem is that it is pretty slow and can’t be used on processing video frames for example but it is fast enough to do different image manipulations without freezing Flash Player.

The lib will contain 2 classes “FFT2D” and “FFT”. First one is to work with two dimensional data mostly presented as images. The second one is for streaming (or not) one dimensional data. Usually used in sound signal manipulations/transformation.
At this very moment only FFT2D class available. I will add one dimensional FFT class in near future.

Simple demo example of FFT usage
Project page at Google Code

FlashSURF -> ASSURF

The shortest blog post of mine.
I’ve received a mail from the Group Product Manager of Adobe Flash Player:
“I’m writing to you because the current name of the library misuses the Flash brand. As you know, Flash® is a well-known trademark of Adobe registered in many countries, and in order for us to protect it we need to be diligent about the proper use of the mark… etc.”

As you may understand Adobe asked me to change the Lib name. I’ve no chance to debate on the subject as far as they are right here and they do what they should do. So after not long conversation new name has been chosen ASSURF. (thanx to Mario Klingemann and Jon Howard for the idea)

So if you have any bookmarks update it:
ASSURF Wiki page
ASSURF SWC Lib development project
ASSURF usage examples project