Plugin like YouUp

[antalgeek]

Hi all

I have made a little program witch permits to replace human face by a template image in a webcam stream.
This program is named concombre and it can be found at http://www.deb-indus.org/concombre.
Sorry everything is in french for the moment.

I think that a little plugin for aMSN could be a good idea, something like ubisoft youup, but under GPL.

Before downloading source code and begin to make some code I would like to know if such an idea have been recalled here, if someone is already working on it.

Regards

[kakaroto]

ohh, c'est cool ca!
welcome to the forums btw!
well, I'm sure there will be some interest for this... I'm not sure however if we will use it for the current webcam feature.. but if you can (or maybe i could) build a gstreamer element for it, it would become very iinteresting for the audio/video conferencing feature i'm working on in amsn's SVN...
I was planning on having an 'effects' feature where you can add effects like those from effectv... this one would probably be a welcomed addition!
I'm curious however, how does it work? does it automatically detect faces and replace them? or do you have to specify where/what to change? what if a user moves? will it follow? I've read the website and it says something about an xml file describing the object to replace.. do we have to generate those or there are some shipped xml files like 'face.xml' that will find/replace any face? would someone need to create an individual XML for his face, or will a generic xml work for everyone's faces? that's the kind of info i'm curious about!
thanks for letting us know about this project!

EDIT: by the way, is your app available as a library too ?  Does it work only on images? or can it work on /dev/video or something.. what we could do in aMSN is to pass the capture image to concombre  after we capture it and before we encode it to be sent over the network.. but how would that work exactly? I don't want to write the image to a file, then launch an external application, then read from the file... if we could link against a library, it would be better.. and just give it RGB data or something... give it multiple pngs to overlay, etc.. kind of like the video shown in youup.com (quite nice)! I know your project (is it yours btw?) is still in early development, but i'm just curious

[antalgeek]

Salut kakaroto

Thanks for your interest
Concombre is free software so you can use it as you wish in aMsn if you think that it is interesting

The software is curently based on OpenCV (Open Computer Vision - http://sourceforge.net/projects/opencvlibrary/). This library permits to make many operations of professionnal industrial vision (that's the way I use it in my job) but also to produce funny effects.
The library gives you an environnment to work on images and stream, in the case of concombre I have to load a xml classifier (for human faces the file is availlable in OpenCV source code), apply a function to your current webcam image, then you get position/size of faces. Then you copy your template. You can also make your own classifier, but it is very long : 3-10 days of computing.
OpenCV is also able to produce many other effects like motion vector detection in a stream, blob tracking...

For the moment concombre uses /dev/video0 via the OpenCV webcam interface but I have a development version witch uses directly ioclts.

If the concombre feature interrests you I can contribute to the aMSN project as you want : creating a library witch take RGB data plus configuration structure , do the job and sent RGB data.
I have planned to make such a libconcombre to serve other applications.

Let me know how I can help aMSN project
 

EDIT : yes concombre is a project of my own, but many people of debian-fr.org have contributed to make it works !

[kakaroto]

ok cool! yeah, i've seen openCV and it's very interesting!!! having a library would defniitely be better because we could use it in amsn then! I think people would love such a feature! And it would be easy to integrate in aMSN once we have a library to do it all for us...
We'd be glad to have you work with us on such a feature, I think it's very interesting and I'd like to integrate it in aMSN!
Gstreamer elements for concombre/openCV would also be awesome and would help many other applications take advantage of that (as well as aMSN when we make video calls use farsight)

gtg!

[antalgeek]

ok

I begin to work on a library to serve aMSN or other applications. I'll look on gstreamer elements too.
I'll come back with some software asap.

Glad to work with you.

[kakaroto]

Any news on this?

[antalgeek]

hi

I had many work since last post on this forum
So I didn't give any new about libconcombre
Please tell me if you have a deadline to test it
I think I can commit a first release in a month

[kakaroto]

nope, no deadline, you can take all the years you need :p (but obvisouly sooner is always better!)... I'm actually not even sure how/if we could integrate it into amsn  (not sure how the UI would work), but I asked just because i was curious!

[alexandernst]

*bump*

Are there any news about that lib?

[kakaroto]

humm.. maybe it wasn't a good idea for me to say "take all the years you need", lol...
yeah, would be nice to have some news from time to time.. still curious about it, would be nice to have something we can play and experiment with!