Audio/Video conversation

[kakaroto]

Hi,
For the interested, you can now do an audio call in aMSN... The audio+video call is still being worked on but it's not complete though...
Here's a small summary of what you need for aMSN to have audio calls working :
1 - get all the dependencies installed.. here's how : http://amsn-project.net/wiki/Farsight
2 - take the latest SVN version of amsn, then ./configure && make (make sure it detected farsight installed during the configure)
3 - run amsn and from a chat window : actions->start audio call


For those interested in the audio+video feature, here's the original post I had in here :

Hi,
here's a copy of a mail I sent to the amsn-devel mailing list :

Hi all,
I don't know if many of you are following the amsn-commits mailing list, but for those who don't, this email is for you!
I want to let you guys know that I'm working on the audio/video conferencing and it's going pretty good so far!
Basically what this is for those who aren't aware, is the 'video call' feature of WLM, which is audio AND video AND bidirectional. In other
words, one click and you start sending and receiving both audio and video.. so no more 2 invites to send/receive webcam + use voice clips...

Here's the current status :
first, it's far from being done, but I can get audio + video decoded. The problem is... to play them...
So here's how it works,
once you do the a/v conf you see a dump.av file in your amsn dir.. then you can do ...

Code:

./demux.tcl dump.av dump

which will create 3 files :
dump.siren, dump.raw and dump.wmv3.
then do

Code:

./wmv3_dec dump.wmv3 out.rgb

which will decode it in rgb and open display to show you each image.
and for the sound.. simply do

Code:

cat dump.raw | ./stream_audio.tcl

Problems are :
1 - i don't send yet
2 - no tk extension to decode the wmv3 yet (it's a little C program)
3 - can't record/play audio unless using snack and snack doesn't like real time streaming (it opens/closes the audio device on every chunk
(every 50ms) so the sound gets chopped so it looks like garbage, not real sound...
4 - i can send sound (not written yet) but there's no wmv3 encoder in open source anywhere..
So that's the current status..

About sending video, implementing a WMV3 encoder is too complicated and can't/won't be done. So our choices are :
1 - don't send video...
2 - encode/send the video with libmimic..
(libmimic is the codec currently used for the 'webcam' feature)

there's an FCC (Four Character Code) being sent with every frame, so I can send ML20 in the FCC instead of WMV3 and hopefully, WLM reads the
FCC and chooses the decoder accordingly. if it does, then we're good, if the wmv3 encoder is hardcoded, then we won't send video


That's about it.. you can test it if you want, it's in the 'video_conf' branch of SVN... you can use the status log to see if you are
receiving video/audio frames... and that's it!
SVN revision 9556 is necessary...

I'd like to know if it works for anyone besides me...

KaKaRoTo

So this is to announce that it's being worked on... and it requires MSNP12, so you don't need to go into MSNP15 to make it work (but it works just as good on MSNP15 so if you already use that, no need to go back to MSNP12 either).
You would need to ask someone to send you an invite, or if you want to 'send' the invite yourself, just do this in the console (ctrl-shift-C) :

Code:

::MSNCAM::StartVideoConferenceQueue $email

You are free to test, no bugs to report please, but just have fun with it.. hopefully, it will evolve into a much better/fuller implementation in the next few days...

[kakaroto]

by the way, the window that appears showing you the webcam is imagemagick's 'display' command, it will show you one frame at a time, you must press the spacebar to see the next frame, etc... In the latest SVN, I just gave it an additional parameter (-delay 1) so that it will show you the next frame after 1sec delay, but it's very slow...
also, you might notice that you can hear the first few seconds of audio in aMSN when you start the A/V conversation, then it stops, it's normal.. to hear it fully, just use the command I gave you above in your xterm.

[iron_maiden89b]

Good!!

[kakaroto]

Hi,
a bit of news...
1 - I found a way to record live audio using snack, that should help us.. but for playing live audio in snack, that's still a problem... we might have to modify snack.. since snack is an abandonned project, we *might* have to take it to ourselves, fork it, enhance it to support live audio and fix some of the bugs people are complaining about in snack... It would be shipped with amsn on windows and mac, and for linux users, it should be compiled automatically when you compiled amsn.
2 - The WMV3 decoding seems to work great BUT I noticed that it might crash (segfault) on some frames, I already fixed a bug in libavcodec for decoding WMV3, and now, I'll just have to fix yet another one.. hopefully it will become more stable soon.. I'll be in contact with the ffmpeg developers in order to make sure no issues will be coming from there..
I don't have a lot of time available lately, but hopefully, I should be able to finish this pretty soon.. hopefully before FOSDEM!
I'll keep you posted!

[why.arent.guests.allowed]

I was wondering why I couldn't get an audiovisual session in aMSN... so that is why.
I had set the webcam for audio... but wondered why I couldn't get or send audio... heheh... I see now that sound clips must be used.

Let me tell you, aMSN developers, that I'm really looking forward for this feature (you see... there's this girl...  :roll:  :oops: ) and I really appreciate your work on this! kudos!
I'll be following this thread closely.
Having bidirectional audiovisual session + yahoo contacts working on aMSN... will be pure bliss!  

I'm curious, will bi-directional audio + video work with yahoo contacts as well, or would that require reverse engineering yahoo's protocol as well  :x And I perfectly understand if the developers don't even think about it.

microsoft is such a nuissance... Communication protocols and data formats should be open sourced or well documented... Hey that's a good topic for FOSDEM
There should be a lobby to force the opening of protocols and data formats which when closed limit the choice of freedom.

BTW (a bit off topic here and maybe silly but), have you ever proposed aMSN improvements to Google's Summer Of Code?

One last thing, you said "I don't have a lot of time available lately, but hopefully, I should be able to finish this pretty soon.. hopefully before FOSDEM!"
You mean item number 2, or the whole bidirectional audiovisual session? That WMV3 audio and video encoder sure seems a real pain


Thank you for everything!

I'll see if I can test it and post some results

[why.arent.guests.allowed]

hmm... I was wondering... is this by any chance usefull for WMV3 encoder purposes: http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/2007-June/031699.html

[Kalinda]

I'm curious, will bi-directional audio + video work with yahoo contacts as well, or would that require reverse engineering yahoo's protocol as well  :x And I perfectly understand if the developers don't even think about it.

No, it won't. I mean, as I understand it, the audio chat/webcam doesn't even work with Yahoo contacts in the official client. All you can do is chat with them through text, that's the extent of the interoperability.

[kakaroto]

yep, as Kalinda said, you can only chat with yahoo contacts, you can't do webcam/audio, you can't even do a file transfer as far as I know, and that's a limitation of the protocol and interop decisions, same applies for the official client.

[kakaroto]

Hi again,
Just want to post up some news here...
We've worked on this A/V thing a bit at FOSDEM and we got it to work a bit better... the audio is still not being processed, but the video is...
so now, you can go grab the latest SVN version of the video_conf branch, then recompile it (along with ffmpeg...) and see for yourself.. in theory, it should work fine for receiving... for sending, I'm sending an ML20 encoded stream, but I don't know yet if it works or not (my WLM blue screens whenever I try something with it other than typing text...).
So, can you guys test it and see how it goes ? you're now allowed to complain about stuff (video related, and NOT audio related please!). I would really like to know if sending video works or not. Most of the stuff should now work, but it's not yet completed... for example, don't try the 'pause', or don't expect it to work if you don't have a webcam (in theory it should just work without sending the webcam), etc...
You can also review the A/V session in the webcam log viewer...

[Kalinda]

Alright, I tried it (just the wbecam bit) with one of my guinea pigs and he said that after I had accepted the invite, his window changed to receive webcam but he never got anything.

On my end, it said he rejected the webcam session and then I saw my own webcam window appear, but it never sent to him. He's using the latest WLM (2008 (Build 8.5.1302.1018))

Now, I'm assuming I don't need to use the console commands anymore (for webcam, anyway), since it looks like you took out the dump.av thing. Am I correct?

Keep up the good work, though I figured you guys would've worked on it at FOSDEM.

[kakaroto]

Hello Kalinda.
I finally got my PC to stop BSOD-ing on the video call. I was able to test and I see that it doesn't work with mimic codec. There's something wrong in there because it also doesn't work with WMV3 (I send a WMV3 webcam log as if it was my own webcam).. but it's really weird because it shows the 'no webcam' icon, but if I make it go into full screen mode, it then shows the webcam correctly, if I go back to non-fullscreen mode, it shows the 'no webcam' icon again...
anyways, so that's that... by the way, you didn't say if you were able to receive webcam from your friend correctly or not. I suppose you did.. if you didn't, make sure you compiled ffmpeg correctly. When doing ./configure, it should tell you if ffmpeg was found or not (if not, it will tell you how to compile it).

[Kalinda]

Hello

Yeah, I didn't receive from my friend. Sorry, I forgot to mention that. I did compile ffmpeg using the command provided and I got no errors, so I assume it worked out alright. Maybe you and I should test it.

I'll have to see about this fullscreen mode thing. It's unfortunate because I don't have two webcams, but I do have a few computers.

[kakaroto]

install webcammax  : virtual webcam for windows..
maybe you didn't receive your friend's webcam because he didn't have a webcam or was not sending it ?

[Kalinda]

Alright, I tried it with myself and it worked better. I was able to receive the WLM webcam, but I still couldn't send the aMSN one to WLM and when I full screened it in WLM, I just got a black screen.

I tried it again with my friend and was able to successfully receive his webcam but, like before, he couldn't receive mine. I dunno what I was doing wrong before, but part of it works, so hooray!

Thanks for the webcammax thing, too, it's really cool and helpful.

[kakaroto]

Cool!
Yes, webcammax is awesome! Too bad it's not free/open source, but it's good..
anyways, about the full screen trick, it only works if you send a WMV3 video. Currently, aMSN sends a mimic video (so it should work between two aMSN instances, but not with WLM). There is no WMV3 encoder available anywhere (open source), so we can't encode in WMV3 right now, even if we wanted to. What I did to get it to send in WMV3 and have it work in full screen was to send a WMV3 webcam log as my own. It's an old trick existing since we first implemented webcam support :p just open the console and type :

Code:

set ::test_webcam_send_log /path/to/webcam.log