aMSN Forums
July 15, 2018, 07:59:13 pm *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: New forum for aMSN !!
 
   Home   Help Search Login Register  
Pages: 1 [2] 3 4 ... 46
  Print  
Author Topic: Audio/Video conversation  (Read 565897 times)
Kalinda
Power user
*
Offline Offline

Posts: 97


View Profile
« Reply #15 on: March 01, 2008, 05:20:06 pm »

Ahh, so are you just going to leave it alone for now and do the audio part instead?

Actually, above someone posted a link about a patch to ffmpeg for a wmv3 encoder. Would that work? I wonder if it's still in their SVN or if it was abandoned...
Logged
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9428


View Profile WWW
« Reply #16 on: March 01, 2008, 10:58:39 pm »

Yes, for now, I'll leave it alone.. well, It does send using mimic, so I'll leave it sending with mimic so that at least it will work between two aMSN clients (since aMSN is not as stupid as WLM and it does check for the FCC and chooses the right decoder depending on how it got encoded, while WLM always uses the WMV3 decoder). There are still some tweaks that need to be fixed though.. for instance, if you don't have a webcam, it will cancel the A/V, and that's because we use the same code for capturing/encoding/sending as for the webcam feature, and it's normal that for the webcam feature, if you are sending and you don't have a webcam, then there's no reason not to cancel the whole thing. So I'll just have to make sure it doesn't do that and have it keep the A/V open but just disable sending webcam. I also need to tweak it to have the long-awaited 'webcam image in the chat window' feature (for both webcam and A/V). This will be useful because of the two way webcam. + we'll need to add a change volume thing to the UI. Also allow for sending webcam+audio or webcam only or audio only or none... etc... all those tweaks to make the feature 'finalized'.
But first, I'll have to work on the audio stuff.. I might have a solution for getting the audio to play correctly so I'll have to try that.. then there's capturing which is also possible as far as I know, but we'll have to wait and see how it works out.

About the wmv3 encoder, it was never committed to SVN, so the code is nowhere to be found apart from the attached patch to the mailing list.. but of course, the attached files are not saved for the web archive of the mailing list... I wrote to the author of the encoder as soon as I read about it but he hasn't answered me yet... I'll wait and maybe try to contact some other ffmpeg developer to see if they still have the patch somewhere on their system or saved in their mail client...

EDIT Ahhhh, found it!! I just asked and they made me realize on how stupid I am.. the attachment IS saved even in the web archive and the link to the patch was in the mail, I just didn't see it!!!  
So here it is : http://lists.mplayerhq.hu/pipermail/ffmpeg-devel/attachments/20070625/bbadb401/attachment.bin Cheesy
Logged

KaKaRoTo
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9428


View Profile WWW
« Reply #17 on: March 02, 2008, 08:01:39 am »

ok.. now I got the encoder in there... if you look at the SVN log... it says :
Quote
 encoder works BUT only for a main profile with sequence layer 0x40010005. the msn profile is a simple profile with sequence layer 0x0ff18001.. This makes it use sequence layer 0x0ff18001 but it doesn't work yet...

In other words.. it works if I encode it, then decode it manually using a specific 'sequence layer' (kind of 'encoding settings' for bitrate, compression used, etc...). If I use the same sequence layer as what WLM uses, it gives us garbage image... which means something is wrong (maybe wrong compression algorithm used or something...).
Other news... now sound should work.. so if someone could test this, update the SVN and try to see if the sound works fine for you... In theory it shouldn't.. here's the svn commit log message. :
Quote
Audio reception should now work BUT with stupid snack lib, audio is play a bit faster than it should so previous buffer finishes playing too early, so we get a small lag between two buffers.. the bigger the buffering, the bigger the gap, the smaller the buffering, the more annoying the gap is... I tried putting a median value... until we wrote our own extension...

In other words, you shouldn't really have any problems.. sound should work fine.. BUT if you test by trying to say a long sentence with no interruptions, you'll notice that there is a small silence every 2 or 3 words, but it's "ok" for now...
To really fix audio we'll have to write our own playing/recording extension and drop snack because it is just too crappy... Sad
I'll wait and see what you think.. if it worked for you or not.. if you notice the lag in the sound or not, if it's big or small, etc... (I know snack behaves differently from one system to another...)
Logged

KaKaRoTo
Kalinda
Power user
*
Offline Offline

Posts: 97


View Profile
« Reply #18 on: March 02, 2008, 06:57:26 pm »

Ok, I tested it...

AUDIO:
It worked for receiving (it made weird noises and occasionally cut out, but I know MSN's audio isn't perfect). It was generally constant, with only a few pauses. So, good going Smiley

However, I couldn't send. Or rather, my friend heard nothing... I dunno if that's my mic or the program, but when I ran it through the audio/video wizard and tested it, the mic worked fine and I know it's on and it works for other things. I installed libsnack, but is there anything special I have to do with it, like when I had to set up ALSA for my mic?

WEBCAM:
Since you haven't quite got the encoder down yet, it didn't work.... and my friend has no webcam so I couldn't test that part. I'll test it with my other friend, though, when he gets on.

I got this terminal output during the whole thing:
Code:
vid-probe: trying: v4l2...
ioctl VIDIOC_QUERYCAP: Invalid argument
open(/dev/video1): No such file or directory
open(/dev/video2): No such file or directory
open(/dev/video3): No such file or directory
vid-probe: trying: v4l...
open(/dev/video1): No such file or directory
open(/dev/video2): No such file or directory
open(/dev/video3): No such file or directory
Found Creative Nx Pro at /dev/video0


Hope that helps Smiley
Logged
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9428


View Profile WWW
« Reply #19 on: March 03, 2008, 03:24:30 am »

Cool, thanks for the feedback, appreciated Smiley
About audio, yes, it's normal, the weird noises are caused by the codec I think, but the 'few pauses' are the big problem caused by libsnack... which is why we'll need to rewrite our own sound extension instead of using snack.. sucks Sad
About sending, don't bother, it's not there yet, I didn't do anything for sending audio (yet), so just wait...
I might do it with snack now, but I think it will be 10 times easier to just wait until we write our sound extension instead of doing the work twice...
About webcam. yeah, it still wouldn't work sending, but I think I temporarly disabled it for now so we never send anything... (I don't have my webcam anymore so I disabled sending to avoid the auto-cancellation when no webcam is found). Also, the encoder isn't fully working yet, so...
About those messages, it's normal, you should get the same when you do a normal 'webcam' session...
Logged

KaKaRoTo
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9428


View Profile WWW
« Reply #20 on: March 06, 2008, 03:32:39 pm »

news news news!!!
Audio : We have a new tcl extension.. this one uses libao, so it should work MUCH better with it than with snack.. you will now need libao and libao-dev packages installed on your system in order to compile amsn. Once you do that, you will get the sound working a lot better... depending on your connection of course... I tried it on local network and it works extremely well.. I was able to watch a movie on my desktop but listen to it on my laptop.. the sound quality is excellent (even with non-voice/music), the lag is 50ms, there is no gap in the sound, it is very smooth, looks like a steady stream of audio, you don't notice any gaps between the packets and everything is fine! Smiley
Also, you should notice that those 'noises' that we heard before disappeared.. which means that the noises in the audio were caused by snack and not by the codec...

Video : I just fixed the wmv3 encoder so that it can work with the msn profile (using implicit quantization instead of explicit which was hardcoded before). This means that it's good news because I can now work on integrating the encoder into aMSN and be able to encode the webcam in WMV3 so we can send it.. it's not done yet, but should be coming pretty soon.

TODO :
1 - write a library/extension for capturing audio and for the mixer.
2 - record audio and send it correctly
3 - Try to find out why the WMV3 encoder lets me encode only one frame
4 - integrate the WMV3 encoding into webcamsn extension so it can be access with aMSN
5 - integrate the WMV3 encoding into aMSN so we can use the correct encoder depending on the session and fix the FCC in the header.
6 - try to find out why WMV3 stream doesn't work with WLM unless you go full screen... is it because we send only the webcam and no audio packets ?

Once it's done, I will merge the branch into the svn trunk so that others can try it and work on it. This doesn't mean the job is done, this is merely the beginning.. we will also need to build an appropriate UI for it. Apart from the menu button to start the call, we should adapt the current audio system (we're dropping snack) and update the assistant, have the webcam windows show up in the chat window, show a 'connecting' when we are connecting, have sliders for the volume, allow to pause/mute the webcam/sound. etc...

Any help will be appreciated of course! Feedback is also helpful Smiley
Enjoy! Smiley
Logged

KaKaRoTo
strickerkr
Newbie

Offline Offline

Posts: 1


View Profile
« Reply #21 on: March 09, 2008, 04:50:43 pm »

That sounds very good. I would like to tes the feature when's it's integrated.

mfg

Stephan
Logged
Fabioamd87
Super Power User
**
Offline Offline

Posts: 467



View Profile
« Reply #22 on: March 09, 2008, 08:08:06 pm »

Quote from: "strickerkr"
That sounds very good. I would like to tes the feature when's it's integrated.

mfg

Stephan

me too
Logged
why.arent.guests.allowed
Power user
*
Offline Offline

Posts: 63


View Profile
« Reply #23 on: March 09, 2008, 09:06:24 pm »

Thank you very much for your hard work on this kakaroto. That sure is impressive.
You sure hack and slash that code like... mmmh.... *looks at Fabioamd87's avatars* .... like Link through hordes of evil monsters.
I'm wondering... is aMSN a hobby or a day time job? :p

Sorry for not giving feedback as I should... Even though this is really one the features I'm looking forward to.
But to give you an idea, I've not used msn for more than a week now.
Nonetheless, I hope to be able to test the SVN version when things are integrated in trunk.

Once again thanks!
Logged

kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9428


View Profile WWW
« Reply #24 on: March 10, 2008, 12:57:40 am »

Hey there why.arent.guests.allowed (your nickname is quite complicated.. if u want to change it PM me with the new username! :p)
Thanks for the appreciation!! hehe, comparing me to Link.. quite nice! You could have compared me to how kakaroto fights against hordes of evil monsters too, no? :p
Anyways, no this is a hobby.. and it's taking a lot of my time, but thankfully I have some free time lately.. some other days I have no time at all...

Right now, I'm working on a new audio output library.. I found libao2 within mplayer's source code, it's like libao but heavily modified (supports volume mixer and has a dsound module for windows!) but it's crappy and needs to be rewritten.. this is going to take a lot of time unfortunately Sad But hopefully, once I'm done we'll have a really good audio processing library! I also found some audio input code for recording, but it only supports alsa and OSS... I'll have to look into this..
Hopefully, you'll soon see this merged into the trunk! It can already be merged as it is safe enough already.. but I want to finish it (new audio lib + sending of voice+video) before I merge it into the trunk...
Unfortunately, I found some bugs in the wmv3 encoder for ffmpeg, so I'll have to fix those bugs first...
Logged

KaKaRoTo
lucianolev
Power user
*
Offline Offline

Posts: 56


View Profile
« Reply #25 on: March 10, 2008, 05:10:56 am »

You're doing an awesome and revolutionary work here!  Cheesy
aMSN will become the first open source application in supporting voice/video conversation for the MSN protocol as far as i know. This is great news for the open source community. Cool

Thanks for your time and keep up the good work!
Logged
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9428


View Profile WWW
« Reply #26 on: March 10, 2008, 06:57:04 am »

thanks! and yes, we'll be the first Smiley
For now, in the branch, reception works pretty good, the only problematic thing is the sending...
Logged

KaKaRoTo
flomar34
Newbie

Offline Offline

Posts: 47


View Profile
« Reply #27 on: April 05, 2008, 08:24:30 am »

Hello,

Let me try it to say in english.  :oops:

I wil have said it exactly as lucianolev Smiley

It's a really very good  news for linux users and hope you will succeed with it.

I think that every body is going to follow your hard job now.

Thanks for all you are doing

 Smiley
Logged
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9428


View Profile WWW
« Reply #28 on: April 05, 2008, 10:50:10 am »

Hi flomar34 and welcome to the forums!
Thanks for the nice comments (and for registering just to say it!).

I'm trying to finish this but it takes more time than what I thought... we'll be having a Google Summer Of Code for the audio input/output library and hopefully we'll soon have a nice library for that... the WMV3 encoder has some stuff missing (p-frames are not encoded, only I-frames, but we need P-frames...), and once those two big tasks are done, we can move forward, but it might take a lot of time... Sad

in the meantime, I just started yesterday on a different 'project'... I'm currently working on a Tcl SIP client that will work with WLM's SIP server... I was able to have quite easily a working audio conference using farsight/gstreamer in less than a day of work, so that's good news.. a lot of work will still remain there but it's going pretty good so far! The big issue is :
1 - add farsight/gstreamer/glib as a new dependency
2 - farsight needs glib's main loop which is incompatible with tcl's main loop.. so either use farsight2, which is a complete rewrite and still new (not much tested.. not very sure of its stability).. but it also uses the main loop although work is currently being done to remove that... or use a separate process for the streaming...  in any case, we'll move to farsight 2 at some point...
We also have another problem.. WLM doesn't play our sound if it doesn't receive RTCP... unfortunately, RTCP support in farsight is poor or requires jrtplib which is a crappy library... but farsight 2 has great RTCP support... so that should be the way to go now, tomorrow I'll try using farsight 2 and enable RTCP to see if it can play my audio!!

Right now, my work is already in amsn's SVN (trunk, not a separate branch) and we'll hopefully soon be able to demo it!

In the meantime, you can receive audio and video in the video_conf branch without any problems, you just can't send...!
Logged

KaKaRoTo
why.arent.guests.allowed
Power user
*
Offline Offline

Posts: 63


View Profile
« Reply #29 on: April 05, 2008, 09:16:36 pm »

Thank you for giving us an update on the status of things kakaroto, or should I say かかろと ;-)
(my big achievement for the day... getting japanese input on Linux... hehe)

Awesome news on the Google SoC!!! But I checked SoC webpage and didn't see aMSN there...
It's just too bad it is more troublesome than it looked like... But we'll wait Cheesy
BTW, I remember seeing a possible SoC project by FFMPEG on WMV3. Maybe this is of interest to you. (don't know if there is any time to take an action given the deadline)

I'm not sure if I quite understand what you said about your other project...
Is it to be integrated in aMSN because it currently doesn't support audio conference? Will bidirectional audio-video conference be implemented through farsight2/gstreamer instead?

Thanks
Logged

Pages: 1 [2] 3 4 ... 46
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!