aMSN Forums
November 30, 2020, 10:06:50 pm *
Welcome, Guest. Please login or register.
Did you miss your activation email?

Login with username, password and session length
News: New forum for aMSN !!
 
   Home   Help Search Login Register  
Pages: 1 2 [3] 4 5 ... 46
  Print  
Author Topic: Audio/Video conversation  (Read 697066 times)
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #30 on: April 06, 2008, 12:51:02 am »

Quote
Thank you for giving us an update on the status of things kakaroto, or should I say かかろと  
(my big achievement for the day... getting japanese input on Linux... hehe)

lol, thanks and congrats on japanese input :p

Quote
Awesome news on the Google SoC!!! But I checked SoC webpage and didn't see aMSN there...

it's because it's under the Tcl/Tk community, you can see the 'ideas' here : http://wiki.tcl.tk/20832
the "Audio input/output library and extension" is the one posted by me... I have a friend who applied for it, and I'll be mentoring.
About WMV3 on FFMPEG, yeah, I know and talked to them, but I can't do it since I'm already a mentor, and any other person I know who could do it are too busy and can't do it...
 
Quote

I'm not sure if I quite understand what you said about your other project...
Is it to be integrated in aMSN because it currently doesn't support audio conference? Will bidirectional audio-video conference be implemented through farsight2/gstreamer instead?

No.. let me explain it to you.. WLM has 5 different A/V features.. you can see a summary that a wrote here : http://imfreedom.org/wiki/index.php/MSN:AV
Here's a summary of that :
- The 'Video Call' feature is a bidirectional Audio and Video conversation which relies on the MSNP2P protocol for the signaling.
- The 'Webcam' feature is a unidirectional Video only conversation which relies on MSNP2P for the signaling.
- The 'Computer Call' feature is a bidirectional Audio only feature which uses SIP and RTP for the signaling.
(the other two being the voice clips and an old A/V system that aMSN can already use if you have the linphone extension (read the FAQ, we don't give support for it))

aMSN already supports "Webcam" as you probably know.. the remaining features is the 'video call' and the 'computer call'... the video call is MSNP2P and is bidirectional audio and video... while the computer call is SIP with RTP and is for audio only (bidirectional).
What I was working on from the start (in this thread) was the video call.. now, I put that on hold, and now I'm working on the computer call feature... I already have the SIP working, and farsight/gstreamer takes care of the RTP... I'll be writing soon a farsight 2 app that does the RTP and I'll be porting libnice to implement ICE draft 6 and make it compatible with MSN's ICE implementation in order to be able to do NAT traversal...

Once I complete these.. We'll have both features in aMSN... if you have glib/gstreamer/gst-plugins-base/gst-plugins-good/gst-plugins-bad/gst-plugins-farsight/farsight2 installed on your system as well as the compiled aMSN code for it, you will be able to do an audio only conversation using SIP and RTP (much better performance, quality, speed, nat traversal, etc...)... Or if you don't, then you can still do the Audio/Video call which doesn't have any dependency (apart from ffmpeg and the new libao/libai which will be shipped with aMSN).
Hope it's clear now!
Logged

KaKaRoTo
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #31 on: April 06, 2008, 10:56:40 am »

Hello.. so again with some news.. I've put a lot of work on this today and I just sent my results (instructions on how to use this) to the amsn-devel mailing list...
For those of you who aren't subscribed to the mailing list, here's a copy/paste of the email...
If you don't understand something.. don't ask, it just means you're not qualified to be my tester :p

Quote

Hi,
I just 'finished' writing the first part of my SIP client.. this is useful for the audio conference that uses SIP and RTP (not the one with audio and video which uses msnp2p and is being developped on a separate branch).

So here's the deal.. I have two phases for the client, first phase is being able to make a call (send an invite) and it already works pretty good! second phase is to be able to receive a call (receive an invite) and that won't be too difficult...
I consider the first phase complete, and I like the API and the design a lot! Smiley What I would need now is to have you guys test it.. so here's how you can do a voice call with aMSN :

- first, you need to update to SVN r9697 or later..
- Download the latest version of farsight and compile it.. (with gst-plugins-farsight of course)
- Open the amsn console and type 'source sip.tcl' Now you have everything you need to make a voice call..
- Enable MSNP15 support and connect so you can have your authentification token...

The easy part :
- type in the console : 'createSIP' which will create a new SIP connection object called 'sip'
- then type 'inviteSIP $email' to invite someone from WLM.
You should then see status messages on the console, once it gets accepted on the other side, you'll see a status OK with the list of codecs and candidates (ip/port combinations).
For example :
Quote
Invite SIP Response : OK -- {{{} 1 {} UDP 1 192.168.1.100 123456} {{} 2 {} UDP 1 192.168.1.100 123457}} {{PCMA 8 8000} {PCMU 0 8000} {telephone-event 101 8000}}

The first list is the candidate list, you take the first one ("the one for RTP has a '1' as second element, RTCP has '2') and you have your ip and port..
- Go to farsight's directory and type : ./tests/test-rtp-7 192.168.1.100 123456 1
That's it.. you should start hearing the WLM audio in your PC..

The complicated part :
In theory, you should create a new SIP connection with :
Code:
SIPConnection create %AUTO% -user $your_email -password [[$::sso GetSecurityTokenByName Voice] cget -ticket]

The returned object is your SIP connection.. it's not conneced yet.. you could do a :
Code:
$sip Invite $email $codec_list $candidate_list

The codec list is a list containing codecs.. each codec is itself a list of the form
Code:
[list $encoding_name $payload_type $bitrate]

The encoding name and payload types should be following the RTP RFCs.. to make it easy you can use PCMA which has a static payload type of 8 and PCMU with payload type 0.
The condidate list is also a list containing candidates (connection info like ip/port), each candidate is a list of the form :
Code:
[list $candidate_id $component_id $password $transport $qvalue $ip $port]

those are ICE candidates, WLM uses ICE draft 6, so the candidate id and password are randomly generated and used to authenticate once the connection is established... you don't need that.. it can work even without ICE, so you can give an empty string as candidate id and password and it will skip ice candidates... the qvalue is just a 'weight'.. the higher the value (between 0.00 and 1.00) the better it is to connect on it (ip/port priority). transport is either UDP or TCP and that's it...

So let's say we want to create an object called 'sip' it can be done like this :
Code:
SIPConnection create sip -user $your_email -password [[$::sso GetSecurityTokenByName Voice] cget -ticket] -error_handler myErrorHandler -request_handler myRequestHandler

The error and request handlers should be procs that handle errors and new requests.. you can skip the -request_handler for now (since we don't handle incoming requests for now) but the error_handler should have a prototype of the form :
Code:
proc myErrorHandler { reason }

anyways, once the object is created, it's not connected yet, so you can do it with :
Code:
sip Connect

but you don't really need to do that.. you mainly need to Register.. if you try to register and it's not connected, it will connect automatically.. so just do a :
Code:
sip Register

it will then register itself on the SIP server...
If you want to invite someone, you can also skip that and just send the invite, it will automatically connect and register before sending out the invite... So with everything I said so far, here's how you can send an invite :
Code:
sip Invite $email [list [list "PCMA" 8 8000] [list "PCMU" 0 8000]] [list [list "" 1 "" UDP 1 $my_ip $my_port]]

Since we don't use ICE, if you're behind a firewall or a NAT, then you just won't be able to make the connection work, so I was testing between WLM and aMSN inside my local network, so the $my_ip was actually my local ip.. which I was getting with :
Code:
[::abook::getDemographicField localip]

The $port is special though, since we're using a 'test' application from farsight, and that test only takes the destination ip/port, it has a hardcoded value for the port to listen to and it's 7078... so until we write our own application that uses farsight, we have to use the port 7078..
One last thing.. if you want to know what is happening (you must!) then you need to specify one last argument : a callback function with the protocol :
Code:
proc InviteCallback { status detail }

The status tells you what's happening and the detail is an extra detail per status.. for example "ERROR" status will have the reason in the detail.. a "OK" will have a list containing the remote user's candidates and codecs in the $detail variable...
So you end up doing this :
Code:
sip Invite $email [list [list "PCMA" 8 8000] [list "PCMU" 0 8000]] [list [list "" 1 "" UDP 1 [::abook::getDemographicField localip] 7078 InviteCallback

Once you do that, the invite gets sent, the other user's WLM starts ringing.. he can accept/decline/not answer/etc... the Tcl client will tell you exactly what happens.
The rest is the same as the 'easy method', you just look at what you receive in your callback, when you get the OK, you parse the remote candidates, get the ip/port and use that with farsight :
Code:
./tests/test-rtp-7 $ip $port 1


Now To summarize everything I've just explained.. let's look at the last few functions in sip.tcl which serve as test and are what are being used for the 'easy method' :
Code:

proc createSIP { {host "vp.sip.messenger.msn.com"} } {
        global sso
        set token [$sso GetSecurityTokenByName Voice]
        catch {sip destroy}
        return [SIPConnection create sip -user [::config::getKey login] -password [$token cget -ticket] -error_handler errorSIP -request_handler requestSIP -host $host]
}


This helper function just creates the 'sip' object for you just like I said before...
Code:

proc inviteSIP { email } {
        sip Invite $email [list [list "PCMA" 8 8000] [list "PCMU" 0 8000]] [list [list "" 1 "" UDP 1 [::abook::getDemographicField localip] 7078]] inviteSIPCB
}

and this one invites the user without having you type in the list of codecs/candidates everytime...

Code:
proc inviteSIPCB { status detail} {
        puts "Invite SIP Response : $status -- $detail"
}


invite callback which tells you what's happening...

Code:

proc requestSIP { request headers body } {
        puts "Received Request : $request"
}


Request handler..
This will tell you if you receive a request from the other user (assuming you are connected to the same SIP proxy but it does nothing else...)

Code:

proc errorSIP { reason } {
        puts "Error in SIP : $reason"
}


Error handler which will tell you what happened when something goes wrong...


That's about it.. if you follow these instructions, you should *in theory* be able to do a voice call with WLM... problem now is that WLM will not play your sound, and that's because farsight doesn't send RTCP and without it, WLM doesn't know if your SSRC (and id in the rtp header) is yours... with farsight 2 (which we'll use), we'll have RTCP so WLM will play our sound too... yeay!

If you have any questions.. you will find me tomorrow after I wake up on IRC... or write to the ML...

Feedback appreciated!!!
Thanks
KaKaRoTo




EDIT :
Oups... use svn revision 9698 instead..
and here's how it looks like for different scenarios... : http://pastebin.com/m52e0c270
Logged

KaKaRoTo
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #32 on: April 09, 2008, 01:31:23 am »

Quote from: "from email"
Hi,
A bit more news.. I have just tested out the Audio stuff between my aMSN and Tom's aMSN and even though we're both firewalled (and obviously not in the same network) it worked! I could hear him very well..
It is currently using PCMA/PCMU (almost uncompressed audio) which uses about 10KB/s of your bandwidth.. I'll soon try to write siren gst elements so we should get less than 1.6KB/s of bandwidth with that codec...

With my latest commit (r9707) things have changed a little.. here's how you can do it :
First, you must open the console and type 'source sip.tcl' then ...
1 - sending an invite :
type : createSip; inviteSip $email
2 - receiving an invite :
One you receive the invite, you should see the invite printed in there, like this :
Received INVITE : 6935BADC0FC1B544C33B62B465969E09 - {{{}..(removed the rest of the line).. all you have to do is type :
acceptSIP 6935BADC0FC1B544C33B62B465969E09
and that's it... (notice that the 'id' 6935BADC0FC1B544C33B62B465969E09 was taken from the 'Received INVITE' line...)

Notice though that to be able to do this you will first need this :
1 - gstreamer
2 - gst-plugins-base
3 - gst-plugins-good
4 - gst-plugins-bad
5 - gst-plugins-farsight
6 - farsight2 with ./configure --disable-python --with-transmitter-plugins=rawudp
7 - go to amsn/utils/farsight and type 'make'
Make sure you have the latest version of all these components, you can try disabling some stuff if you don't need them (like for farsight2)

This will work bidirectional between two aMSNs but with WLM you can only receive and not send.. That should be fixed soon..

TODO :
1 - use of ICE for having WLM receive our stream
2 - add a Siren encoder/decode/payloader/depayloader for gstreamer
3 - clean the ugly hacky functions I used for testing...
4 - a proper configure/makefile/runtime dependency check...
5 - a proper UI

That's it! have fun!

KaKaRoTo
Logged

KaKaRoTo
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #33 on: April 10, 2008, 06:10:30 am »

Hi Again!
For those who read this thread (doesn't look like many of you are...), I have some really great news!
1 - I was able to write Siren encoder. decoder. RTP payloader and depayloader elements for gstreamer...
What this means is simply that we can now use a more 'advanced' codec which uses only 1.9KB/s !!!
This also means :
2 - It looks like WLM now accepts the sound I'm sending it, it actually didn't want to play PCMA/PCMU codecs, But with the SIREN codec, it now works!
This simply means that you can now audio chat with any amsn OR WLM users without problems!

Now, the next step is what will take a bit more time, it's about cleaning the code, fixing some stuff, making it 'UI-wise' (you click accept/refuse instead of typing commands), etc... but for those who want to try it out, the previous instructions still apply..
I'll keep you posted on when/where you can find the siren gstreamer elements so you can make it use less bandwidth and work with WLM.
Logged

KaKaRoTo
rowanparker
Super Power User
**
Offline Offline

Posts: 234



View Profile WWW
« Reply #34 on: April 10, 2008, 07:38:10 am »

Quote from: "kakaroto"
For those who read this thread (doesn't look like many of you are...)


Some of us do man.
I think its very good what you're doing.
This is one of the features people want baddly.
I'm just not replying because I don't understand all of it Smiley
Logged

flomar34
Newbie

Offline Offline

Posts: 47


View Profile
« Reply #35 on: April 10, 2008, 07:52:56 am »

Sure, i read this post everyday, and of course ...........   i 'm not able to  understand it

I just understand that what you are doing is great and very exciting for us.  Smiley
Logged
Fabioamd87
Super Power User
**
Offline Offline

Posts: 467



View Profile
« Reply #36 on: April 10, 2008, 09:39:49 am »

I read this post every time, but im still waiting for a friendly gui, because i dont wont to broken my amsn :p

anyway GOOD JOB!
Logged
kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #37 on: April 10, 2008, 04:49:50 pm »

hehe, hi,
thanks for the answers :p I got tired of answering myself, lol...
And sorry if you don't understand it, it's pretty technical, but I guess more info is better than not enough info.
Anyways, it should soon be integrated with aMSN so you can use it with the UI. There isn't much work left to do Smiley
Logged

KaKaRoTo
rowanparker
Super Power User
**
Offline Offline

Posts: 234



View Profile WWW
« Reply #38 on: April 10, 2008, 06:12:02 pm »

One quick question: Does this have to be using version 15 of the protocol?
Logged

kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #39 on: April 10, 2008, 09:39:51 pm »

for sending a request, no, for receiving one, yes...  (although you'll need to authenticate first with msnp15, then your authentication token can last a couple of days, even if you disconnect or use msnp11 afterwards)
Logged

KaKaRoTo
rowanparker
Super Power User
**
Offline Offline

Posts: 234



View Profile WWW
« Reply #40 on: April 10, 2008, 10:09:24 pm »

Quote from: "kakaroto"
for sending a request, no, for receiving one, yes...  (although you'll need to authenticate first with msnp15, then your authentication token can last a couple of days, even if you disconnect or use msnp11 afterwards)


Oh right, I see.
Because for some reason I cannot connect using msnp15 (it just timesout, no matter what I try).
And I would like to try this out (once it gets a bit simpler to do so obviously).
Logged

kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #41 on: April 11, 2008, 12:34:21 am »

hummm... go to the MSNP15 thread and report the problem there, the status and protocol logs are important... thx
Logged

KaKaRoTo
why.arent.guests.allowed
Power user
*
Offline Offline

Posts: 63


View Profile
« Reply #42 on: April 12, 2008, 05:12:48 pm »

Hi かかろと :)
Yes, I also do read this thread... I check it (usually) at least once a week. But like most users here... most technicalities are out of my grasp especially because I'm not familiar with MSN protocol... and don't have much free time to study it Sad
But I learn a bit more everytime I read your posts.

Those are great news about Siren... < 2KB/s for audio chatting. Cheesy

BTW, thank you very much for your explanation (1st post of this page... post number 30?) on the MSN A/V features. It did clear things up for me!  Cheesy
Logged

kakaroto
Administrator
Super Power User
*****
Offline Offline

Posts: 9425


View Profile WWW
« Reply #43 on: April 13, 2008, 01:22:28 am »

hi (don't expect to write your nick in jap),
thx all for reading :p and sorry for all the technical stuff that makes you feel even n00b-er :p hehe.. well, anyways, I'm working on integrating this into the UI and it's almost done now.

arg.. seems I forgot to finish/send this post.. anyways, now I have already finishes the integration lol...
so... starting revision 9729.. here are the new instructions :

1 - get the dependencies you need (http://amsn-project.net/wiki/Farsight)
2 - type ./configure && make
3 - run amsn and open a chat window..
4 - menu Actions->Start Audio call...

as simple as that! Smiley

You don't need to be using MSNP15 anymore because it now automatically authentifies you if you're not using msnp15... BUT if you don't use MSNP15 you will not be able to *receive* invites, but you can still call someone.
The current issues are that if you receive a call from WLM, he doesn't seem to work because he expects ICE connections (which will be fixed someday), you we answer him that we're temporarly unavailable and we give you the choice to call him back (if you call WLM, it works, if you get a call from WLM, it doesn't).
If you call a WLM user, you should be able to receive him, but he will not be able to receive your sound *unless you're in the same network* (it disables ICE so it can't find its external IP and gives you only the internal IP).. note though that if the WLM user is not inside a NAT, it should work.
The best compatibility now is between two aMSN users... simply because we send our external ip instead of the internal ip.
If you call someone (amsn or WLM) from the same NAT (local network), it will not work because we send the external ip...

that's the only issues I found so far.. they should *all* be fixed (apart from the 'you have to use msnp15 to receive a call') once I fix libnice and make aMSN use ICE connectivity to get through firewalls.


The configure will tell you whether it found the dependencies or not and will compile the farsight utility if it finds them.
aMSN will also check this at startup and will tell you if a runtime dependency is missing...

Here are the use cases I found and tested to be behaving correctly (use case - message amsn prints in the chat window):
1 - send an invite - Calling $user
2 - receive an invite - Received Audio Call invitation from $1
3 - accept an invite - Audio Call Accepted
4 - your invite gets accepted - User accepted your Audio Call
5 - decline an invite - Audio Call Rejected
6 - your invite gets declined - User Declined your Audio Call invitation
7 - you receive an invite while being already in a call
8 - you send an invite to someone who is already in a call - User is currently Busy
9 - You cancel an invitation - Your Audio Call has ended
10 - The invitation you received was canceled - User has canceled the Audio call Invitation
11 - You don't accept/reject a call for 1 minute - You missed an Audio Call from $user - Call Back
12 - the other contact hangs up - User has ended your Audio Call
13 - You hang up a call - Your Audio Call has ended
14 - your invitation did not get a reply in 1 minute - User did not answer your Call
15 - invite sent to an incompatible or busy client  - User is not available right now
16 - farsight 2 is not installed or crashes on init - You cannot make an Audio Call. Please install Farsight2 and try again.
17 - other client doesn't support SIP or is not using MSNP15 - User does not support Audio Calls.
18 - receive an invite from WLM with ICE connectivity which is not currently working : You have received an Audio Call invitation that is not currently supported. You may call back with a new invitation. The Audio Call will then work
19 - you try to call multiple people at the same time - You can only have one Audio Call at a time.


That's it.. have fun trying it!!!
Logged

KaKaRoTo
flomar34
Newbie

Offline Offline

Posts: 47


View Profile
« Reply #44 on: April 13, 2008, 10:23:54 am »

Hello, this is a great news!

I had no problem to compile amsn and i have the menu for the audi conversation

the only problem i have is with farsight2

I tried to compile it but on the ./configre  it asked for gstreamer 0.10.17 and with Gutsy we have the 0.10.14 release Sad
Logged
Pages: 1 2 [3] 4 5 ... 46
  Print  
 
Jump to:  

Powered by MySQL Powered by PHP Powered by SMF 1.1.11 | SMF © 2006-2009, Simple Machines LLC Valid XHTML 1.0! Valid CSS!