mediastream2使用指南(转载)

2009/6/29

                                       

转载，出处。(http://eatdrinkmanwoman.spaces.live.com/blog/cns!97719476F5BAEDA4!1003.entry)

mediastream.c是mediastream2库自带的一个test，也是最为复杂的一个test，学习它有助于加深对mediastreamer2的理解。

简介一下它的功能
1 利用mediastreamer2库封装的filter完成：从声卡捕捉声音，编码后通过rtp发送给远端主机，同时接收远端主机发来的rtp包，解码到声卡回放。
  filter graph如下：
  soundread -> ec -> encoder -> rtpsend
  rtprecv -> decode -> dtmfgen -> ec -> soundwrite

2 利用mediastreamer2库封装的filter完成：从摄像头捕捉图像，编码后通过rtp发送给远端主机（有本地视频预览），同时接收远端主机发来的rtp包，解码后视频回放。
  filter graph如下：
  source -> pixconv -> tee -> encoder -> rtpsend
                       tee -> output
  rtprecv -> decoder -> output

这个程序没有实现：用2个session来分别同时传送视频和音频。所以不要造成误解。
它实现的是：用1个全双工的session来传送视频或者音频，不管是本机还是远端主机，运行的都是同一个程序，一次只能选择一种payload。
牢 记rfc3550 Page 17中的所说“Separate audio and video streams SHOULD NOT be carried in a single RTP session and demultiplexed based on the payload type or SSRC fields. ”

程序中audio_stream_new() video_stream_new()内使用create_duplex_rtpsession()建立起监听端口。
比较奇怪的是video_stream_start()最后没有attach上rtprecv。而audio_stream_start_full()里有attach rtprecv。

编译的时候，别忘了加-D VIDEO_ENABLED启用视频支持。

程序命令参数
mediastream --local <port> --remote <ip:port> --payload <payload type number>
          [ --fmtp <fmtpline>] [ --jitter <miliseconds>]

这里fmtp和jitter是可选
fmtp的介绍如下：
Sets a send parameters (fmtp) for the PayloadType. This method is provided for applications using RTP with SDP, but actually the ftmp information is not used for RTP processing.
jitter就是设定缓冲时间，也就是队列的阀值。具体可以参见Comer所著TCP/IP 卷一的RTP一章。默认是80ms（还是50ms？），没必要修改它。

举一个使用的例子。
主机A IP 10.10.104.198
主机B IP 10.10.104.199
主机A 运行 ./mediastream --local 5010 --remote 10.10.104.199:6014 --payload 110
主机B 运行 ./mediastream --local 6014 --remote 10.10.104.198:5010 --payload 110

这里我使用的是音频传输，speex_nb编码。视频没有使用，怀疑是SDL有些问题，视频预览的时候是绿屏。

注意：程序代码里提到的音频编码有lpc1015，speex_nb，speex_wb，ilbc等，视频编码有h263_1998，theora，mp4v，x_snow，h264等。但是你却不一定能用得起来。这要看之前编译ffmpeg时，究竟是否指定了如上编码。
如果你的机子上并没有这些库，却又指定了这些库的payload type，那么mediastreamer2初始化的时候，会在终端输出错误信息找不到xxx.so之类，那么请换另一种payload type。
一般来说，speex，theora,xvid(h264)这三个比较容易编译。


关于rtp session，不知道你会不会像我有一些误解（尤其是双工的session）。
我 觉得session是一个难以一言蔽之的概念（否则rfc3550上也不会唠唠叨叨说一大堆,Page 9），虽然也可以精炼说成“The distinguishing feature of an RTP session is that each maintains a full, separate space of SSRC identifiers”，但是这样没什么意义，让人理解起来反而更困难。
我个人认为不必深究session的确切定义，而要细细体会rfc3550 Page 68 Section 11。(我每次有疑问时，就再来看一遍这一节)
1.RTP relies on the underlying protocol(s) to provide demultiplexing of RTP data and RTCP control streams.  For UDP and similar protocols,RTP SHOULD use an even destination port number and the corresponding RTCP stream SHOULD use the next higher (odd) destination port number.
2.For applications in which the RTP and RTCP destination port numbers are specified via explicit, separate parameters (using a signaling protocol or other means), the application MAY disregard the restrictions that the port numbers be even/odd and consecutive although the use of an even/odd port pair is still encouraged. 
3.The RTP and RTCP port numbers MUST NOT be the same since RTP relies on the port numbers to demultiplex the RTP data and RTCP control streams.
4.In a unicast session, both participants need to identify a port pair for receiving RTP and RTCP packets.  Both participants MAY use the same port pair.  A participant MUST NOT assume that the source port of the incoming RTP or RTCP packet can be used as the destination port for outgoing RTP or RTCP packets.
5.RTP data packets contain no length field or other delineation,therefore RTP relies on the underlying protocol(s) to provide a length indication.  The maximum length of RTP packets is limited only by the underlying protocols.
6.（原话找不到了）RTP本身并不知道该使用remote host的什么端口来传输，这需要用Non-RTP means来告知（比如和远端主机之间的信令交互得知），而本程序中没有信令交互，是显式指定。


最后把主机A上的运行结果贴一下

[atom@localhost code]$ ./mediastream --local 5010 --remote 10.10.104.199:6014 --payload 110 > atom
ortp-message-Registering all filters...
ortp-message-Registering all soundcard handlers
ortp-message-Card ALSA: default device added
ortp-message-Card ALSA: Ensoniq AudioPCI added
ortp-message-Card OSS: /dev/dsp added
ortp-message-Card OSS: /dev/dsp added
ortp-message-Loading plugins
ortp-message-Cannot open directory /usr/lib/mediastreamer/plugins: No such file or directory
ortp-message-ms_init() done
ortp-message-Setting audio encoder network bitrate to 8000
ortp-message-ms_filter_link: MSAlsaRead:0x93f0db0,0-->MSSpeexEnc:0x93f0ea0,0
ortp-message-ms_filter_link: MSDtmfGen:0x93f0d30,0-->MSAlsaWrite:0x93f0e28,0
ortp-message-ms_filter_link: MSSpeexEnc:0x93f0ea0,0-->MSRtpSend:0x93f0c18,0
ortp-message-ms_filter_link: MSRtpRecv:0x93f0c88,0-->MSSpeexDec:0x93f0f60,0
ortp-message-ms_filter_link: MSSpeexDec:0x93f0f60,0-->MSDtmfGen:0x93f0d30,0
ortp-message-Using bitrate 2150 for speex encoder.
ortp-message-alsa_open_r: opening default at 8000Hz, bits=16, stereo=0
ortp-message-synchronizing timestamp, diff=960
ortp-message-synchronizing timestamp, diff=320
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=47
 number of rtp bytes sent=1636 bytes
 number of rtp packet received=49
 number of rtp bytes received=1927 bytes
 number of incoming rtp bytes successfully delivered to the application=1739
 number of times the application queried a packet that didn‘t exist=107
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=99
 number of rtp bytes sent=3254 bytes
 number of rtp packet received=102
 number of rtp bytes received=3683 bytes
 number of incoming rtp bytes successfully delivered to the application=3629
 number of times the application queried a packet that didn‘t exist=213
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=152
 number of rtp bytes sent=5554 bytes
 number of rtp packet received=154
 number of rtp bytes received=5077 bytes
 number of incoming rtp bytes successfully delivered to the application=5038
 number of times the application queried a packet that didn‘t exist=318
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=205
 number of rtp bytes sent=6671 bytes
 number of rtp packet received=207
 number of rtp bytes received=5964 bytes
 number of incoming rtp bytes successfully delivered to the application=5925
 number of times the application queried a packet that didn‘t exist=424
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=258
 number of rtp bytes sent=8632 bytes
 number of rtp packet received=260
 number of rtp bytes received=7422 bytes
 number of incoming rtp bytes successfully delivered to the application=7292
 number of times the application queried a packet that didn‘t exist=530
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-Receiving RTCP SR
ortp-message-interarrival jitter=121
ortp-message-Receiving RTCP SDES
ortp-message-Found CNAME=unknown@unknown
ortp-message-Found TOOL=oRTP-0.14.2
ortp-message-Found NOTE=This is free sofware (LGPL) !
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=312
 number of rtp bytes sent=10280 bytes
 number of rtp packet received=314
 number of rtp bytes received=10202 bytes
 number of incoming rtp bytes successfully delivered to the application=9970
 number of times the application queried a packet that didn‘t exist=636
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=364
 number of rtp bytes sent=12382 bytes
 number of rtp packet received=365
 number of rtp bytes received=11527 bytes
 number of incoming rtp bytes successfully delivered to the application=11501
 number of times the application queried a packet that didn‘t exist=742
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=416
 number of rtp bytes sent=14337 bytes
 number of rtp packet received=419
 number of rtp bytes received=14520 bytes
 number of incoming rtp bytes successfully delivered to the application=14346
 number of times the application queried a packet that didn‘t exist=847
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=470
 number of rtp bytes sent=16216 bytes
 number of rtp packet received=472
 number of rtp bytes received=15832 bytes
 number of incoming rtp bytes successfully delivered to the application=15752
 number of times the application queried a packet that didn‘t exist=953
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=523
 number of rtp bytes sent=18399 bytes
 number of rtp packet received=524
 number of rtp bytes received=16604 bytes
 number of incoming rtp bytes successfully delivered to the application=16578
 number of times the application queried a packet that didn‘t exist=1059
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-Receiving RTCP SR
ortp-message-interarrival jitter=133
ortp-message-Receiving RTCP SDES
ortp-message-Found CNAME=unknown@unknown
ortp-message-Found TOOL=oRTP-0.14.2
ortp-message-Found NOTE=This is free sofware (LGPL) !
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=575
 number of rtp bytes sent=20072 bytes
 number of rtp packet received=578
 number of rtp bytes received=17986 bytes
 number of incoming rtp bytes successfully delivered to the application=17754
 number of times the application queried a packet that didn‘t exist=1164
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=629
 number of rtp bytes sent=22106 bytes
 number of rtp packet received=630
 number of rtp bytes received=20753 bytes
 number of incoming rtp bytes successfully delivered to the application=20637
 number of times the application queried a packet that didn‘t exist=1270
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=681
 number of rtp bytes sent=24917 bytes
 number of rtp packet received=683
 number of rtp bytes received=21885 bytes
 number of incoming rtp bytes successfully delivered to the application=21859
 number of times the application queried a packet that didn‘t exist=1376
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=734
 number of rtp bytes sent=27703 bytes
 number of rtp packet received=736
 number of rtp bytes received=22594 bytes
 number of incoming rtp bytes successfully delivered to the application=22555
 number of times the application queried a packet that didn‘t exist=1481
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=788
 number of rtp bytes sent=28977 bytes
 number of rtp packet received=789
 number of rtp bytes received=23637 bytes
 number of incoming rtp bytes successfully delivered to the application=23483
 number of times the application queried a packet that didn‘t exist=1587
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-Receiving RTCP SR
ortp-message-interarrival jitter=127
ortp-message-Receiving RTCP SDES
ortp-message-Found CNAME=unknown@unknown
ortp-message-Found TOOL=oRTP-0.14.2
ortp-message-Found NOTE=This is free sofware (LGPL) !
ortp-message-oRTP-stats:
   Global statistics :
 number of rtp packet sent=840
 number of rtp bytes sent=30072 bytes
 number of rtp packet received=837
 number of rtp bytes received=25564 bytes
 number of incoming rtp bytes successfully delivered to the application=25564
 number of times the application queried a packet that didn‘t exist=1692
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-oRTP-stats:
   Audio session‘s RTP statistics :
 number of rtp packet sent=840
 number of rtp bytes sent=30072 bytes
 number of rtp packet received=837
 number of rtp bytes received=25564 bytes
 number of incoming rtp bytes successfully delivered to the application=25564
 number of times the application queried a packet that didn‘t exist=1693
 number of rtp packet lost=0
 number of rtp packets received too late=0
 number of bad formatted rtp packets=0
 number of packet discarded because of queue overflow=0

ortp-message-ms_filter_unlink: MSAlsaRead:0x93f0db0,0-->MSSpeexEnc:0x93f0ea0,0
ortp-message-ms_filter_unlink: MSDtmfGen:0x93f0d30,0-->MSAlsaWrite:0x93f0e28,0
ortp-message-ms_filter_unlink: MSSpeexEnc:0x93f0ea0,0-->MSRtpSend:0x93f0c18,0
ortp-message-ms_filter_unlink: MSRtpRecv:0x93f0c88,0-->MSSpeexDec:0x93f0f60,0
ortp-message-ms_filter_unlink: MSSpeexDec:0x93f0f60,0-->MSDtmfGen:0x93f0d30,0
ortp-message-MSTicker thread exiting

Remote addr: ip=10.10.104.199 port=6014
Starting audio stream.
Bandwidth usage: download=26.614873 kbits/sec, upload=24.850787 kbits/sec
Bandwidth usage: download=24.757891 kbits/sec, upload=23.288695 kbits/sec
Bandwidth usage: download=21.915512 kbits/sec, upload=28.845805 kbits/sec
Bandwidth usage: download=18.254627 kbits/sec, upload=19.849715 kbits/sec
Bandwidth usage: download=22.314123 kbits/sec, upload=26.907152 kbits/sec
Bandwidth usage: download=33.280316 kbits/sec, upload=24.266766 kbits/sec
Bandwidth usage: download=21.465322 kbits/sec, upload=27.740203 kbits/sec
Bandwidth usage: download=34.365668 kbits/sec, upload=26.550016 kbits/sec
Bandwidth usage: download=21.669883 kbits/sec, upload=26.035174 kbits/sec
Bandwidth usage: download=17.581689 kbits/sec, upload=28.368437 kbits/sec
Bandwidth usage: download=22.050596 kbits/sec, upload=23.840863 kbits/sec
Bandwidth usage: download=32.469631 kbits/sec, upload=27.263791 kbits/sec
Bandwidth usage: download=19.910096 kbits/sec, upload=32.475279 kbits/sec
Bandwidth usage: download=16.712168 kbits/sec, upload=32.902848 kbits/sec
Bandwidth usage: download=19.092574 kbits/sec, upload=21.455217 kbits/sec
Bandwidth usage: download=24.896035 kbits/sec, upload=19.984568 kbits/sec
stoping all...

 

 

用mediastream+ortp库使用本机摄像头的简单程序

http://blog.sina.com.cn/s/blog_6a9032c10100lyjz.html

用mediastream+ortp库使用本机摄像头的简单程序

#include "mediastreamer2/mediastream.h" 
#include "mediastreamer2/msvideoout.h" 
#include "mediastreamer2/msv4l.h"

int main(int argc, char *argv[]){ 
VideoStream *vs; 
MSWebCam *cam; 
MSVideoSize vsize; 
int i;

vsize.width=MS_VIDEO_SIZE_CIF_W; 
vsize.height=MS_VIDEO_SIZE_CIF_H;

ortp_init(); 
ortp_set_log_level_mask(ORTP_MESSAGE|ORTP_WARNING|ORTP_ERROR|ORTP_FATAL); 
ms_init(); 
cam=ms_web_cam_manager_get_default_cam(ms_web_cam_manager_get()); 
//vs=video_preview_start(cam,vsize); 
//while(1); 
 
for(i=0;i<1;++i){ 
int n; 
vs=video_preview_start(cam,vsize);

for(n=0;n<1000;++n){ 
#ifdef WIN32 
MSG msg; 
Sleep(100); 
while (PeekMessage(&msg, NULL, 0, 0,1)){ 
TranslateMessage(&msg); 
DispatchMessage(&msg); 
} 
#else 
struct timespec ts; 
ts.tv_sec=0; 
ts.tv_nsec=10000000; 
nanosleep(&ts,NULL);

if (vs) video_stream_iterate(vs); 
#endif

 
if (n==400) 
{ 
printf("this is 400\n"); 
ms_ticker_detach (vs->ticker, vs->source);

vs->tee = ms_filter_new(MS_TEE_ID);

ms_filter_unlink(vs->pixconv,0, vs->output,0);

ms_filter_link(vs->pixconv,0,vs->tee,0); 
ms_filter_link(vs->tee,0,vs->output,0); 
ms_filter_link(vs->tee,1,vs->output,1); 

//ms_filter_unlink(vs->tee,0,vs->output,0); 
ms_ticker_attach (vs->ticker, vs->source);

} 
if (n==500) 
{ 
printf("this is 500\n"); 
int corner=1; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 
} 
if (n==600) 
{ 
printf("this is 600\n"); 
int corner=2; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 
} 
if (n==700) 
{ 
printf("this is 700\n"); 
int corner=3; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 
} 
if (n==800) 
{ 
printf("this is 800\n"); 
int corner=-1; 
ms_filter_call_method(vs->output,MS_VIDEO_OUT_SET_CORNER,&corner); 
} 
if (n==900) 
{ 
printf("this is 900\n"); 
ms_ticker_detach (vs->ticker, vs->source);

ms_filter_unlink(vs->pixconv,0,vs->tee,0); 
ms_filter_unlink(vs->tee,0,vs->output,0); 
ms_filter_unlink(vs->tee,1,vs->output,1); 
ms_filter_destroy(vs->tee); 
vs->tee=NULL;

ms_filter_link(vs->pixconv,0, vs->output,0);

ms_ticker_attach (vs->ticker, vs->source); 
} 
} 
video_preview_stop(vs); 
} 
return 0; 
}