
                          SER Mediaproxy

                        Copyright Dan Pascu
                       2002-2004 AG Projects



SERMediaproxy is a NAT traversal solution for IPTEL SIP Express router.

The proxy server is meant to be used in conjunction with SER's mediaproxy
module. In the rest of this document they will be named 'proxy server' and
'mediaproxy module' or 'module' to avoid confusion between them.


For information about installing the proxy server and the mediaproxy module
as well as for support information please consult the INSTALL file.
For licensing information please read the LICENSE file.


This proxy server represents one of the parts that compose the NAT traversal
solution implemented by mediaproxy. The other is the SER mediaproxy module
about which you can find more reading the documentation available with it.
The mediaproxy module is distributed with SER and is currently available from
the SER cvs only (it will be present in the SER version 0.8.13 when it will
be released).


1. Architecture
   ------------

The proxy server is designed around the concept of having different proxy
servers available for different domains as well as being able to have
multiple proxy servers for a domain that will do load balancing of the media
traffic that passes through them, or offer fallback capabilities if some of
them go offline (the others will absorb their traffic share redistributing
the load balancing between the remaining servers).

To implement this the SERMediaproxy is divided in 2 components:

i.  the proxy server itself (dealing with forwarding the RTP streams between
    the SIP parties).
ii. the proxy dispatcher which determines which of the available proxies
    will handle a given conversation (session) and forward the request to
    that proxy.

We will describe these components, however we will start with the dispatcher
as it is the core of the system.

1.1 Proxy dispatcher
    ----------------

The dispatcher implemented here is based on distributing the sessions based
on the SIP domain of the caller/destination using DNS SRV records. If a
domain doesn't define any mediaproxy server, a default will be used.

However it should be fairly easy to implement another dispatcher that uses a
different mean to determine the proxy server (like using a database mapping),
or even to distribute the traffic by other means than SIP domain name.

The proxy dispatcher is contacted by the mediaproxy module and passed the
commands to create/lookup a session information.
The dispatcher will receive among other informations the domain names for the
caller and the destination as well as the information if they are local or
not (i.e. if they are registered with the SER server in cause).

The dispatcher will use the sender's domain if the sender is local, else will
use the destination's domain if the destination is local. If none of them is
local, it will refuse to use a proxy server at all.
Once it has determined which domain to use to obtain the list of available
proxy servers, it will make a DNS lookup for the following SRV records:

_mediaproxy._tcp.selected.domain.name

If this lookup yields no record, the dispatcher decides that the domain
doesn't define any mediaproxy servers and will fallback to use a default
mediaproxy server as configured (see section 3.1 about how is the default
proxy server configured into the dispatcher).

However if the lookup yields one or more SRV records a list of servers will
be made where their order is based on the priority and weight properties of
the SRV records found.
From this list it will return the first mediaproxy server that responds.

The end result of this algorithm is that RTP traffic will be load balanced
between the servers at the same priority but with different weights. If some
of them are not responding their traffic share will be distributed between
the other servers at the same priority. When none of the servers at a given
priority is reachable will fallback to the next priority where the same rules
will apply.

After the proxy server has been determined the request will be forwarded to
this proxy server and the reply will be sent back to the mediaproxy module.
It will remember the proxy associated with the session and further requests
for the same session will go to the same proxy.

1.2 The proxy server
    ----------------

The proxy server does the actual RTP traffic forwarding between the parties
involved in the conversation.

It allocates a pair of sockets for each media stream a session has (one for
RTP the other for RCTP) and returns it's own IP address and a list of ports
that the mediaproxy module will use to replace the original values in the SDP
message. After this mangling is done, the parties involved in the conversation
will contact the proxy server thinking they contact the other party.
This is needed because the proxy server (which must have a public Internet
address) is able this way to determine the addresses from where the media
streams originate. This information is unknown when the SIP signaling takes
place, because it cannot be determined until the RTP streams actually start
to flow.
After it has allocated the sockets for each stream, the proxy will listen for
an incoming packet from the 2 parties. Once these arrive, the proxy is able
to know where the packets should be forwarded and can start forwarding them
between the 2 parties. Please note however that if one party is not behind
NAT the proxy server is able to send packets to it even before it receives a
packet from it, since the IP/port is already known. Because of this, our
mediaproxy server is able to work even when chained with another mediaproxy
server. There will be no blocking of the media streams because both mediaproxy
servers are passive and wait forever for a packet from each other. Instead our
mediaproxy server will only be passive and wait for a packet if the party it
talks with is behind NAT making it possible to chain as many mediaproxy
servers without blocking.

The proxy server is able to work directly with the mediaproxy module without
having the dispatcher in between them, however this way one looses the
ability to use different mediaproxy servers for different SIP domains, the
ability to do load balancing of RTP traffic between multiple mediaproxy
servers as well as the ability to run a remote proxy server (the mediaproxy
module is only able to talk with someone listening on a local UNIX filesystem
socket). However this can be done if one only needs a single proxy server
that runs on the same host as SER itself. Since both the dispatcher and the
proxy server use the same command set any of them can talk with the
mediaproxy module directly. In fact the dispatcher passes the commands it
receives from the mediaproxy module to the proxy server directly without
modifying them at all (it only interprets some information present in the
command to take the decision to which proxy server to forward the command).

In short, if one needs to run a remote mediaproxy server, wants to distribute
the RTP traffic based on the SIP domain of the caller/destination, wants to
do a load balancing of the RTP traffic on multiple proxy servers (be they
local or remote) or a combination of the reasons mentioned here he needs to
run the dispatcher too.


2. Features
   --------

- Geographically distributed proxy servers
- Ability to have different proxies for different SIP domains
- Ability to use multiple proxy servers per SIP domain
- Load balancing and redundancy
- Display active media sessions
- Accounting of network traffic
- Configurable IP and UDP port range
- Support for audio and video (handles all media streams in a session)
- Independent of SER version
- Only use 2 ports per media stream (one for RTP one for RTCP) for both
  endpoints of a call


3. Command line options
   -------------------

3.1 proxydispatcher.py
    ------------------

If you run proxydispatcher.py --help it will show the help screen as seen
below:

usage: proxydispatcher.py [options]

options:
  --version          show program's version number and exit
  -h, --help         show this help message and exit
  --no-fork          run the process in the foreground
  --socket=FILE      control socket (/var/run/proxydispatcher.sock)
  --group=GROUP      allow this group to write to the control socket (ser)
  --proxy=Host|File  default mediaproxy address (/var/run/mediaproxy.sock)
  --pid=FILE         pid file (/var/run/proxydispatcher.pid)
  --log=FILE         log to this file too, along with syslog (None)


In this help screen it describes the options it accepts, what they means and
inside the parentheses the default values for those options in case they are
not specified.

--socket specifies the path to the UNIX file socket where the dispatcher will
         listens for commands. If modified, you should also update the
         mediaproxy_socket option for the mediaproxy module in ser.cfg
         By default the mediaproxy module talks with the dispatcher (if you
         don't specify the --socket option and set no mediaproxy_socket option
         in ser.cfg this is what happens)

--proxy specifies where the default mediaproxy is located. By default it
        talks with the proxy server running on the same host through the 
        /var/run/mediaproxy.sock UNIX socket.
        The default proxy/proxies are used whenever a domain doesn't define
        it's own servers using SRV records in DNS.
        You can use the following forms to specify the default mediaproxy
        server to use:

        - /some/path

          To specify a local UNIX socket.

          The proxy server will run on the same host in this case

        - hostname:port or hostname or ipaddress:port or ipaddress

          To specify a network address by hostname or IP address

          The proxy server can run on the same host or a remote host in this
          case. If port is not specified 25060 is used by default.

        - None

          To specify that no default proxy server will be used

          By using the keyword `None' instead of an address the use of a
          default media proxy will be inhibited. Calls from domains that
          don't define their own mediaproxy servers using DNS SRV records
          will not go through a proxy server at all.

        - domain://some.domain.name

          To use the servers defined by domain some.domain.name

          In this case the default proxy servers to use will be taken from
          the some.domain.name domain.
          This allows one to specify multiple load balanced and redundant 
          default proxy servers (unlike the previous cases where the default
          proxy server was single)

The other options to proxydispatcher.py should be obvious.

3.2 mediaproxy.py
    -------------

If you run mediaproxy.py --help it will show the help screen as seen below:

usage: mediaproxy.py [options]

options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  --no-fork             run the process in the foreground
  --ip=IP               use this IP to talk to the RTP parties (first found)
  --ports=min:max       rtp port range (35000:65000)
  --socket=File         control socket or 'None' (/var/run/mediaproxy.sock)
  --group=Group         allow this group to write to the control socket (ser)
  --listen=Host[:Port]  address to listen on for remote control (None)
  --allow=Host[,...]    who is allowed to control this mediaproxy (127.0.0.1)
  --expire=Secs         expire idle sessions after this many seconds (60)
  --tos=TOS             set TOS value on IP packets (0xb8)
  --pid=File            pid file (/var/run/mediaproxy.pid)
  --log=File            log to this file too, along with syslog (None)


In this help screen it describes the options it accepts, what they means and
inside the parentheses the default values for those options in case they are
not specified.


--ip This option is used to specify the the IP address used by the proxy
     engine to exchange the RTP streams with the clients.
     If not specified first found will be used. Please note however that the
     one it picks may not be what you want. What it picks depends very much
     on the sanity of /etc/hosts. If that file is messed it can pick anything
     including the localhost address (127.0.0.1) which is obviously not what
     you want since it won;t be able to talk with anyone external using that
     address. Also it is possible that it will not be able to find any IP
     address at all. 
     As a simple rule, unless you need it to use a very specific IP address,
     you should let it choose the IP address. When it starts, mediaproxy will
     display what it has chosen (in syslog) and if you're not satisfied with
     that choice you can always use --ip=your.own.ip to overwrite.

--socket the socket to listen on for local connections through the
         filesystem. This is used (and preferable) when the proxy server runs
         on the same host as the dispatcher or the mediaproxy module.
         If you only need to listen for remote commands over TCP/IP you can
         use the keyword `None' (without quotes) to disable the UNIX socket
         control port.

--listen specifies the network address to listen on for remote control
         It can be one of hostname:port, hostname, ipaddress:port, ipaddress.
         In addition to these, the keyword `Any' can be used in place of
         a hostname or IP address to specify to listen on all network
         addresses the host has.
         If a hostname or IP address is used, it will only listen on that
         specific IP address.
         If port is not present, 25060 will be used by default.
         Examples are: proxy.test.com, 1.2.3.4:7890 or any:5678
         If no --listen option is specified at all the proxy server will not
         listen on any network address and will not be available for remote
         control over TCP/IP.

--allow  specified a list of hosts that are allowed to control this proxy
         server from remote (over TCP/IP). Obviously this option only applies
         if you have also used a --listen option.
         hosts can be specified by hostname, IP address or network classes
         using the CIDR notation: 10.0.0.0/24
         If multiple are needed they should be separated by a command and no
         spaces, like this: work.test.com,1.2.3.4,10.0.0.0/24
         In addition to specify that anyone is allowed to control it use the
         keyword any (beware!!!)
         By default if no --allow is specified only localhost (127.0.0.1) can
         connect to the network port and control it. The default is for
         security reasons, obviously one need to use some real life values
         for this option.


All the other options should be clear from the help message.

3.3 sessions.py
    -----------

This script is used to show the sessions currently running and information
about them (like status, codec type, duration, traffic, ...) in a terminal.

If you want a nicer formatted page check the media_sessions.phtml page in the
web/ subdirectory that formats the same info into a web page.
The php script can be placed on a web server and displayed in a web browser.

If you run sessions.py --help it will show the help screen as seen below:

usage: sessions.py [options]

options:
  --version          show program's version number and exit
  -h, --help         show this help message and exit
  --proxy=Host|File  mediaproxy's control port (/var/run/mediaproxy.sock)

--proxy specifies where the proxy server to query is located. By default it
        talks with the proxy server running on the same host through the 
        /var/run/mediaproxy.sock UNIX socket.
        You can use the following forms to specify the proxy server to query:

        - /some/path

          To specify a local UNIX socket.

          The proxy server will run on the same host in this case

        - hostname:port or hostname or ipaddress:port or ipaddress

          To specify a network address by hostname or IP address

          The proxy server can run on the same host or a remote host in this
          case. If port is not specified 25060 is used by default.


3.4 rtpgenerator.py
    ---------------

This is a utility script found under the utils/ subdirectory.
It's purpose is to emulate a pair of SIP phones making a conversation with a
specific codec type. It's purpose is to test the load that the mediaproxy
server can handle. Please note however that the RTP media streams do not
contain any valid RTP data. They are all filled with 0, and all that matters
about them is the amount of traffic they generate which the same as one
generated by a real phone.
The script is able to simulate the request/lookup commands that come from the
mediaproxy module to initiate a RTP session without the need to actually have
the call come through real SIP signaling from SER. As a matter of fact you
don't even need SER running to test this. All that is needed is a running
mediaproxy server that listens on an accessible control port.

If you run rtpgenerator.py --help it will show the help screen as seen below:

usage: rtpgenerator.py [options]

options:
  --version          show program's version number and exit
  -h, --help         show this help message and exit
  --ip=IP            IP used to send/receive the RTP streams (127.0.0.1)
  --proxy=Host|File  mediaproxy control socket (/var/run/mediaproxy.sock)
  --g711             create a load equivalent of a G711 codec
  --g729             create a load equivalent of a G729 codec
  --g723             create a load equivalent of a G723 codec
  --gsm              create a load equivalent of a GSM codec
  --rate=Rate        data rate in kpbs 0-512 (64)
  --count=Count      simulate this many streams of the given rate 1-30 (1)


--ip specify from which IP address to pretend the calls are from (must be a
     valid IP address of the host you run the script on). You actually need
     to specify one, as the default (127.0.0.1) will only work when testing
     a proxy server that is on the same host as the generator script.

--proxy specifies where the proxy server to test is located. By default it
        talks with the proxy server running on the same host through the 
        /var/run/mediaproxy.sock UNIX socket.
        You can use the following forms to specify the proxy server to test:

        - /some/path

          To specify a local UNIX socket.

          The proxy server will run on the same host in this case

        - hostname:port or hostname or ipaddress:port or ipaddress

          To specify a network address by hostname or IP address

          The proxy server can run on the same host or a remote host in this
          case. If port is not specified 25060 is used by default.

--g711
--g729
--g723
--gsm
--rate  used to specify what RTP stream to simulate. They will generate a
        traffic load equivalent to that codec type. With --rate you can
        manually specify what rate you wish between 0 and 512 kbps.
        If none is specified the equivalent of a G711 codec is generated

--count simulate this many streams. This is like making as many separate
        calls using 2xcount phones. This option allows easy generation of
        multiple media streams running the script only once.
        There is a limit of 30 streams a single process can generate.
        If you need more, run the script multiple times.


