define gdb_strcmp
        dont-repeat
        set $result = 1 
        set $_i = 0 
        if ($arg0[0] == 0x0 && $arg1[0] != 0x0)
                set $result = 0 
        end 
        if ($arg0[0] != 0x0 && $arg1[0] == 0x0)
                set $result = 0 
        end 
        while ($result == 1 && $arg0[$_i] != 0x0 && $arg1[$_i] != 0x0)
                if ($arg0[$_i] != $arg1[$_i])
                        set $result = 0 
                end 
                set $_i = $_i + 1 
        end 
end
document gdb_strcmp
Determines if two C strings match
end

Note that gdb user commands are annoying because you don’t really have return values (they are not really functions/macros), so you have to set a global variable (yuck!) to hold the result. This macro sets $result to 0 if the strings are not equal and $1 if they are. I contemplated using the same return value than the C counterpart, but since I was interested in just a ‘yes or no’ answer I sticked to use 1 for equal and 0 for non equal.

You can then go ahead and use this macro in other macros to do useful things, such as scan a linked list and verify if a given member has certain string value.

PD. I know it’d be cleaner to start using Python for these things but I have not really looked yet into that

It is sometimes required to analyze protocol messages in a given link in order to troubleshoot call problems. Wanpipemon makes possible to create a pcap file with all D-channel data which can then be examined using Wireshark.

What makes it so convenient is that this method is independent from the application you’re using (Asterisk, FreeSWITCH or whatever). You basically have the capability to tap into the D-channel, pretty much like using tcpdump to monitor TCP/IP traffic.

wanpipemon -i w1g1 -pcap -pcap_file isdn.pcap -prot ISDN -full -systime -c trd

You can refer to the Sangoma Wiki for other details (and SS7 tracing commands too): http://wiki.sangoma.com/wanpipe-wireshark-pcap-pri-bri-wan-t1-e1-tracing

Next time your D-channel link does not come up, fire up wanpipemon and take a look in Wireshark to what’s going on!

http://area51.stackexchange.com/proposals/12932/telephony

I was clearly able to switch to the correct directory with cd /path/to/repo && git pull, but it kept failing.

It seems git hooks run with some git environment variables like GIT_DIR that affect the new instance of git being spawned. The easiest solution I found was to specify –git-dir when doing the pull:

cd /path/to/repo
git --git-dir=/path/to/repo/.git pull

And finally I could move on after being stuck like half an hour

This time I’ll show you how to check whether an FXO line is plugged into the the card via the command line. You can also check if the FXO is OnHook (http://en.wikipedia.org/wiki/On-hook) or OffHook (http://en.wikipedia.org/wiki/Off-hook)

In order to check the analog status for a given analog channel in a Sangoma card you can do the following:

# wanpipemon -i w1g1 -c astats -m 1

The -i option is used in most commands to specify the wanpipe interface in which will be run. You can find your wanpipe interfaces using ifconfig, normally they are named in the form “wXg1”.

The -c option is used to specify the command to run in the given interface.

The -m is used to specify the analog channel.

Typical output for an FXO channel not connected will be:

root@sigchld:/home/moy # wanpipemon -i w1g1 -c astats -m 4

        ------- Voltage Status  (FXO,port 3) -------

VOLTAGE : 1 Volts

        ------- Line Status  (FXO,port 3) -------

Line    : disconnected

The voltage comes handy if you want to check if the line is OnHook or OffHook. The typical voltage when the line is connected and OnHook is around 53, 53 volts. When the line is OffHook and the circuit is closed, the voltage drops to 8, 9 volts.

Even when the output of the command tells you if the line is connected or not, you can also appreciate that the voltage is very low when the line is not connected (1 volt is reported in this case).

The binary compatibility report is at
http://www.libopenr2.org/reports/abi_report_1.3.0_to_1.3.1.html

The most significant changes are:

– Proper DTMF R2 support, including new timers to tweak the end of DTMF timeout.
– Indonesian variant
– Test for lib64 directory for installation in 64 bit
– Fixed some build issues including support for Linux/Sparc

The next week I will also be releasing openr2 2.0.0 which breaks API
and ABI compatibility with openr2 1.x series. Releases 1.x have proven
to be stable now and they will enter maintenance mode (no new variants
nor features will be added there).

Releases 2.x will be used for future Asterisk versions (1.10??) and
for FreeSWITCH and Windows support.

Anyhow, I had never really spent some quality time understanding Makefiles, I knew the basics to create targets and stuff, but this time I decided it was time to really dig into the fascinating and sometimes frustrating world of Makefiles. While I was doing this refactoring, I had the need to arbitrarily find the topmost directory of our svn tree. I’ve seen some projects require you to execute a shell script at the top directory to set such variables, something like:

export ROOT_SVN=$(pwd)

Then it is required that you execute that setup.sh script before building anything. Then makefiles that can be included from different directory levels can use that variable to refer to the root directory of the project and not depend on their location using relative paths (which can change depending on who is including that file).

I had to ask myself, do I really need a setup.sh script before building just to figure out what the topmost svn directory is? Being an amateur Makefile writer it took me one hour figure out a function to do it, and this is what I came out with:

find_root = $(if $(wildcard $1/.svn),$(if $(filter-out /,$1),$(call find_root,$(realpath $1/..),$1),$1),$2)
SVN_ROOT := $(call find_root,$(shell pwd),$(shell pwd))

Ta da!

That should recursively look for the topmost directory with an .svn entry in it, if / is reached, then / is returned. It is easily changed to look for any other file (like .git).

I could not find any solution using google, so I thought in dropping this here.

I use the native Makefile functions $(if ), $(wildcard ), $(filter-out ) and $(realpath ), you can find documentation for such functions in the Make documentation here: http://www.gnu.org/software/hello/manual/make/Functions.html

Presentation in PDF and PPT available here:

http://www.moythreads.com/congresos/astricon2010/asterisk-pri-passive-recording.pdf

http://www.moythreads.com/congresos/astricon2010/asterisk-pri-passive-recording.ppt

I am here to warn you, select() has some important limitations to be aware of. I must confess I used select for a long time without realizing its limitations, until, of course, I hit the limits.

About a year ago I started porting a heavily threaded networking real-time voice application for Windows to Linux. When the code was compiling and running apparentely without issues, we started doing scalability and stress tests. We used 32 telephony E1 cards (pretty much like a network card) where each E1 port can handle up to 30 calls. So we’re talking about 32 * 30 = 960 calls in a single server. Knowing in advance that I would need lots of file descriptors (Linux typically defaults to 1024 per process), we used the setrlimit() system call to increase the limit up to 10,000 which should be more than enough because a telephony call in this system requires about 4 file descriptors (for the network VoIP connection and the E1 side devices etc).

At some point during the stress test, calls stopped working and some threads were going crazy eating up 99% CPU. After finding out using “ps” and “pstack” which threads were the ones going crazy, I found out that were the ones waiting for I/O in some file descriptors using select(), like the embedded HTTP server or the Network-related code.

Reading carefully the select documentation you will find the answer by yourself. “man select” says:

“An fd_set is a fixed size buffer. Executing FD_CLR or FD_SET with a value of fd that is negative or is equal to or larger than FD_SETSIZE will result in undefined behavior.”

So, big deal you may say, you can split across threads the load to not have more than 1024 file descriptors in your select() call, right? WRONG! read it twice, the problem is with the highest file descriptor value provided in a fd_set structure, not the number of file descriptors in the fd set.

These 2 numbers are related but are in no way the same. Let’s say you have program that opens 2000 text files (either with open() or fopen()) to read from them and scan for a list of words, at the same time each time you hit a word you must connect and send a network message to some TCP server and read data from the TCP server too. Probably you would launch some threads for reading on the files and another thread to handle the network connection related data. Event though only 1 thread is using select() and that thread is providing select() with just 1 file descriptor (the TCP server connection), you cannot guarantee which file descriptor number will be assigned to that network connection. You could try to ensure that you always start the TCP connection before opening any files so you will get a lower-number file descriptor, but that is a very shaky design, if you later want to do other stuff that requires the use of file descriptors (use pipes, unix sockets, etc) you may hit the problem again.

The limitation comes from the way select() works, most concretely the data type used to represent the list of file descriptors, fd_set. Let’s take a look at fd_set in /usr/include/sys/select.h

You will see a definition pretty much like:

typedef struct  {
    long int fds_bits[32];
} fd_set;

I removed a bunch of macros to make the code more clear. As you see you have a static array of 32 long ints. This is just a bit map, where each bit represents a file descriptor number. 32 * sizeof(long int), for 32 bit platforms is 1024. So, if you do fd_set(&fd, 10), to add file descriptor 10 to an fd_set, it will just set to 1 the 10th bit in the bit map, what happens then if you do fd_set(&fd, 2000) ?, you guessed right, unpredictable. May be some sort of array overflow (fd_set is, at least on my system, implemented using the assembly instruction btsl, bit test and set).

Be aware also, all of this is on Linux, I am not sure about how select is implemented in other operating systems, like Windows. Given that we did not notice this problem on Windows servers, probably select is implemented differently.

Solution? use poll (or may be epoll when available). These 2 system calls are not as popular and available in most operating systems as select is, but whenever possible, I recommend using poll. There may be differences in the performance, for example, using FD_ISSET() is faster (just checking if a given bit is set in the bitmap) than iterating over the poll array, but I have found in my applications that the difference is just not critical.

In short, next time you find using select(), think it twice before deciding that is what you need.

– To disable SELinux. Edit /etc/selinux/config and change the SELINUX policy from enforcing to

SELINUX=disabled

Be aware that you are disabling security “features”, whatever that means. So you may want to read this other article about disabling SELinux.

I wasn’t lucky enough to have SELinux disabled as an option. Developing with SELinux enabled is a good idea so you can notice compatibility problems with SELinux in your development environment before a customer that really *needs* SELinux enabled discovers them in your software, which, from the customer point of view is a plain bug.

The first problem I noticed after some CentOS upgrade, change the hard-drive or something along those lines, was that the command “ping” wasn’t working, it’s been a while since I had the problem so I don’t quite remember the exact error when pinging other systems, but it was most likely something like permission denied. Probably other network commands did not work either, but I could just notice ping at that moment. So, I used the good old strace to find out what was causing the problem.

The underlying ping problem was because the open() system call was failing with EACCESS when trying to open /etc/hosts. However I was able to “cat /etc/hosts”. So it wasn’t a simple permission problem, but a bit more complex SELinux problem. Eventually I found out that the solution was:

restorecon reset /etc/hosts

At which point the SELinux security context for this file got screwed up? I certainly don’t know. But that command restored it. The Z option of the ls command will show you the SELinux security context for any file.

ls -Z /etc/hosts

The second problem was that some libraries of one of the programs I am responsible for were not being loaded. Again, the problem was due to permission denied errors, this time when loading the shared libraries that required text relocation.

The solution was to recompile the shared libraries with the -fPIC option.

I am sure SELinux has its uses, however I have the feeling that sometimes makes things more complicated than needed in some environments. I recommend reading this blog post and particularly the comments in there.