Sending UDP packets from the command line

Although is pretty easy to write a perl/python script to send UDP data, I wanted to be able to send a UDP message from the command line to test some code quickly. After 15 seconds of googling I found the awesome command “sendip”. After a minute of reading the man page I got what I needed:

sendip -p ipv4 -is 192.168.1.81 -p udp -us 5070 -ud 5060 -d "Hello" -v 192.168.1.81

This should send a “Hello” message over udp to port 5060 using an ipv4 address.

Here the result:

Added 26 options
Initializing module ipv4
Initializing module udp
Finalizing module udp
Finalizing module ipv4
Final packet data:
45 00 00 21   E..!
34 E0 00 00   4...
FF 11 02 F9   ....
C0 A8 01 51   ...Q
C0 A8 01 51   ...Q
13 CE 13 C4   ....
00 0D 30 7D   ..0}
48 65 6C 6C   Hell
6F   o
Sent 33 bytes to 192.168.1.81
Freeing module ipv4
Freeing module udp

It is pretty awesome that you can build about any protocol packet for quick testing using just bash …

Posted in linux | 20 Comments

Busy November 2011

I attended 3 Linux/Telephony events in this past November 2011. All of them are focused on the Latino American community.

Find the presentations below:

– Elastix World 2011, Mexico D.F. Nov 3-4 – “Negociación de codecs en Asterisk

– FSL, Vallarta, México. Nov 5 – “FreeSWITCH – Asterisk con esteroides

– 4K Conference, Buenos Aires, Argentina, Nov 24  – “Manejo de medios en FreeSWITCH

Posted in asterisk, freeswitch, linux, voip | Leave a comment

Thanks, Wikipedia

I use Wikipedia all the time. Not directly, but an awful lot of the stuff I search in Google ends up being a Wikipedia web page. If I can spend 10 bucks a month in Netflix, 100 or more in TV/internet etc, why not spend 50 a year in supporting a piece of the internet that helps me do my work better? So I donated, and so should you! Thanks Wikipedia.

Support Wikipedia

Posted in Uncategorized | Leave a comment

Linux Core Dumps

Note that most of the theory (particularly the low level segmentation fault details) are also valid for Windows platforms and other operating systems. The commands to configure core dumps and retrieve them are Linux specific though. I also assume that the program you are trying to debug is “FreeSWITCH”, but you can easily change the program name to the one is misbehaving in your case and you should be fine.

What is a core dump?

Sometimes problems with FreeSWITCH, Asterisk or just about any other program in Linux are hard to debug by just looking at the logs. Sometimes the process crashes and you don’t have a chance to use the CLI to look at stats. The logs may not reveal anything particularly interesting, or just some minimal information. Sometimes you need to dig deeper into the state of the program to poke around and see what is going on inside. Linux process core dumps are meant for that.

The most typical case for a core dump is when a process dies violently and unexpectedly. For example, if a programmer does something like this:

*(int *)0 = 0;

Is usually not that straight forward, it may be the that the programmer did not expect certain variable to contain a NULL (0) value. The process is killed by the Linux kernel because by default, a Linux process does not map the memory address 0 to anything. This causes a page fault in the processor which is trapped by the kernel, the kernel then sees that the given process does not have anything mapped at address 0 and then sends the SIGSEGV (Unix signal) to the process. This is called a segmentation fault.

The default signal handler for SIGSEGV dumps the memory of the process and kills the process. This memory dump contains all the memory for this process (and all the threads belonging to that process) and that is what is used to determine what went wrong with the program that attempted to reference an invalid address (0 in this case, but any invalid address can cause it).

You can learn more about it here: http://en.wikipedia.org/wiki/Segmentation_fault

How can I make sure the core dump will be saved?

Each process has a limit for how big this core can be. If the limit is exceeded no core dump will be saved. By default this limit is 0!, that means no core will be dumped by default.

Before starting the process you must use “ulimit“. The “ulimit” command sets various limits for the current process. If you execute it from the bash shell, that means the limits are applied to your bash shell process. This also means any processes that you start from bash will inherit those limits (because they are child processes from your bash shell).

ulimit -a

That shows you all the limits for your bash shell. In order to guarantee that a core will be dumped you must set the “core file size” limit to “unlimited”.

ulimit -c unlimited

If you are starting the process from an init script or something like that, the init script has to do it. Some programs are smart enough to raise their limits themselves, but is always better to make sure you have unlimited core file size for your bash shell. You may then want to add those ulimit instructions inside your $HOME/.bashrc file.

Where is the core dump saved?

Each process has a “working directory”. See http://en.wikipedia.org/wiki/Working_directory

That is where the process core dump will be saved by default. However, some system-wide settings affect where the core is dumped.

“/proc/sys/kernel/core_pattern” and “/proc/sys/kernel/core_uses_pid” are 2 files that control the base file name pattern for the core, and whether the core name will be appended with the PID (process ID).

The recommended settings are:

mkdir -p /var/core
echo "/var/core/core" > /proc/sys/kernel/core_pattern
echo 1 > /proc/sys/kernel/core_uses_pid

You can confirm what you just did with:

cat /proc/sys/kernel/core_pattern
 
cat /proc/sys/kernel/core_uses_pid

This settings will cause any process in the system that crashes, to dump the core at:

/var/core/core.<pid>

What if I just want a core dump without killing the process?

In some situations, if a process becomes unresponsive or the response times are not ideal. For example, you try to execute CLI commands in Asterisk or FreeSWITCH but there is no output, or worst, your command line prompt gets stuck. The process is still there, it may be even processing calls, but some things are taking lot of time or just don’t get done. You can use “gdb” (The GNU debugger) to dump a core of the process without killing the process and almost with no disruption of the service. I say almost because for a large process, dumping a core may take a second or two, in that time the process is freezed by the kernel, so active calls may drop some audio (if you’re debugging some real time audio system like Asterisk or FreeSWITCH).

The trick to do it fast is to first create a file with the GDB commands required to dump the core.

Latest versions of CentOS include (with the gdb RPM package) the “gcore” command to do everything for you. You only need to execute:

gcore $(pidof freeswitch)

To dump the core of the running process.

If you are in a system that does not include gcore, you can do the following:

echo -ne "generate-core-file\ndetach\nquit" > gdb-instructions.txt

The 3 instructions added to the file are:

generate-core-file
detach
quit

This will do exactly what we want. Generate the core file for the attached process, then detach from the process (to let it continue) and then quit gdb.

You then use GDB (you may need to install it with “yum install gdb”) to attach to the running process, dump the core, and get out as fast as possible.

gdb /usr/local/freeswitch/bin/freeswitch $(pidof freeswitch) -x gdb-instructions.txt

The arguments to attach to the process include the original binary that was used to start it and the PID of the running process. The -x switch tells GDB to execute the commands in the file after attaching.

The core will be named core.<pid> by default and the path is not affected by the /proc/sys/kernel settings of the system.

This core can now be used by the developer to troubleshoot the problem.

Sometimes though, the developer will be more interested in a full back trace (stack trace), because the core dump itself can’t be easily examined in any other box than the one where was generated, therefore it might be up to you to provide that stack trace, which you can do with:

gdb /usr/local/freeswitch/bin/freeswitch core.<pid>
(gdb) set logging file my_back_trace.txt
(gdb) thread apply all bt full
(gdb) quit

Then send the file my_back_trace.txt to the developer, or analyze it yourself, sometimes is easy to spot the problems even without development experience!

Posted in asterisk, freeswitch, linux | 2 Comments

Cluecon 2011 – By Developers For Developers

I am back from ClueCon once again. I’m getting lazy, I used to post before going.

Anyways, the topic of my session this time was media handling in FreeSWITCH. We’ll be doing some work in in that area at Sangoma in the following months so I thought I’d kill two birds with one stone.

 

 

 

Presentation here:
PPT: http://www.moythreads.com/congresos/cluecon2011/media-handling-freeswitch-final.ppt

PDF: http://www.moythreads.com/congresos/cluecon2011/media-handling-freeswitch-final.pdf

Posted in freeswitch, voip | Leave a comment

Wanpipemon cookies: ISDN pcap traces

PRI and BRI telephony links use the Q.931 and Q.921 protocols in its respective D-channel for call control and link reliability respectively.

It is sometimes required to analyze protocol messages in a given link in order to troubleshoot call problems. Wanpipemon makes possible to create a pcap file with all D-channel data which can then be examined using Wireshark.

What makes it so convenient is that this method is independent from the application you’re using (Asterisk, FreeSWITCH or whatever). You basically have the capability to tap into the D-channel, pretty much like using tcpdump to monitor TCP/IP traffic.

wanpipemon -i w1g1 -pcap -pcap_file isdn.pcap -prot ISDN -full -systime -c trd

You can refer to the Sangoma Wiki for other details (and SS7 tracing commands too): http://wiki.sangoma.com/wanpipe-wireshark-pcap-pri-bri-wan-t1-e1-tracing

Next time your D-channel link does not come up, fire up wanpipemon and take a look in Wireshark to what’s going on!

Posted in linux, wanpipemon | Leave a comment

Telephony Stack Exchange Q&A Site

A nice proposal for a telephony Q&A site has been done in Stack Exchange. If you’re involved in telephony go show your support for the proposal!

http://area51.stackexchange.com/proposals/12932/telephony

Posted in voip | Leave a comment

GIT hooks

I started using git hooks to update a clone repository with git pull just after every push to the repository. It was a bit of a pain to get it to work. The hook kept complaining about “fatal: Not a git repository: ‘.'”

I was clearly able to switch to the correct directory with cd /path/to/repo && git pull, but it kept failing.

It seems git hooks run with some git environment variables like GIT_DIR that affect the new instance of git being spawned. The easiest solution I found was to specify –git-dir when doing the pull:

cd /path/to/repo
git --git-dir=/path/to/repo/.git pull

And finally I could move on after being stuck like half an hour 🙁

Posted in git, linux | Leave a comment

Wanpipemon cookies: Checking Sangoma FXO status

This is the first post in what I expect to be a long series of posts called “Wanpipemon cookies”. They aim to be small and easy to swallow posts with tips on how to use “Wanpipemon”, the debugging tool provided with Sangoma Wanpipe drivers.

This time I’ll show you how to check whether an FXO line is plugged into the the card via the command line. You can also check if the FXO is OnHook (http://en.wikipedia.org/wiki/On-hook) or OffHook (http://en.wikipedia.org/wiki/Off-hook)

In order to check the analog status for a given analog channel in a Sangoma card you can do the following:

# wanpipemon -i w1g1 -c astats -m 1

The -i option is used in most commands to specify the wanpipe interface in which will be run. You can find your wanpipe interfaces using ifconfig, normally they are named in the form “wXg1”.

The -c option is used to specify the command to run in the given interface.

The -m is used to specify the analog channel.

Typical output for an FXO channel not connected will be:

root@sigchld:/home/moy # wanpipemon -i w1g1 -c astats -m 4

        ------- Voltage Status  (FXO,port 3) -------

VOLTAGE : 1 Volts

        ------- Line Status  (FXO,port 3) -------

Line    : disconnected

The voltage comes handy if you want to check if the line is OnHook or OffHook. The typical voltage when the line is connected and OnHook is around 53, 53 volts. When the line is OffHook and the circuit is closed, the voltage drops to 8, 9 volts.

Even when the output of the command tells you if the line is connected or not, you can also appreciate that the voltage is very low when the line is not connected (1 volt is reported in this case).

Posted in wanpipemon | Leave a comment

OpenR2 1.3.1 released

I just released openr2 1.3.1
(http://openr2.googlecode.com/files/openr2-1.3.1.tar.gz)

The binary compatibility report is at
http://www.libopenr2.org/reports/abi_report_1.3.0_to_1.3.1.html

The most significant changes are:

– Proper DTMF R2 support, including new timers to tweak the end of DTMF timeout.
– Indonesian variant
– Test for lib64 directory for installation in 64 bit
– Fixed some build issues including support for Linux/Sparc

The next week I will also be releasing openr2 2.0.0 which breaks API
and ABI compatibility with openr2 1.x series. Releases 1.x have proven
to be stable now and they will enter maintenance mode (no new variants
nor features will be added there).

Releases 2.x will be used for future Asterisk versions (1.10??) and
for FreeSWITCH and Windows support.

Posted in openr2 | 4 Comments