C/C++ – Moy Blog https://moythreads.com/wordpress Abandon All Hope, Ye Who Read This Blog Mon, 15 Feb 2021 22:51:26 +0000 en-US hourly 1 https://wordpress.org/?v=5.1.9 FreeSWITCH Internals Notes https://moythreads.com/wordpress/2015/11/19/freeswitch-internals-notes/ https://moythreads.com/wordpress/2015/11/19/freeswitch-internals-notes/#respond Thu, 19 Nov 2015 18:28:52 +0000 http://www.moythreads.com/wordpress/?p=304 Continue reading ]]> At Sangoma we use FreeSWITCH as a communications development platform. I decided to write some notes and guide lines on the FreeSWITCH internal architecture so our developers can use the FreeSWITCH framework effectively. I decided to publish them here as well so they’re useful to others developing modules or core enhancements for FreeSWITCH.

Core

The FreeSWITCH core is contained in the src/*.c files, everything that you see there is considered “core”. The FreeSWITCH core provides common services and protocols that are re-used among many different call control or media protocols. For example, the core provides several useful API abstractions:

  • OS abstraction based on top of the APR project (Apache Portable Runtime). There is a thin layer of abstraction (src/switch_apr.c) that prevents introducing a direct dependency between the rest of the system and the APR library. This means the rest of the system *DOES NOT* use APR directly, they use the API exposed in src/switch_apr.c for operating system abstraction primitives to create sockets, threads, manage memory etc.
  • RTP stack (src/switch_rtp.c) and an API to create rtp streams, read/write to them etc. This provides access to RTP as RTP is used by many other signaling protocols such as SIP, H.323, Megaco etc, so it makes sense to have it in the core.
  • Call switching (src/switch_ivr.c, src/switch_ivr_originate.c) API routines to create a new call (session) towards a destination without explicitly knowing the low-level protocol details. A module can therefore use switch_ivr_originate() API to place a call in SIP, H.323, SS7, PRI etc, using the same consistent API provided by the core.
  • Media playing API (src/switch_ivr_play_say.c, src/switch_ivr_say.c) that allows you to play a file, insert tones etc etc on a given session (regardless of the signaling protocol)
  • Session and channel management (src/switch_core_session.c, src/switch_channel.c) that allows to perform call control operations (answer a channel, hangup a channel) and low-level I/O operations (write a frame of media, read a frame of media) on sessions (regardless of the underlying signaling and media protocols).

fs_architecture
(Image taken from the FreeSWITCH wiki long ago, can’t recall the URL anymore)

Sessions and Channels

The core abstraction for a call leg is the switch_core_session_t opaque structure. This contains all the necessary information for a given call leg. The session contains a channel. The channel is simply a lower-level representation of the call leg that contains information about the underlying protocols and data structures used to interface with that call leg (ie, contains function pointers to functions that allow the call to be answered, hangup, provide media etc) and the state of the session.

Sessions in FreeSWITCH live in their own thread. A typical call involves 2 sessions (the inbound session and then an outbound session, both of them “bridged”). Each session lives in its own thread, looping through a finite state machine (the session channel states go from CS_INIT, last state is CS_DESTROY, there are other states such as CS_ROUTING, CS_EXCHANGE_MEDIA etc etc. These states determine what the session is doing, each of the state handlers to perform the operation (ie, hangup) is executed in the session thread. It is possible however that other threads want to send a message to indicate something, or read/write information about a session in it. In order to do this in a safe way, there are several APIs that can be used to get a locked pointer to a session that allows you to peek into the session data safely without fearing the session thread will terminate and destroy the session while you’re looking at it or doing something with it from another thread, the main core function for this is:

switch_core_session_t *switch_core_session_locate(const char *uuid);

Every session has a UUID (universal identifier) that is unique across computers and across time (at least for practical purposes). You can use that UUID to ask the core to find a session in the system with that UUID. The core will attempt to find the session (in an internal hash table) and if it’s found, it will lock it and return it to you. You can then use the session to retrieve data, set data or perform operations on it or in its underlying switch_channel_t (which is obtained through switch_core_session_get_channel()) without fear of the session thread to destroy the session (for example if someone sets the state to CS_DESTROY). You must remember to unlock the session when done, and you should do that fast enough (not more than a few milliseconds) otherwise you’re blocking other threads from accessing that session if they need to do something with it. Unlock the session with:

switch_core_session_rwunlock();

This function unlocks the session and this releases it so other threads can use it (including destroy it).

Be aware that there are other core functions that return a session locked, such as switch_core_session_get_partner(). It is your responsibility to verify if the function you are using returns a locked session and then use switch_core_session_rwunlock() to unlock it when done.
Modules

FreeSWITCH is highly modular. The FreeSWITCH engine/core source code is contained in the src/ directory. The plugins or modules source code are contained in the src/module/ directory (the only exception is mod_freetdm, which is contained at libs/freetdm/mod_freetdm, there are some legacy reasons why this was done this way, but there is talks in the FreeSWITCH open source community about moving it to the right place in src/mod/endpoints/ folder).

There are different types of modules/plugins (in the future, I’ll refer to them simply as modules). There are endpoints, formats, loggers, event handlers, dialplans, TTS engines etc etc. Pretty much every directory under src/mod/ is a type of module. Each type of module registers an “interface” with the FreeSWITCH core. Great care has been taken to make proper abstractions and do not expose “module-specific” data into the FreeSWITCH core (src/*.c) files. The modules make use of FreeSWITCH core API primitives to request core services, the FreeSWITCH core uses the abstract interface exposed by different type of modules for performing operations. For example, the module mod_sofia is an endpoint module. Therefore mod_sofia code is contained at src/mod/endpoints/mod_sofia/

FreeTDM

The FreeTDM library is a modular API that allows applications to initialize different types of TDM/Analog signaling stacks, place/receive calls and read/write media. It supports signaling modules for SS7, ISDN(PRI/BRI), Analog, MFCR2 etc. It also supports a I/O modules to support multiple different hardware manufacturers. The 2 most prominent I/O modules are src/ftmod/ftdm_wanpipe/ftmod_wanpipe.c (For Sangoma Wanpipe cards) and the src/ftmod/ftmod_zt/ftmod_zt.c (For DAHDI-enabled cards such as Sangoma and Digium).

The FreeTDM project is part of the same source tree as FreeSWITCH, however FreeTDM does not depend on FreeSWITCH. FreeSWITCH also does not depend on FreeTDM, the glue that links them together is mod_freetdm, which is an endpoint module for FreeSWITCH that allows FreeSWITCH to place calls in SS7, PRI, MFC-R2 and Analog telephony networks. Note that the mod_freetdm module is just a plugin/extension to FreeSWITCH and it is a “user” of the freetdm library. As such, you should never expose internal opaque details of freetdm to mod_freetdm or worst, to FreeSWITCH. The freetdm.h header is the main public header that is used by freetdm API users. There are other headers under a private/ folder in freetdm that are not meant to be used by the API users, only for internal use of other freetdm C files/components.

Memory Allocation

When allocating memory within FreeSWITCH you must ask yourself where to allocate from. FreeSWITCH uses APR memory pools (wrapped on the switch_memory_pool_t) data structure. You can request a completely new memory pool for yosuelf, but that may be a waste, instead ask yourself whether the memory you’re allocating will be associated to a particular session (call leg) and whether needs to persist. For example, memory for a data structure to keep track of RTCP statistics of an RTP stream, will always be associated to the RTP stream, which in turn is associated to the session. In such cases it is recommended to use the session memory pool, which is automatically destroyed at the end of the call.

In the other hand, if the memory you are allocating must persist beyond the call life cycle, you will be better off requesting a new memory pool and then allocating from it, but now you’re responsible to destroy the pool when you’re done with your object life cycle. Remember it is not possible to free memory allocated from a pool until you destroy the whole pool completely.

Some functions to remember (from src/include/switch_core.h and src/switch_core_memory.c):

switch_core_session_get_pool() – Returns the memory pool associated with a given session
switch_core_session_alloc() – Allocate x amount of bytes from a given session memory pool
switch_core_session_sprintf() – Allocates a new string with the provided format using the session memory pool
switch_core_session_strdup() – Duplicates a new string using the session memory pool
switch_core_new_memory_pool() – Creates a new pool ready to start allocating objects
switch_core_destroy_memory_pool() – Destroy the given pool, you better be sure no one will be needing any of the objects allocated from that pool
switch_core_alloc() – Allocate memory from a given memory pool
switch_safe_free() – Free memory allocated directly using malloc()/calloc(), testing for NULL pointer
For all the session-related allocation functions, you must be sure the allocated object will NOT outlive the session.

Remember all switch allocation functions already initialize memory, you do not need to call memset() after allocation

There are a few cases where it’s justifiable to use standard memory allocation functions such as strdup()

strdup() is ok when you need a quick duplicate of another string but you have no session pointer or it’s something called often and short-lived, and therefore you do not want to use the session pointer with switch_core_session_strdup() because that would leave the memory allocated until the end of the call

String Manipulation

switch_copy_string() – Safely copy a string, use this instead of strcpy() or strncpy(). It will guarantee the output is null-terminated.
switch_set_tring() – Use this to initialize a string buffer of a fixed length. DO NOT USE THIS with char* pointers as the implementation uses sizeof() and sizeof(char*) is only 4 or 8!!
switch_toupper() – Convert a string to uppercase
switch_strstr() – Find a string in a substr
switch_safe_atoi() – Turn a string into a number, testing for NULL and defaulting to a given value in case of NULL
More gems can be found in src/include/switch_utils.h, please check there before hand-coding your own function or check if you can use one from switch_utils.h instead of using direct libc functions!!

Booleans

switch_true() – Use this to test if a string is true (ie, “true”, “enabled”, “on”, etc, all are considered true), this is useful when parsing configuration files
switch_false() – Use this to test if a string is false (ie “false”, “disabled” etc), this is useful when parsing configuration files
SWITCH_TRUE / SWITCH_FALSE – Use this along with switch_bool_t instead of using integers

]]>
https://moythreads.com/wordpress/2015/11/19/freeswitch-internals-notes/feed/ 0
GDB strcmp in a core dump https://moythreads.com/wordpress/2015/06/25/gdb-strcmp-in-a-core-dump/ https://moythreads.com/wordpress/2015/06/25/gdb-strcmp-in-a-core-dump/#comments Fri, 26 Jun 2015 00:05:00 +0000 http://www.moythreads.com/wordpress/?p=286 Continue reading ]]> I tried finding an answer on google about how to do a strcmp operation in a core dump and could not find any solution. The answers available focused on using “call” to call the libc strcmp() function on a live process. Most interesting debugging for me happens on core dumps, so I decided to write my own gdb user defined command (sort of like a macro):

define gdb_strcmp
        dont-repeat
        set $result = 1 
        set $_i = 0 
        if ($arg0[0] == 0x0 && $arg1[0] != 0x0)
                set $result = 0 
        end 
        if ($arg0[0] != 0x0 && $arg1[0] == 0x0)
                set $result = 0 
        end 
        while ($result == 1 && $arg0[$_i] != 0x0 && $arg1[$_i] != 0x0)
                if ($arg0[$_i] != $arg1[$_i])
                        set $result = 0 
                end 
                set $_i = $_i + 1 
        end 
end
document gdb_strcmp
Determines if two C strings match
end

Note that gdb user commands are annoying because you don’t really have return values (they are not really functions/macros), so you have to set a global variable (yuck!) to hold the result. This macro sets $result to 0 if the strings are not equal and $1 if they are. I contemplated using the same return value than the C counterpart, but since I was interested in just a ‘yes or no’ answer I sticked to use 1 for equal and 0 for non equal.

You can then go ahead and use this macro in other macros to do useful things, such as scan a linked list and verify if a given member has certain string value.

PD. I know it’d be cleaner to start using Python for these things but I have not really looked yet into that

]]>
https://moythreads.com/wordpress/2015/06/25/gdb-strcmp-in-a-core-dump/feed/ 3
Sangoma Tapping Solution for FreeSWITCH https://moythreads.com/wordpress/2013/09/30/sangoma-tapping-solution-for-freeswitch/ https://moythreads.com/wordpress/2013/09/30/sangoma-tapping-solution-for-freeswitch/#respond Mon, 30 Sep 2013 04:33:47 +0000 http://www.moythreads.com/wordpress/?p=213 Continue reading ]]> About 4 years ago I wrote a post about Sangoma Tapping with Asterisk. Many people has been interested in that and I’ve done a few implementations with it.

Having said that, still showed some stability issues and it became a burden to maintain because it is a patch to Asterisk.

Something you may find interesting is that a bit later after I wrote the tapping feature for Asterisk, I also did it for FreeSWITCH 🙂

It has been more stable in FreeSWITCH, mostly due to the fact that FreeSWITCH TDM abstraction is modular and it has been much more easy to maintain a tapping module rather than a patch. You can find the module in FreeSWITCH’s tree at libs/freetdm/src/ftmod/ftmod_pritap/ftmod_pritap.c

In addition to the TDM tapping module, I also wrote an RTP tapping module called mod_oreka that can be used for tapping any media stream on FreeSWITCH (SIP, TDM, H.323 etc)

See press release from OrecX

 

]]>
https://moythreads.com/wordpress/2013/09/30/sangoma-tapping-solution-for-freeswitch/feed/ 0
select system call limitation in Linux https://moythreads.com/wordpress/2009/12/22/select-system-call-limitation/ https://moythreads.com/wordpress/2009/12/22/select-system-call-limitation/#comments Tue, 22 Dec 2009 21:26:26 +0000 http://www.moythreads.com/wordpress/?p=108 Continue reading ]]> Try finding a network sample code of how to accept TCP connections and most likely you will find the select() system call in such code. The reason being that select() is the most popular (but not the only one as we will see) system call to wait for I/O in a list of file descriptors.

I am here to warn you, select() has some important limitations to be aware of. I must confess I used select for a long time without realizing its limitations, until, of course, I hit the limits.

About a year ago I started porting a heavily threaded networking real-time voice application for Windows to Linux. When the code was compiling and running apparentely without issues, we started doing scalability and stress tests. We used 32 telephony E1 cards (pretty much like a network card) where each E1 port can handle up to 30 calls. So we’re talking about 32 * 30 = 960 calls in a single server. Knowing in advance that I would need lots of file descriptors (Linux typically defaults to 1024 per process), we used the setrlimit() system call to increase the limit up to 10,000 which should be more than enough because a telephony call in this system requires about 4 file descriptors (for the network VoIP connection and the E1 side devices etc).

At some point during the stress test, calls stopped working and some threads were going crazy eating up 99% CPU. After finding out using “ps” and “pstack” which threads were the ones going crazy, I found out that were the ones waiting for I/O in some file descriptors using select(), like the embedded HTTP server or the Network-related code.

Reading carefully the select documentation you will find the answer by yourself. “man select” says:

“An fd_set is a fixed size buffer. Executing FD_CLR or FD_SET with a value of fd that is negative or is equal to or larger than FD_SETSIZE will result in undefined behavior.”

So, big deal you may say, you can split across threads the load to not have more than 1024 file descriptors in your select() call, right? WRONG! read it twice, the problem is with the highest file descriptor value provided in a fd_set structure, not the number of file descriptors in the fd set.

These 2 numbers are related but are in no way the same. Let’s say you have program that opens 2000 text files (either with open() or fopen()) to read from them and scan for a list of words, at the same time each time you hit a word you must connect and send a network message to some TCP server and read data from the TCP server too. Probably you would launch some threads for reading on the files and another thread to handle the network connection related data. Event though only 1 thread is using select() and that thread is providing select() with just 1 file descriptor (the TCP server connection), you cannot guarantee which file descriptor number will be assigned to that network connection. You could try to ensure that you always start the TCP connection before opening any files so you will get a lower-number file descriptor, but that is a very shaky design, if you later want to do other stuff that requires the use of file descriptors (use pipes, unix sockets, etc) you may hit the problem again.

The limitation comes from the way select() works, most concretely the data type used to represent the list of file descriptors, fd_set. Let’s take a look at fd_set in /usr/include/sys/select.h

You will see a definition pretty much like:


typedef struct  {
    long int fds_bits[32];
} fd_set;

I removed a bunch of macros to make the code more clear. As you see you have a static array of 32 long ints. This is just a bit map, where each bit represents a file descriptor number. 32 * sizeof(long int), for 32 bit platforms is 1024. So, if you do fd_set(&fd, 10), to add file descriptor 10 to an fd_set, it will just set to 1 the 10th bit in the bit map, what happens then if you do fd_set(&fd, 2000) ?, you guessed right, unpredictable. May be some sort of array overflow (fd_set is, at least on my system, implemented using the assembly instruction btsl, bit test and set).

Be aware also, all of this is on Linux, I am not sure about how select is implemented in other operating systems, like Windows. Given that we did not notice this problem on Windows servers, probably select is implemented differently.

Solution? use poll (or may be epoll when available). These 2 system calls are not as popular and available in most operating systems as select is, but whenever possible, I recommend using poll. There may be differences in the performance, for example, using FD_ISSET() is faster (just checking if a given bit is set in the bitmap) than iterating over the poll array, but I have found in my applications that the difference is just not critical.

In short, next time you find using select(), think it twice before deciding that is what you need.

]]>
https://moythreads.com/wordpress/2009/12/22/select-system-call-limitation/feed/ 9
I hate SELinux https://moythreads.com/wordpress/2009/12/17/i-hate-selinux/ https://moythreads.com/wordpress/2009/12/17/i-hate-selinux/#comments Thu, 17 Dec 2009 06:08:23 +0000 http://www.moythreads.com/wordpress/?p=98 Continue reading ]]> I am not a security-savvy person, even though I know pretty well how to code defensively to avoid security issues in my C code, my security knowledge in a Linux system is pretty average (use firewall, do not run services as root etc). No wonder I really don’t know how to use or configure SELinux. But there is one thing I know about it. It can be a pain in the ass sometimes. This blog post is to describe 2 problems I have faced with SELinux. The easy solution would have been to disable SELinux. So I’ll start by showing you how to disable it in case you don’t want to mess with it at all.

– To disable SELinux. Edit /etc/selinux/config and change the SELINUX policy from enforcing to

SELINUX=disabled

Be aware that you are disabling security “features”, whatever that means. So you may want to read this other article about disabling SELinux.

I wasn’t lucky enough to have SELinux disabled as an option. Developing with SELinux enabled is a good idea so you can notice compatibility problems with SELinux in your development environment before a customer that really *needs* SELinux enabled discovers them in your software, which, from the customer point of view is a plain bug.

The first problem I noticed after some CentOS upgrade, change the hard-drive or something along those lines, was that the command “ping” wasn’t working, it’s been a while since I had the problem so I don’t quite remember the exact error when pinging other systems, but it was most likely something like permission denied. Probably other network commands did not work either, but I could just notice ping at that moment. So, I used the good old strace to find out what was causing the problem.

The underlying ping problem was because the open() system call was failing with EACCESS when trying to open /etc/hosts. However I was able to “cat /etc/hosts”. So it wasn’t a simple permission problem, but a bit more complex SELinux problem. Eventually I found out that the solution was:

restorecon reset /etc/hosts

At which point the SELinux security context for this file got screwed up? I certainly don’t know. But that command restored it. The Z option of the ls command will show you the SELinux security context for any file.

ls -Z /etc/hosts

The second problem was that some libraries of one of the programs I am responsible for were not being loaded. Again, the problem was due to permission denied errors, this time when loading the shared libraries that required text relocation.

The solution was to recompile the shared libraries with the -fPIC option.

I am sure SELinux has its uses, however I have the feeling that sometimes makes things more complicated than needed in some environments. I recommend reading this blog post and particularly the comments in there.

]]>
https://moythreads.com/wordpress/2009/12/17/i-hate-selinux/feed/ 2
Quick tip for debugging deadlocks https://moythreads.com/wordpress/2009/09/27/quick-tip-for-debugging-deadlocks/ https://moythreads.com/wordpress/2009/09/27/quick-tip-for-debugging-deadlocks/#comments Sun, 27 Sep 2009 04:34:14 +0000 http://www.moythreads.com/wordpress/?p=84 Continue reading ]]> If you ever find yourself with a deadlock in your application, you can use gdb to attach to the application, then sometimes you find one of the threads that is stuck trying to lock mutex x. Then you need to find out who is currently holding x and therefore deadlocking your thread (and equally important why the other thread is not releasing it).

At least on recent libc implementations in Linux, the mutex object seems to have a member named “__owner”. Let me show you what I recently saw when debugging a deadlocked application.

(gdb) f 4
#4  0x0805ab46 in ACE_OS::mutex_lock (m=0xa074248) at include/ace/OS.i:1406
1406      ACE_OSCALL_RETURN (ACE_ADAPT_RETVAL (pthread_mutex_lock (m), ace_result_),
(gdb) p *m
$8 = {__data = {__lock = 2, __count = 0, __owner = 17828, __kind = 0, __nusers = 1, {__spins = 0, __list = {__next = 0x0}}},
  __size = "\002\000\000\000\000\000\000\000�E\000\000\000\000\000\000\001\000\000\000\000\000\000", __align = 2}

We can see that the __owner is 17828. This number is the LWP (Light-weight process) id of the thread holding the lock. Now you can go to examine that thread stack and find out why that thread is also stuck.

This example also brings up a regular point of confusion for some Linux application developers. What is the difference between LWP and POSIX thread id ( the pthread_t type in pthread.h)?. The difference is that pthread_t is a user space concept, is simply an identifier for the thread library implementing POSIX threads to refer to the thread and its resources, state etc. However the LWP is an implementation detail of how the Linux kernel implements threads, which is done through the “thread group” concept and LWP’s, that are processes that share memory pages and other resources with the other processes in the same thread group.

From the Linux kernel point of view the pthread_t value doesn’t mean anything, the LWP id is how you identify threads in the kernel, and they share the same numbering as regular processes, since LWPs are just a special type of process. Knowing this is useful when using utilities like strace. When you want to trace a particular thread of a multi threaded application, you need to provide the LWP of the thread you want to trace, a common mistake is to provide the process id, which in a multithreaded application the process id is just the LWP of the first thread in the application (the one that started executing main()).

Here is how you get each identifier in a C program:

#include 
#include 
#include 

int main()
{
  pthread_t tid = pthread_self();
  int sid = syscall(SYS_gettid);
  printf("LWP id is %d\n", sid);
  printf("POSIX thread id is %d\n", tid);
  return 0;
}

It’s important to note that getting the POSIX thread id is much faster than the LWP, because pthread_self() is just a library call and libc most likely has this value cached somewhere in user space, no need to go down to the kernel. As you can see, getting the LWP requires a call to the syscall() function, which effectively executes the requested system call, this is expensive (well, compared with the time required to enter a simple user space function).

]]>
https://moythreads.com/wordpress/2009/09/27/quick-tip-for-debugging-deadlocks/feed/ 1
New Project – Sangoma Bridge https://moythreads.com/wordpress/2009/09/09/new-project-sangoma-bridge/ https://moythreads.com/wordpress/2009/09/09/new-project-sangoma-bridge/#comments Wed, 09 Sep 2009 21:31:40 +0000 http://www.moythreads.com/wordpress/?p=64 Continue reading ]]> A couple of months ago I wrote a little application for Regulus Labs. The application is a simple daemon bridge between Sangoma E1 devices receving ISDN PRI calls and a TCP IP server. Everything received on the telephony side was simply bridged to the configured TCP IP server. The bridge supports PRI voice calls and V.110 data calls.

Even when the application is simple in nature, learning about V.110 to get it to work was interesting 🙂

Today I made the project public ( thanks to Tzury Bar Yochay from Regulus Labs) in google code:

http://code.google.com/p/sbridge/

Hopefully somebody else will find it useful.

]]>
https://moythreads.com/wordpress/2009/09/09/new-project-sangoma-bridge/feed/ 2
Debugging information in separate files https://moythreads.com/wordpress/2009/08/31/debugging-information-in-separate-files/ https://moythreads.com/wordpress/2009/08/31/debugging-information-in-separate-files/#comments Mon, 31 Aug 2009 04:00:54 +0000 http://www.moythreads.com/wordpress/2009/08/31/debugging-information-in-separate-files/ Continue reading ]]> Debugging information in Linux ELF binaries is usually stored in the binary itself. This had been really convenient to me, for example, I always compile my openr2 library with -ggdb3 -O0. I don’t care about optimizations nor the increase in size in the binary and users can always change those flags using CFLAGS when configuring openr2. Is convenient because if my users ever get a core dump, I was able to jump right in and get a useful backtrace and examine the stack. Alternatively they could get the stack trace themselves and send it to me without worrying about anything else than launching gdb with the right arguments.

However, when you ship non-open source software or you’re just concerned with the size of all the debugging information in lots of libraries, you want to separate the debugging information from the binary holding the program/library itself. In Windows this is the default behavior you get with the well known PDB (Program Data Base) files. For Linux though, you need some tricks to get the debugging information separate. This is of course what most distributions do, they include an extra package with debugging information, so when you install a package you get just the binary code, then, if you need to debug it you download the debugging package.

If you ever need this, you can follow the instructions in this web page to get it to work:

http://sources.redhat.com/gdb/current/onlinedocs/gdb_17.html#SEC166

The way I solved it for our internal build system is just to always compile with -ggdb3 and then:

1. Create a copy of the debugging symbols in a separate binary

objcopy --only-keep-debug somelibrary.so somelibrary.so.dbg 

2. Remove the debugging information from the code binary.

objcopy --strip-debug somelibrary.so

3. Add a reference to the code binary so gdb knows where to look for the debugging information

objcopy --add-gnu-debuglink somelibrary.so.dbg somelibrary.so

This last step is simply putting a file name reference inside the ELF binary so GDB (or some other debugger) knows which file name will have the debugging information for this .so (or an executable if that’s what you’re building). In the red hat web page more advanced techniques are explained to make sure you don’t end up with a version mismatch between the debugging information and your library or executable.

]]>
https://moythreads.com/wordpress/2009/08/31/debugging-information-in-separate-files/feed/ 1
A Tale of Two Bugs https://moythreads.com/wordpress/2008/05/25/a-tale-of-two-bugs/ Sun, 25 May 2008 05:05:11 +0000 http://www.moythreads.com/wordpress/2008/05/25/a-tale-of-two-bugs/ Continue reading ]]> It was the best of times, it was the worst of times, it was the age of wisdom, it was the age of foolishness, it was the age of bug-hunting!

Recently I fixed 2 bugs, yeah, I know I spent a lot of time fixing bugs but this 2 were quite interesting to me, not because of the bugs itself, but rather because of some stuff I learned in the process like the implementation of variadic functions and how the C++ compiler optimizes certain stuff unveiling odd bugs.

Bug 1

Let’s analyze the first one, it was a bug I had with some Unicall R2 installation in 64 bits. The problem was simple, as soon as I loaded chan_unicall.so Asterisk crashed 🙂

After running Asterisk with gdb I found the crash happened inside libc function strlen that was being called by uc_log(), the Unicall logging function. As most logging C functions, uc_log is a variadic function. uc_log does not do any complicated stuff, is mostly just a wrapper to vsnprintf and the variable arguments were just passed on to vsnprintf and there the crash was occurring, so, how can one see the arguments a variadic function receives using gdb? First, one must know how variadic functions are implemented by the compiler and platform you are working on.

Most common implementation of variadic functions in C is just to define va_list as an unsigned char* pointing to the last argument of the function and each call to va_arg() retrieves the next chunk of memory of the specified size and increment va_list to point to the start of the next argument, therefore, displaying arguments is just matter of printing the memory area after the last argument. However, AMD64 has a different implementation, va_list is an array of 1 structure with members:

.gp_offset
.fp_offset
.overflow_arg_area
.reg_save_area

gp_offset is how many bytes after reg_save_area the first argument is. To print the first variable argument that we know is an “int” we do:

(gdb) p *(int *)(((char *)arg_ptr[0].reg_save_area)+arg_ptr[0].gp_offset)

however, gp_offset will be only incremented after calling va_arg() macro, if you want to see more arguments you must increment reg_save_area by the number of bytes you know arguments take, in the case of uc_log, initial value of gp_offset is 24, probably because it receive 3 fixed arguments (8 bytes * 3). So, the first variable argument starts at .reg_save_area + 24, the second at .reg_save_area + 32 (we’re in a 64 bit machine).

So, what about .fp_offset and .overflow_arg_area?, well, it seems .reg_save_area is quite limited (possibly limited by the number of the processor registers) and you can never go beyond .gp_offset == 40, therefore that will only work for up to 6 arguments (including the fixed ones). .overflow_arg_area is used for any subsequent argument and .fp_offset is the pointer to the next argument on that memory area. Well, that’s enough, let’s get straight to the point, the crash was caused because unicall.h include the following prototype:

extern const char *uc_statet2str(int state);

That function returned the value passed to uc_log(…., uc_state2str()) … so what’s the issue? well, read once again the prototype and how uc_log used it. Is not a typo here in my blog, the prototype really is uc_statet2str, and the function call is uc_state2str, indeed there is a typo in the header file causing the compiler to default to the return value “int” and not const char* when compiling libmfcr2, for 64 bit platform there is 4 bytes of difference between char* and int causing a crash due to invalid memory read.

Bug 2

This one is easier to explain with a chunk of code, can you tell what’s wrong with it and what possible outputs will have when running it as “./test t”?


#include <stdio.h>
#include <string.h>

#define SIZE 100

int main(int argc, char *argv[])
{

        char *bufptr = NULL;
        if (argc == 2) {

                char inblock_buff[SIZE];
                bufptr = inblock_buff;
                strcpy(bufptr, "some buffer");
        }

        printf("buffer: %s\n", bufptr);
        if (argc == 2 && argv[1][0] == 't') {

                char otherbuff[SIZE];
                otherbuff[0] = 0;
        }

        printf("buffer: %s\n", bufptr);
        return 0;
}

Indeed, the output will depend on how you compile it and even probably will depend on the compiler implementation? The thing is, that if you compile this code in Linux with gcc 4.1.2 as gcc -O2 bug.c -o bug, and then run it as ./bug t

The output is:

buffer: some buffer
buffer: some buffer

But, compiling without optimizations gcc -O0 bug.c -o bug the output is:

buffer: some buffer
buffer:

When the second if() block is optimized-out the value of the block variable inblock_buff is not overwritten and therefore bufptr remains pointing to “some buffer” and the code seems to “work”, but when -O0 the second if() block is not optimized and the bug arise, bufptr will point to char 0 printing nothing. In my particular case this buffer was the input of the keyboard of a 5250 session, hence, in some cases the keyboard input was just ignored.

]]>
IJW (It Just Works) and COM Interop https://moythreads.com/wordpress/2008/03/26/ijw-it-just-works-and-com-interop/ https://moythreads.com/wordpress/2008/03/26/ijw-it-just-works-and-com-interop/#comments Wed, 26 Mar 2008 05:05:24 +0000 http://www.moythreads.com/wordpress/2008/03/26/ijw-it-just-works-and-com-interop/ Continue reading ]]> As I promised in my last post, today I am going to give an overview about how to call C# code from C++. After googling around a bit you will find there are at least 2 well-known ways to call C# from C++.

It Just Works

The first one I want to mention is “IJW” that stands for “It Just Works”. As the name implies, there is nothing much to discuss about IJW, is pretty easy to use, all one have to do is specify the /clr switch in the Microsoft VS compiler. This switch will cause your C++ code to be compiled to IL (Intermediate Language) and therefore will run in a managed environment. Nice isn’t it? However there is a catch, even though your code will be compiled to IL, the classes are not managed, which means among other things that the CLR will not take care of the memory. This is kind of obvious, since the original code was meant to take care of the memory itself, all the /clr switch does is give the opportunity to that old C++ code to run in managed environment.

Once that we have this C++ code working on a managed environment, there is some new fun stuff we can do with it.


#using <mscorlib.dll>
#using <myassembly.dll>

#include <stdio.h>

using namespace System;
using namespace MyNameSpace;

void callSomethingInMyAssembly()
{

    Console::WriteLine("In Managed Environment!");
    MyAssemblyClass::MyStaticMethod();
}

#pragma unmanaged


void oldAndCrappyFunction()
{
    callSomethingInMyAssembly();
}

The compiler pragma directive “managed” and “unmanaged” will help you to switch from managed to unmanaged and viceversa. This way you can call managed code from your old unmanaged code. Be aware that this solution will not work when you have an old DLL that other components depend on, since the resulting DLL will be a DLL with IL code and therefore not callable from other unmanaged DLLs.

COM Interop

To use COM Interop, the managed code, in this case, a C# class needs to allocate an UUID to be used when creating a COM instance. The following code shows how to create code with an interface and class with UUIDs.

using System;

using System.Runtime.InteropServices;

namespace ManagedNameSpace 
{

/* The GUID should be generated using guidgen.exe */
[Guid("EF870CF7-DA0F-4bf7-89DD-DE21E4701E21")]
public interface IManagedComponent
{
        void ManagedMethod();
}

[Guid("687EADD7-0B02-457a-85E5-84BEF198F7BA")]

public class ManagedClass : IManagedComponent
{
        public void ManagedMethod()
        {

                Console.WriteLine("in managed environment!\n");
        }
}

}

You can create a UUID using the tool guidgen.exe. The class should be compiled with

C:\csc /target:library mycomponent.cs

That will generate a mycomponent.dll assembly. In order to other COM components to use our C# class we must register the DLL, we can do so using regasm utility:

C:\regasm mycomponent.dll /tlb:mycomponent.tlb

Now that the DLL is registered you can use it from any language that supports COM, C++ included. Before proceeding to call this C# code from C++, you may be wondering about that mycomponent.tlb file specified as argument to regasm /tlb switch. That file is known as a “type library” and is used by the COM infrastructure to save meta-data about the types involved in the exported interfaces, that way any language that supports COM can interact with any other COM component, in this case, written in C#.

Finally let’s call our C# component from C++ code.

#include <windows.h>
#include <iostream>


using namespace std;

#import <mscorlib.tlb> raw_interfaces_only

#import "dtcom.tlb" no_namespace named_guids

int main(int argc, char *argv[])
{

        IManagedComponent *component;
        HRESULT hr;

        /* COM require an initialize routine */
        CoInitialize(NULL);

        hr = CoCreateInstance(CLSID_ManagedClass, NULL, 
                        CLSCTX_INPROC_SERVER, IID_IManagedComponent, reinterpret_cast<void **>(&component));

        if ( FAILED(hr) ) {
                cerr << "Could not create component instance." << endl;

                return -1;
        } 

        cout << "Calling managed methods." << endl;
        component->ManagedMethod();

        /* COM Require to release the instance */
        component->Release();

        /* Close COM */
        CoUninitialize();

        return 0;
}

As you can see we use the CoCreateInstance COM function to create an object that references (indirectly via CCW ) to the C# object.

So, that’s it, you can learn more about COM at Wikipedia.

]]>
https://moythreads.com/wordpress/2008/03/26/ijw-it-just-works-and-com-interop/feed/ 2