How to debug core file using gdb in linux ?

Here we will look at the process how to debug a crash dump in Linux using gdb.

What all tools are required and how to start debugging the core file generated.

1. Install GNU Debugger GDB

[root@nglinux ~]# yum install gdb
Failed to set locale, defaulting to C
Loaded plugins: fastestmirror, refresh-packagekit, security
Setting up Install Process
Loading mirror speeds from cached hostfile
epel/metalink                                                                                                                |  17 kB     00:00     
 * base: mirror.riverfrontnetworks.com
 * epel: ewr.edge.kernel.org
 * extras: mirror.team-cymru.com
 * updates: mirror.metrocast.net
base                                                                                                                         | 3.7 kB     00:00     
base-debuginfo                                                                                                               | 2.5 kB     00:00     
base-debuginfo/primary_db                                                                                                    | 1.8 MB     00:05     
extras                                                                                                                       | 3.3 kB     00:00     
updates                                                                                                                      | 3.4 kB     00:00     
updates/primary_db                                                                                                           |  14 kB     00:00     
Resolving Dependencies
--> Running transaction check
---> Package gdb.i686 0:7.2-92.el6 will be installed
--> Finished Dependency Resolution

Dependencies Resolved

====================================================================================================================================================
 Package                          Arch                              Version                                 Repository                         Size
====================================================================================================================================================
Installing:
 gdb                              i686                              7.2-92.el6                              base                              2.3 M

Transaction Summary
====================================================================================================================================================
Install       1 Package(s)

Total download size: 2.3 M
Installed size: 5.1 M
Is this ok [y/N]: y
Downloading Packages:
gdb-7.2-92.el6.i686.rpm                                                                                                      | 2.3 MB     00:02     
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : gdb-7.2-92.el6.i686                                                                                                              1/1 
  Verifying  : gdb-7.2-92.el6.i686                                                                                                              1/1 

Installed:
  gdb.i686 0:7.2-92.el6                                                                                                                             

Complete!
[root@nglinux ~]#

2. Check the core file pattern if defined in /etc/sysctl.conf or in /etc/kdmp.conf

[root@nglinux ~]# (sysctl -p; cat /etc/kdump.conf) | grep -i core
kernel.core_pattern = /var/crash/core.%e.%p.%u

To generate a sample core file, you can do following as an example:
1. On Terminal 1, run command: # top
2. On Terminal 2, kill the process using: kill -ABRT 3835

For more details refer to article: http://ngelinux.com/how-to-enable-core-dump-collection-in-linux-for-every-user/

3. Now check out the generated core file.

[root@nglinux crash]# ls -ltr
total 952
-rw-------. 1 root root 581632 Jul  9 23:59 core.0.top.3835
-rw-------. 1 root root 581632 Jul 10 00:01 core.0.top.3848
drwxr-xr-x. 2 root root   4096 Jul 10 20:51 127.0.0.1-2018-07-10-20:51:46
drwxr-xr-x. 2 root root   4096 Jul 11 20:27 127.0.0.1-2018-07-11-20:26:50
-rw-------. 1 root root 577536 Jul 18 06:07 core.top.5604.0

4. Debug the generated core file using the binary file which generated this core file.

[root@nglinux crash]# gdb `which top` /var/crash/core.top.5604.0 
GNU gdb (GDB) Red Hat Enterprise Linux (7.2-92.el6)
Copyright (C) 2010 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later 
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "i686-redhat-linux-gnu".
For bug reporting instructions, please see:
...
Reading symbols from /usr/bin/top...(no debugging symbols found)...done.
[New Thread 5604]
Reading symbols from /lib/libproc-3.2.8.so...(no debugging symbols found)...done.
Loaded symbols for /lib/libproc-3.2.8.so
Reading symbols from /lib/libncursesw.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libncursesw.so.5
Reading symbols from /lib/libc.so.6...(no debugging symbols found)...done.
Loaded symbols for /lib/libc.so.6
Reading symbols from /lib/libtinfo.so.5...(no debugging symbols found)...done.
Loaded symbols for /lib/libtinfo.so.5
Reading symbols from /lib/libdl.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libdl.so.2
Reading symbols from /lib/ld-linux.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/ld-linux.so.2
Reading symbols from /lib/libnss_files.so.2...(no debugging symbols found)...done.
Loaded symbols for /lib/libnss_files.so.2
Core was generated by `top'.
Program terminated with signal 11, Segmentation fault.
#0  0x00979424 in __kernel_vsyscall ()
Missing separate debuginfos, use: debuginfo-install procps-3.2.8-45.el6.i686
(gdb) 

Tags: corecorefiledebugfilegdbkdump.conflinuxunix