Kernel Crash Dumping Mechanism on KVM Compute Resources

Kdump is a kernel crash dumping mechanism designed for saving the system's memory contents for later analysis. It is especially helpful for the purposes of debugging when you want to discover the cause of a crash. At OnApp, Kdump feature allows you to create and collect kernel crash dumps on the CloudBoot compute resources. This feature is implemented for CentOS6 KVM and CentOS7 KVM compute resources only.

Enable KDump

The feature availability is controlled by the crashkernel kernel option for the particular compute resource in its /tftpboot/pxelinux.cfg/<MACADDR> file. The crashkernel option should be added to the config file manually. 

To enable Kdump:

  1. Access the /tftpboot/pxelinux.cfg/<MACADDR> file and set the amount of memory to “crashkernel=auto”. 

    # cat /tftpboot/pxelinux.cfg/<MACADDR>
    default centos7-ramdisk-kvm
    
    label centos7-ramdisk-kvm
        kernel images/centos7/ramdisk-kvm/vmlinuz
        append initrd=images/centos7/ramdisk-kvm/initrd.img NFSNODEID=<NFSNODEID> NFSROOT=<NFSROOTIP>:/tftpboot/export/centos7/kvm CFGROOT=<CFGROOTIP>:/tftpboot/images/centos5/diskless/snapshot ADDTOBRIDGE=mgt pcie_aspm=off selinux=0 cgroup_disable=memory net.ifnames=0 biosdevname=0 NTPSERVER=<CFGROOTIP> crashkernel=auto
  2. Reboot the compute resource. 

Dump(s) are automatically generated if the compute resource crashes (with the enabled kdump). You can view the generated file in NFS resources in the /data/clb-hv-crash directory:
# ls -1 /data/clb-hv-crash
<HVIPADDR>-2018-04-16-20:32:49
<HVIPADDR>-2018-04-16-21:15:26

Note that the feature requires about 550 MB of extra RAM space on a compute resource:

  • 128 MB of RAM is reserved to capture a kernel crash dump.
  • The Kdump service creates custom initramfs/initrd image which is about 20 MB.
  • The /boot's content (kernel and initrd) requires extra 400 MB.