Trisul Network Analytics
Developer Zone

**This is an old revision of the document!** ----

====== Debugging crashes and other problems on the Trisul Probe nodes ====== Despite our best efforts with testing , sometimes the Trisul-Probes can stop for a number of reasons. This document describes how you can debug such events. ===== Where to look ===== If you find Trisul Probe has crashed, you can collect the following information before restarting. If nothing jumps out , you can contact [[https://www.trisul.org/support/|Trisul Support]] and send us the logs. ==== Log files ==== The log files on the trisul probe are stored under ''/usr/local/var/log/trisul-probe/domain0/probe0/context0'' The latest ''ns-xxx.log'' contains the most recent logs before the crash. You can also use the helper bash aliases : <code bash> source /usr/local/share/trisul-probe/trisbashrc # to change to probe log dir cd.l # to tailf probe log file tailf.ns # the latest log file cd.l ls -lrt ns* </code> When contacting Trisul Support send us the latest log file ns-xxx.log ==== dmesg : view messages from the kernel ==== Sometimes useful clues can be found in the kernel message log. For example, when trisul is stopped by an OOM - Out of Memory killer. To view dmesg output <code bash> # to dump the kernel message log dmesg -T # to view trisul messages dmesg -T | grep trisul </code> ==== Software versions ==== To get probe version ''trisul --version'' To get platform version ''uname -a'' To get installed packages on probe <code bash> # on ubuntu dpkg -l | grep trisul # on centos rpm -qa | grep trisul </code> ===== Debugging with a binary with symbols ===== If we find nothing in the Trisul and Kernel log files, we (Trisul Support) will send you a special binary image with symbols. You can then run it under ''gdb'' to help pinpoint the location of the crash. This could be a bug <note tip>We recommend you run this under the 'screen' utility, so you can detach the terminal and log off. </note> <code> # install gdb apt install gdb # run the special trisul binary we send you like this gdb --args /home/mike3/trisul -nodemon /usr/local/etc/trisul-probe/domain0/probe0/context0/trisulProbeConfig.xml -mode online_rxring (...wait...) # after it crashes - type bt on the gdb prompt gdb: bt ..output here.. send us this output </code>