Monitor remote service via LibreNMS

I wanted to monitor with LibreNMS if transmission-daemon process is running . On the remote host I had to install https://www.monitoring-plugins.org/ due check_procs command

Test monitoring command

$ cd /usr/lib/monitoring-plugins
$ ./check_procs -c 1:1 -a transmission-daemon
PROCS OK: 1 process with args 'transmission-daemon' | procs=1;;1:1;0;

Also on the remote host I had to install LibreNMS agent Check_MK which pushes data to LibreNMS server. Had to download and configure check_mrpe

wget https://raw.githubusercontent.com/librenms/librenms-agent/master/agent-local/check_mrpe
mv check_mrpe /usr/lib/check_mk_agent/local
chmod +x /usr/lib/check_mk_agent/local/check_mrpe
echo "transmission /usr/lib/monitoring-plugins/check_procs -c 1:1 -a transmission-daemon" > /etc/check_mk/mrpe.cfg

To monitor transmission service I had to Add service in LibreNMS service, select type mrpe and fill parameters

Simple SSH login email notify

Wanted to track remote SSH port forward login and the following is a great solution

	$ cat /etc/profile.d/ssh-login-notify.sh
	if [ -n "$SSH_CLIENT" ]; then
	TEXT="$(date): ssh login to ${USER}@$(hostnamectl –static)"
	TEXT="$TEXT from $(echo $SSH_CLIENT|awk '{print $1}')"
	echo $TEXT|mail -s "ssh login" info@example.com
	fi

view raw

ssh-login-notify.sh

hosted with ❤ by GitHub

Source https://askubuntu.com/a/702743/168459

Monitor progress of cp, mv, gzip, mysql and CLI commands

To monitor progress of an ongoing copy or mysql dump import you might try progress CLI tool. Track progress of commands like cp, mv, dd, tar, cat, rsync, grep, fgrep, egrep, cut, sort, md5sum, sha1sum, sha224sum, sha256sum, sha384sum, sha512sum, adb, gzip, gunzip, bzip2, bunzip2, xz, unxz, lzma, unlzma, 7z, 7za, zcat, bzcat, lzcat, split, gpg, etc.

Hikvision NVR camera stream watching on LG TV

LG Smart World contains the Smart IPTV app

What’s needed is creating a m3u playlist file with the RTSP stream URLs

#EXTM3U
#EXTINF:0,group-title="Kamera",Hatso resz
rtsp://admin:PASSWORD@192.168.88.3:6911/Streaming/Channels/101
#EXTINF:0,group-title="Kamera",Utca
rtsp://admin:PASSWORD@192.168.88.3:6911/Streaming/Channels/201

The m3u file can be uploaded via https://siptv.app/mylist/. You could upload it on a shared web hosting or on your own public web server that can realtime provide updated m3u file.

Tried mitmproxy & mitmweb

Today I came by https://mitmproxy.org/ and gave it a shot just to know what it is capable of. For example for REST API communication debugging. The mitmweb is in beta, but looks good and also worked for me 🙂

The best is that they provide a docker image so trying it out was really simple.

1. Just run docker run --rm -it -p 8080:8080 -p 8081:8081 --net="host" mitmproxy/mitmproxy mitmweb
2. Open http://127.0.01:8081
3. I changed Firefox to use proxy at 127.0.0.1 and port 8080 and browsed the web with mitmproxy intercepting the HTTP communication

Tinkering with GNU parallel and wget for broken link checking

Finally found a parallel spidering solution. Online solutions didn’t really fit, because I don’t want to overload the production site and they can’t reach http://localhost. Trying out parallel + wget snippet from https://www.gnu.org/software/parallel/man.html#EXAMPLE:-Breadth-first-parallel-web-crawler-mirrorer looks promising.

#!/bin/bash

URL=$1
 # Stay inside the start dir
 BASEURL=$(echo $URL | perl -pe 's:#.*::; s:(//.*/)[^/]*:$1:')
 URLLIST=$(mktemp urllist.XXXX)
 URLLIST2=$(mktemp urllist.XXXX)
 SEEN=$(mktemp seen.XXXX)

# Spider to get the URLs
 echo $URL >$URLLIST
 cp $URLLIST $SEEN

while [ -s $URLLIST ] ; do
 cat $URLLIST |
 parallel lynx -listonly -image_links -dump {} \; \
 wget -qm -l1 -Q1 {} \; echo Spidered: {} \>\&2 |
 perl -ne 's/#.*//; s/\s+\d+.\s(\S+)$/$1/ and
 do { $seen{$1}++ or print }' |
 grep -F $BASEURL |
 grep -v -x -F -f $SEEN | tee -a $SEEN > $URLLIST2
 mv $URLLIST2 $URLLIST
 done

rm -f $URLLIST $URLLIST2 $SEEN

Great exercise for the CPUs

When the command finishes then the next step is parsing access_log

grep -r ' 404 ' /var/log/httpd/access_log | cut -d ' ' -f 7 | sed -r 's/^\//http\:\/\/localhost\//g'

First experience with RAID0

In my new desktop PC I have Intel Rapid Storage technology so gave it a try. Bought a second Kingston V300 SSD to put it in RAID0.

Intel Rapid Storage technology

I was curious about the performance with and without RAID0 of the SSDs. So I run hdparm -tT on them.
The results:

Kingston SSD performance

The old SSD slowed down the RAID array so I sold it and installed OS on the new SSD, because it was 3x faster and I didn’t need the extra space with two SSDs in RAID.
Unfortunately I don’t remember the speed of the old SSD when I bouht it. Now I saved the new ones to be able to compare after some years.
It’s mindblowing how different/super fast is my PC with the new SSD.

Check out this awesome article about swapping https://rudd-o.com/linux-and-free-software/tales-from-responsivenessland-why-linux-feels-slow-and-how-to-fix-that

For disk benchmarking on Linux checkout https://wiki.archlinux.org/index.php/Benchmarking/Data_storage_devices

Browser memory usage

I was curious why did Chromium “eat” my system memory and so I checked chrome://chrome-urls/ to see what’s there for memory usage debugging.

Saw chrome://memory-internals/ and checked it out. There’s the proof for browser tab and extension usage.

Chrome memory usage data from chrome://memory-internals

Tried closing and reopening some tabs, but before that I typed free -h to check memory usage.

$ free -h
              total        used        free      shared  buff/cache   available
Mem:           3.9G        1.9G        1.3G         53M        682M        1.8G
Swap:          2.8G        276M        2.5G
$ free -h
              total        used        free      shared  buff/cache   available
Mem:           3.9G        1.9G        1.3G         53M        684M        1.9G
Swap:          2.8G        275M        2.5G
$ free -h
              total        used        free      shared  buff/cache   available
Mem:           3.9G        2.0G        1.2G         58M        694M        1.7G
Swap:          2.8G        273M        2.5

The numbers were not lying. So if you working with a lot of tabs and extensions more RAM will benefit your system.

Similiar goodies are available also in Firefox. At http://kb.mozillazine.org/About_protocol_links you can see a list of internal about:* sites of Firefox.

For memory usage visit about:memory and click Measure

Firefox memory usage

Enable MySQL file logging

Wanted to know what queries are executed so I had to enable it for MariaDB.

SET GLOBAL general_log = 'ON';
SET GLOBAL general_log_file = '/var/log/mysql.log';
SET GLOBAL slow_query_log = 'On';
SET GLOBAL slow_query_log_file = '/var/log/mysql-slow.log';

To see the changes:

SHOW VARIABLES LIKE 'general_log';
SHOW VARIABLES LIKE 'slow_query_log';

logrotate error: stat of /var/log/xferlog failed

In Arch Linux the logrotate service was failing:

user@host# systemctl start logrotate.service 
Job for logrotate.service failed. See "systemctl status logrotate.service" and "journalctl -xe" for details.

So I ran it by hand to debug:

user@host# logrotate /etc/logrotate.conf

And the following error appeared:

logrotate error: stat of /var/log/xferlog failed

I fired up a grep for xferlog in the /etc directory:

user@host:/etc# grep -r xferlog *
logrotate.d/proftpd:/var/log/xferlog

Solution was commenting the xferlog rule in logrotate.d/proftpd. I don’t need the transfer log.