Maintaining my first open source package

I just became the package maintainer of https://aur.archlinux.org/packages/desktime/

Advertisements

Tinkering with GNU parallel and wget for broken link checking

Finally found a parallel spidering solution. Online solutions didn’t really fit, because I don’t want to overload the production site and they can’t reach http://localhost. Trying out parallel + wget snippet from https://www.gnu.org/software/parallel/man.html#EXAMPLE:-Breadth-first-parallel-web-crawler-mirrorer looks promising.

#!/bin/bash

URL=$1
 # Stay inside the start dir
 BASEURL=$(echo $URL | perl -pe 's:#.*::; s:(//.*/)[^/]*:$1:')
 URLLIST=$(mktemp urllist.XXXX)
 URLLIST2=$(mktemp urllist.XXXX)
 SEEN=$(mktemp seen.XXXX)

# Spider to get the URLs
 echo $URL >$URLLIST
 cp $URLLIST $SEEN

while [ -s $URLLIST ] ; do
 cat $URLLIST |
 parallel lynx -listonly -image_links -dump {} \; \
 wget -qm -l1 -Q1 {} \; echo Spidered: {} \>\&2 |
 perl -ne 's/#.*//; s/\s+\d+.\s(\S+)$/$1/ and
 do { $seen{$1}++ or print }' |
 grep -F $BASEURL |
 grep -v -x -F -f $SEEN | tee -a $SEEN > $URLLIST2
 mv $URLLIST2 $URLLIST
 done

rm -f $URLLIST $URLLIST2 $SEEN

Great exercise for the CPUs
htop gnu parallel

When the command finishes then the next step is parsing access_log

grep -r ' 404 ' /var/log/httpd/access_log | cut -d ' ' -f 7 | sed -r 's/^\//http\:\/\/localhost\//g'

Video cutting from start to end time using ffmpeg

Sometimes I need pieces from movies to upload to YouTube for example.

It’s possible with the following nice oneliner:

ffmpeg -i IN.mp4 -ss 01:12:55 -t 35 -async 1 OUT.mp4

The solution came from http://stackoverflow.com/questions/18444194/cutting-the-videos-based-on-start-and-end-time-using-ffmpeg

Overwriting httpd.service PrivateTmp

While setting up Xdebug I had to change the httpd.service systemd unit file to don’t use private tmp dir in /tmp

For locating the unit file I just used

locate httpd.service

Then needed to copy it, edit and reload the daemon config.

sudo cp /usr/lib/systemd/system/httpd.service /etc/systemd/system
sudo vim /etc/systemd/system/httpd.service
sudo systemctl daemon-reload
sudo systemctl restart httpd.service