Link Search Menu Expand Document

Linux-Sysadmin Cheat Sheet




    dmidecode -s system-product-name
    hdparm -I /dev/....
    smartctl -a /dev/...



  • fio: Synthetic IO stress test
  • iozone: Synthetic FS benchmark

I/O Scheduler

Get available and selected disk I/O scheduler

cat /sys/block/sda/queue/scheduler

Set Linux disk IO scheduler

echo deadline > /sys/block/sda/queue/scheduler

How to enable BFQ scheduler with udev


CPU Scheduler

Check and change scheduler settings with schedtool

schedtool <pid>                  # Print settings for given PID
schedtool -r                     # Show scheduler class prio ranges
schedtool -R -p 20 -e <command>  # Run command with SCHED_RR and nice 20

List per-process scheduler settings

ps -wweALo pid,spid,user,priority,ni,pcpu,vsize,time,args

Check CPU freq governor

cat /sys/devices/system/cpu/cpufreq/policy*/scaling_governor

Check SCHED_ISO settings

cat /proc/sys/kernel/iso_cpu


Human readable sar output

sar -n DEV 1 1 # e.g. show recent Device I/O 

# Save one complete sample if you have no collection enabled 
sar -o /tmp/sar.out -n ALL 1 1 

Machine readable: you need to use sadf

sadf -j /tmp/sar.out -- -n DEV    # All in nice JSON


Swap Files

  • Create swap file and mount it

    dd if=/dev/zero of=/swapfile bs=1024000 count=1024          # Create ~1GB file
    mkswap /swapfile
    chmod 0600 /swapfile
    swapon /swapfile

    To persist mounting the swapfile add an entry in /etc/fstab like this

    /swapfile    none     swap    sw              0       0


  • Using hypervisor clock in VMs

    timedatectl set-ntp false
    modprobe ptp
    apt-get install chrony
    echo "refclock PHC /dev/ptp0 poll 3 dpoll -2 offset 0" >> /etc/chrony/chrony.conf
    sed -i '/ntp/s/^/#/g' /etc/chrony/chrony.conf
    sed -i "/makestep 0.1 3/c\makestep 0.1 -1" /etc/chrony/chrony.conf
    systemctl restart chronyd
  • Check for pending leap seconds:

      ntpq -c"rv 0 leap"

Avoid leap second problems

  • Variant 1: Disable NTP, skip over second

    # before leap second
    /etc/init.d/ntp stop
    # after leap second
    date -s "$(LC_ALL=C date)"    Note: check correct locale!
    /etc/init.d/ntp start
  • Variant 2: Ensure NTP is up-to-date and running with -x
  • Variant 3: Converging solution


  • Find broken links

    find . -xtype l
  • supervisor - Start CLI with


    CLI Commands

    start <name>
    stop <name>
    restart all
    tail <name>   # JSON log
  • pv - Visualize pipe progress:

    pv -cN source < linux-3.4.0.tar.bz2 | bzcat | pv -cN bzcat | gzip -9 | pv -cN gzip > linux-3.4.0.tar.gz
  • Fix broken text encoding:

    iconv -c -t ASCII input.txt
    iconv -c -t latin1 input.txt
  • screen - Detect other users sessions

    screen -ls <user name>/
  • screen - Allow othe users in our screen

    Ctrl-A :multiuser on
    Ctrl-A :acladd <user to grant access>
  • screen - Attach to other users screen session

    screen -x <user name>/<session name>
    screen -x <user name>/<pid>.<ptty>.<host>
  • screen - Solve “Cannot open your terminal ‘/dev/pts/0’”:

    # Sign in as user who opened the screen
    script /dev/null
    screen -x
  • rsyslog - Modify rate imux limiting

    $SystemLogRateLimitInterval 2
    $SystemLogRateLimitBurst 50  # increase this
  • Running parallel SSH sessions using xargs

    echo "$hosts" | xargs --replace={} -t -n 1 -P 15 sh -c "ssh -o StrictHostKeyChecking=no -o PreferredAuthentications=publickey {} date || :"


  • update-rc.d Examples

    update-rc.d apache2 defaults          # creates S20/K20 links
    update-rc.d apache2 defaults <nr>       # S<nr>/K<nr>
    update-rc.d apache2 defaults <nr1> <nr2>  # S<nr1>/K<nr2>
    # Really specific 
    update-rc.d apache2 start 20 2 3 4 5 . stop 80 0 1 6 .
    update-rc.d -f apache2 remove             # Cleanup

Linux CPU Scheduler

Check if CFS is default scheduler

grep cfs_rq /proc/sched_debug

Check if there are processes not running with CFS (SCHED_OTHER)

ps -ef|grep [0-9]|awk '\{system("chrt -p " $2);print $0}' | grep -Ev 'priority|SCHED_OTHER' |grep -A1 SCHED