Sysadmin

Getting rid of Bash Ctrl+R

Today was a good day, as I stumbled over this post (at http://architects.dzone.com) hinting on the following bash key bindings:

bind '"\e[A":history-search-backward'
bind '"\e[B":history-search-forward'

It changes the behaviour of the up and down cursor keys to not go blindly through the history but only through items matching the current prompt. Of course at the disadvantage of having to clear the line to go through the full history. But as this can be achieved by a Ctrl-C at any time it is still preferrable to Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R Ctrl+R ....

How Common Are HTTP Security Headers Really?

A recent issue of the German iX magazin featured an article on improving end user security by enabling HTTP security headers

  • X-XSS-Protection,
  • X-Content-Type-Options MIME type sniffing,
  • Content-Security-Policy,
  • X-Frame-Options,
  • and HSTS Strict-Transport-Security.

The article gave the impression of all of them quite common and a good DevOps being unreasonable not implementing them immediately if the application supports them without problems.

This lead me to check my monthly domain scan results of April 2014 on who is actually using which header on their main pages. Results as always limited to top 200 Alexa sites and all larger German websites.

Usage of X-XSS-Protection

Header visible for only 14 of 245 (5%) of the scanned websites. As 2 are just disabling the setting it is only 4% of the websites enabling it.

Website Header
www.adcash.com X-XSS-Protection: 1; mode=block
www.badoo.com X-XSS-Protection: 1; mode=block
www.blogger.com X-XSS-Protection: 1; mode=block
www.blogspot.com X-XSS-Protection: 1; mode=block
www.facebook.com X-XSS-Protection: 0
www.feedburner.com X-XSS-Protection: 1; mode=block
www.github.com X-XSS-Protection: 1; mode=block
www.google.de X-XSS-Protection: 1; mode=block
www.live.com X-XSS-Protection: 0
www.meinestadt.de X-XSS-Protection: 1; mode=block
www.openstreetmap.org X-XSS-Protection: 1; mode=block
www.tape.tv X-XSS-Protection: 1; mode=block
www.xing.de X-XSS-Protection: 1; mode=block; report=https://www.xing.com/tools/xss_reporter
www.youtube.de X-XSS-Protection: 1; mode=block; report=https://www.google.com/appserve/security-bugs/log/youtube

Usage of X-Content-Type-Options

Here 15 of 245 websites (6%) enable the option.

Website Header
www.blogger.com X-Content-Type-Options: nosniff
www.blogspot.com X-Content-Type-Options: nosniff
www.deutschepost.de X-Content-Type-Options: NOSNIFF
www.facebook.com X-Content-Type-Options: nosniff
www.feedburner.com X-Content-Type-Options: nosniff
www.github.com X-Content-Type-Options: nosniff
www.linkedin.com X-Content-Type-Options: nosniff
www.live.com X-Content-Type-Options: nosniff
www.meinestadt.de X-Content-Type-Options: nosniff
www.openstreetmap.org X-Content-Type-Options: nosniff
www.spotify.com X-Content-Type-Options: nosniff
www.tape.tv X-Content-Type-Options: nosniff
www.wikihow.com X-Content-Type-Options: nosniff
www.wikipedia.org X-Content-Type-Options: nosniff
www.youtube.de X-Content-Type-Options: nosniff

Usage of Content-Security-Policy

Actually only 1 website in the top 200 Alexa ranked websites uses CSP and this lonely site is github. The problem with CSP obviously being the necessity to have a clear structure for the origin domains of the site elements. And the less advertisments and tracking pixels you have the easier it becomes...

Website Header
www.github.com Content-Security-Policy: default-src *; script-src https://github.global.ssl.fastly.net https://ssl.google-analytics.com https://collector-cdn.github.com; style-src 'self' 'unsafe-inline' 'unsafe-eval' https://github.global.ssl.fastly.net; object-src https://github.global.ssl.fastly.net

Usage of X-Frame-Options

The X-Frame-Options header is currently delivered by 43 of 245 websites (17%).

Website Header
www.adcash.com X-Frame-Options: SAMEORIGIN
www.adf.ly X-Frame-Options: SAMEORIGIN
www.avg.com X-Frame-Options: SAMEORIGIN
www.badoo.com X-Frame-Options: DENY
www.battle.net X-Frame-Options: SAMEORIGIN
www.blogger.com X-Frame-Options: SAMEORIGIN
www.blogspot.com X-Frame-Options: SAMEORIGIN
www.dailymotion.com X-Frame-Options: deny
www.deutschepost.de X-Frame-Options: SAMEORIGIN
www.ebay.de X-Frame-Options: SAMEORIGIN
www.facebook.com X-Frame-Options: DENY
www.feedburner.com X-Frame-Options: SAMEORIGIN
www.github.com X-Frame-Options: deny
www.gmx.de X-Frame-Options: deny
www.gmx.net X-Frame-Options: deny
www.google.de X-Frame-Options: SAMEORIGIN
www.groupon.de X-Frame-Options: SAMEORIGIN
www.imdb.com X-Frame-Options: SAMEORIGIN
www.indeed.com X-Frame-Options: SAMEORIGIN
www.instagram.com X-Frame-Options: SAMEORIGIN
www.java.com X-Frame-Options: SAMEORIGIN
www.linkedin.com X-Frame-Options: SAMEORIGIN
www.live.com X-Frame-Options: deny
www.mail.ru X-Frame-Options: SAMEORIGIN
www.mozilla.org X-Frame-Options: DENY
www.netflix.com X-Frame-Options: SAMEORIGIN
www.openstreetmap.org X-Frame-Options: SAMEORIGIN
www.oracle.com X-Frame-Options: SAMEORIGIN
www.paypal.com X-Frame-Options: SAMEORIGIN
www.pingdom.com X-Frame-Options: SAMEORIGIN
www.skype.com X-Frame-Options: SAMEORIGIN
www.skype.de X-Frame-Options: SAMEORIGIN
www.softpedia.com X-Frame-Options: SAMEORIGIN
www.soundcloud.com X-Frame-Options: SAMEORIGIN
www.sourceforge.net X-Frame-Options: SAMEORIGIN
www.spotify.com X-Frame-Options: SAMEORIGIN
www.stackoverflow.com X-Frame-Options: SAMEORIGIN
www.tape.tv X-Frame-Options: SAMEORIGIN
www.web.de X-Frame-Options: deny
www.wikihow.com X-Frame-Options: SAMEORIGIN
www.wordpress.com X-Frame-Options: SAMEORIGIN
www.yandex.ru X-Frame-Options: DENY
www.youtube.de X-Frame-Options: SAMEORIGIN

Usage of HSTS Strict-Transport-Security

HSTS headers can only be found on a few front pages (8 of 245). Maybe it is visible more on the login pages and is avoided on front pages for performance reasons, maybe not. That would require further analysis. What can be said is only some larger technology leaders are brave enough to use it on the front page:

Website Header
www.blogger.com Strict-Transport-Security: max-age=10893354; includeSubDomains
www.blogspot.com Strict-Transport-Security: max-age=10893354; includeSubDomains
www.facebook.com Strict-Transport-Security: max-age=2592000
www.feedburner.com Strict-Transport-Security: max-age=10893354; includeSubDomains
www.github.com Strict-Transport-Security: max-age=31536000
www.paypal.com Strict-Transport-Security: max-age=14400
www.spotify.com Strict-Transport-Security: max-age=31536000
www.upjers.com Strict-Transport-Security: max-age=47336400

Conclusion

Security headers are not wide-spread on website front pages at least. Most used is the X-Frame-Option header to prevent clickjacking. Next following is X-Content-Type-Options to prevent MIME sniffing. Both of course are easy to implement as they most probably do not change your websites behaviour. I'd expect to see more HSTS on bank and other online payment service websites, but it might well be that the headers appear only on subsequent redirects when logging in, which this scan doesn't do. With CSP being the hardest to implement, as you need to have complete control over all domain usage by application content and partner content you embed, it is no wonder that only Github.com has implemented it. For me it is an indication how clean their web application actually is.

Website Technology Changes in March 2014

As in the last months let's look into changes visible at the frontend pages of the biggest websites. This time I compared the changes between February to April.

These last two months saw the usual lot of version upgrades, along with some probably unintended un-hiding of server versions, several sites going to CloudFlare as well as a premiere with IPv6 being available on the first adult movie site.

The detailed results can be found here:

What Changed?

DNS-Prefetching The HTML header based DNS prefetching is expanding once more and for the first time used on adult site: redtube.com
IPv6 An AAAA record was sighted for the first time for xhamster.com. That makes IPv6 available for the first time on a major adult site!
CDN Changes
Version Upgrades
  • dooyoo.de upgrades from PHP 5.3.2-1ubuntu4.15 to recent 4.23
  • adf.ly upgrades from PHP 5.4.21 to 5.5.8
  • duden.de upgrades from PHP 5.3.3-7+squeeze18 to 19
  • duden.de upgrades from PHP 4.4.6 to 5.4.16
  • wikipedia.de upgrades from PHP 5.3.3-7+squeeze9 to 19
  • wikipedia.org upgrades from PHP 5.3.19-1ubuntu3.9+wmf1 to 3.10+wmf1
  • xhamster.com upgrades from PHP 5.3.21 to 5.3.26
  • jquery.com upgrades from nginx 1.4.4 to 1.4.7
  • xhamster.com upgrades from nginx 1.4.4 to 1.4.7
  • qq.com upgrades from squid 3.1.18 to 3.2.1
Hiding Server Version Against the trend in the last month this month three sites have unhidden the previously hidden server details:
  • edarling.de normally not showing any server version displayed "PHP/5.4.4-14+deb7u7" in February.
  • greenpeace.de previously not indicating the Apache version now shows CentOS Apache 2.2.15
  • jamba.de previously hiding the server version in April indicated "Servlet 2.4; JBoss-4.3.0.GA_CP06 (build"

Note: the website links lead to a history page for the different sites were you can see the change details.

Caution!

All the results listed above are based on a simple scanning script. The results present a snapshot of the websites and a single response only. This is of course not necessarily an indicating for what techniques the site uses in daily operations!

Who is using which CDN in 04/2014

Recent CDN usage for top 200 Alexa ranked sites and major German sites. For measurement method read more here...

CDN Sites
Akamai web.mit.edu adobe.com aol.de apple.com ask.com autobild.de avg.com bbc.co.uk bigpoint.com bild.de bing.com buch.de buzzfeed.com chip.de cnet.com cnn.com computerbild.de conduit.com crunchbase.com dailymotion.com dooyoo.de ebay.de edarling.de erento.com facebook.com finanzen.net flickr.com flipkart.com focus.de forbes.com godaddy.com huffingtonpost.com hungryhouse.co.uk ifeng.com imdb.com immonet.de indeed.com jamba.de java.com last.fm lieferheld.de linkedin.com mashable.com microsoft.de mjam.at morgenpost.de mozilla.org msdn.com msn.com mtv.com mytoys.de mywebsearch.com netflix.com n-tv.de otto.de paypal.com pizza.de qq.com reddit.com reference.com rp-online.de rtl.de salesforce.com skype.com skype.de slashdot.org sourceforge.net spiegel.de sport1.DE stepstone.de stern.de stumbleupon.com sueddeutsche.de superantojo.com.mx taobao.com theguardian.com welt.de wetter.com wetteronline.de xing.de zdf.de zedo.com
cachefly arstechnica.com cdnplanet.com
CDNetworks ifeng.com
Cedexis autobild.de bild.de computerbild.de finanzen.net morgenpost.de welt.de
Cloudflare crunchbase.com foodpanda.in foodpanda.pl foodpanda.ru imgur.com pingdom.com statcounter.com
CloudFront web.mit.edu addthis.com amazon.com amazon.de arstechnica.com bannersdontwork.com berlinonline.de cdnplanet.com chefkoch.de crunchbase.com foodarena.ch foodpanda.in foodpanda.ru hungryhouse.co.uk imdb.com instagram.com kicker.de neobux.com nzz.ch onlinepizza.se pizzaportal.pl samsung.com spotify.com stumbleupon.com superantojo.com.mx theguardian.com tivo.com tumblr.com tvinfo.de
EdgeCast web.mit.edu buzzfeed.com dailymotion.com erento.com forbes.com soundcloud.com sueddeutsche.de tumblr.com twitter.com
fastly web.mit.edu github.com samsung.com theguardian.com twitter.com
Level3 go.com groupon.de kicker.de linkedin.com photobucket.com stern.de sueddeutsche.de theguardian.com wetter.com zalando.de
Limelight arstechnica.com arte.tv softpedia.com swissre.com xnxx.com xvideos.com zeit.de
phncdn.com pornhub.com redtube.com tube8.com youporn.com
rncdn1.com thepiratebay.sx tube8.com
WaveCDN upjers.com

How to Munin Graph JVM Memory Usage with Ubuntu tomcat

The following description works when using the Ubuntu "tomcat7" package:

Grab the "java/jstat__heap" plugin from munin-contrib @ github and place it into "/usr/share/munin/plugins/jstat__heap".

Link the plugin into /etc/munin/plugins

ln -s /usr/share/munin/plugins/jstat__heap /etc/munin/plugins/jstat_myname_heap

Choose some useful name instead of "myname". This allows to monitor multiple JVM setups.

Configure each link you created in for example a new plugin config file named "/etc/munin/plugin-conf.d/jstat" which should contain one section per JVM looking like this

[jstat_myname_heap]
user tomcat7
env.pidfilepath /var/run/tomcat7.pid
env.javahome /usr/

Website Technology Changes in January 2014

As in the last four months I'm looking into changes visible at the frontend pages of the biggest websites. The last month saw a lot of version upgrades and hiding server versions.

The detailed results can be found here:

What Changed?

DNS-Prefetching The HTML header based DNS prefetching is expanding once more and for the first time used on adult site: xnxx.com
IPv6 An AAAA record was sigthed for the first time for yandex.ru
Version Upgrades
  • bildblog.de upgrades from Gentoo PHP 5.4.13 to 5.5.7
  • flipkart.com upgraded nginx 1.4.1 to 1.4.4
  • gomez.com switched OS and upgraded Apache 2.2.13 (Win32) to 2.2.15 (Red Hat)
  • jquery.com upgraded PHP 5.3.27 to 5.3.28
  • kickass.to upgraded Gentoo nginx 1.5.1 to bleeding edge 1.5.10 same for PHP 5.5.5 to 5.5.9
  • taz.de upgraded Apache 2.2.16 to 2.2.22
  • wetteronline.de upgraded Apache 2.4.2 to 2.4.7
  • xhamster.com upgraded nginx 1.4.1 to 1.4.4 and PHP 5.3.15 to 5.3.21
Hiding Server Version Additional sites are now hiding the webserver version

Note: the website links lead to a history page for the different sites were you can see the change details.

Caution!

All the results listed above are based on a simple scanning script. The results present a snapshot of the websites and a single response only. This is of course not necessarily an indicating for what techniques the site uses in daily operations!

Sharing Screen With Multiple Users

How to detect screen sessions of other users:

screen -ls <user name>/

How to open screen to other users:

  1. Ctrl-A :multiuser on
  2. Ctrl-A :acladd <user to grant access>

Attach to other users screen session:

With session name

screen -x <user name>/<session name>

With PID and tty

screen -x <user name>/<pid>.<ptty>.<host>

Website Technology Changes in December 2013

As in the last three months I'm looking into changes visible at the frontend pages of the biggest websites. As to be expected changes during December are a bit limited as probably everyone favours stability over the holidays.

The detailed results can be found here:

What Changed?

DNS-Prefetching The HTML header based DNS prefetching is still there and gained yet another site: flipkart.com
IPv6 No change in AAAA records.
Version Upgrades fu-berlin.de upgrades from Apache 2.2.17 to 2.2.22
conduit.com upgrades from IIS 7.5 to 8.5
microsoft.com upgrades from IIS 8.0 to 8.5
Webserver Change Documentation site w3schools.com now shows nginx 1.2.6 instead IIS 7.5 as user facing webserver
Hiding Server Version One more site hostgator.com stopped reporting the Apache version

Note: the website links lead to a history page for the measurements.

Caution!

All the results listed above are based on a simple scanning script. The results present a snapshot of the websites and a single response only. This is of course not necessarily an indicating for what techniques the site uses in daily operations!

How to dry-run with chef-client

The answer is simple: do not "dry-run", do "why-run"!

chef-client --why-run
chef-client -W

And the output looks nicer when using "-Fmin"

chef-client -Fmin -W

As with all other automation tools, the dry-run mode is not very predictive. Still it might indicate some of the things that will happen.

Large Website Technology Changes in November 2013

As in the last two months I performed another a indexing of the information reported by major websites. It covers mostly request header, HTML and DNS based information of the top 200 sites listed by Alexa and the top 100 German websites. All the information is freely available and only extracted from the website responses!

The detailed results can be found here:

What Changed?

DNS-Prefetching The HTML header based DNS prefetching is still there and gained another site: German job portal stepstone.de
IPv6 IPv6 support did not spread in the last month. Several sites have flapping visibility of their AAAA records.
Version Upgrades adf.ly upgrades from PHP 5.3.8 to 5.4.21.
rtl.de upgrades from Apache 2.2.21 to 2.4.6.
Webserver Change German health portal imedo.de switched from Mongrel to Apache
Hiding Server Version Porn site xnxx.com stopped reporting usage of PHP 5.3.6
CDN In 11/2013 conduit.com switched from Cotendo to Akamai CDN.
Hoster German couples portal parship.de seems to have changed hoster and is using BigIP F5 load balancers now.

Note: the website links lead to a history page for the measurements.

Caution!

All the results listed above are based on a simple scanning script. The results present a snapshot of the websites and a single response only. This is of course not necessarily an indicating for what techniques the site uses in daily operations!

Syndicate content