Large Website Technology Changes in October 2013
Last month I did a first indexing of mostly request header, HTML and DNS based information about the top 200 sites listed by Alexa and the top 100 German websites. All the information is freely available and extracted from the website responses.
The detailed results can be found here:
What Changed In A Month?
Now roughly a month later I repeated the scan to see what changes do happen and what trends might be noticable:
|DNS-Prefetching||THE HTML header based DNS prefetching is spreading. Two of the large adult content sites added it and Chinese search engine baidu.com introduced it too. As in my experience it provides a small but measurable latency improvement on many sites I'd guess usage will further spread...
Added by baidu.com, pornhub.com, tube8.com, www.wer-kennt-wen.de
|IPv6||IPv6 support did not spread in the last month. Just two sides that dropped their AAAA records.
Removed by www.mail.ru, www.volkswagen.de
|Varnish||One of the most significant changes is more and more sites announcing the usage of Varnish as a cache. This is sometimes combined with a CDN, sometimes without.|
|Hiding Server Version||Two more websites are now hiding their Apache version.
Removed version now: www.weibo.com, www.ftd.de
|XSS Header||XSS headers are not widely spread. No real change in adoption.
Added by www.craigslist.org
All the results listed above are based on a simple scanning script. The results present a snapshot of the websites and a single response only. This is of course not necessarily an indicating for what techniques the site uses in daily operations!