Ubuntu 12.04 on Xen: blkfront: barrier: empty write xvda op failed

When you run Ubuntu 12.04 VM on XenServer 6.0 (kernel you can get the following errors and your local file systems will mount read-only. Also see Debian #637234 or Ubuntu #824089.

blkfront: barrier: empty write xvda op failed
blkfront: xvda: barrier or flush: disabled

You also won't be able to remount read-write using "mount -o remount,rw /" as this will give you a kernel error like this

EXT4-fs (xvda1): re-mounted. Opts: errors=remount-ro

This problem more or less sporadically affects paravirtualized Ubuntu 12.04 VMs. Note that officially Ubuntu 12.04 is not listed as a supported system in the Citrix documentation. Note that this problem doesn't affect fully virtualized VMs.

The Solution:

  • According to the Debian bug report the correct solution is to upgrade the dom0 kernel to at least 2.6.32-41.
  • To solve the problem without upgrading the dom0 kernel: reboot until you get a writable filesystem and add "barrier=0" to the mount options of all your local filesystems.
  • Alternatively: do not use paravirtualization :-)

Patch to Create Custom FLV Tags with Yamdi

As described in the Comparison of FLV and MP4 metadata tagging tools (injectors) post yamdi is propably the best and fastest Open Source FLV metadata injector.

Still yamdi is missing the possibility to add custom FLV tags. I posted a patch upstream some months ago with no feedback so far. So if you need custom tags as provided by flvtool2 you might want to merge this patch against the yamdi sources (tested with 1.8.0.

--- ../yamdi.c	2010-10-17 20:46:40.000000000 +0200
+++ yamdi.c	2010-10-19 11:32:34.000000000 +0200
@@ -105,6 +105,9 @@
 	FLVTag_t *flvtag;
 } FLVIndex_t;
+// max number of user defined tags
+#define MAX_USER_DEFINED	10
 typedef struct {
 	FLVIndex_t index;
@@ -168,6 +171,8 @@
 	struct {
 		char creator[256];		// -c
+		char *user_defined[MAX_USER_DEFINED];	// -a
+		int user_defined_count;		// number of user defined parameters
 		short addonlastkeyframe;	// -k
 		short addonlastsecond;		// -s, -l (deprecated)
@@ -288,8 +293,15 @@
-	while((c = getopt(argc, argv, "i:o:x:t:c:lskMXh")) != -1) {
+	while((c = getopt(argc, argv, "a:i:o:x:t:c:lskMXh")) != -1) {
 		switch(c) {
+			case 'a':
+				if(flv.options.user_defined_count + 1 == MAX_USER_DEFINED) {
+					fprintf(stderr, "ERROR: to many -a options\n");
+					exit(1);
+				}
+				printf("Adding tag >>>%s<<<\n", optarg);
+				flv.options.user_defined[flv.options.user_defined_count++] = strdup (optarg);
 			case 'i':
 				infile = optarg;
@@ -1055,6 +1067,7 @@
 int createFLVEventOnMetaData(FLV_t *flv) {
 	int pass = 0;
+	int j;
 	size_t i, length = 0;
 	buffer_t b;
@@ -1073,6 +1086,21 @@
 	if(strlen(flv->options.creator) != 0) {
 		writeBufferFLVScriptDataValueString(&b, "creator", flv->options.creator); length++;
+	printf("Adding %d user defined tags\n", flv->options.user_defined_count);
+	for(j = 0; j < flv->options.user_defined_count; j++) {
+		char *key = strdup (flv->options.user_defined[j]);
+		char *value = strchr(key, '=');
+		if(value != NULL) {
+			*value++ = 0;
+			printf("Adding tag #%d %s=%s\n", j, key, value);
+			writeBufferFLVScriptDataValueString(&b, key, value);
+			length++;
+		} else {
+			fprintf(stderr, "ERROR: Invalid key value pair: >>>%s<<<\n", key);
+		}
+		free(key);
+	} 
 	writeBufferFLVScriptDataValueString(&b, "metadatacreator", "Yet Another Metadata Injector for FLV - Version " YAMDI_VERSION "\0"); length++;
 	writeBufferFLVScriptDataValueBool(&b, "hasKeyframes", flv->video.nkeyframes != 0 ? 1 : 0); length++;

Using the patch you then can add up to 10 custom tags using the new "-a" switch. The syntax is

-a <key>=<value>

How to Merge CSV Files

When reading the meager manpage of the "join" command many Unix users propably give up immediately. Still it can be worth using it instead of scripting the same task in your favourite scripting language.

Here is an example on how to merge 2 CSV files:

CSV File 1 "employees.csv"

# <employee id>;<name>;<location>

CSV File 2 "task.csv"

# <task id>;<employee id>;<task description>
1000;2;Setup new Project
1001;3;Call customer X
1002;3;Get package from post office

And now some action:

A naive try...

The following command

join employee.csv tasks.csv

... doesn't produce any output. This is because it expects the shared key to reside at the first column in both file, which is not the case. Also the default separator for 'join' is a whitespace.

Full Join

join -t ";" -1 1 -2 2 employee.csv tasks.csv

We need to run join with '-t ";"' to get tell it that we have CSV format. Then to avoid the pitfall of not having the common key in the first column we need to tell join where the join key is in each file. The switch "-1" gets the key index for the first file and "-2" for the second file.

2;Marie;28;Paris;1000;Setup new Project
3;Tom;25;London;1001;Call customer X
3;Tom;25;London;1002;Get package from post office

Print only name and task

join -o1.2,2.3 -t ";" -1 1 -2 2 employee.csv tasks.csv

We use "-o" to limit the fields to be printed. "-o" takes a comma separated list of "<file nr>.<field nr>" definitions. So we only want the second file of the first file (1.2) and the third field of the second file (2.3)...

Marie;Setup new Project
Tom;Call customer X
Tom;Get package from post office


While the syntax of join is not that straight forward, it still allows doing things quite quickly that one is often tempted to implement in a script. It is quite easy to convert batch input data to CSV format. Using join it can be easily grouped and reduced according to the your task.

If this got you interested you can find more and non-CSV examples on this site.

Determine Memory Configuration of HP servers

Use dmidecode like this

dmidecode 2>&1 |grep -A17 -i "Memory Device" |egrep "Memory Device|Locator: PROC|Size" |grep -v "No Module Installed" |grep -A1 -B1 "Size:"

The "Locator:" line gives you the slot assignements as listed in the HP documentation, e.g. the HP ProLiant DL380 G7 page.

Of course you can also look this up in the ILO GUI.

How to apply changes to /etc/security/limits.conf immediately?

Sometimes you need to increase the open file limit for some application server or the maximum shared member for you ever-growing master database. In such a case you edit your /etc/security/limits.conf and then wonder how to get the changed limits to be visible to check wether you have set them correctly. You do not want to find out that they were wrong after your master DB doesn't come up after some incident in the middle of the night...

The best way is to change the limits and check them by running

ulimit -a

with the affected user.

Workaround: Re-Login with "sudo -i"

But often you won't see the changes. The reason: you need to re-login as limits are only applied on login!

But wait: what about users without login? In such a case you login as root (which might not share their limits) and sudo into the user: so no real login as the user. In this case you must ensure to use the "-i" option of sudo:

sudo -i -u <user>

to simulate an initial login with sudo. This will apply the new limits.

Make it work for sudo without "-i"

Wether you need "-i" depends on the PAM configuration of your Linux distribution. If you need it then PAM probably loads "pam_limit.so" only in /etc/pam.d/login which means at login time but no on sudo. This was introduced in Ubuntu Precise for example. By adding this line

session    required   pam_limits.so

in /etc/pam.d/sudo limits will also be applied when running sudo without "-i". Still using "-i" might be easier.

Instant Applying Limits to Running Processes

Actually you might want to apply the changes directly to a running process additionally to changing /etc/security/limits.conf. In bleeding edge Linux versions there is a tool "prlimit" to get/set limits. To find out if you already have it run


It should print a list of all active limits. So if you have prlimit installed (comes with util-linux-2.21) use it like this

prlimit --pid <pid> --<limit>=<soft>:<hard>

for example

prlimit --pid 12345 --nofile=1024:2048

If you are unlucky and do not have prlimit yet check out this instruction to compile your own version because despite missing user tool the prlimit() system call is in the kernel for quite a while (since 2.6.36).

Bad and Good Extraction with Regular Expressions in Perl

Again and again I find myself writing stuff like this:

if($str =~ /(\w+)\s+(\w+)(\s+(\w+))?/) {
      $result{id} = $1;
      $result{status} = $2;
      $result{details} = $4 if(defined($4));

when I should write:

if($str =~ /(?<id>\w+)\s+(?<status>\w+)(\s+(?<details>\w+))?/) {
      %result = %+;

as described in the perlre manual:

Capture group contents are dynamically scoped and available to you outside the pattern until the end of the enclosing block or until the next successful match, whichever comes first. (See Compound Statements in perlsyn.) You can refer to them by absolute number (using "$1" instead of "\g1" , etc); or by name via the %+ hash, using "$+{name}".

strptime() implementation in Javascript

If you need a simply strptime() implementation for Javascript feel free to use the following. I needed this for more sane date formatting in SpurTracer.

If you find this useful or find bugs please post a comment!

// Copyright (c) 2012 Lars Lindner <[email protected]>
// GPLv2 and later or MIT License - http://www.opensource.org/licenses/mit-license.php

var dayName = new Array("Mon", "Tue", "Wed", "Thu", "Fri", "Sat", "Sun");
var monthName = new Array("Jan", "Feb", "Mar", "Apr", "May", "Jun", "Jul", "Aug", "Sep", "Oct", "Nov", "Dez");
/* simulates some of the format strings of strptime() */
function strptime(format, date) {
	var last = -2;
	var result = "";
	var hour = date.getHours();

	/* Expand aliases */
	format = format.replace(/%D/, "%m/%d/%y");
	format = format.replace(/%R/, "%H:%M");
	format = format.replace(/%T/, "%H:%M:%S");

	/* Note: we fail on strings without format characters */

	while(1) {
		/* find next format char */
		var pos = format.indexOf('%', last + 2);

		if(-1 == pos) {
			/* dump rest of text if no more format chars */
			result += format.substr(last + 2);
		} else {
			/* dump text after last format code */
			result += format.substr(last + 2, pos - (last + 2));

			/* apply format code */
			formatChar = format.charAt(pos + 1);
			switch(formatChar) {
				case '%':
					result += '%';
				case 'C':
					result += date.getYear();
				case 'H':
				case 'k':
					if(hour < 10) result += "0";
					result += hour;
				case 'M':
					if(date.getMinutes() < 10) result += "0";
					result += date.getMinutes();
				case 'S':
					if(date.getSeconds() < 10) result += "0";
					result += date.getSeconds();
				case 'm':
					if(date.getMonth() < 10) result += "0";
					result += date.getMonth();
				case 'a':
				case 'A':
					result += dayName[date.getDay() - 1];
				case 'b':
				case 'B':
				case 'h':
					result += monthName[date.getMonth()];
				case 'Y':
					result += date.getFullYear();
				case 'd':
				case 'e':
					if(date.getDate() < 10) result += "0";
					result += date.getDate();
				case 'w':
					result += date.getDay();
				case 'p':
				case 'P':
					if(hour < 12) {
						result += "am";
					} else {
						result += "pm";
				case 'l':
				case 'I':
					if(hour % 12 < 10) result += "0";
					result += (hour % 12);
		last = pos;
	return result;

Follow file with tail until it gets removed...

Instead of

tail -f /var/log/myserver.log


tail --follow=name /var/log/myserver.log

Using the long form --follow instead of -f you can tell tail to watch the file name and not the file descriptor. So shortly after the file name was removed tail will notice it and terminate itself.

GCC linking and mixed static and dynamic linking

GCC syntax schema to link some libraries statically and others dynamically:

gcc <options> <sources> -o <binary> -Wl,-Bstatic <list of static libs> -Wl,Bdynamic <list of dynamic libs>

Regex in Postgres UPDATE Statement

Want to use regular expressions in Postgres UPDATE statements?

UPDATE table SET field=regexp_replace(field, 'match pattern', 'replace string', 'g');

Syndicate content Syndicate content