Solving 100% CPU usage of Chef beam.smp (RabbitMQ)

Search for chef 100% cpu issue and you will find a lot of sugestions ranging from reboot the server, to restart RabbitMQ and often to check the kernel max file limit.

All of those do not help! What does help is checking RabbitMQ with

rabbitmqctl report | grep -A3 file_descriptors

and have a look at the printed limits and usage. Here is an example:

 {file_descriptors,[{total_limit,8900},
                    {total_used,1028},
                    {sockets_limit,8008},
                    {sockets_used,2}]},

In my case the 100% CPU usage was caused by all of the file handles being used up which for some reason causes RabbitMQ 2.8.4 to go into a crazy endless loop rarely responding at all.

The "total_limit" value is the "nofile" limit for the maximum number of open files you can check using "ulimit -n" as RabbitMQ user. Increase it permanently by defining a RabbitMQ specific limit for example in /etc/security/limits.d/rabbitmq.conf:

rabbitmq    soft   nofile   10000

or using for example

ulimit -n 10000

from the start script or login scripts. Then restart RabbitMQ. The CPU usage should be gone.

Update: This problem only affects RabbitMQ releases up to 1.8.4 and should be fixed starting with 1.8.5.

Hotkeys to Efficiently Use Bash in Emacs Mode

When watching other people familiar with Linux type commands on a bash shell what one often needs is a lot of patience. It seems that many of us (and I include myself) do not know or regularily forget about the biggest timesavers.

Here is a list of maybe trivial things I consider absolutely helpful to efficiently use Bash in Emacs mode.

1. Jump Words not Characters!

When scrolling through a long command: save time by holding Ctrl to skip words. Alternatively press ESC before each cursor press or use Alt-B and Alt-F.

2. Kill Words not Characters!

The same when deleting stuff. Don't press Backspace 30 times. Use a hotkey to do it in one step:

  • Alt-Backspace or Ctrl-w to delete a word backwards (the difference lies in delimiters)
  • Alt-d to delete a word from start

3. Lookup Filenames while Writing

Do not cancel writing commands just because you forgot a path somewhere on the system. Look it up while writing the command. If paths are in quotes, temporarily disable the quotes to make expansion work again.

4. Save Cancelled Commands

If you still decide to cancel a command because you want to look something up, then keep it in the history. Just type a "#" in front of it and Enter to safely store it for later reference.

5. Use Ctrl-A and Ctrl-E

Most beginners do scroll around a lot for no good reason. This hurts on slow connections. Do never scroll to the start/end of the command. Use Ctrl-a for start of line and Ctrl-E for end of line. When in screen use Ctrl-a-a instead of Ctrl-a!

6. Do not overuse Ctrl-R!

Many users heavily rely on Ctrl-R for reverse history search and use it to go back >50 times to find a command of 10 characters! Just type the 10 characters! Or if you thought it was only 10 steps away in the history just cancel when you do not find it early on. This is mostly a psychological thing: "it might be around the corner" but wastes a lot of time.

When your command is only a few steps away in the history use the Up/Down leys until you find it. Cancel this once you press as many times as your command is long!

7. Use Undo!

Avoid rewriting stuff if you mess up.

Case 1: If you delete something and want to restore it use Ctrl+_ (Ctrl+Shift+Dash) to undo the last change.

Case 2:If you have edited a history entry you got using Ctrl-R or Up/Down navigation and want to restore it to it's original value use Alt-R. In general try to avoid editing history commands as this is confusing!

8. Use History with Line Numbers

Often I see people calling "history" and then mouse copy&pasting a command the find. The faster thing to do is to type "!<number>" to execute a specific command. Usually just 4-5 characters eliminating the possibility of pasting the wrong copy buffer or a command with line breaks after a lot of fiddling with the mouse.

What Else?

There might be other useful commands, but I think I've listed the time savers above. Feel free to comment your suggestions!

Media Player with GStreamer and PyGI

When trying to implement a media plugin for Liferea I learned at lot from Laszlo Pandy's session slides from the Ubuntu Opportunistic Developer Week (PDF, source). The only problem was that it is for PyGtk, the GTK binding for Python, which is now more or less deprecated for PyGI, the GTK+ 3.0 GObject introspection based binding.

While it is easy to convert all pieces manually following the Novacut/GStreamer1.0 documentation I still want to share a complete music player source example that everyone interested can copy from. I hope this saves the one or the other some time in guessing how to write something like "gst.FORMAT_TIME" in PyGI (actually it is "Gst.Format.TIME").

So here is the example code (download file):

from gi.repository import GObject
from gi.repository import GLib
from gi.repository import Gtk
from gi.repository import Gst

class PlaybackInterface:

    def __init__(self):
	self.playing = False

	# A free example sound track
	self.uri = "http://cdn02.cdn.gorillavsbear.net/wp-content/uploads/2012/10/GORILLA-VS-BEAR-OCTOBER-2012.mp3"

	# GTK window and widgets
	self.window = Gtk.Window()
	self.window.set_size_request(300,50)

	vbox = Gtk.Box(Gtk.Orientation.HORIZONTAL, 0)
	vbox.set_margin_top(3)
	vbox.set_margin_bottom(3)
	self.window.add(vbox)

	self.playButtonImage = Gtk.Image()
	self.playButtonImage.set_from_stock("gtk-media-play", Gtk.IconSize.BUTTON)
	self.playButton = Gtk.Button.new()
	self.playButton.add(self.playButtonImage)
	self.playButton.connect("clicked", self.playToggled)
	Gtk.Box.pack_start(vbox, self.playButton, False, False, 0)

	self.slider = Gtk.HScale()
	self.slider.set_margin_left(6)
	self.slider.set_margin_right(6)
	self.slider.set_draw_value(False)
	self.slider.set_range(0, 100)
	self.slider.set_increments(1, 10)

	Gtk.Box.pack_start(vbox, self.slider, True, True, 0)

	self.label = Gtk.Label(label='0:00')
	self.label.set_margin_left(6)
	self.label.set_margin_right(6)
	Gtk.Box.pack_start(vbox, self.label, False, False, 0)

	self.window.show_all()

        # GStreamer Setup
        Gst.init_check(None)
        self.IS_GST010 = Gst.version()[0] == 0
	self.player = Gst.ElementFactory.make("playbin2", "player")
	fakesink = Gst.ElementFactory.make("fakesink", "fakesink")
	self.player.set_property("video-sink", fakesink)
	bus = self.player.get_bus()
	#bus.add_signal_watch_full()
	bus.connect("message", self.on_message)
	self.player.connect("about-to-finish",  self.on_finished)

    def on_message(self, bus, message):
	t = message.type
	if t == Gst.Message.EOS:
		self.player.set_state(Gst.State.NULL)
		self.playing = False
	elif t == Gst.Message.ERROR:
		self.player.set_state(Gst.State.NULL)
		err, debug = message.parse_error()
		print "Error: %s" % err, debug
		self.playing = False

	self.updateButtons()

    def on_finished(self, player):
	self.playing = False
        self.slider.set_value(0)
	self.label.set_text("0:00")
        self.updateButtons()

    def play(self):
	self.player.set_property("uri", self.uri)
	self.player.set_state(Gst.State.PLAYING)
	GObject.timeout_add(1000, self.updateSlider)

    def stop(self):
	self.player.set_state(Gst.State.NULL)
	
    def playToggled(self, w):
        self.slider.set_value(0)
	self.label.set_text("0:00")

	if(self.playing == False):
		self.play()
	else:
		self.stop()

	self.playing=not(self.playing)
	self.updateButtons()

    def updateSlider(self):
	if(self.playing == False):
	   return False # cancel timeout

	try:
	   if self.IS_GST010:
	      nanosecs = self.player.query_position(Gst.Format.TIME)[2]
	      duration_nanosecs = self.player.query_duration(Gst.Format.TIME)[2]
	   else:
	      nanosecs = self.player.query_position(Gst.Format.TIME)[1]
	      duration_nanosecs = self.player.query_duration(Gst.Format.TIME)[1]

	   # block seek handler so we don't seek when we set_value()
	   # self.slider.handler_block_by_func(self.on_slider_change)

           duration = float(duration_nanosecs) / Gst.SECOND
	   position = float(nanosecs) / Gst.SECOND
	   self.slider.set_range(0, duration)
	   self.slider.set_value(position)
           self.label.set_text ("%d" % (position / 60) + ":%02d" % (position % 60))

	   #self.slider.handler_unblock_by_func(self.on_slider_change)
	
	except Exception as e:
		# pipeline must not be ready and does not know position
		print e
		pass

	return True

    def updateButtons(self):
        if(self.playing == False):
           self.playButtonImage.set_from_stock("gtk-media-play", Gtk.IconSize.BUTTON)
        else:
           self.playButtonImage.set_from_stock("gtk-media-stop", Gtk.IconSize.BUTTON)

if __name__ == "__main__":
    PlaybackInterface()
    Gtk.main()

and this is how it should look like if everything goes well:

Example Player Screenshot

Please post comments below if you have improvement suggestions!

Further Improving the Liferea Music Player Plugin

Yesterday I found time to improve the music player plugin. It now provides a slider showing the playing progress and the time position in the track.

The plugin is written in Python and uses GStreamer. So far I tested with GStreamer 0.10. The plugin is currently missing seeking and GStreamer 0.11 testing.

I'm wondering if some reader would like to apply his Python skills and improve the player further? Check out the so far simple script: media-player.py

How to Run VACUUM on SQLite

in

On any SQLite based database (e.g. Firefox or Liferea) you can run "VACUUM" to reduce the database file size. To do this you need the SQLite command line client which can be run using

sqlite3 <database file>

When called like this you get a query prompt. To directly run "VACUUM" just call

sqlite3 <database file> "VACUUM;"

Ensure that the program using the database file is not running!

Alternatives to Manual VACUUM

If you are unsure how to do it manually you can also use a helper tool like BleachBit which along many other cleanup jobs also performs SQLite database compaction.

Liferea 1.8.10 released

This release fixes a search folder counting and rebuilding, it fixes crash when removing nodes from Google Reader subscriptions and it prevents sorting feeds and adding unsupported node types to Google Reader subscriptions.

Please upgrade!

Grab the newest release from the homepage!

Liferea 1.9.7 released

This release introduces a new preference to set the default viewing mode (email, wide or 2pane), adds Google Chrome as supported browser preference and brings an overhaul of the search folder implementation to fix counting and rebuilding issues.

Grab the newest release from the homepage!

chef-server "Failed to authenticate."

If your chef GUI suddenly stops working and you see something like the following exception in both server.log and server-webui.log:

merb : chef-server (api) : worker (port 4000) ~ Failed to authenticate. Ensure that your client key is valid. - (Merb::ControllerExceptions::Unauthorized)
/usr/share/chef-server-api/app/controllers/application.rb:56:in `authenticate_every'
/usr/lib/ruby/vendor_ruby/merb-core/controller/abstract_controller.rb:352:in `send'
/usr/lib/ruby/vendor_ruby/merb-core/controller/abstract_controller.rb:352:in `_call_filters'
/usr/lib/ruby/vendor_ruby/merb-core/controller/abstract_controller.rb:344:in `each'
/usr/lib/ruby/vendor_ruby/merb-core/controller/abstract_controller.rb:344:in `_call_filters'
/usr/lib/ruby/vendor_ruby/merb-core/controller/abstract_controller.rb:286:in `_dispatch'
/usr/lib/ruby/vendor_ruby/merb-core/controller/abstract_controller.rb:284:in `catch'
/usr/lib/ruby/vendor_ruby/merb-core/controller/abstract_controller.rb:284:in `_dispatch'
[...]

Then try stopping all chef processes, remove

/etc/chef/webui.pem
/etc/chef/validation.pem

and start everything again. It will regenerate the keys.

The downside is that you have to

knife configure -i

all you knife setup locations again.

Chef: Which nodes have role X / recipe Y

Why is it so hard to find out which nodes have a given role or recipe in chef?

The only way seems to loop yourself:

for node in $(knife node list); do
   if knife node show -r $node | grep 'role\[base\]' >/dev/null; then       
     echo $node;
   fi;
done

Did I miss some other obvious way? I'd like to have some "knife run_list filter ..." command!

Added Google Chrome to Browser Preferences

Chrome Logo

After reading this recent review of Liferea over at TechNewsWorld I found that we somehow missed adding Google Chrome to the browser preferences!

Thanks to review author Jack M. Germain this is now added in 1.9 git branch to be released with 1.9.7.

As a workaround you can still use Chrome if you select "Default Browser" if your system preferences default to Chrome or you can specify the following "Manual Command" in the browser settings tab:

google-chrome "%s"

Have fun!

Syndicate content Syndicate content