Category Archives: Sysadmin

Topics related to system administration or IT.

Memory-backed Filesystem for Temporary Storage of Whisper Data

Instead of buying and installing SSDs, storing Graphite’s whisper files in a memory-backed filesystem can be a good way to go if you have the RAM to spare. Depending on your environment, you may or may not care about losing a few minutes (or hours) of metric data. I know we certainly don’t care about losing 30 minutes, so there’s no reason for our carbon-cache instances to be scrawling to persistent storage 24/7. Continue reading

Documenting bash scripts

For those of us still in the bash trenches now and then, I learned you can avoid the hassle of hash/octothorpe (#) commenting blocks of documentation in your scripts by using a standard “here” doc prefixed by the no-op expression (:).

Clearly this is something you’d only want to do for major blocks of text, like a large documentation block at the start of a script.



My free-form documentation here.


Bootstrapping new Chef nodes without knife bootstrap

There may be a time when you don’t want to (or cannot) make use of knife bootstrap to set up a new Chef node or a whole fleet of hundreds of nodes. If that’s the case, and you already have a “hook” into the hosts you want to turn into Chef nodes (such as an existing CM tool), you have an option. Continue reading

CIFS read performance on Linux < 3.2

Here’s what you can expect from read performance using the Linux CIFS kernel module prior to kernel 3.2.

See the ‘rsync’ graph in the lower right.

Abysmal CIFS read performance

Relevant context:

  1. Each of the colors in the rsync graph represent remote WAN sites. Each data point represents a transfer of ~200MB.
  2. No amount of dickering with the cifs.ko kernel module options improved anything
  3. All sites have at least 1.5Mbps links
  4. An HTTP get to each remote site ran at ~900kbps
  5. Using a Windows 2008 client to each remote site’s CIFS share gave ~940kbps
  6. Performance did not improve when using cp in place of rsync, nor did it improve with tweaks to the rsync options.
  7. The client of the CIFS shares was 96% CPU idle. A textbook case for illustrating to novices the reduced value of looking at load averages (see graph) to indicate how hard a host is working. The CPU graph was removed as it was irrelevant.

Thankfully, this is a largely solved issue and we’ll see it in future distributions. Here is the thread from the linux-cifs list.

Another case for in-place file editing in CM tools

Our UNIX/Linux directory services (nss) data was migrated from NIS in LDAP in 2009. Those who never administered a NIS domain may not even know what a netgroup[1] is, but we continue to make significant use of netgroups (stored in LDAP) to define groups of users and hosts for various needs. They’ve worked great for decades and serve a good purpose. Continue reading