The problem is actually that today’s ideology of choice-led capitalism, the idea that everyone is a maker of his or her life, which goes very much against the reality of the social situation, actually pacifies people and makes us constantly turning criticism towards ourselves instead of organizing ourselves and making a critique of the society we live in.” ~ Renata Salecl

However, if you told me tomorrow that everyone currently working in Perl, Python, or Ruby was going to standardize on one and only one of them, make it rock solid with a phenomenal standard library, and have a major feature release ever year, I would happily ditch the Python I enjoy so much and join the crowd.

I haven’t started the World Scripting Language Standardization Consortium or something, so don’t take all of this too seriously, but I tweeted this today as something for folks to consider that they might never have:

Can you imagine the benefits possible to everyone if everyone agreed to do dev only in ONE of #perl #python #ruby ? ENORMOUS.

The first responder stated:

very little actually because of the lack of diversity

Let’s put that in direct context and make a full sentence from it so we’re clear what he’s saying: “There would be very little benefit to everyone doing scripting development in 1 language because of the lack of language diversity.”

To me, that makes no sense whatsoever.

While your language of choice is a religious or emotional topic to you, it really shouldn’t be to the extent you take it (whomever). But Perl allows me to parse X. And in Ruby I have blocks and hoo-hahs! And Python has a nice way of Y! Who cares? It’s really all the same shit, folks. It’s all Smalltalk, Pascal, Lisp, awk, sed, and 20 other things, and the 3 languages all do the same damn thing in the end. We all have our preferences, but my point is that we pay an astounding cost for that small freedom of preference. Is it really worth it here?

What if 3 versions of TCP/IP had grown up with the internet, all with the same real end result to 95% accuracy?

Standards bodies exist for a very very good reason, and it’s not to be The Man keeping you down.

Consider how ridiculous it is that the US still doesn’t use the metric system and consider how much time and money is wasted dealing directly and indirectly with imperial units.

Now, I wouldn’t consider myself a web developer, because it sure isn’t what I do for my day job, but I’ve been there every part of the way since installing NCSA httpd 0.9, writing HTML in 1994, and having wrapped up a Django 1.3 app last week… but I don’t think I need to remind you all of the absolutely hell on earth that was web development from 1998-2010 due to waffling standards and shoddy adherence to the standards that existed during that time.

How many tens of thousands of hours have been wasted by developers implementing the same functionality in new language X that existed in last year’s language Y? I wish I knew, but I have to guess it’s a staggering amount of effort that could have made a single fantasticly stable and thorough language.

How do we practically weigh the value of language choice freedom in the face of global familiarity and standardization for the sake of productivity?

And here we also get into the “artistic expression” side of programming and whether that’s sufficient enough ground to warrant myriad language choices to tickle each developer. That’s a huge can of worms. Are you really that phenomenal an artist that your masterpiece must have Blue shade 102 and not Blue shade 103 only offered by MasterPaintCo? No, you’re not.

Oh how we like to feel special and part of something cool.

TID[1] pointed out that an analogy to all of this can be made in the situation of Esperanto which went nowhere (and I was ALL FOR, BTW). It’s a similar situation, but only on the surface. I don’t think it’s fair to consider the failure of changing the spoken and written language of everyone on the planet, in the face of thousands of years of history… to Jimmy who chose write his applications in Python since 2001.

If you’re a programmer’s programmer, I’m pretty sure you wouldn’t stop programming because the language you like was no longer an option. You’d bitch and moan, learn it because your livelihood depended on it, become very proficient with it, forget what seemed so enticing about the previous language in a year, and ultimately (the goal) continue to create cool applications that made you happy.

So is the world a better, more productive, and more progressive place with a 5%-new-ideas language every year? I think it’s pretty clear where I stand. We don’t need another divisive general purpose programming language with a new syntax, and we didn’t need the last one we got, or probably the one before that, or…

I’d be super happy with everyone on the same team using a single amazingly solid and amazingly documented general purpose language. The language would grow to meet changing needs in the world and all of the time everyone spent previously implementing a new language, learning a new language, or reinventing wheels in some new language, would be spent doing something actually innovative.

I dunno. Just something to think about. Or not.

[1] Thanks to The Insane Dreamer (who may totally disagree with all of the above). We had a thoughtful exchange via twitter on this subject.

The World Community Grid, whose mission is to “create the world’s largest public computing grid to tackle projects that benefit humanity”, reports 466,581 years of compute time has been racked up as of 5/13/2011 9AM.

466,581 years is 4,087,249,560 hours

At 100W assumed for a computing device running at full steam, that’s 408,724,956 kWh.

According to carbonfund.org, an average kWh of electricity translates to 1.297 lbs CO2 emissions.

The carbon footprint of the WCG, using my numbers, is 530,116,268 lbs of CO2 since its inception.

To compile Persistence of Vision Raytracer under Linux ( > 0.99pl4 )

FTP to alfred.ccs.carleton.ca and grab
/pub/pov-ray/POV-Ray1.0/povsrc.tar.Z in binary mode.

I really didn't think it was necessary to ftp the POV sources from
the above address and put them on sunsite or tsx.  You all are big
kids, you can grab it yourself rather than clogging up sunsite and
tsx anymore than they need to be.

Download povsrc.tar.Z to your Linux partition.

1.  mkdir /build
2.  mv povsrc.tar.Z to /
3.  zcat < povsrc.tar.Z | tar xvf -
    this will create a dir called /povsrc
4.  cp /povsrc/machine/unix/* /build
5.  cp /povsrc/source/* /build
6.  cd /build
7.  edit unix.c and comment out the whole function "matherr"
        (I am pretty sure matherr is in unix.c....if you can't find it
        type grep matherr *.c to find which file matherr may be in.

8.  Do ONE of the following:
         -  If you want the +d option on POV to open and display your
            raytrace in an X-window:
                cp xwindows.mak makefile
         -  If you do NOT want the X11 previewer compiled in:
                cp unix.mak makefile

9.      cp unixconf.h config.h  (note: if you use gccconf.h as config.h
        instead, don't forget to change the CC= line in the makefile to
        gcc)  It is safest to just use cc.

10. type make and press enter.....
11. if all goes well, you will have a povray binary file now.
12. move this to wherever you want it (/usr/bin or /usr/local/bin are
good places)

That's it!
You will need to get the include files that povray uses at render time
(textures.inc, shapes.inc, shapesq.inc, colors.inc etc etc) from
alfred.ccs.carleton.ca or just copy them over from your DOS partition
if you use POV under DOS.

Jeff Blaine
jblaine@garnet.acns.fsu.edu

Update 1/11/2011: Nearly all of this article is bunk now, as it turns out the GNU libc developers consider getent (the basis of everything below) to be a debugging tool only. As such, it does abnormal things. I suggested the man page should indicate as much. Anyway, here it is…

Let’s examine hostname resolution on a RHEL 5.5 box on a Sunday night. I was inspired from reading Down the ‘ls’ Rabbit Hole 2 weeks ago. I suspect any other modern Linux distro will provide nearly identical results.

The short summary is:

1. Read /etc/resolv.conf
2. Try to use nscd
3. Try to use nscd again
4. Read /etc/nsswitch.conf
5. Load libnss_files.so
6. Read /etc/host.conf
7. Try to find IPv6 address in /etc/hosts
8. Load libnss_dns.so
9. Load libresolv.so
10. Perform DNS IPv6 ‘AAAA’ query
11. Try to find IPv4 address in /etc/hosts
12. Perform DNS IPv4 ‘A’ query

Read on for the full trace with commentary.

strace -f getent hosts www.puppetlabs.com
...
open("/etc/resolv.conf", O_RDONLY)      = 3
...
close(3)                                = 0

Looking at the source for GNU libc 2.5 (which is what is installed on this box), it appears that /etc/resolv.conf is loaded in resolv/res_init.c and the explanation is given as:

/*
 * Resolver state default settings.
 */

/*
 * Set up default settings.  If the configuration file exist, the values
 * there will have precedence.  Otherwise, the server address is set to
 * INADDR_ANY and the default domain name comes from the gethostname().
 *
 * An interrim version of this code (BIND 4.9, pre-4.4BSD) used 127.0.0.1
 * rather than INADDR_ANY ("0.0.0.0") as the default name server address
 * since it was noted that INADDR_ANY actually meant ``the first interface
 * you "ifconfig"'d at boot time'' and if this was a SLIP or PPP interface,
 * it had to be "up" in order for you to reach your own name server.  It
 * was later decided that since the recommended practice is to always
 * install local static routes through 127.0.0.1 for all your network
 * interfaces, that we could solve this problem without a code change.
 *
 * The configuration file should always be used, since it is the only way
 * to specify a default domain.  If you are running a server on your local
 * machine, you should say "nameserver 0.0.0.0" or "nameserver 127.0.0.1"
 * in the configuration file.
 *
 * Return 0 if completes successfully, -1 on error
 */

Okay. I guess. Let’s move on.

...
socket(PF_FILE, SOCK_STREAM, 0)         = 3
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT
(No such file or directory)
close(3)                                = 0
socket(PF_FILE, SOCK_STREAM, 0)         = 3
fcntl(3, F_SETFL, O_RDWR|O_NONBLOCK)    = 0
connect(3, {sa_family=AF_FILE, path="/var/run/nscd/socket"...}, 110) = -1 ENOENT
(No such file or directory)
close(3)                                = 0

Why did you check nscd twice?

GNU libc nscd/nscd_helper.c is the only place with a connect() call referencing /var/run/nscd/socket (aka _PATH_NSCDSOCKET as defined in nscd/nscd-client.h). The connect() is in open_socket(), which is referenced in two places:

One:

/* Try to get a file descriptor for the shared memory segment
   containing the database.  */
static struct mapped_database *
get_mapping (request_type type, const char *key,
             struct mapped_database **mappedp)

Two:

/* Create a socket connected to a name. */
int
__nscd_open_socket (const char *key, size_t keylen, request_type type,
                    void *response, size_t responselen)

Here I took it upon myself to try to build the GNU libc code I was referencing. I figured I’d build it with debug symbols and then run getent again under gdb. The build with CFLAGS=-g spit out an error saying that it must be built with optimization. So much for that, but I did at least throw in some syslog() calls. For one, the two attempts to connect to an nscd socket above are in fact from both referenced functions.

Update 1/11/2011: This shows my lack of gdb knowledge. One doesn’t need to build in debugging symbols to see what I am trying to see. Commenter Dave W. shows that below with his traces.

Jan  3 03:57:33 new-host-2 getent: get_mapping() trying to open nscd socket with
open_socket()
Jan  3 03:57:33 new-host-2 getent: __nscd_open_socket() trying to open nscd socket
with open_socket() with open_socket()

Is that correct behavior? Could it be better? Beats me. I’m only taking it that far, but it doesn’t seem ideal.

...
open("/etc/nsswitch.conf", O_RDONLY)    = 3
...
close(3)                                = 0

Now we actually get somewhere. At least we’re reading the right configuration file at this point.

This is generated from GNU libc nss/nsswitch.c

int
__nss_database_lookup (const char *database, const char *alternate_name,
                       const char *defconfig, service_user **ni)
{
...
    service_table = nss_parse_file (_PATH_NSSWITCH_CONF);

Fine, moving on.

open("/lib64/libnss_files.so.2", O_RDONLY) = 3
...
close(3)                                = 0

This is due to “files” being first in /etc/nsswitch.conf. Fine.

...
open("/etc/host.conf", O_RDONLY)        = 3
...
close(3)                                = 0

The hell? You already found a valid /etc/nsswitch.conf. Why would you query this stupid old legacy file?

nss/getXXbyYY_r.c causes this read of /etc/host.conf

#ifdef NEED__RES_HCONF
          if (!_res_hconf.initialized)
            _res_hconf_init ();
#endif /* need _res_hconf */

Turns out this is hardcoded and not managed/overriden in any way by configure.

[jblaine@new-host-2 glibc-2.5]$ grep "#define NEED__RES_HCONF" */*
inet/gethstbyad_r.c:#define NEED__RES_HCONF     1
inet/gethstbynm2_r.c:#define NEED__RES_HCONF    1
inet/gethstbynm_r.c:#define NEED__RES_HCONF     1

??? – feel free to provide a comment on this below. I don’t understand the need for this nowadays when we have /etc/nsswitch.conf.

...
open("/etc/hosts", O_RDONLY)            = 3
...
close(3)                                = 0

Makes sense finally, at least if this was the result of doing what our /etc/nsswitch.conf said (“files dns”).

Update 1/11/2011: Oddly, this first opening of /etc/hosts is due to trying to resolve www.puppetlabs.com via an IPv6 address.

open("/lib64/libnss_dns.so.2", O_RDONLY) = 3
...
close(3)                                = 0
...
open("/lib64/libresolv.so.2", O_RDONLY) = 3
...
close(3)                                = 0

Fine.

socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.1")}, 28) = 0
...
sendto(3, "uw\1\0\0\1\0\0\0\0\0\0\3www\npuppetlabs\3com\0"..., 36, MSG_NOSIGNAL, NULL, 0) = 36
...
recvfrom(3, "uw\201\200\0\1\0\1\0\1\0\0\3www\npuppetlabs\3com\0"..., 1024, 0,
{sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.1")}, [16]) = 120
close(3)                                = 0

DNS traffic, finally.

Update 1/10/2011: After coming back to this little exercise tonight armed with Wireshark, I’ve found that this DNS request is for an IPv6 “AAAA” record. Commenter Dave W. confirmed this below. Again, this is odd to me that it would try IPv6 first.

open("/etc/hosts", O_RDONLY)            = 3
...
close(3)                                = 0

Why? What did this and what is the reason?

Update 1/11/2011: This is the attempt to look it up as an IPv4 address. The following lack of expected syslog() output is still a bit mysterious though.

Opening /etc/hosts happens in 2 GNU libc functions:

One:

void
_sethtent(f)
        int f;
{
        if (!hostf)
                hostf = fopen(_PATH_HOSTS, "r" );
        else
                rewind(hostf);
        stayopen = f;
}

Two:

struct hostent *
_gethtent()
{
...
        if (!hostf && !(hostf = fopen(_PATH_HOSTS, "r" ))) {
                __set_h_errno (NETDB_INTERNAL);
                return (NULL);
        }
...

Let’s assume our “problem” is _gethtent(). It’s referenced 3 places:

One:

struct hostent *
_gethtbyname2(name, af)
        const char *name;
        int af;

Two:

struct hostent *
_gethtbyaddr(addr, len, af)
        const char *addr;
        size_t len;
        int af;

Three:

struct hostent *
gethostent()

Oddly, with plenty of syslog() calls in _sethtent() and _gethtent() around where the fopen() of /etc/hosts happens, I cannot get them to be reached. This odd opening of /etc/hosts remains a mystery.

Moving on.

...
socket(PF_INET, SOCK_DGRAM, IPPROTO_IP) = 3
connect(3, {sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.1")}, 28) = 0
...
sendto(3, "\256\261\1\0\0\1\0\0\0\0\0\0\3www\npuppetlabs\3com\0"..., 36, MSG_NOSIGNAL, NULL, 0) = 36
...
recvfrom(3, "\256\261\201\200\0\1\0\2\0\0\0\0\3www\npuppetlabs\3com\0"..., 1024, 0,
{sa_family=AF_INET, sin_port=htons(53), sin_addr=inet_addr("192.168.1.1")}, [16]) = 66
close(3)                                = 0
...
write(1, "74.207.250.144  puppetlabs.com w"..., 5074.207.250.144  puppetlabs.com
www.puppetlabs.com) = 50
exit_group(0)                           = ?

Another DNS query before we get our screen output and getent exits. Why?

Update 1/11/2011: This is the IPv4 query of an “A” record finally.

Feel free to chime in.

facter-1.5.8:beijing> ruby install.rb --destdir=/net/silmaril/export/puppet
...
facter-1.5.8:beijing>

Good.

puppet-2.6.4:beijing> more README
Documentation (and detailed install instructions) can be found
online at http://docs.puppetlabs.com/.

Additional documentation can also be found at the Puppet Wiki:

http://projects.puppetlabs.com/projects/puppet/wiki/

Generally, you need the following things installed:

* Ruby >= 1.8.1 (earlier releases might work but probably not)

* The Ruby OpenSSL library.  For some reason, this often isn't included
  in the main ruby distributions.  You can test for it by running
  'ruby -ropenssl -e "puts :yep"'.  If that errors out, you're missing the
  library.

  If your distribution doesn't come with the necessary library (e.g., on Debian
  and Ubuntu you need to install libopenssl-ruby), then you'll probably have to
  compile Ruby yourself, since it's part of the standard library and not
  available separately.  You could probably just compile and install that one
  library, though.

* The Ruby XMLRPC client and server libraries.  For some reason, this often
  isn't included in the main ruby distributions.  You can test for it by
  running 'ruby -rxmlrpc/client -e "puts :yep"'.  If that errors out, you're
  missing the library.

* Facter => 1.5.1
  You can get this from < http://puppetlabs.com/projects/facter >

Maybe there’s a good reason, but why are you telling me all of these minimum requirements instead of just checking for it in the build/install process?

puppet-2.6.4:beijing> ruby install.rb --destdir=/net/silmaril/export/puppet
which: no ronn in (/usr/bin:/bin:/sbin:/usr/sbin:/net/silmaril/export/puppet/usr/bin)
Could not load facter; cannot install
puppet-2.6.4:beijing>

WTF is ‘ronn’? This is a fully loaded RHEL 5.5 box I am on.

puppet-2.6.4:beijing> ruby install.rb --help
which: no ronn in (/usr/bin:/bin:/sbin:/usr/sbin:/net/silmaril/export/puppet/usr/bin)
Could not load facter; cannot install
puppet-2.6.4:beijing>

Awesome!

puppet-2.6.4:beijing> ruby install.rb --fdgfsdgfds
which: no ronn in (/usr/bin:/bin:/sbin:/usr/sbin:/net/silmaril/export/puppet/usr/bin)
install: invalid option: --fdgfsdgfds
install.rb:92:in `mkdir': Permission denied - /etc/puppet (Errno::EACCES)
        from install.rb:92:in `do_configs'
        from install.rb:492
puppet-2.6.4:beijing>

So you got an invalid argument, and continued to try to mkdir? That doesn’t seem smart.

I guess I’ll look at the source for install.rb in order to find the ronn reference?

# install.rb
# ...
begin
  if $haverdoc
    ronn = %x{which ronn}
    $haveman = true
  else
    $haveman = false
  end
rescue
  puts "Missing ronn; skipping man page creation"
  $haveman = false
end
# ...

So much for displaying that error message. Why wouldn’t you also display that along with ‘$haveman = false’ after the else clause?

Whatever, let’s just try to address the fact that Facter is not found. I’m not chasing the ronn rabbit down a hole, even though it’s not clear to me if I need it or not or what the hell it is.

puppet-2.6.4:beijing> find /net/silmaril/export/puppet/usr -name fac\*
/net/silmaril/export/puppet/usr/bin/facter
/net/silmaril/export/puppet/usr/lib/ruby/site_ruby/1.8/facter.rb
/net/silmaril/export/puppet/usr/lib/ruby/site_ruby/1.8/facter
/net/silmaril/export/puppet/usr/lib/ruby/site_ruby/1.8/facter/facterversion.rb
/net/silmaril/export/puppet/usr/lib/ruby/site_ruby/1.8/facter/util/fact.rb
puppet-2.6.4:beijing>

Apparently /net/silmaril/export/puppet/usr/bin being in my PATH does not mean anything to the Ruby library search logic. This obviously isn’t Puppet’s fault.

Digging around, I find that I can set RUBYLIB for non-standard places holding Ruby libraries/modules.

export RUBYLIB=/net/silmaril/export/puppet/usr/lib
    ruby install.rb --destdir=/net/silmaril/export/puppet = FAIL
export RUBYLIB=/net/silmaril/export/puppet/usr/lib/ruby
    ruby install.rb --destdir=/net/silmaril/export/puppet = FAIL
export RUBYLIB=/net/silmaril/export/puppet/usr/lib/ruby/site_ruby
    ruby install.rb --destdir=/net/silmaril/export/puppet = FAIL
export RUBYLIB=/net/silmaril/export/puppet/usr/lib/ruby/site_ruby/1.8
SUCCESS

And apparently search for Ruby libraries only works at some unknown-to-me level of directory traversal. This, too, obviously isn’t Puppet’s fault.

So far, this is a pantload of fun.

We are angry because we are overly optimistic, insufficiently prepared for the frustrations endemic to existence. A man who screams every time he loses his keys or is turned away at an airport is evincing a touching but recklessly naive belief in a world in which keys never go astray and our travel plans are invariably assured.

- Alain de Botton, “A Week at the Airport” p. 33

Found: ryanfreitas.tumblr.com.