Django: Taking note of ManyToManyField changes

January 18, 2012 – 9:05 pm

I’ve gone through a bit of a mess lately, trying to do something I considered very possible, only to ultimately fail to find a working solution. This write-up will hopefully save someone else several hours of effort, asking for help, waiting, etc.

Those reading the article, please be sure to also read any comments to the article, as others may have additional ideas, solutions, or helpful information.

As you can see from the documentation links in this post, I am working with version 1.3. I am not aware of any differences in 1.3.1 or 1.4.0 regarding the topic covered in this post.

Scenario

You have a Django model. That model has a field which is a ManyToManyField. When an instance of your model is modified via any mechanism (custom web form, Django admin interface, code using the ORM, etc), you want to execute some custom code based on what happened to the model instance’s fields (including the ManyToManyField).

Specifically, what if you need to keep a non-Django source in sync with a Django model’s data? In my case, I have a “simple” need to push changes to a Django model’s data to an LDAP server. Read More »

Thunderbird 8 rewrote my subject?

December 14, 2011 – 12:41 pm

I have no Thunderbird extensions installed.

I sent this message (in blue below) with the subject shown. I received it too, as I was a recipient.

I then selected it (as you see above) and replied to it (replying to my own message, to add new info). I didn’t touch the subject line. I sent the new reply out. I receieved it too, as I was a recipient. Here is how I received it. Take note of the changed subject.

Update: Testing shows that the subject is being altered by Thunderbird as soon as I click on the original message and hit reply. The reply window appears with the subject altered.

Never assume customers will register complaints

November 21, 2011 – 4:46 pm

We’re an enterprisey shop. Our department has no public-facing services. Our shop’s customers are engineer employees at our company, and you could basically consider our department a large lab operations group at a deep-but-slow think tank. This will be a pretty moronic post to anyone involved in providing performant web services for a salary, but I’m always readily willing to look moronic to tell a story.

As an aside, in an unrelated ticket, a customer mentioned slow page load times for one of the pages served by our GForge Advanced Server instance. He indicated that this particular page always takes nearly 20 seconds to load and offered that it might be tied to the fact that he is a member of roughly 20 projects hosted via that GForge server.

This aside from the customer was a remarkable coincidence, as I had created an internal task ticket 2-3 weeks ago for us to implement metrics gathering for this service because “we should”. We already were performing the most basic of up/down monitoring for the host and service via Nagios. Now we get to have a problematic baseline of metrics and watch things improve from here.

After tweaking our Apache logging to log request service time (%D via mod_log_config, we noticed some (too many) problematic pages for certain users and projects. One of those pages was, of course, the page the customer had reported.

So far, we’ve instrumented metric gathering for each block of PHP (…) code in the most commonly accessed problematic page and tracked down the specific section where the slowness happens. The metric gathering is simplistic: for each major block of execution in the PHP file, store a start time, store an end time, and calculate total seconds to execute that block. Finally, syslog() the accumulated metrics as one line.

Anyway, that’s not the point of this post. The point is: Never stand up a service that works overall and assume your users will complain of slowness. Turns out 10-20 users have been quietly suffering through seriously long page load times for years now without saying a word to us. According to other admins in other departments (who responded to our nice-but-essentially “WTF people?” message to all customers of the GForge service), they’ve experienced the same phenomenom.

On Pithy Bullshit Analogies

November 14, 2011 – 3:05 pm

I’ve had it with the pithy bullshit analogies, especially via Tweets.

Belittling those who productively scrutinize and debate similar software offerings is LAME.

I’m so sick of hearing people dismiss “Chef vs. Puppet” discussions (and hundreds of other very valid comparison-style blog posts, etc) as worthless, un-enlightened, and several other holier-than-thou attributions, as if the content is a simplistic bantering about Coke vs. Pepsi.

My camel’s back broke when John Allspaw (@allspaw) recently tweeted:

Sometimes I imagine web engineers as carpenters; too busy being angry about each other’s brand of hammers to actually build the damn house.

Greg Fodor (@gfodor) replied with:

amateurs argue over tools. journeymen argue over techniques. experts argue if we should be building the house in the first place.

Both of these statements are nothing more than horriffic broccoli farts with the hive-minded in the DevOps world apparently smelling them as roses. I don’t know Greg Fodor, but I’m pretty positive John Allspaw is a lot more intelligent than that bullshit tweet of his, and he should be concerned about saying what he actually means, in detail. Because, you know, people listen to him.

Decisions are all based on something. Just because you don’t need to bother yourself with making the “small” decisions anymore doesn’t give you a new license to be an asshole toward those involved in discussions about those “small” decisions. Those people making the “small” decisions now, with debates and careful evaluations surrounding them, MAKE SHIT ULTIMATELY HAPPEN. I understand that everyone, more and more, just wants stuff to magically happen, but the reality still is that things need to be planned, discussed, scrutinized, evaluated, and finally decided on. Don’t get pissy because that part still has to happen and your Idea-to-Realization time can’t be measured in minutes.

God damn.

Graphite for RHEL 5.7

November 1, 2011 – 8:14 pm

Graphite install for RHEL 5.7 from scratch.

Official Graphite documentation is at http://graphite.readthedocs.org/

export PATH=/usr/bin:/bin:/usr/sbin:/sbin
mkdir -p /graphite/src
cd /graphite/src

# Download Python 2.7.x source from python.org
# Unpackage

cd Python-2.7.2
./configure --prefix=/graphite
make
make install
cd ..

# Now make sure you get your python before RHEL's

export PATH=/graphite/bin:$PATH

# Download Django 1.x source from django.org
# Unpackage

cd Django-1.3
python setup.py install
cd ..

# Download django-tagging from http://code.google.com/p/django-tagging/
# Unpackage

cd django-tagging-0.3.1
python setup.py install
cd ..

# Download Twisted 11.x source here from http://twistedmatrix.com/
# Unpackage

cd Twisted-11.0.0
python setup.py install
cd ..

# Download setuptools from http://pypi.python.org/pypi/setuptools
# Unpackage

cd setuptools-0.6c11
python setup.py install
cd ..

# Download pip from http://pypi.python.org/pypi/pip
# Unpackage

cd pip-1.0.2
python setup.py install
cd ..

# Now you have to build the Cairo graphics library, as RHEL's
# is too old for py2cairo's needs.

# Start with pixman, a dependency for building cairo!

# Download pixman from http://www.cairographics.org/releases/
# Unpackage

cd pixman-0.22.2
./configure --prefix=/graphite
make
make install

# Download cairo from http://www.cairographics.org/releases/
# Unpackage

cd cairo-1.10.2
# Make sure we find our shit and don't have runtime linker problems
export LDFLAGS="-L/graphite/lib -Xlinker -rpath -Xlinker /graphite/lib"
# PKG_CONFIG_PATH is set below so pixman is found
export PKG_CONFIG_PATH=/graphite/lib/pkgconfig
# Turn off all the X shit and gobject since RHEL glib2 is too old
./configure --prefix=/graphite --without-x --enable-xlib=no \
    --enable-xlib-xrender=no --enable-xcb-shm=no --enable-qt=no \
    --enable-gl=no --enable-gobject=no
make
make install

# Download Pycairo (aka py2cairo) from http://www.cairographics.org/pycairo/
# Unpackage

# Had to add these to LDFLAGS or ./waf configure bombs on Python.h test
# due to not including them when it should be in its test
export LDFLAGS="$LDFLAGS -lm -ldl -lutil"
cd py2cairo-1.10.0
./waf configure --prefix=/graphite
./waf build
./waf install
cd ..

# Download carbon, whisper, and graphite-web from
# https://launchpad.net/graphite
# Unpackage

cd whisper-0.9.9
python setup.py install
cd ..

cd carbon-0.9.9
python setup.py install
cd ..

cd graphite-web-0.9.9
./check-dependencies.py # Ignore WARNINGs as you see fit
python setup.py install
cd ..