Those reading the article, please be sure to also read any comments to the article, as others may have additional ideas, solutions, or helpful information.

As you can see from the documentation links in this post, I am working with version 1.3. I am not aware of any differences in 1.3.1 or 1.4.0 regarding the topic covered in this post.

Scenario

You have a Django model. That model has a field which is a ManyToManyField. When an instance of your model is modified via any mechanism (custom web form, Django admin interface, code using the ORM, etc), you want to execute some custom code based on what happened to the model instance’s fields (including the ManyToManyField).

Specifically, what if you need to keep a non-Django source in sync with a Django model’s data? In my case, I have a “simple” need to push changes to a Django model’s data to an LDAP server.

class Interface(models.Model):
    fqdn = models.CharField(primary=True,
                            verbose_name="Fully Qualified Domain Name",
                            max_length=80)

class Netgroup(models.Model):
    name = models.CharField(primary=True,
                            max_length=80)

    interfaces = models.ManyToManyField(model=Interface,
                                        null=True,
                                        blank=True)

Did NetgroupInstance.interfaces change? What changed? Was 1 interface removed from 500? What was that 1 interface? Were 9 interfaces added to the existing 15 interfaces? What were the 9 interfaces? Was this a brand new Netgroup creation with interfaces added at creation-time? What were the interfaces?

Solution 1?

Just override the Netgroup save() method! Call the superclass’s save first, then run your custom code against the saved data.

FAILS: The ManyToManyField data is not reliable at this point in the code flow for some reason. See the first few posts in my thread here.

Solution 2?

Maybe try Django’s signals? Register your custom code as a callback function for the Django post_save signal?

FAILS: Does not work properly in the ‘Admin’ interface at a minimum. Creating a new Netgroup with some Interface relations selected does not show the Interface relations inside the post_save callback function. Another similar case to above. See the final post here (as of 1/18/2012).

Solution 3?

Try solution 2 in combination with another signal callback function for the m2m_changed Django signal. This was recommended to me in #django-users IRC as the way to do it.

FAILS: Does not work properly in the ‘Admin’ interface. Removing Interface relationships does not present itself as data associated with the pre_remove and/or post_remove “actions”. Instead, it appears the ‘Admin’ interface clears all interfaces from the field and then adds the correct interfaces back in. See bug 16073, bug 14482, and this thread.

Sad Conclusion

If there wasn’t a bug, we would obviously use solution 3, even though I find it goofy to have to catch an extra signal (m2m_changed) when a model happens to have a ManyToManyField on it.

As it is now, we just use “Solution 2″ and tell users of the ‘Admin’ interface, via extra help_text on the Netgroup.name field, to enter the name, save, then add items to the Netgroup instance.

class Interface(models.Model):
    fqdn = models.CharField(primary=True,
                            verbose_name="Fully Qualified Domain Name",
                            max_length=80)

class Netgroup(models.Model):
    name = models.CharField(primary=True,
                            max_length=80,
                            help_text="WARNING: save new netgroups with just the name, then add interfaces and save again.  Sorry!")

    interfaces = models.ManyToManyField(model=Interface,
                                        null=True,
                                        blank=True)

from django.db.models.signals import post_delete, post_save, m2m_changed

def netgroup_delete(sender, **kwargs):
    pass # DO STUFF HERE

def netgroup_save(sender, **kwargs):
    pass # DO STUFF HERE

post_delete.connect(netgroup_delete, sender=Netgroup,
                    dispatch_uid="netgroup_delete")
post_save.connect(netgroup_save, sender=Netgroup,
                  dispatch_uid="netgroup_save")

I sent this message (in blue below) with the subject shown. I received it too, as I was a recipient.

I then selected it (as you see above) and replied to it (replying to my own message, to add new info). I didn’t touch the subject line. I sent the new reply out. I receieved it too, as I was a recipient. Here is how I received it. Take note of the changed subject.

Update: Testing shows that the subject is being altered by Thunderbird as soon as I click on the original message and hit reply. The reply window appears with the subject altered.

As an aside, in an unrelated ticket, a customer mentioned slow page load times for one of the pages served by our GForge Advanced Server instance. He indicated that this particular page always takes nearly 20 seconds to load and offered that it might be tied to the fact that he is a member of roughly 20 projects hosted via that GForge server.

This aside from the customer was a remarkable coincidence, as I had created an internal task ticket 2-3 weeks ago for us to implement metrics gathering for this service because “we should”. We already were performing the most basic of up/down monitoring for the host and service via Nagios. Now we get to have a problematic baseline of metrics and watch things improve from here.

After tweaking our Apache logging to log request service time (%D via mod_log_config, we noticed some (too many) problematic pages for certain users and projects. One of those pages was, of course, the page the customer had reported.

So far, we’ve instrumented metric gathering for each block of PHP (…) code in the most commonly accessed problematic page and tracked down the specific section where the slowness happens. The metric gathering is simplistic: for each major block of execution in the PHP file, store a start time, store an end time, and calculate total seconds to execute that block. Finally, syslog() the accumulated metrics as one line.

Anyway, that’s not the point of this post. The point is: Never stand up a service that works overall and assume your users will complain of slowness. Turns out 10-20 users have been quietly suffering through seriously long page load times for years now without saying a word to us. According to other admins in other departments (who responded to our nice-but-essentially “WTF people?” message to all customers of the GForge service), they’ve experienced the same phenomenom.

Belittling those who productively scrutinize and debate similar software offerings is LAME.

I’m so sick of hearing people dismiss “Chef vs. Puppet” discussions (and hundreds of other very valid comparison-style blog posts, etc) as worthless, un-enlightened, and several other holier-than-thou attributions, as if the content is a simplistic bantering about Coke vs. Pepsi.

My camel’s back broke when John Allspaw (@allspaw) recently tweeted:

Sometimes I imagine web engineers as carpenters; too busy being angry about each other’s brand of hammers to actually build the damn house.

Greg Fodor (@gfodor) replied with:

amateurs argue over tools. journeymen argue over techniques. experts argue if we should be building the house in the first place.

Both of these statements are nothing more than horriffic broccoli farts with the hive-minded in the DevOps world apparently smelling them as roses. I don’t know Greg Fodor, but I’m pretty positive John Allspaw is a lot more intelligent than that bullshit tweet of his, and he should be concerned about saying what he actually means, in detail. Because, you know, people listen to him.

Decisions are all based on something. Just because you don’t need to bother yourself with making the “small” decisions anymore doesn’t give you a new license to be an asshole toward those involved in discussions about those “small” decisions. Those people making the “small” decisions now, with debates and careful evaluations surrounding them, MAKE SHIT ULTIMATELY HAPPEN. I understand that everyone, more and more, just wants stuff to magically happen, but the reality still is that things need to be planned, discussed, scrutinized, evaluated, and finally decided on. Don’t get pissy because that part still has to happen and your Idea-to-Realization time can’t be measured in minutes.

God damn.

Official Graphite documentation is at http://graphite.readthedocs.org/

export PATH=/usr/bin:/bin:/usr/sbin:/sbin
mkdir -p /graphite/src
cd /graphite/src

# Download Python 2.7.x source from python.org
# Unpackage

cd Python-2.7.2
./configure --prefix=/graphite
make
make install
cd ..

# Now make sure you get your python before RHEL's

export PATH=/graphite/bin:$PATH

# Download Django 1.x source from django.org
# Unpackage

cd Django-1.3
python setup.py install
cd ..

# Download django-tagging from http://code.google.com/p/django-tagging/
# Unpackage

cd django-tagging-0.3.1
python setup.py install
cd ..

# Download Twisted 11.x source here from http://twistedmatrix.com/
# Unpackage

cd Twisted-11.0.0
python setup.py install
cd ..

# Download setuptools from http://pypi.python.org/pypi/setuptools
# Unpackage

cd setuptools-0.6c11
python setup.py install
cd ..

# Download pip from http://pypi.python.org/pypi/pip
# Unpackage

cd pip-1.0.2
python setup.py install
cd ..

# Now you have to build the Cairo graphics library, as RHEL's
# is too old for py2cairo's needs.

# Start with pixman, a dependency for building cairo!

# Download pixman from http://www.cairographics.org/releases/
# Unpackage

cd pixman-0.22.2
./configure --prefix=/graphite
make
make install

# Download cairo from http://www.cairographics.org/releases/
# Unpackage

cd cairo-1.10.2
# Make sure we find our shit and don't have runtime linker problems
export LDFLAGS="-L/graphite/lib -Xlinker -rpath -Xlinker /graphite/lib"
# PKG_CONFIG_PATH is set below so pixman is found
export PKG_CONFIG_PATH=/graphite/lib/pkgconfig
# Turn off all the X shit and gobject since RHEL glib2 is too old
./configure --prefix=/graphite --without-x --enable-xlib=no \
    --enable-xlib-xrender=no --enable-xcb-shm=no --enable-qt=no \
    --enable-gl=no --enable-gobject=no
make
make install

# Download Pycairo (aka py2cairo) from http://www.cairographics.org/pycairo/
# Unpackage

# Had to add these to LDFLAGS or ./waf configure bombs on Python.h test
# due to not including them when it should be in its test
export LDFLAGS="$LDFLAGS -lm -ldl -lutil"
cd py2cairo-1.10.0
./waf configure --prefix=/graphite
./waf build
./waf install
cd ..

# Download carbon, whisper, and graphite-web from
# https://launchpad.net/graphite
# Unpackage

cd whisper-0.9.9
python setup.py install
cd ..

cd carbon-0.9.9
python setup.py install
cd ..

cd graphite-web-0.9.9
./check-dependencies.py # Ignore WARNINGs as you see fit
python setup.py install
cd ..

Support, what RHEL service is logging this?  rhnsd?  yum-updatesd?
It should show the process name in the syslog output.  Please fix?

Oct 27 10:00:04 rcf-monitor : 4 updates available

Response:

I think that rfc-monitor is likely a third-party service of some sort. I haven’t been able to find any mention of it internally or any package or file with that name on RHN. So I don’t think that’s yum or rhnsd talking.

Support,

Syslog logs the hostname.  rcf-monitor is the hostname.

Proper syslogging looks like this, where you will note
"ntpd" as the service responsible for generating the
syslog line.

Nov  1 03:52:51 rcf-monitor ntpd[27018]: message here...

Designed by yours truly. If you’d like to do something else with the design, here it is in multiple formats, and I’d love to hear if you liked it.

• OpenDocument Drawing
• Big-ass PNG @ 300dpi
• Encapsulated PostScript
• Plain SVG
• Inkscape SVG

I’d never ever delved into this sort of thing before (I had to look up what a PostgreSQL sequence was) as I’m not a DBA by trade. I also won’t likely have to do this again for quite some time, so here are my notes:

Query the last value for the sequence (well call this SEQLAST below, and assume it returned 3):

SELECT last_value FROM mytable_sequence;

Query for the largest id (our primary key, we’ll call this LARGESTID below and assume it returned 18133):

SELECT id FROM mytable ORDER BY id DESC;

If SEQLAST and LARGESTID match, you’re all set. If they don’t, you need to adjust mytable_sequence so that the next primary key issued is LARGESTID + 1 for mytable. You do this with the setval sequence function.

SELECT setval('mytable_sequence', 18133);

This reads as, “Set the last value of mytable_sequence to 18133 and increment before handing out the next number when nextval() is called for this sequence.” The incrementing is implied, though you could also specify the clearer and functionally equivalent statement via:

SELECT setval('mytable_sequence', 18133, true);

You could also specify “Set the next value of mytable_sequence to 18134 and do not increment before handing out the next number when nextval() is called for this sequence.”

SELECT setval('mytable_sequence', 18134, false);

Reference: Google Search: site:postgresql.org postgresql setval

From The Robustness Principle Reconsidered: