Blog

Remote logging with Apache 2

Peter McCurdy, 9 December 2009

Just a quick note, since this didn’t immediately come up in a Google search.

Say you have yourself an Apache 2 server on Unix (naturally), but you’d like the logs to end up on a remote machine, for any number of reasons (security, backup, just plain having all your logs in one place, what have you). So you set up rsyslogd on your log machine, set up syslog on your web servers to point to it… and you get nothing from Apache. Apache doesn’t go through syslog by default, writing its own log files instead.

The solution for Apache’s error logs is obvious enough: just add “ErrorLog syslog” to your config file. But mysteriously, there’s no analogous directive for the access logs. The solution is still straightforward enough, and it looks like:

CustomLog "|/usr/bin/logger -p local1.info -t apache2" common

I suppose this shouldn’t seem shocking, but when we were looking this up, many people seemed to forget that (a) the “logger” program exists already, and (b) you can pass options to it directly from the Apache config file, without an intermediate script. In other words, there’s no need at all to be writing additional scripts, which you might not have known based on the Google results I was getting earlier.

Google Wave Whiteboard Gadget

Peter Zion, 2 December 2009

I’ve been experimenting with writing Google Wave extensions, and I’ve built an extension that adds a shared virtual whiteboard to a Wave. If you have an account at Google Wave and would like to try it out, see instructions at http://wwb.navarra.ca.

Protocol https not supported or disabled in libcurl

David Carney, 13 November 2009

This is the second time I had to figure this out, so I’m documenting it for prosperity. It’s certainly nothing special, but I’m sure that others have wasted time searching for the solution as well. Assuming you’re using Macports, read on.

I’ve been automating a bunch of stuff with various rake tasks in a Rails project. One task in particular, has a line similar to the following:

curl --user bob@foo.com:password https://foo.com/db.sql > db.sql

Which, when run, yields the following error:

curl: (1) Protocol https not supported or disabled in libcurl

To fix it, do the following:

sudo port deactivate curl
sudo port install curl +ssl

And voila.

“git grep”, another grep alternative

Peter McCurdy, 1 November 2009

So as I was recently saying, ack is a fine grep alternative for programming tasks, but I should also mention another useful tool: git grep. It’s extremely simple, it just searches through all your checked-in files for the given expression. The advantages are that it’s extremely fast due to git’s efficient disk layout, and it’s reasonably common to want to know where a given string has been checked in (especially if you have a lot of generated source code). The drawbacks are that its command line options are a bit limited, it’s missing all of ack’s nice features, and of course it only works on git repositories. Still, a handy little tool to have at hand.

ack: grep for programmers

Peter McCurdy, 26 October 2009

If you find yourself using grep a lot to search through code files, you should probably take a look at ack (packaged as ack-grep in Debian and Ubuntu). It’s like grep, but optimized for searching through codebases, particularly ones that involve multiple languages.

As an example, say you have a project that involves a mix of C++ and Python files, and you want to find all the occurrences of “foo” in just the Python files (this happens to me all the time these days). With grep, you have to say find . -name \*.py | xargs grep "foo", or thereabouts, whereas with ack it’s just ack "foo" --python. For languages with multiple possible extensions (e.g. Perl, C++), this is even nicer.

OK, maybe you’ve already set up all the grep aliases you could ever want, so never type those ugly command lines any more. Even so, I still love ack, because the search results are that much more readable: the filename only shows up once per file, with each match within the file having only the line number taking up space, like so:

$ ack Hello
hello.c
3:// Prints "Hello, world!"
6:    printf("Hello, world!");

I find this makes the results significantly more readable. And what I haven’t tried to replicate above is how it highlights both matches and the filename with a bit of colour, making it even easier to scan.

And on top of all that, “ack” ignores your editor’s swap files by default, and is 25% less typing than “grep”! So give it a spin, I started liking it right away.

PyChecker

Mark Côté, 15 October 2009

Present Python to a group of coders used to C/C++, and the objections are pretty familiar. They almost all revolve around Python’s typing, specifically that it is dynamic and thusly only evaluated at run time. “You’ll be delivering products with all sorts of potential bugs!” My knee-jerk reactions when put on the defensive like this also follow standard patterns, such as “a compiler isn’t a substitution for thorough testing” and “so you never type cast?” But those reactions never seem to win the argument on their own because there really is no denying that Python introduces a source of errors that are caught very quickly in statically/normatively typed languages. One friend of mine objected to using Python for a weekend programming event because he didn’t want to be searching for errors like this at 2:00 am:

broken = False
...
try:
    thingy.configure()
except:
    borken = True
...
return broken

He pictured himself debugging thingy.configure(), trying to figure out why it wasn’t generating an exception while it was obviously failing, never noticing that he wasn’t setting the correct variable in the exception handler. I’m sure that many Python programmers would loudly suggest that the potential of such errors is minimal if one is careful, and indeed is an acceptable price given the speed of development conferred by Python, but would secretly be thinking of times they were burned by exactly such a mistake.

If you can, however, find these errors by carefully scanning your code, why can’t this be automated? And indeed it already has been! PyChecker “finds problems that are typically caught by a compiler for less dynamic languages, like C and C++.” And it does a pretty bang-up job too. I just found a few silly mistakes in my app, down code paths that (apparently) aren’t used very often. Hey maybe inadvertently it’s also a good tool to determine how much effort I’ve wasted coding features and error handlers that are never used…

Chaining named_scope in Rails

David Carney, 18 September 2009

This is neat. In a model, you can have something like this:

class Foo < ActiveRecord::Base
named_scope :default, {:order => "priority DESC, last_modified DESC "}
named_scope :offset, lambda{|offset| {:offset => offset}}
named_scope :limit, lambda{|limit| {:limit => limit}}
...

Then, in a controller:

class FooController < ActionController::Base
def recent
offset = params[:offset] || 0
limit = params[:limit] || 100

foos = Foo.default.offset(offset).limit(limit)

The last line chains all of the parameters together, which builds a compound query. Moreover, you can define (and chain) much more complex named_scopes!

Very cool.

iPhone hackathon 4 charity: Halifax edition

William Lachance, 28 July 2009

I think it’s fair to say that Halifax’s first iPhone hackathon for charity was a big success. The idea was pretty simple: get a group of people (developers, marketers, artists) together over a weekend and try to produce as many iPhone apps as possible over the course of a weekend. Sell the apps on the app store (or otherwise monetize them), then donate the proceeds to charity.

I think we managed to get a group of about 15 together. After the weekend was over, we had three apps in various stages of completion. They are:

  • PostCard: Send post cards, with a local twist.
  • Meet me here: A streamlined way to tell your friends where you are.
  • Civic Snitch: Report on problems in your neighbourhood using your phone. A front end to the amazing fixmystreet.ca (this is the one I worked on).

As usual for a hackfest, the energy level was amazing. In addition to seeing the familiar faces of MindSea, Applied Logic, Hand Puppet and Say Hi There, it was fantastic to meet the new faces at North Knight and an amazing group of unaffiliated (yet crazy competent) developers. A weekend is a bit too short a time to do anything but a trivial iPhone application, but we got a good start on all of them. Rumor has it that the postcard application is quite close to completion. Another few hacking sessions and we should have some apps that are good for release.

It’s hard to do justice to the overwhelming feeling of WIN that came out of this. Since co-founding Navarra a year ago, I’ve been at a ton of conferences, hack weekends, and other networking events and this has by far been the one I’ve felt the best about. What made it so great?

  • First and foremost, the feeling that the work you’re doing will be used for good.
  • The opportunity to take part in something untested and different. In these difficult times, charities are looking for new ways to fill gaps in fundraising– can software developers help?
  • The Halifax Hub’s open concept space which did so much to facilitate collaboration and communication (as it always does).
  • The amazing catered food from Local Source Organic (Splice Training also provided some tasty home-baked cookies).
  • The awesome high-quality t-shirts, featuring an amazing design by Nick Brunt (printing courtesy of Mindsea).
  • The free massages from Be Wellness.
  • DJ Rich.Ness spinning tunes for us to enjoy all of Saturday night

So what’s next? Well, that’s something we’re working out with a lawyer. :) The idea is to create some kind of legal structure that allows us to safely collect any app store proceeds and get them sent to charity, though we haven’t yet finalized on what that will look like. The hope is that we can create a model that can be reused in other cities (iHackMTL anyone?).

Likewise, the final decision on which local charities will be receiving the proceeds has not yet been made. Something like ten organizations submitted proposals before the hackfest. It’s great to see so much interest, but it’s obviously not possible to accomodate everyone this round. It’s fair to say that at least one app will be going directly to an organization which helps in some way to address poverty in the HRM. I think there’s a collective understanding among the participants at the hackfest that we’ve been quite blessed by circumstance and good fortune and that there’s a responsibility to help those who haven’t been so lucky.

As for the apps themselves, the plan is to put the source up on github ASAP under the MIT License. I’ll be sure to post an announcement when this happens (though this is of course only of interest to the hardcore geeks).

Thanks again to the participants and the sponsors (The Hub, Local Source Organic, Be Wellness, Splice Training, Say Hi There!, Mindsea, innovacorp, Nova Scotia Rural BroadBand and Development, Humina Huminah) for the amazing weekend. Most especially, Dale Zak, the event organizer (and happy hacker) deserves huge kudos for the amazing idea and the perseverance to make it happen.

Easily extensible command module through reflection

Mark Côté, 15 July 2009

The wikipedia describes reflection as “the process by which a computer program can observe and modify its own structure and behavior.” Sounds on the surface like some crazy theoretical computer science, but it’s actually easy to use reflection in real-life applications. At the risk of making this blog Python specific, I’ll show you a simple example of how you can use Python’s reflective character to create an easily extensible module for executing commands. This isn’t particularly advanced, but it does show the power of the current generation of programming languages.

Say you have some sort of application that supports a number of different commands–through network protocols or parsed files or the like. It’s not a heavy program, so you have one class that is responsible for executing the code associated with each command: you want to avoid the boilerplate of having an object for each command. In C++, each time you’d want to add a command, in addition to writing the function (and of course the declaration in the header file), you would have to add to a growing switch statement an association between the command name (probably declared as a literal somewhere to avoid spelling mistakes) and the command function. So that’s 3 or 4 things that you need to change just to make your app support a new command, and you get the ugliness of a long switch statement.

In Python, thanks to its reflective capabilities, you can create a command-execution function that requires exactly one change to add a new command: adding a new function with an appropriate name (and of course Python has no header files). This is due to the magic of getattr:

class CommandRunner(object):
    ...
    def RunCommand(self, cmd_name, args):
        cmd_func_name = 'Do_' + cmd_name
        try:
            cmd_func = getattr(self, cmd_func_name)
        except AttributeError:
            # handle non-existent command
        try:
            cmd_func(args)
        except:
            # handle exception executing command

You want to support a new command named “foo”? Just create CommandRunner.Do_foo()! If you’ve properly set up your exception handling for AttributeErrors, you can easily error out on nonexistent commands, while seamlessly executing supported commands.

Metaprogramming–no longer just for eggheads!

A Banner Day for Thunderbird

Peter McCurdy, 22 June 2009

This is a day I’ve been looking forward to for a long, long time: someone has finally checked in a patch to Mozilla Thunderbird to enable a reply-to-list button by default. I wrote some of the backend infrastructure code for this years ago, and then wrote an extension to expose the functionality, but the extension still had a few issues (and I stopped maintaining it when I stopped using Thunderbird), so having this in the default build is good news.

Now we just have to wait for the Thunderbird 3.0 release….