Recent uploads from my Flickr photostream...

Archive for the 'programming' Category

Several Perl scripts released to Github

I’ve been playing with Github lately, and rather liking it.

I did briefly try it about a year ago, and was impressed, but wasn’t driven enough to consider moving away from Subversion.

I tried it out more when I started contributing to the Dancer Perl framework, and got to like it; since then, I’ve moved several of my modules over to Github already.

I finally got round to moving some miscellanous Perl scripts too, ones which I’ve been meaning to release as open-source in case they’re of use to anyone, but didn’t get round to doing.

They’re now online at: http://github.com/bigpresh/misc-scripts.

Github made sharing my code easy – Github++;

Now, that’s enough geekery for one evening.

Dancer 1.150 released – a flexible, lightweight web framework for Perl

Version 1.150 of the Dancer web framework has just been released, so this seemed like a good time to write up this post.

Recently, I’ve been wanting to find a Perl web framework that I really got on with. I’ve used Catalyst, which is very powerful and popular, but it’s quite heavy (a lot of dependencies, and reasonably high memory usage and startup time), and I felt as though it forced me to code “the Catalyst way”, rather than staying out of my way and getting on with writing my code.

I took a look around at the current Perl web frameworks (e.g. CGI-Application, Jifty, Catalyst, Mojo…) – all good in their own ways, but for various reasons, none of them really struck me as something I’d be particularly happy to work with.

I did briefly consider trying to write my own, but that’s a wheel I do not want to re-invent – there’s enough odd-shaped wheels out there already.

When I found Dancer (a port of Ruby’s Sinatra framework), I immediately liked the fact it looked simple and stays out of the way as much as possible, so I gave it a try – and, I must say, I’m impressed.
Continue reading ‘Dancer 1.150 released – a flexible, lightweight web framework for Perl’

OpenDNS vs Google – speed comparison

I read a Twitter post earlier mentioning Google’s public DNS service, and suggesting that it could displace the popular OpenDNS

I thought it would be interesting to do a performance comparison between Google and OpenDNS, to see how they compare. I also decided to include the nameservers of my ISP, Virgin Media, to illustrate whether there are performance gains to be had by changing to OpenDNS (which I primarily use, along with others) or Google, or whether staying with defaults works. Continue reading ‘OpenDNS vs Google – speed comparison’

Configure CPAN.pm to use sudo to install

I prefer to run CPAN.pm as a normal user and have it use sudo just for the actual installation, rather than running tests etc as root.

I have an annoying habit of forgetting the option names , so I’m posting this here for my reference, but might be useful for other people too.

To do that, do the following from a CPAN shell:


o conf make_install_make_command 'sudo make'
o conf mbuild_install_build_command 'sudo ./Build'
o conf commit

Also, whilst documenting that, to set your preferred CPAN mirror:


o conf urllist unshift ftp://mirrors.uk2.net/pub/CPAN/
o conf commit

Lyrics::Fetcher::LyricWiki updated

LyricWiki.org used to provide the ability to retrieve lyrics via an API, which was handy.

Recently, LyricWiki effectively ceased, but the project and all the contributed lyrics so far were migrated to Wikia.

Unfortunately, it seems the record labels demanded that lyrics no longer be available via the API, and can only be served up on a web page.

Therefore, I’ve updated Lyrics::Fetcher::LyricWiki appropriately.

Subversion – show commit details when editing commit message

Something I’ve wanted to do for a while is get the list of changed files and a diff into the commit message in my editor when I make a commit with Subversion.

With Git, you can pass the -v (verbose) option when committing, and the commit message you edit will include diffs as well as the list of modified files.

Subversion provides no such option, so I put together a little wrapper shell script to do this for me.

The script provides a function named svncommit (which I alias to just ‘ci’ for supreme shortness :) ).

When used, after the “–This line, and those below, will be ignored–” marker line, the list of files and then diffs will be inserted, as shown in the screenshot below (click for full size):

Subversion commit message being edited

The script itself is relatively simple (it was knocked up quickly; I’ll probably improve on it sometime):

# Do an svn commit, with diffs included in the commit message
svncommit() {
 
    # Start preparing the commit message which we'll then edit
    COMMITMSG=/tmp/$USER-commitmsg
    echo > $COMMITMSG
    echo "--This line, and those below, will be ignored--" >> $COMMITMSG
    svn status "$@" >> $COMMITMSG
    echo >> $COMMITMSG
 
    # Now do a diff; work out stats on lines added/removed by looking at
    # the diff, add that info, then the diff itself
    svn diff "$@"   > /tmp/$USER-svndiff
    LINESADDED=$(  grep '^+[^+]' /tmp/$USER-svndiff | wc -l)
    LINESREMOVED=$(grep '^-[^-]' /tmp/$USER-svndiff | wc -l)
    echo "Added $LINESADDED lines, removed $LINESREMOVED lines" >> $COMMITMSG
    echo >> $COMMITMSG
    cat /tmp/$USER-svndiff >> $COMMITMSG
    echo >> $COMMITMSG
 
    ORIGMD5=$(md5sum $COMMITMSG)
    $VISUAL $COMMITMSG
 
    if [[ "$(md5sum $COMMITMSG)" == "$ORIGMD5" ]]; then
        echo "Commit message unchanged, commit aborted";
    else
        svn commit "$@" -F $COMMITMSG
    fi
 
    rm $COMMITMSG
    rm /tmp/$USER-svndiff
}

Retagging audio tracks based on filename

I had some audio tracks which weren’t tagged, but did have filenames containing the artist, title etc, so I whipped up a quick Perl script to sort them out – retag-by-filename.pl.

It takes a regular expression with named captures for track, title, artist and comment, and sets the tgs on the file as appropriate.

It makes use of Music::Tag to do the actual tagging and Getopt::Lucid to read the options supplied, and requires Perl 5.10.0 for named regex captures (and ’say’).

A --dry-run option allows you to check that the filenames are being parsed correctly by your regex before actually writing tags.

See retag-by-filename.pl for the full details.

Easy CLI option parsing with Getopt::Lucid

I often write Perl scripts which need to read options given on the command line. Normally I turn to the venerable old Getopt::Long which does the job.

However, I was writing a script which needed to be able to accept only certain parameters, which were mostly optional, and also take a list of filenames. I wanted this to be possible in any order, e.g.:

myscript --foo=foo --bar=bar file1 file2
myscript file1 file --foo foo

Getopt::Lucid makes this all pretty easy, and also makes the code pretty self-documenting, too. Straight from the documentation, showing the various types of parameters it can parse:

@specs = (
    Switch("version|V"),
    Counter("verbose|v"),
    Param("config|C"),
    List("lib|l|I"),
    Keypair("define"),
    Switch("help|h"),
);
 
$opt = Getopt::Lucid->getopt( \@specs );
 
$verbosity = $opt->get_verbose;
@libs = $opt->get_lib;
%defs = $opt->get_define;

A real-world example,from one of my scripts which handles ID3-tagging:

# The options we can take - these correspond to the name of the tag that
# they'll set:
my @options = qw(track title artist album comment);
 
my @option_specs = (
    Getopt::Lucid::Switch('verbose|v'),
    map { Getopt::Lucid::Param($_) } @options,
);
my $opt = Getopt::Lucid->getopt(\@option_specs);
 
my @tags_to_set = grep { $opt->{seen}{$_} } @options;
my @files = @{ $opt->{target} };
 
if (!@tags_to_set) {
    say "Nothing to do.  Use one or more of the options:\n" .
        join ', ', map {'--'.$_} @options;
    exit;
}

(The script then goes on to loop over all files, and use Music::Tag to set the ID3 tags requested).

Easy file finding with File::Find::Rule

Recently I found File::Find::Rule on the CPAN, and I’m impressed how easy it makes it to get a list of files to work on.

A fairly common way to do this in Perl would be something like:

my $dirh = new DirHandle($somedir);
while (my $entry = $dirh->read) {
    # Skip hidden files and directories:
    next if ($entry =~ /^\./ || !-f $entry);
 
    # Skip if it doesn't match the name we want:
    next if ($entry !~ /\.txt$/);
 
    print "Found: $somedir/$entry\n";
}

File::Find::Rule makes things rather easier:

my @files = File::Find::Rule->file()->name('*.txt')->in($somedir);

Various conditions can be chained together to find exactly what you want.

Another example, showing combining rules with ->any() to find files matching any of those conditions:

# find avis, movs, things over 200M and empty files
my @files = File::Find::Rule->any(
    File::Find::Rule->name( '*.avi', '*.mov' ),
    File::Find::Rule->size( '>200M' ),
    File::Find::Rule->file->empty,
)->in('/home');

There’s plenty of other ways to do this, but I think File::Find::Rule gives a way to clearly and concisely state what you want and get the job done.

Favourite new Perl features

I’ve been starting to make use of the new features introduced in perl 5.10 recently (after being constrained by my main dev environments still running perl 5.8.8, and not having the time to upgrade).

My favourite features so far are:

The smart match operator

The new smart-match operator, ~~, is a great example of DWIM.

A few examples:

if (@a ~~ 'foo')  # list contains at least one item equalling 'foo'
if (@a ~~ /foo/) # list contains at least one item matching /fo+/
if (@a ~~ @b)   # lists contain same values

That’s just a brief overview; there’s plenty more documentation

say

Not a big change, but the new say keyword acts just like print, but adds an implicit newline to the end – so say 'Hello'; is just the same as print "Hello\n";

It’s more useful in cases where you would have had to add parenthesis to get correct precedence – something like: print join(';', @foo) . "\n"; can now be written more concisely as just say join ';', @foo;.

Switch (given) statement

given ($foo) {
    when (/^abc/) { abc(); }
    when (/^def/) { def(); }
    when (/^xyz/) { xyz(); }
    default { die "Unrecognised foo"; }
}

Defined-or

// is now the defined-or operator.

It's pretty common to use conditional assignments like: $a ||= $b to assign to $a unless $a already has a value. Now you can use $a //= $b to test for definedness rather than truthiness.

Likewise, if ($hash{foo} // $hash{bar}) will be true if either of them is defined (even if they're defined but have a false value).

Named regex captures

Parenthesised sub-expressions in regular expressions can now be given a name, and accessed via the special %+ hash:

if ($foo =~ m{ (?<year> \d{4} ) - (?<month> \d{2}) - (?<day> \d{2}) }xms) {
    say "Year: $+{year}";
}

The features above are my own personal favourites, in no particular order. The full (large) set of changes can be found in the perldelta for 5.10.0.


Random photos from my Flickr photostream...