Posts on Free Range Bits

Automatic Theme Switching in Emacs

Mon, 24 Feb 2025 12:23:47 +0100

I’m a dark theme nerd. Partially because of the hacker aesthetic, but mostly because I prefer to work in dark environments and light themes turn computer screens into floodlights.

However, in well lit rooms (or when working outdoors) dark themes become completely unreadable. Such is the case in my office during the morning hours. Naturally, I figured out how to change the system theme from the command line, immediately updating running GTK and Qt applications. The next step was getting Emacs to respond to the same event.

Theme Switching and D-Bus

Linux and other Unix-like operating systems use D-Bus for interprocess communication. Applications can use D-Bus to register methods and properties which can be accessed and monitored from any D-Bus client. For example, to find out whether the system is currently using a light or dark theme, call the org.freedesktop.portal.Settings.Read method from the command line using the dbus-send tool:

dbus-send --session --print-reply \
  --dest=org.freedesktop.portal.Desktop \
  /org/freedesktop/portal/desktop \
  org.freedesktop.portal.Settings.Read \
  string:org.freedesktop.appearance string:color-scheme

This command will print 1 when the system is using a dark theme and 2 when using a light theme (plus some other stuff you can ignore).

With that information in hand we need to get Emacs to do two things:

Call this method when starting up and set its theme to match the system, and
Monitor the color-scheme property and change its theme automatically when the system theme changes.

Emacs Groundwork

In order to get Emacs to change its theme we’ll need a way to store our preferred dark and light themes, and a way to change the current theme based on the results of a D-Bus call:

(defvar pjones:dark-theme 'ef-maris-dark
  "Default dark theme.")

(defvar pjones:light-theme 'ef-maris-light
  "Default light theme.")

(defun pjones:theme-from-dbus (value)
  "Change the theme based on a D-Bus property.

VALUE should be an integer or an arbitrarily nested list that
contains an integer.  When VALUE is equal to 2 then a light theme
will be selected, otherwise a dark theme will be selected."
  (load-theme (if (= 2 (car (flatten-list value)))
                  pjones:light-theme
                pjones:dark-theme)
              t))

You might be wondering what flatten-list is for. It turns out that D-Bus gives two different types of answers depending on how you requested a property. When invoking a D-Bus method, the result comes back to Emacs as a list. For example, when requesting the current color scheme using the Read method the response will be (1) or (2) depending on whether the current system theme is dark or light.

However, when the property changes and D-Bus notifies clients that are listening for change events, Emacs sees the property as a nested list. In this case the color scheme value will be ((1)) or ((2)). The call to flatten-list ensures that both of these situations look the same to the callback function (pjones:theme-from-dbus).

The only thing left is to convince Emacs to set its theme using information from D-Bus.

Using D-Bus from within Emacs

It should come as no surprise that Emacs ships with its own D-Bus library. To find out what the current color scheme is we can call the Read method using dbus-call-method-asynchronously. And then to be notified when the system theme changes we can use dbus-register-signal to be notified when the SettingChanged event is emitted. The arguments to these functions are a bit crazy but they mirror those given to the dbus-send tool.

(require 'dbus)

;; Set the current theme based on what the system theme is right now:
(dbus-call-method-asynchronously
   :session "org.freedesktop.portal.Desktop"
   "/org/freedesktop/portal/desktop"
   "org.freedesktop.portal.Settings"
   "Read"
   #'pjones:theme-from-dbus
   "org.freedesktop.appearance"
   "color-scheme")

;; Register to be notified when the system theme changes:
(dbus-register-signal
   :session "org.freedesktop.portal.Desktop"
   "/org/freedesktop/portal/desktop"
   "org.freedesktop.portal.Settings"
   "SettingChanged"
   (lambda (path var value)
     (when (and (string-equal path "org.freedesktop.appearance")
                (string-equal var "color-scheme"))
       (pjones:theme-from-dbus value))))

That’s it. Switching the system color scheme should now cause Emacs to update its theme too. And when Emacs starts it should automatically set its theme to match the system theme.

Command Line Theme Switching

I mentioned earlier that I figured out how to change the system theme using the command line. The most reliable way that I have found so far is to use gsettings, which also emits the SettingChanged event via D-Bus:

gsettings set org.gnome.desktop.interface color-scheme prefer-dark
gsettings set org.gnome.desktop.interface gtk-theme Adwaita-dark

# or

gsettings set org.gnome.desktop.interface color-scheme prefer-light
gsettings set org.gnome.desktop.interface gtk-theme Adwaita

The second invocation of gsettings (changing the GTK theme) isn’t strictly necessary. However, I configured Qt to use the GTK theme and this is the only way I could get Qt applications to also change their color scheme as they don’t appear to respect the color-scheme setting.

I also wrapped these commands up in XDG desktop entries so I could invoke them from tools like Rofi.

Appendix: GTK and Qt Themes

It probably matters how I configured GTK and Qt, although these don’t have anything to do with Emacs. Basically, GTK is set to use the Adwaita-dark theme, and Qt is configured to use its gtk3 engine which should follow the current GTK theme. This works for almost all Qt applications, although some refuse to update to the light theme.

All of my GTK/Qt settings are done using Home Manager in this file.

Patching Instead of Pinning

Sat, 06 Jan 2024 11:44:19 -0700

In a previous post I showed how to work around a deadlock between GnuPG and Emacs. As a brief recap, GnuPG 2.4.1 introduced a change in its output which breaks a protocol that Emacs relied on, so I pinned GnuPG to version 2.4.0 on my system to avoid the bug.

Damien Cassou reached out to me and expressed a preferred way of dealing with this issue: patch the current version of GnuPG so it doesn’t contain the bug. This means we get to keep any bug fixes and security updates in the most recent version without having to suffer with the deadlock bug.

I’m going to demonstrate how to apply a patch to GnuPG in nixpkgs, but before that let’s review the Nix code for pinning GnuPG to version 2.4.0.

Pinning the GnuPG Package

In the last post we looked at three different options for pinning GnuPG:

A nixpkgs overlay to replace GnuPG globally
Replace the version of GnuPG that shows up in the system PATH via the NixOS module
Replace the version of GnuPG that shows up in the user environment via the Home Manager module

My preference was for the last option, using Home Manager:

{ pkgs }:

{
  programs.gpg.package = pkgs.gnupg.overrideAttrs (orig: {
    version = "2.4.0";
    src = pkgs.fetchurl {
      url = "mirror://gnupg/gnupg/gnupg-2.4.0.tar.bz2";
      hash = "sha256-HXkVjdAdmSQx3S4/rLif2slxJ/iXhOosthDGAPsMFIM=";
    };
  });
}

However, I didn’t really explain what was going on in that Nix code. I’d like to give that a try now.

Overriding Nix Packages

One way to think about a package written in Nix is as a set of attributes. (Of course, this isn’t true, but it’s a useful analogy and allows us to continue without having to talk about lazy evaluation and fixpoints.) These attribute sets are immutable so we can’t change them. But what we can do is build new packages from existing ones.

This is done with the overrideAttrs function, which allows us to create a new package by replacing attributes in an existing package. The argument to overrideAttrs is a function that is given the existing attributes and should return replacement attributes.

In the Nix code above that pins GnuPG to version 2.4.0, I replaced the version and src attributes. The result is a new package based on the existing GnuPG 2.4.1 package, except that the upstream source code is now coming from the GnuPG 2.4.0 tarball.

This won’t always work, but it does in this case because GnuPG 2.4.0 and 2.4.1 are so similar. If I tried to update the GnuPG package by replacing its source code with that of the Linux kernel things wouldn’t turn out so nicely.

The most interesting thing about overrideAttrs is that we can replace any package attribute we want. I’ve used this to great effect over the years: adding shell commands to the end of install scripts, changing compiler flags, installing files that were missed by make install, to name a few.

And of course, we can use this to apply patches to the source code before building.

Patching the Package Instead

Nix packages have an optional patches attribute which is a list of patch files to apply to the source code before building. And thanks to overrideAttrs we can add patches to that list:

final: prev:

{
  gnupg_plus_960877b = prev.gnupg.overrideAttrs (orig: {
    patches = (orig.patches or [ ]) ++ [
      (prev.fetchurl {
        url = "https://github.com/gpg/gnupg/commit/960877b10f42ba664af4fb29130a3ba48141e64a.diff";
        sha256 = "0pa7rvy9i9w16njxdg6ly5nw3zwy0shv0v23l1mmi0b7jy7ldpvf";
      })
    ];
  });
}

This example is an overlay and is straight out of my Nix configuration. It creates a new package (gnupg_plus_960877b) from the exiting gnupg package by replacing the patches attribute with one that includes the fix for the deadlock bug.

I ensure that I’m including patches from the package I’m modifying by using the original patches list (orig.patches), or if that’s missing, an empty list. Then I append ("++") a list with one element which uses fetchurl to get the patch I want.

The only thing left to do is tell Home Manager to use this new package:

{ pkgs }:

{
  programs.gpg.package = pkgs.gnupg_plus_960877b;
}

Conclusion

Nix and nixpkgs make it very easy to alter an existing package thanks to the overrideAttrs function. Using nixpkgs overlays we can even replace a package globally throughout the package set.

Sometimes we may want to alter a package’s src attribute to downgrade (or upgrade) it to another version. In this case we took Damien Cassou’s advice and applied a patch to the exiting GnuPG package.

Update (February 23, 2024): GnuPG 2.4.4 has been released and it contains the patch mentioned in this post. In other words, GnuPG 2.4.4 and Emacs work nicely together once again.

Archiving in Org Mode

Thu, 04 Jan 2024 12:45:09 -0700

As some of you may know, I practically live inside Emacs. And like a lot of Emacs users a good portion of that time is spend in org-mode, a package that is hard to describe due to its overwhelming large number of features.

One of the many ways I use org-mode is to implement a Getting Things Done (GTD) workflow. That is, managing all of the projects that I’m trying to push forward as well as all of the routine recurring tasks that I have to deal with on a daily basis.

The issue that every org-mode user eventually has to wrestle with is what to do with all those completed projects and obsolete tasks. For some projects the obvious answer is to simply delete them. This is especially true if you’re using a version control system to track your org-mode files. But if you regularly need to refer to completed projects/tasks (i.e. “what was that crazy query I ran last time?”) then archiving is a better option.

Archiving Options in org-mode

In org-mode, archiving is the process of moving a subtree somewhere else when it’s no longer needed¹. The destination for an archived subtree is taken from the org-archive-location variable.

By default, org-mode will move the subtree to another file whose name is derived from the name of the current file by appending “_archive” to the name, but before the extension. That said, like everything in Emacs, the options for configuring the archive destination are virtually limitless.

A really common way to configure archiving is to use a dedicated file, usually archive.org. Whether you use a single archive file or the default strategy of using a file named after the current one, you will ultimately run into the same problem, huge unwieldy org-mode files.

I think there’s a better way to archive projects and tasks: use a lot of small files. Specifically, when I archive a subtree in org-mode I send the subtree to the current day’s journal entry.

Using the org-roam Journal

I maintain my personal knowledge base in org-mode thanks to org-roam. One of its features is the ability to manage journal entries. The journal system in org-roam is simply a collection of subtrees in a one-file-per-day organization scheme. In other words, each day gets a new org-mode file and throughout the day I add headings to the file in order to record certain events.

When I archive a subtree from somewhere in my vast collection of notes, org-mode stores it in one of my journal files. The specific journal file used is based on the date chosen from a prompt when I run the following archiving function:

(defun pjones:org-archive-subtree-to-daily (&optional _find-done)
  "Arhive the current subtree to the roam daily file."
  (interactive)
  (require 'org-roam)
  (when-let* ((today (save-excursion
                       (org-roam-dailies-goto-date nil "d")
                       (buffer-file-name)))
              (org-archive-location
               (concat today "::* Archived From %s")))
    (org-archive-subtree 0)))

You can tell org-mode that you want this to be your archiving function by setting org-archive-default-command:

(custom-set-variables
  '(org-archive-default-command #'pjones:org-archive-subtree-to-daily))

After which you can archive the current subtree with the normal key binding “C-c C-x C-a”. This leaves “C-c C-x C-s” alone if you want to use the built in system which relies on org-archive-location.

(Note: I have both daily and monthly journal files. To avoid being asked which one I want to archive to I pass “d” as the second argument to org-roam-dailies-goto-date. This bypasses the prompt and goes directly to the daily journal file.)

Conclusion

I find it easier to manage many small files verses a small number of huge files. To avoid a massive archive.org file I archive subtrees into my org-roam journal. This makes it a lot easier to find what I’m looking for and provides a bit of context when found. And it pairs perfectly with rg-project and consult-ripgrep.

You can also tag the current subtree with the ARCHIVE tag which darkens the heading and keeps the tree collapsed. This can be done with the org-toggle-archive-tag function or the “C-c C-x a” key binding. ↩︎

Pinning GnuPG with nixpkgs

Sun, 31 Dec 2023 10:49:30 -0700

GnuPG 2.4.1 was released all the way back in May but I failed to notice until I upgraded to NixOS 23.11 a few weeks ago. And I only noticed because Emacs froze when I tried to save an encrypted file. It didn’t take me long to realize it was due to a change in GnuPG.

Emacs and Encrypted Files

When working with encrypted files Emacs goes to great lengths to ensure the unencrypted cleartext is never written to the file system. It does this by invoking GnuPG directly and communicating with it using pipes. Thus, to write to an encrypted file, Emacs sends the cleartext down the pipe and GnuPG writes the encrypted contents to a file.

Just before Emacs sends the cleartext down the pipe to GnuPG it waits for GnuPG to signal that it’s ready to read cleartext. The problem with GnuPG (versions 2.4.1 through 2.4.3) is that it sends this signal after it reads from standard input. In other words, both Emacs and GnuPG are waiting for one another at the same time, resulting in a classic I/O deadlock.

The Workaround

The Emacs maintainers consider this situation a bug in GnuPG. And after a few days the GnuPG maintainers fixed the bug. It will be part of GnuPG 2.4.4 release. Until then the solution is to downgrade GnuPG. The Emacs PROBLEMS file says:

The only known workaround is to downgrade to a version of GnuPG older than 2.4.1 (or, in the future, upgrade to a newer version which solves the problem, when such a fixed version becomes available).

Pinning Packages in nixpkgs

This presents the perfect opportunity to demonstrate several ways that nixpkgs ecosystem allows us to solve the problem of downgrading a package, sometimes called package pinning.

The easiest solution is to use an overlay, a function that takes the current package set and returns a new package set. The overlay below replaces the gnupg package with one pinned at version 2.4.0:

final: prev:

{
  gnupg = prev.gnupg.overrideAttrs (orig: {
    version = "2.4.0";
    src = prev.fetchurl {
      url = "mirror://gnupg/gnupg/gnupg-2.4.0.tar.bz2";
      hash = "sha256-HXkVjdAdmSQx3S4/rLif2slxJ/iXhOosthDGAPsMFIM=";
    };
  });
}

Notice that the overlay function actually takes two arguments. The second argument (prev) is the previous package set, the one we want to alter. Since overlays can be chained together we may be manipulating the original package set or one from a previous overlay.

Sometimes we need access to the package set the way it will be after all overlays have been applied. For example, wanting to use a dependency from a later overlay in the current overlay. In this case the first argument (final) comes in handy, it’s the final package set after all overlays have been applied. (This is possible thanks to Nix being a lazy language and the use of fixpoints.)

Livin’ La Vida Gentoo

Unfortunately, using an overlay to pin GnuPG comes at a cost. It turns out that GnuPG is a dependency in a lot of packages. This means that after applying the overlay, nixpkgs will determine that all of those packages need to be rebuilt, and we won’t be able to take advantage of the binary cache (because the cache was built with GnuPG 2.4.1).

I don’t have the patience of a Gentoo user and compiling a ton of packages from source isn’t my idea of a good time. Besides, it’s a terrible use of the power grid. Thankfully, we have other options.

The NixOS Module

If you use NixOS then the easiest way to install and configure GnuPG is via the NixOS module:

{
  programs.gnupg.enable = true;
}

The NixOS module also has a package option so users can override the GnuPG package. Taking the overlay code from above we can write our own NixOS module and load it into our system configuration (via the normal imports list mechanism):

{ pkgs }:

{
  programs.gnupg.package = pkgs.gnupg.overrideAttrs (orig: {
    version = "2.4.0";
    src = pkgs.fetchurl {
      url = "mirror://gnupg/gnupg/gnupg-2.4.0.tar.bz2";
      hash = "sha256-HXkVjdAdmSQx3S4/rLif2slxJ/iXhOosthDGAPsMFIM=";
    };
  });
}

Similar to the overlay method, pinning GnuPG through the NixOS module affects the version of GnuPG in the system PATH. But unlike with overlays, other packages still see the original version of GnuPG. Since we’re not infecting other packages with a changed GnuPG, nothing needs to be recompiled.

Now, I’m a very happy user of NixOS, but this isn’t my preferred solution. And, of course, it doesn’t work at all for non-NixOS users. The last way we’ll look at pinning GnuPG works for any nixpkgs user and also doesn’t suffer from the overhead of the overlay technique.

The Home Manager Module

I’m a huge fan of Home Manager. It allows nixpkgs users take all the awesomeness of NixOS to other Linux distributions and even to closed operating systems like macOS, Android, and even Windows.

The Home Manager module for GnuPG is very similar to the NixOS module and also includes a package option. That means we can pin GnuPG in our user environment similar to how the NixOS module allowed us to pin it at the system level. The following Nix code can be loaded into your Home Manager configuration through the normal imports list mechanism, just like with NixOS modules:

{ pkgs }:

{
  programs.gpg.package = pkgs.gnupg.overrideAttrs (orig: {
    version = "2.4.0";
    src = pkgs.fetchurl {
      url = "mirror://gnupg/gnupg/gnupg-2.4.0.tar.bz2";
      hash = "sha256-HXkVjdAdmSQx3S4/rLif2slxJ/iXhOosthDGAPsMFIM=";
    };
  });
}

A Warning About Other Workarounds

It’s important to point out that the correct solution to this problem between Emacs and GnuPG is to fix GnuPG.

I’ve seen people on social media sites like Reddit suggesting a hack to Emacs to resolve the problem. Specifically, people are putting the following in their Emacs configuration:

(fset 'epg-wait-for-status 'ignore)

Don’t do this! This is not a valid fix and may lead to file corruption when reading/writing encrypted files.

This bit of hackery replaces the epg-wait-for-status function with one that ignores its arguments and always returns nil. That function is used throughout the EPG library in Emacs, not just when writing files through GnuPG. Again, please don’t do this.

Conclusion

We’ve looked at three different ways of pinning a package in nixpkgs:

Using an overlay function.

This is usually my go-to solution. But in this case GnuPG is used in so many other packages that it caused massive rebuilds.
Pinning via the NixOS programs.gnupg.package option.

This works by only changing the version of GnuPG installed in the system PATH. If you use NixOS (and not Home Manager) this is the way to go.
Pinning via the Home Manager programs.gpg.package option.

This is my preferred solution because it works nearly everywhere thanks to Home Manager. It replaces the version of GnuPG in your user PATH, leaving the system PATH and the rest of nixpkgs alone.

Like many others, my frustration with GnuPG is nearing its breaking point. Over the last 20 years I’ve sent/received no more than a handful of encrypted emails. My use of GnuPG at this point is limited to encrypting files in my password store and signing Git commits. It’s becoming less clear why I put up with the burden of managing my GnuPG key and the occasional upgrade problem.

Perhaps I’ll finally switch to age over the next major holiday break. (I probably won’t find the time, but here’s hoping for the best.)

Update (January 6, 2024): A better solution may be to patch the current version of GnuPG so it doesn’t have the deadlock bug. I explore that option in this post.

Update (February 23, 2024): GnuPG 2.4.4 has been released and fixes the problems mentioned in this post.

Why the world needs Haskell

Tue, 09 Jul 2013 00:00:00 -0700

This is a technical review of Haskell and why software developers should care about functional programming. For a non-technical review of why your company should be using Haskell you might want to watch this introduction to Haskell video by FP Complete.

TL;DR: Writing high-quality, bug-free software isn’t easy. Haskell (and similar languages) provide the programmer with the tools necessary to get closer than ever before. By removing entire categories of bugs (such as those caused by null pointers and nil objects, type coercions, accidental mutation, etc.) and introducing a powerful type system, programmers can write better code in the same amount of time (and often less) than more traditional languages.

Null Pointers and Undefined Values

The code we write eventually executes in the real world and things don’t always go as we would like. Network connections drop while you’re reading from sockets, disk space goes to zero while writing to files, and people trip over power cables while balancing huge cups of double foam lattes.

These are exceptional situations that we don’t have much control over and which we have to deal with gracefully. But what about errors that we do have control over? You know, the exceptions we create as programmers every day when we dereference a null pointer or call a method on a nil object. These types of errors should be completely unacceptable yet they plague production systems all the time.

Sometimes undefined values propagate up from bad database constraints, null columns that leak right out of our data types as nil objects or null pointers. This is just one symptom of a larger problem, the widespread design idiom that overloads nil to stand in for failure, missing values, and the inability to compute a value. When compared to raising an exception this seems very reasonable indeed. But is it?

There are 3 alternatives when a function can either return a valid result or a nil object:

Test the result of every function to make sure it’s valid. Of course this leads to verbose code that sucks to write and when a programmer is feeling a bit lazy it’s easy to leave out.
Pretend that the function always returns valid data and use it without checking. I’m surprised how often this happens (I’m certainly guilty of it) and that most languages can’t detect this and slap us right across the face like we deserve. This happens a lot in object oriented scripting languages where we chain a bunch of method calls together, any one of which may return nil.
Use a complete hack such as the try method from Ruby on Rails which obscures the source of the problem, leaving it for someone else to deal with. This seems to be a popular favorite these days.

Of course the correct approach is method 1 but it comes at a cost. In order handle this situation properly we have to write a bunch of boilerplate code and we’re forced to make decisions about what to do in the event a function actually returns nil. Often the only sensible thing to do is propagate the failure up the call stack which is why method 3 is so popular.

It’s also why ruthless testing and 100% test coverage have become so important in mainstream languages. But even with 100% test coverage you can’t be sure that your code will work correctly if a function unexpectedly returns nil unless you’re also mocking things out or using fuzz testing. That’s a lot of extra work and most of us don’t go to such great lengths. Have no fear my friends, there’s a better way.

Haskell has a solution to this problem that is both simple and elegant. First and foremost there’s no such thing as null or nil. If a function is declared as returning an integer then the compiler won’t let it return anything else¹. If you’re brave and try to return anything that isn’t an integer the code won’t compile and the day will be saved.

If a function needs to either return a valid value or the absence of one it’s easy to encode that using the type system which then allows the compiler to verify that the programmer is dealing with the missing value. If it’s not explicitly dealt with the code won’t compile. We’ll get more into using the Haskell type system to our advantage in the next section.

This entire class of run-time errors is completely eliminated in Haskell for free, at compile time. After you’ve been using Haskell for a bit of time you’ll be paranoid and frustrated when using languages that have nil values or null pointers because the responsibility to deal with all those potentially missing values sits squarely on your shoulders. Those languages don’t help you out one bit in this regard.

What Has Your Compiler Done for You Lately?

As a programmer it’s likely you know at least a handful of programming languages and there’s one in particular you gravitate towards. It’s also likely that over the years languages have come into and gone out of your favor. Think of your current favorite language, what makes it different than your previous favorite language?

I’ve noticed a pattern in myself, I tend to toggle between lower level and higher level languages. I love the raw power and control of the lower level static languages such as C++, but after a bit of time the verbosity of the language and the amount of code to get something simple done starts to wear on me and I switch to a high-level dynamic language like Ruby.

With higher level languages it feels like you can get more done with significantly less code and the development cycle of writing code and running it becomes so much easier and faster. The experience is really enjoyable until things start to go wrong in production because stupid typos and nil objects that somehow slipped through testing. That’s when I start to long for a static language with a compiler that can catch silly mistakes well before anything is deployed to production.

From the perspective of a software developer it’s a zero-sum game. All the gains I think I’m getting from a dynamic high-level language are eventually eroded by all the test writing and general care that’s necessary to have confidence that things aren’t going to blow up in a user’s face. And all the hand-holding, nudging, and extra code needed in a static low-level language just to get the same amount of work done can be infuriating.

The obvious solution is a hybrid language, one that is abstract enough that you can work faster and write less code and at the same time uses a static type system with a compiler that can catch stupid typos and enforce design choices at compile time. In the world of functional programming, languages like this are plentiful and Haskell happens to be one of them.

Haskell has a very strong static type system. Every expression has a type and the way expressions are used is fully checked at compile time to ensure type safety. If you’ve used a static type system before you might have just thrown up in your mouth and had flashbacks of spoon-feeding the compiler type annotations and extra code that it should have been able to figure out for itself. Have no fear, Haskell has an excellent type inference system which means you get type safety without having to explicitly tell it what you’re up to all the time.

But type inference isn’t all that special really. Heck, even C++11 has the auto keyword that allows the type of a variable to be inferred by the compiler. What you should take away from this, however, is that Haskell provides a lot of type safety at compile time without placing any additional burdens on the programmer. Type inference is just one example.

The type system in Haskell is quite a bit more powerful than in most languages. If you haven’t been exposed to a language like Haskell you probably haven’t thought about how to use the type system to make code safer or how to enforce design decisions at compile time but that’s exactly what Haskell programmers do every day.

Consider again the topic of null pointers and nil objects. What can you (and the compiler especially) gather from the following C++ function prototype²?

User* get_user_by_name (const std::string &name);

I see a function that returns a pointer to a User object. Had it returned a reference I could be somewhat assured that it won’t fail at run time (but it could still throw an exception.) Setting aside the ambiguity of who’s responsible for memory management in this case, it’s probably safe to assume that this function can potentially return a null pointer.

The C++ compiler doesn’t make a distinction between a valid User pointer and a null pointer, it doesn’t even spit out a warning if you try to dereference a known null pointer, and it certainly doesn’t force the programmer to test the pointer to see if it’s NULL. This is a case of “the programmer knows best” even when the compiler can see otherwise. I don’t know about you but when I’m writing production software and the compiler can potentially detect a mistake I’ve made I want to know about it.

There are two popular ways to write this function in Haskell. The first clearly tells the compiler and the programmer that this function might not return a valid User:

getUserByName :: String -> Maybe User

And the following means that the function can either return a string containing an error message or a valid User:

getUserByName :: String -> Either String User

And here’s the kicker, this is all done in the type system. Programmers who don’t know Haskell might be tempted to think that Maybe and Either are language keywords or some sort of type qualifier for the compiler but they’re just data types that happen to be in the standard library.

The closest comparison to C++ that comes to mind is the union type. Take the Either type in Haskell, implemented in C++ it would be a union with two fields, a string and a user. Since it’s a union only one of the fields can be used at once and the programmer has to work out which one can be accessed. In Haskell it’s not so ambiguous, it’s actually not ambiguous at all.

In both Haskell examples above the compiler won’t let you directly access the User object because it might not even be there. You have to write a little more code to handle the case where the function failed to return a user. This might seem like extra work for the programmer but in reality it doesn’t work out that way. Haskell even provides some syntactic sugar that allows you to chain function calls of this nature together so that the first failure stops the chain, sort of like the try method mentioned earlier, but with much more flexibility.

If the Haskell type system can be used to encode information like “this function might return a user but it also might return nothing” can you imagine what else you could use it for? What about “this function only accepts validated user input” or something we’ll go over in the next section “this function does I/O”.

What this boils down to is that you can clearly articulate invariants using the Haskell type system and then know that the compiler will confirm that no programmer has broken the rules. It also means that the compiler acts as an automated test system for type related parts of the code. Now that’s a level of confidence you’re probably not used to, at least not yet.

Side Effects and Spaghetti

I’ve spent a large portion of my career using object oriented programming languages to great effect. The ability to manage the complexity of large projects by segmenting code into classes and hiding implementation details through encapsulation and abstraction has proved to be very helpful. Certainly more so than the abstractions in procedural languages.

Over the last few years something started bothering me and other object oriented practitioners, the fact that sending a single message to an object might change its state in several non-obvious ways. This led, in part, to conventions such as the single responsibility principle. In my opinion, however, this problem strikes at the heart of OOP (and more generally imperative programming.)

You might not think about it very often but a method call can do any of the following:

Alter instance variables for the current object either directly or by invoking other instance methods that do so.
Change the state of any of its parameters.
Modify class variables, global variables, or any other variable that might be in scope.
Perform I/O (open files, write to a socket, communicate with another application, launch missiles, etc.)

The point is, going by the laws of encapsulation an object should be a black box that provides an interface and when calling a method you only care that it performs its promised task. You don’t really know what other magic it’s doing behind the scenes and you really shouldn’t care. Except that if you don’t know then you can’t write robust code because you can’t be sure what a method is capable of doing.

When invoking a method you know what inputs it takes (parameters) and what outputs it produces (return values) but you don’t necessarily know what its side effects are going to be. If fact, invoking the same method over and over again may produce different results and may alter the outside world in different ways. None of this information, however, is conveyed by the signature of the method. Most languages leave it up to the programmer to document these side effects in one way or another.

This all makes for a big mess.

Consider global variables, for a very long time now they have been the black sheep of the family, and for good reason. But what really makes global variables different than instance variables? Sure, instance variables have a much smaller scope, but in the context of all the code that makes up a single class don’t they essentially cause the same problems? Have you ever written a method that failed because some other method accidentally screwed up the value of an instance variable?

There’s a big difference between passing in an invalid parameter to a method and the object having an invalid internal state. The former is fairly easy to track down while the latter can be just like hunting for a bug involving a global variable. Instance variables in big classes also tend to create spaghetti code where side effects are hard to trace.

With the ability to indirectly (and often accidentally) change the state of any object and affect the outside world in any way, it’s like we’re living in the wild west of programming. While side effects are absolutely necessary for our programs to do anything useful, we need a way to manage them much better than we do, especially in imperative languages.

Let’s start by partitioning side effects into two buckets, mutable state and the ability to interact with the outside world (A.K.A. I/O). We’ll tackle how Haskell handles these starting with the latter.

Communicating with the outside world presents several problems, the least of which is changing the environment in a way that can’t easily be undone (deleting a file, sending a network packet, etc.) By default Haskell doesn’t allow any of these but instead provides a special I/O compartment or an escape hatch if you will.

The bridge to the outside, unpredictable world in Haskell is through a data type called IO. And since IO is a data type we can use the power of the type system to enforce this so-called compartment! In other words, a function can’t perform I/O unless its return type is the IO data type. Here’s a simple example:

getLine :: IO String

The getLine function reads a single line of characters from standard input. It doesn’t take any parameters and returns an IO type³. The function’s type contains enough information to convey to the programmer that it might produce side effects. As a bonus feature it conveys the same information to the compiler which can make better optimization choices for functions that don’t have side effects.

This is so important that I want to say it one more time, you can’t do any I/O in Haskell unless your function returns an IO type. That also means that any function that uses an IO function must also be itself an IO function, which is why the main function is of type IO. Theoretically you could write all of your functions this way but you’d totally miss out on the benefits of compartmentalized I/O.

Getting back to the other major kind of side effects, mutations, Haskell has a simple solution to this one as well, they’re not allowed outside of the I/O compartment. In practice this isn’t an issue because mutations are very rare in Haskell. If you need to change one field of a record you just create a copy with the changes applied. The Haskell optimizer and garbage collector remove any of the downsides you might be thinking of.

Haskell does not have variables in the sense that you’re used to, which is a good thing. It means that everything you need to know about a function can be gleaned from its type signature. You don’t have to worry that it might change something you’re not aware of or affect the outside world behind your back.

More importantly, you can’t accidentally mutate something. Yet another class of problems completely eliminated and the only thing you have to do is slightly change the way you write code.

Haskell is Hard to Learn, Right?

There’s definitely some truth to Haskell’s reputation as a language that’s hard to learn. Fortunately it’s less about Haskell and more about us. As imperative programmers we bring a lot of baggage to Haskell. We expect to mutate variables, do I/O at will, and work with very simplistic type systems.

Then there’s the whole imperative vs. declarative issue to deal with. Haskell is a high-level language, very high-level in fact. In the imperative programming world even the highest level languages still require the programmer to structure code as a sequence of steps to execute on a CPU. In functional declarative languages like Haskell you write your algorithms similar to how they are structured in mathematics and let the compiler work out how to translate that into a series of steps. Often the Haskell compiler can generate binaries that are close to if not as fast as hand written C.

All of this adds up to Haskell being very different than we’re used to. At this point in my career I can usually learn a new programming language in a few hours and write production quality code in about a week. The way I’ve learned to assimilate programming languages is by comparison. Ruby and C++ are more similar than they are different so using what I know about C++ greatly reduced the cost of learning Ruby.

Haskell is a very different beast. It approaches programming from a completely different angle. It’s almost like learning to program all over again. Thankfully it’s not as bad as it sounds.

Hopefully I’ve demonstrated why Haskell is important and why you should take the time to learn it, even if it seems difficult at times.

It’s often said that learning Haskell will make you a better programmer no mater which language you choose to use. I’m a firm believer in that. I’d be surprised, however, if Haskell doesn’t become your new favorite language.

Learn Haskell

For those of you who want to learn more I’ve compiled a recommended reading list and study resources at the bottom of this page.

Updates and Discussion

If you want to discus this article head over to reddit.

You can, however, return an undefined value which when evaluated will terminate the program. Haskell programmers use this to stub out parts of the program during development and they’re easy to detect during a production build. ↩︎
Reddit user LampCarpet points out that this isn’t idiomatic C++ code and that competent C++ programmers know better than to return raw pointers or let null pointers go unchecked. I don’t think that changes the fact that even competent programmers make mistakes and working with raw pointers is still very popular with C++ programmers. ↩︎
More accurately, getLine returns an I/O action that when performed yields a string. ↩︎

Parsing with Haskell and Attoparsec

Wed, 26 Jun 2013 00:00:00 -0700

If you’re a Haskell programmer and haven’t gotten around to using the wonderful Attoparsec library this video will give you a much needed kick in the pants. For those of you who haven’t taken the plunge with Haskell it might be interesting to see how straight forward it is to write high performance parsers with Haskell.

The video also includes a short discussion on the major differences between Parsec and Attoparsec. They have very similar APIs which can make it hard for newcomers to decide which library to use.

Learn to read and write Scheme expressions

Mon, 29 Apr 2013 00:00:00 -0700

With this article you’ll begin your journey with the Scheme programming language by first learning how to read and understand expressions and then how to write them. When you reach the end of this article you’ll be ready to group your expressions into larger components such as functions.

It will be most helpful if you follow along with the article using one of the free Scheme interpreters such as Chicken Scheme or Racket. Both are easy to install and support all the popular operating systems (Windows, Mac, and Linux). If you don’t have a preference I recommend you download the Chicken Scheme installer.

Building programs from smaller pieces

Expressions are the fundamental building blocks of a programming language. As a programmer you’ll spend a great deal of your time combining expressions together in order to build larger components that eventually make up an entire application just like when you create buildings from small toy bricks.

Usually, however, programmers will spend some time thinking about the application they are going to write and how it should fit together as a whole before they start dealing with the smaller details like expressions, this is called top-down design. This article series uses the opposite approach to teach programming, namely bottom-up.

Using a bottom-up approach we’ll start with the smaller components (i.e. expressions) and work our way up to more complicated programming structures. For now I just want you to be aware that taking an idea for a big application and jumping directly to writing expressions in a programming language without first thinking about how things fit together might be a recipe for trouble.

Breaking apart a simple Scheme expression

Let’s start with the example code from the previous article which added two numbers together. I’ll continue to use simple arithmetic throughout this article since it gives us common ground to start from, but you should keep in mind that expressions in Scheme can do more than just simple mathematical operations.

(+ 2 3)

In Scheme an expression is actually a list of items, much like a grocery list or a to-do list. A list in Scheme begins with an opening parenthesis, ends with a closing parenthesis, and in between contains one or more items separated from one another with whitespace. In the list above there are 3 items (often referred to as elements or members):

There’s not much more to it than that. Pretty simple right?

Evaluating simple expressions

An expression is pretty useless just sitting there on your screen unless you actually make it do something. Making an expression perform its task is called evaluation. Let’s evaluate this expression in a Scheme interpreter. If you’re a good student you installed Chicken Scheme as instructed at the beginning of the article. Open a terminal window and run the csi tool. You should see the #;1> prompt which means csi is waiting for you to enter an expression.

Type in our example expression from above (don’t forget the parentheses) and then press enter/return. The csi tool should evaluate the expression and print out the result (5), after which it will return you to a new prompt and wait for your next expression.

When expressions are evaluated they will either return a result, produce a side effect (e.g. draw a rainbow), or both. Even though drawing rainbows is fun we’ll focus on expressions that return a result like our example expression that takes some numbers and returns their sum. Let’s break apart how our expression is evaluated.

Do you remember that an expression in Scheme is a list of items? The first item in the list (known as the head of the list, or more technically, the “car” of the list) is special. When the expression is being evaluated the interpreter begins by looking at the first item of the list which should be the name of a function. We’ll get into functions in the next article but for now you can think of a function as a task that you can ask Scheme to execute for you.

With the example expression we’ve been working with so far, the first item of the list is +, the name of the function that adds numbers. The remaining items of the list (known as the tail of the list, or more technically the “cdr” of the list) are given to the + function as arguments, in this case the numbers you want to add together.

Arguments are the inputs to the function. Each function defines what it wants for arguments (and some don’t want any). If you read the documentation for the + function you’d find that it wants one or more numbers as its arguments. Most programming languages come with a set of predefined functions and allow you to define your own. Again, we’ll get to that in the next article.

After an expression is evaluated it is replaced with its result. Don’t worry if that doesn’t make sense just yet, it will become clear in the next section when we start to put expressions together to form more complicated building blocks.

Combining expressions to form complex expressions

To spice things up a bit, pretend you’re a school teacher faced with grading a rather large stack of homework sheets. Your student aid already marked the incorrect answers but she didn’t calculate the percentages that the students are expecting to see.

Let’s write a Scheme expression to calculate the score for the first student’s worksheet. There are a total of 15 questions and the first student correctly answered 12. The arithmetic is pretty simple, divide 12 by 15 and multiple the result by 100:

(* 100 (/ 12 15))

Don’t panic, let’s work through this expression step by step.

First look at the parentheses, do you see that there are two lists in the code above? The second list is inside the first, that is to say one is nested inside the other. Both lists begin with the name of a function (* is the name of a function that multiples its arguments and / is the name of the function that divides its first argument by its second argument).

When evaluating compound expressions like the one above the interpreter will first evaluate the inner-most expression. In our case that’s the expression that does the division. If you evaluate the inner expression in csi you’ll find that the result is 0.8 (12/15 = 0.8).

Once you have the result of the inner expression you can replace it with its resulting value:

(* 100 0.8)

Now you’re left with a single expression and we already know how to evaluate those.

Unfortunately the only thing we’ve managed to do is turn Scheme into a boring calculator and haven’t really made grading papers any easier. Come to think of it, we’ve actually made it harder to grade papers because we have to hand type these weird nested Scheme expressions, repeating two of the numbers for each worksheet we grade.

Not to worry though, we’ll improve this quite a bit in the next article. The important thing is that you’ve learned about expressions and how to put them together to accomplish more complicated tasks.

List items, atoms, and constants

Before we wrap up let’s take a closer look at those things I’ve been calling list items. Did you notice from the code above that lists can contain other lists? Technically speaking lists are made up of atoms and/or other lists. An atom is anything that is not a list, so all the numbers and function names we’ve been using as list items are considered atoms.

More specifically, the numbers are called numeric constants because they can’t change (they are said to be hard-coded). Besides being constant values, numbers and text that are hard-coded are also referred to as literals.

While literal values in your code are useful there’s another way to get values into a running program, things called variables. With variables you essentially have a placeholder that you’ll fill in later when your program is running. Variables can be given values by asking the user a question, reading the current time from the computer’s clock, communicating with another computer, etc.

Scheme (like most programming languages) has support for different types of values. We’ve already seen numbers and identifiers (e.g. + and *). Using the following table, experiment with the Scheme interpreter you installed. You can type the examples directly into something like csi and it should give you back the value you entered.

Type	Description	Literal Examples
Integer	Whole numbers	`-1`, `0`, and `5`
Real	Fractional numbers	`10.1` and `0.8`
Character	A single textual character	`#\a`, `#\b`, and `#\c` (the first 3 letters of the English alphabet)
String	A sequence of characters	`"Hello World!"`
Boolean	True and False	`#t` is true, `#f` is false
List	A list of values	`'(1 2 3)`

There are a few other types of values that Scheme supports but we won’t be getting into those just yet. With the list of types above you can write complex and expressive programs. And don’t worry too much about literals, types, and variables right now. With more practice and through this series you’ll develop an intuition for using them.

Programs and values go hand in hand. From an abstract point of view programs are things that take some input values, translate those values, and then produce some sort of output based on those translated values.

Conclusion and next steps

In Scheme expressions are represented as lists that are delimited by parentheses and the items of the list are separated using whitespace. The first item in the list is the name of a function to execute and the remaining items are the arguments to that function.

When you nest an expression inside another expression the inner expression needs to be evaluated before the outer. You can think about evaluation as the act of replacing an expression with its resulting value.

In the next article we’ll look at how to group a set of expressions together to form a larger building block, a function. For now I’ll leave you with a teaser of what we’ll be working on next.

(define grade
  (lambda (correct total)
    (* 100 (/ correct total))))

(grade 12 15)

If you’re interested in following this series of articles I recommend that you subscribe to the RSS feed or keep an eye on the Learn to Program section of this site.

Picking your first programming language

Sun, 24 Mar 2013 00:00:00 -0700

When teaching an introductory programming course an overwhelming majority of the boot camps and instructors (including myself) have made the mistake of starting with a popular object-oriented language such as Ruby or JavaScript. I love these languages and eventually they are something students should learn but they’re not a good choice as a first language. The same can be said for most programming languages, especially object-oriented languages.

Learning to program incrementally

Object-oriented programming languages like Objective-C and Java don’t make good first languages due to the extra layers of abstraction. Of course, these same abstractions are what make programming in these languages more productive for the experienced software developer, but they add unnecessary difficulties when you approach them as your first programming language. You can’t isolate the basics of programming and ignore the abstractions, they slap you in the face and force you to deal with them.

When you strip away the abstractions, learning to program works best as an incremental process, just like learning the sciences and the arts. Vincent van Gogh spent a great deal of time perfecting his drawing and shading skills before even considering the use of color. The same should go for programming, you need to start with the fundamentals and build your way up. Only then can you understand and appreciate programming languages with advanced abstractions.

Given that the languages all the cool kids are using don’t lend themselves well as a first language, what should a new student learn instead? If you’re familiar with past computer science curriculum my suggestion won’t come as a surprise: Scheme.

A small, rich, and consistent language

You don’t have to look very far in the history of many universities to find a time when Scheme was the teaching language of choice, but that has mostly gone the way of the dodo, being supplanted by more “modern” languages such as Java. This is a real shame considering my points above and that Scheme and other dialects of Lisp are still a relatively popular language in the real world of programming (Clojure being another one of them.)

What is it that makes Scheme such a good teaching language? Scheme is a high-level language like Ruby and Java. This means that beginners don’t have to take a boring detour to learn about computer hardware, registers, pointers, etc. Unlike other popular high-level languages Scheme is relatively minimalistic, that is it doesn’t have all the extra layers of abstraction that can get in the way of learning the basics.

For example, take a look at this snippet of code written in Scheme and see if you can figure out what it does:

(+ 2 3)

Obviously this adds 2 and 3 together.

I know what you’re thinking, you can do this in any programming language and it would look more natural. And you’d be right of course, in nearly every other language the code would be 2 + 3. However, once you understand how the above Scheme works you’ve learned quite a bit about the language and the basics of programming in general.

You can’t say that about most other languages because the various special cases that are part of the syntax (functions vs. operators, infix vs. prefix, statements vs. expressions, etc.). When learning to program using Scheme you don’t have to worry about all that, effectively postponing more advanced syntax until you’re ready.

Making programming fundamentals approachable

Getting back to the Scheme example code from above, it looks weird at first glance. This is because Scheme doesn’t have any special syntax for arithmetic. No rules with a bunch of exceptions. When you reach that moment where you understand that this example code is an invocation of a function named + and that it expects a list of numbers as its arguments, you can apply that knowledge to call any other function. That’s it. Adding numbers, launching missiles, and setting variables all use the same syntax.

And guess what, defining your own function is nearly as simple as the example code because the same rules apply, it’s a simple extension of the same syntax. This is another reason why Scheme is so great as a teaching language, the rules are easy to understand and completely consistent throughout. Best of all, these rules can be taught incrementally.

So what are these programming basics and fundamentals that I keep talking about? I’m going to save that for the next article but in the meantime I’ll point you towards some popular Scheme resources.

Chicken Scheme might have a funny name but it’s a complete Scheme system under active development. It’s small, fast, and fun to use.
Racket is a popular programming language in the Scheme family geared towards learning but powerful enough to write commercial-grade software.
One of the best computer science books of all time is Structure and Interpretation of Computer Programs. It’s available for free and uses Scheme.
Another book that teaches Scheme is How to Design Programs which is also free.

Next steps

Over the course of the next few months I’ll dig deeper into topics covering how to become a programmer using Scheme. If you’re interested in following the series I recommend that you subscribe to the RSS feed or keep an eye on the Learn to Program section of this site.

JavaScript as a virtual machine?

Thu, 03 Jan 2013 00:00:00 -0700

When languages that compile/translate to JavaScript first started becoming popular I rolled my eyes and kept on using JavaScript. Adding a layer of indirection on top of an already high-level language seemed like adding a spork to a picnic basket already stocked with a full set of flatware.

You might be able to make the same argument when comparing C to assembly, and maybe I’m just an old fart yelling at those damn CoffeeScript kids to get off my lawn. Is JavaScript really that bad?

I’ll agree that implicit type conversion is a naught little feature but if you know about it then you should also know how to prevent it. Personally, I think once you actually learn JavaScript and prototype based programming it’s a pretty decent language. And since you have to debug the generated JavaScript and not the higher level language you might as well just learn JavaScript.

But then I started coming across projects like Fay and Emscripten which allow existing languages to be translated to JavaScript. Why is that important? Consider being able to share code from your server-side application (classes, data types, validation) with your front-end code. That’s a pretty cool idea.

Over the holiday break I read an interesting article (which I can’t for the life of me find now) in which the author suggests that we think about JavaScript as very verbose bytecode for a virtual machine built into every web browser. That makes a bit of sense, at least until browsers actually support some sort of universal bytecode interpreter.

Unix cheat sheets

Mon, 17 Sep 2012 00:00:00 -0700

Peteris Krumins is at it again with two *nix cheat sheets that should help you explore your Unix-like system:

Not too long ago I wrote a shell script that gives you a brief description of every tool in your PATH:

#!/bin/sh

whatisthere () {
  for dir in `echo $PATH | sed 's/:/ /g'`; do
    for file in $dir/*; do
      base=`basename $file`
      desc=`whatis $base 2>&1`
      [ $? -eq 0 ] && echo $desc
    done
  done
}

whatisthere | sort | less -SR

Update: The electronic version of The Linux Command Line has been released for free under a creative commons license. Thanks for the tip Norman.

Being explicit with your code

Tue, 17 Apr 2012 00:00:00 -0700

Christian Heilmann has a well reasoned gripe for developers omitting semicolons in JavaScript:

Why would any intelligent person want to make it harder for others to understand what they’ve done, keep booby-traps in their code that will cause errors or deliberately write in a way that might make others stumble? Is this some kind of code-trolling I don’t get?

I have the same exact complaints when people don’t use parentheses in their Ruby code. You can argue all day about your reasoning but when you run your code through ruby -w you’ll see all sorts of warnings about ambiguous code.

RubyGem Version Specifiers

Tue, 03 Apr 2012 00:00:00 -0700

When writing an application in Ruby it’s likely that you’ll make use of several open source libraries that come packaged as RubyGems. But do you know which versions of those gems are getting loaded into your application? Will all versions work with your application?

It doesn’t matter if you are loading gems into your application directly using require statements or using a tool like Bundler, you should always use version specifiers to future proof your application.

Using Version Specifiers

It’s often tempting to load a library without regard for which version you are about to use:

# In your application's source code
require('nokogiri')

# Or, in a Bundler Gemfile
gem('nokogiri')

But that’s equivalent to using the following version specifiers:

# In your application's source code
gem('nokogiri', '>= 0.0')
require('nokogiri')

# Or, in a Bundler Gemfile
gem('nokogiri', '>= 0.0')

As you can see, version specifiers are comprised of a version operator and a version number. Omitting the version specifier or using the >= version operator is almost always a bad idea.

The code above means that you’ll happily use any version of the Nokogiri gem. But is that a true statement? Will your code really work with any version? What about really old versions of Nokogiri? What about the version of Nokogiri that’s going to be released in 2 years?

It’s easy to imagine a situation where you upgrade a gem to support a new application and in the process break an old application. If the older application had a restrictive version specifier it would keep working as long as you had a compatible version of the gem installed.

I said it’s almost always a bad idea to use >= because there’s at least one legitimate use for it. RubyGems allows you to give more than one version specifier and Bundler supports this as well:

gem('nokogiri', '>= 1.0.0', '< 2.0.0')

Which means that you’ll take any version of Nokogiri starting at 1.0.0 but not 2.0.0 or higher. Combining version specifiers like this gives you a lot of control over which version is going to be loaded into your application, but it’s a bit cumbersome. If all you want to do is restrict a gem to a specific major release (e.g. 1.x but not 2.x) then you can use the pessimistic version operator (~>).

The Pessimistic Version Operator

The pessimistic version operator (~>) allows you to state that your application works with future versions of a gem in a safe way. Of course this only works if the author of that gem introduces changes that break backward compatibility in a predictable way (e.g. going from version 1.0.0 to 1.0.1 doesn’t break your application).

For example, if you trust that the authors of the Nokogiri gem won’t break backwards compatibility until version 2.0.0 you can load the gem using the pessimistic version operator:

gem('nokogiri', '~> 1.0')

But, if you only trusted them to maintain backwards compatibility in the 1.5.x releases you could specify the version as:

gem('nokogiri', '~> 1.5.0')

The pessimistic version operator can be controlled by how specific you make the version number. It works by stripping off the last digit you specify and incrementing the remaining version. So 1.5.0 becomes 1.6 and 1.0 becomes 2.0. It then restricts the version of the gem starting with the version you specified up to but not including the incremented version. Here are some examples:

Pessimistic Version	Range Restriction
`gem('nokogiri', '~> 1.0')`	`gem('nokogiri', '>= 1.0', '< 2.0')`
`gem('nokogiri', '~> 1.5.0')`	`gem('nokogiri', '>= 1.5.0', '< 1.6.0')`
`gem('nokogiri', '~> 1.5.3')`	`gem('nokogiri', '>= 1.5.3', '< 1.6.0')`

Available Operators

RubyGems provides a complete set of version operators that allow you to specify which versions of a gem your application can work with. If you don’t use an operator and just use a version number you lock your application to that specific version, it’s shorthand for using the equal (=) operator.

Here’s a list of the of the operators supported in RubyGems:

Operator	Meaning
`=`	Equal to (default)
`!=`	Not equal to
`>`	Greater than
`>=`	Greater than or equal to
`<`	Less than
`<=`	Less than or equal to
`~>`	Pessimistically greater than or equal to

Ruby/Rails Compatibility Matrix

Wed, 14 Mar 2012 00:00:00 -0700

I found myself needing to figure out which versions of Ruby on Rails run under which versions of Ruby. I couldn’t find a compatibility chart so I put this together:

Rails Version	Possible Ruby Versions	Recommended Ruby Version
1.0–2.1	1.8.6	1.8.6
2.2	1.8.6 or 1.8.7	1.8.7
2.3	1.8.6, 1.8.7, or 1.9.1	1.8.7
3.0–3.2	1.8.7, 1.9.2, or 1.9.3	1.9.3
4.0–…	1.9.3 or 2.0.x	2.0.x

Updated: Rails 4.0 adds support for Ruby 2.0.

Ruby's catch/throw, goto's little brother

Fri, 09 Mar 2012 00:00:00 -0700

Last week Pat Shaughnessy discovered that there’s a pre-processor definition in the Ruby 1.9 source code to enable goto and label statements in a rather ugly way:

The __goto__ statement will cause the MRI Ruby 1.9 interpreter to jump up to the __label__ statement on the first line, since they both refer to the same symbol :loop.

I don’t think there’s much use for goto in a language like Ruby considering that it’s garbage collected and you can release resources using closures and ensure blocks. However, if you’re dying to play with this, why not use goto’s little brother, catch and throw:

class Test
  def foo
    throw(:label, "foo")
    "should never get here"
  end

  def bar
    "bar"
  end
end

test = Test.new

puts("foo -> " + catch(:label) {test.foo})
puts("bar -> " + catch(:label) {test.bar})

There’s not much to know about catch and throw. You give a symbol and a block to catch, which acts like a goto label. Any code that uses throw in that block, no matter how deep, will unwind and cause the catch block to terminate, returning the value given to throw.

You can nest multiple catch and throw expressions using different values for the label symbol. Here’s another example:

def thrower
  10.times do |i|
    10.times do |j|
      $stdout.puts("#{i}.#{j}")
      throw(:label, j) if i > 2 and j > 2
    end
  end
end

val = catch(:label) do
  thrower
  raise("should never get here")
end

$stdout.puts("val is #{val}")

Fun stuff for Friday Feb. 24

Fri, 24 Feb 2012 00:00:00 -0700

It’s Friday, and an especially good one at that. At least for me since it marks the start of a much needed vacation. Hopefully this weekend will be good for everyone, and what better way to kick it off than some fun and geeky articles and videos.

If you’re in the market for a new keyboard and think you want to create one yourself, look no further than the Humble Hacker Keyboard. This thing looks very cool. And if you think the layout is a bit crazy then you need to watch this video of Tim Tyler’s keyboard. Via Hack a Day.
Nathan Rice posted some pictures comparing programming languages to cars which is cute.
Eric Moritz put together a list of books that every self-taught developer should read. It seems like a pretty reasonable list. Of course, I would add The Pragmatic Programmer.
There’s a fairly interesting article over at the BBC about the myth of the eight-hour sleep. For whatever reason I’ve found that software developers seem to be interested in sleep patterns and sleep experiments.
Longtime fans of Apache will be glad to hear that version 2.4 has been release with some interesting features and performance changes that match up with Nginx.
Finally, you may want to remove your search history from Google before their new privacy policy goes into effect on 3/1.

As always, have a wonderful weekend. Recharge your batteries and prepare yourself for a return to hacking on Monday.

Fun stuff for Friday Feb. 10

Fri, 10 Feb 2012 00:00:00 -0700

This is going to be fun. I haven’t written a Fun Friday article this year and my queue of things to share is sizable. To top it off, I haven’t been keeping my self-imposed quota of at least one article a day, so my general queue of articles to comment on has grown out of control. This post will contain some fun things I want to share as well as a few backlogged articles that are pretty interesting.

Want to relive the fun of rebooting older operating systems? The Restart Page has you covered.
Developers are religious about the text editor they use, that’s no surprise. Need to vent your anger for Eclipse? Look no further than I Hate Eclipse. If you’re looking for a new text editor Sublime Text 2 Beta is out. Oh, and Emacs is the one true editor.
Parse has found an interesting way to filter through resume overload often accompanied by job postings. To apply you have to submit JSON to their API. Smart.
Ash Moran wrote an article to help you get your crap together instead of just throwing more developers at the problem.
If you’re not up on your diff and patch ninja skills this article walks you through creating and applying patches.
Do you find the Java syntax to be too verbose? Eleftheria Drosopoulou says she likes the verbosity of Java because it forces you to understand the code you are using. I can see where she’s coming from but I think the verbosity of the syntax actually impedes its readability.
Speaking of Java, Marc Kuo has wrote a series of articles about why he loves Common Lisp and hates Java. Part 1 and Part 2.
All Programmers Are Self-Taught is short post about how a computer science education doesn’t actually teach you how to program. I tend to agree but the author is still a student, so maybe he needs to take a few more classes.

That should give you plenty to read over the weekend.

Using Haskell to improve your C++

Mon, 06 Feb 2012 00:00:00 -0700

I find myself drawn more and more to Haskell. Any language that helps a developer write solid code without making you feel like you’re using training wheels is a huge plus in my book. Functional programming has also opened my eyes to all sorts of pitfalls with imperative programming.

Even if you’re put off by Haskell or functional programming there’s a lot you can learn and both can have a positive influence on your imperative programming skills. At least that’s what they say.

The folks over at Valletta Ventures have a short blog post about how their C++ code has changed for the better after thinking about problems functionally:

There is no doubt that time spent playing with Haskell has taught me to exploit the type system when coding in C++, but the most lasting effect may be when I abandon C++ altogether.

I would have liked to see a lot more code and a more complete discussion on how functional thinking can improve imperative code. As I dig deeper into Haskell I’ll post some thoughts on how it’s making my daily imperative programming better or just different.

If you haven’t exposed yourself to Haskell yet I highly recommend watching A Taste of Haskell the next time you have 3 hours to completely geek off. Make sure you download the slides and follow along.

Rails 4.0 will drop support for Ruby 1.8.7

Thu, 02 Feb 2012 00:00:00 -0700

Back in 2010 when Ruby on Rails 3.0 was released the core team dropped support for Ruby 1.8.x except for version 1.8.7. With 4.0 scheduled for a release sometime in the Summer of 2012 all versions of Ruby 1.8 will officially be dropped:

There’s not a lot of details about what we’re going to include in Rails 4.0 yet as the primary purpose for bumping the major version number is to drop Ruby 1.8.7 support.

This coincides with the Ruby core team dropping support for 1.8 this Summer as well.

Why estimating is hard

Tue, 31 Jan 2012 00:00:00 -0700

Software developers are notoriously bad at estimating, to the point where project managers and bosses regularly double or even triple our estimates. Tools like calculating developer velocities help average out these errors but also tend to cover them up. So why is it so hard to estimate development work?

Diego Basch hits the nail on the head in his opposition to the idea that we get better at estimating over time because we’re doing the same sort of things over and over again:

Real software development is about doing something that you’ve never done before. That’s why all stupid analogies about routine real-life stuff break.

As a freelance software developer I tend to do a lot of the same things over and over again but with small changes in the details. This has probably given me a false sense of being a good estimator. I think I need better problems to solve.

Speaking of estimating, if software developers are so bad at estimating why do managers and stakeholders think they are so much better? How often does your boss/client suggest that your current task should only take you 30 minutes? What insight do they have that we don’t?

I used to work in the cube next to my manager. I would hear him on the speakerphone with other managers committing me to tasks and giving them estimates. This guy could barely use a computer yet here he was saying stuff like “that should take a day”.

Maybe that’s why we have such a bad reputation for estimating.

Fixing programming by besting current languages

Thu, 26 Jan 2012 00:00:00 -0700

Jon BeltranDeHeredia’s rant on why programming is broken has turned into a very interesting thought experiment. First, Jon describes why he thinks programming is broken:

Every step of the way, in every statement, line of code, function call or sequence-point, if you want to write good code, you need to think about all the different possible states of the whole program. This is invisible and impossible to define. Provably. Always. In all existing languages. That’s the reason 100% code-coverage unit-testing doesn’t guarantee bug-free code, and it never will. This is also the reason bad programmers don’t become good: they just can’t think about all those possible cases in a structured way.

Then he describes his vision for a new programming language that looks like the love child of Haskell and Prolog.

If you’re looking for something geeky to spend your free time on I suggest taking some ideas from the comments on Jon’s article. Readers are posting links to all sorts of interesting things such as executable specifications, the failures of Prolog, proof management systems, and the 4GL fiasco.

I took some programming in school != programmer

Tue, 24 Jan 2012 00:00:00 -0700

How often does a boss/client tell you they have a good sense of how long something should take because they “took some programming in school”. I can’t tell you all the times I’ve picked up a train wreck project written by someone who “knew enough to be dangerous”. Being good with a computer or taking a Java course in college does not make one a programmer.

Yossi Kreinin has a more thoughtful rant in his article:

When something useful can not be done quickly and isn’t the average person’s idea of fun, it becomes the business of professionals - or hardcore hobbyists indistinguishable from professionals.

In my experience most non-programmers don’t have the perseverance to really understand something when things get tough. They’re more interested in the end result than the process. All of the good programmers I know enjoy the process just as much as the end result.

2012 seems to be the year for learning to code

Mon, 02 Jan 2012 00:00:00 -0700

If your new year’s resolutions include learning how to write code, or if that one buddy who keeps talking about learning to program is still bugging you, this might be your lucky year.

Mike Gehard thinks you should start with the Ruby programming language so he created an outline of assignments that will put you on the right path:

Welcome to my attempt at coming up with a reproducible outline that will allow any non-programmer to become a software developer in the Ruby programming language and get a job writing web applications using the Ruby on Rails framework.

Pulkit Arora put together a similar outline last year that is much more exhaustive than Mike’s. It’s probably safe to say that Pulkit’s list is more like a software developer’s road map.

Finally, the developers behind Codecademy are at it again, this time with Code Year, a site that sends you a weekly interactive lesson designed to teach you how to program. There doesn’t seem to be much more information than that, so take the plunge and type in your email address.

Hashing algorithm vulnerable to denial of service

Thu, 29 Dec 2011 00:00:00 -0700

It’s likely that the hashing algorithm in your favorite language is vulnerable to a denial of service attack. While Perl fixed this forever ago, other languages have been caught with their pants down:

… the flaw affects a long list of technologies, including PHP, ASP.NET, Java, Python, Ruby, Apache Tomcat, Apache Geronimo, Jetty, and Glassfish, as well as Google’s open source JavaScript engine V8. The vendors and developers behind these technologies are working to close the vulnerability, with Microsoft warning of “imminent public release of exploit code” for what is known as a hash collision attack.

There’s already a new release of Ruby that fixes the vulnerability, ruby 1.8.7-p357. Ruby 1.9.x is not vulnerable to this attack.

Java is dead, long live Java

Wed, 21 Dec 2011 00:00:00 -0700

If you only caught the headline from the recent OMGUbuntu article about Java being removed from Ubuntu, you may want to catch the whole story from Simon Phipps:

Java is not being removed from Ubuntu. OpenJDK is the open source version Java, is developed primarily by Oracle, is becoming the reference implementation and is available in Ubuntu. The Sun-Java packages under the DLJ were a temporary stop-gap we put in place before Java was liberated as GPL. Removing these packages is just housekeeping.

I guess Java lives on.

Fun stuff for Friday Dec. 16

Fri, 16 Dec 2011 00:00:00 -0700

The holiday season is in full swing and for a lot of people this Friday marks the start of a long vacation. To celebrate while keeping your mind sharp, here are a few things to check out over the weekend:

If you missed it, Python was voted best language for the third year running by readers of Linux Magazine. C++ was the runner up, which sort of makes sense. A lot of C++ developers embed Python in their applications.
TextMate (the popular Mac OS X text editor) is getting a refresh with 2.0-Alpha out. Matt Gemmell wrote a letter to TextMate explaining why their affair had to end.
Speaking of Mac OS X, the crew over at We Love MacRuby is trying to persuade Apple to support MacRuby on iOS. If you’d like to write iOS applications using Ruby you should head over to their site and join the campaign.
David Tchepak, like a lot of software developers, has decided to move away from good ol’ QWERTY and try Colemak instead. He wrote an interesting article about how the transition has been going.
John Larson wrote a thought provoking essay about cheap software development labor (mostly overseas). As a freelance software developer I can attest to having to rewrite a lot of software that was originally developed overseas.
Some videos to watch: The micromouse won a maze race in 4 seconds. Of course, it previously had exposure to the maze for training (via Hack a Day). This hexabot looks like a Jellyfish (via Hack a Day). And finally, a lizard uses its tongue to play a touchscreen game (via kottke).

Have an excellent weekend!

PostgreSQL as a key-value store

Tue, 13 Dec 2011 00:00:00 -0700

I just read an interesting article about using PostgreSQL as a key-value store using the hstore module:

Yes I know you are really happy with your “persistent” Key Value store. But did anybody notice hstore that comes along Postgresql. I find Postgresql to be a really great RDBMS that has been ignored all the time.

Okay, maybe the English in this article isn’t that great, but if you haven’t looked into hstore before it might spark your interest.

Improving your skills by reading commits

Wed, 07 Dec 2011 00:00:00 -0700

Pat Shaughnessy has a great idea for people looking to improve their Ruby skills, study commits to Rails:

There are plenty of examples of elegant code, cool testing techniques and great new features in Rails if you go and look for them, but what impressed me the most were the small things. Like the small, in-between notes and simple harmonies in the midst of a large symphony, the Rails commits that caught my eye were tiny changes made by the Rails core team that were easy to miss, but that showed their real passion and love for Rails.

Pat highlights some of his favorite commits and even takes a closer in-depth look a few of the good ones.

VendorKit: bundler for Objective-C

Mon, 05 Dec 2011 00:00:00 -0700

Having trouble maintaining dependencies in your Objective-C application? Love the way Bundler deals with Ruby Gem dependencies? You may want to take a look at VendorKit:

VendorKit makes the process of using and managing libraries in iOS easy. VendorKit is modeled after Bundler. VendorKit streamlines the installation and update process for dependent libraries. It also tracks versions and manages dependencies between libraries.

Bundler has driven me to insanity more times than I care to admit, but VendorKit could be a real lifesaver.

Fun stuff for Friday Dec. 2

Fri, 02 Dec 2011 00:00:00 -0700

I’m slowly getting back on track after taking an extended holiday last week and then catching a cold this week. Needless to say, I’m pretty happy that it’s Friday, and I have a whole bunch of fun stuff to share, so let’s get to it.

Hackerbuddy has relaunched with a bunch of improvements and some new features. Full details in this article.
Doom 3 was open sourced and there’s already instructions on how to build it on a Mac using Xcode.
Google Code Search is getting shut down on Jan. 15, 2012. Miguel de Icaza compares alternatives.
Users of the beautiful Protovis visualization library that are making the switch to D3 (its replacement) might want to take a look at Luke Francl’s D3 for Mere Mortals.
Have you seen the new web API gateway from eBay that they’re calling ql.io? Apparently it uses a SQL-like syntax and allows you to communicate with multiple web APIs in the same call. Oh, and it’s not just for eBay APIs.
I must absolutely make myself a Jabba the Cupcake (via Boing Boing).
I’m a mechanical pencil nerd and the Kuru Toga is a must have.

Have a great weekend!

HAML users repent before it's too late

Thu, 01 Dec 2011 00:00:00 -0700

I used to have a coworker that would constantly bug me about how awesome HAML is, but whenever I looked at HAML code all I saw was noise. Brandon Keepers has done a much better job at explaining why you should avoid HAML:

I agree that HAML is easier to write than HTML, simply because it involves less typing. But I feel it is infinitely harder to read. While the indentation makes it easy to see the nesting, the extremely overloaded synax requires careful attention to each line.

In my opinion, if you hate writing HTML a much better option is Zen Coding.

Seven databases in seven weeks

Thu, 01 Dec 2011 00:00:00 -0700

If you enjoyed Bruce Tate’s book Seven Languages in Seven Weeks you might also enjoy Seven Databases in Seven Weeks by Eric Redmond and Jim Wilson:

Data is getting bigger and more complex by the day, and so are your choices in handling it. From traditional RDBMS to newer NoSQL approaches, Seven Databases in Seven Weeks takes you on a tour of some of the hottest open source databases today.

Tate’s book also sparked a whole bunch of Github repositories with the code people write as they make their way though the book.

Don't be affraid to learn vi or emacs

Wed, 23 Nov 2011 00:00:00 -0700

Over at the thoughtbot blog there’s an article about the myth of the vim learning curve. I think everything said about vi/vim can equally be applied to Emacs. The path to getting up and running isn’t as daunting as you might think:

Go to your shell and type vimtutor. The tutorial that’s presented is excellent and you’ll be through it in no time. Once you’re done, you’ll have the rudiments needed to get your work done. You won’t be fast yet, no; but you’ll be competent.

There’s a similar feature in Emacs. To start the tutorial just type C-h t (that’s Control-h and then t).

Git workflows and configuration

Tue, 22 Nov 2011 00:00:00 -0700

If you’re new to Git or are looking for some branching workflow guidance Kumar McMillan has put together an article about topic branches and rebasing using Git:

The nice thing about working in a topic branch is it’s isolated from master and no one else is tracking that branch so I can use git rebase to create the best commit before merging into master.

Need more detail? Yehuda Katz wrote an article last year that covers his workflow compared to how someone would do the same tasks with Subversion.

Finally, to promote his Git workshop Ben Hoskings wrote an article explaining how to configure git log to be more useful.

Simple code makes good software

Mon, 21 Nov 2011 00:00:00 -0700

Alex Obenauer has been rustling some feathers with his recent article entitled “Stop writing good code; start writing good software”:

Do you really need a full objet-oriented API right now? Do you really need to make a dozen interwoven classes, when it’s possible just a hundred or so lines in one class will do fine? Can you do all the same error checking and unit tests in a much smaller code base?

I think Alex should have used the title “Simple code makes good software” and focused on the simple vs. elaborate code debate. At least that’s the theme I see just under the surface in his article.

Fun stuff for Friday Nov. 18

Fri, 18 Nov 2011 00:00:00 -0700

This past week has been a blur for me so I’m totally ready for the weekend. The kids and I have been making our way through the Alien films and tonight we’re on number 3. I don’t have fond memories of the movie but I’m sure we’ll have fun.

Hopefully you have some fun planned yourself and the links below will help you kick off the weekend.

Anders Evenrud has released what he’s calling OS.js, or what I’d call an operating system emulator. It almost looks like you VNC into his Linux box, but nope, it’s all JS. It even has apps written for it.
Emacs users who want to start using version 24 (to be released this spring) should take a look at Bozhidar Batsov’s Emacs Prelude. It’s a set of configuration files and packages sort of like the Emacs starter kit.
Itching to write code nobody can read? Back after a rather long hiatus is the International Obfuscated C Code Contest!
Looking for a geeky way to put your keys away for the night? Why not build an RJ45 key chain. I recommend wiring it so you can plug it right into a hub or switch.
Videos: Astronauts tripping and falling on the moon (via GeekDad) and Simultaneous time lapse video of the sky over 360 days (via kottke).

Have a fun weekend!

Concurrency in Ruby 1.9

Thu, 17 Nov 2011 00:00:00 -0700

Ruby 1.9 introduced an additional concurrency mechanism called Fibers. If you haven’t played with Fibers or still haven’t discovered Ruby continuations William Morgan has written an article that you might want to take a look at:

Ruby 1.9 has both fibers and continuations. The two are often mentioned in the same breath. They do vaguely similar-sounding things, and are implemented in Ruby 1.9 with similar mechanics underneath the hood, much as how continuations and threads were implemented with the same underlying mechanics in Ruby 1.8.

Another way to deal with concurrency is with the Actor Model. Mike Perham demonstrates the Actor Model built into Rubinius in his article on concurrency. He also points to other resources for using the Actor Model in Ruby.

The Objective-C object model

Wed, 16 Nov 2011 00:00:00 -0700

If you’re an Objective-C developer you can get a lot done without understanding the C roots in Objective-C or the object model employed by Objective-C. In fact, trying to learn the object model can be intimidating and confusing because some peculiar complexities.

Even if it’s a little complex, it’s a worthy endeavor to learn the Objective-C object model because you may end up needing to interact with the Objective-C runtime someday, and understanding the object model will be very useful in that situation.

Grab a cup of tea or coffee and follow along with me while we explore the Objective-C object model.

Objects are instances of a class

Sure, this may sound obvious, but it’s worth taking a few seconds to really consider what it means when we say that an object is an instance of a class. Take the following code for example:

NSString *name =
  [NSString stringWithFormat:@"%@", @"Peter"];

This is a rather contrived example but it’s important to demonstrate some of the moving parts in the Objective-C runtime. When the code above is executed we end up with a variable called name that is an instance of the NSString class. Please consider the following:

Objects (instances of a class) are used to hold state.
An object’s state is held in its instance variables. For example, the name object above owns some memory for storing the bytes that make up the string “Peter”.
The name object also contains a pointer back to its class (NSString) as part of its state.
In Objective-C objects don’t contain any behavior, only state.
Classes contain behavior and do not contain any state. The behavior they hold is the methods that their instances can respond to.
Let’s repeat that once again for clarity. Objects are containers that store state (instance variables) and classes are containers that store behavior (methods).

To make this more concrete take a look at the following diagram:

The blue box represents the name variable (an instance of the NSString class). Using the diagram above let’s look at what happens if you send the length message to the name variable:

NSUInteger len = [name length];

The first thing that Objective-C does is ask the name variable which class it was instantiated from. Since all the methods are held in the class, that’s where it will start looking for the length method.
Since the class of name object is NSString, Objective-C goes to the NSString class and searches for the length method.
If Objective-C finds the length method in the NSString class it will stop searching and execute the method that it found. Otherwise it will continue searching the inheritance hierarchy for the method.

I should note that the process I’m describing has been simplified. I’m not considering other places that methods could be stored (e.g. Objective-C categories).

Classes are really objects too

Given that a class holds methods for its instances, you might be wondering where class methods are stored. Let’s look at the code that created the name variable again:

NSString *name =
  [NSString stringWithFormat:@"%@", @"Peter"];

In the code above, the stringWithFormat: message was sent to the NSString class. Can classes receive messages, and if so where are those methods kept? In this case, the Objective-C syntax is covering something up, it’s hiding the fact that NSString is also an object.

If NSString is an object, that means that it must have been instantiated from a class that is holding its methods. This is true, but before we dig into it let’s look at some more code:

@interface Person : NSObject {}
+ (void) aClassMethod;
- (void) anInstanceMethod;
@end

You’ve probably seen this many, many times. But what is really going on?

A new class is being declared, the Person class.
This class will be used to hold methods for instances.
A second class is also being created behind the scenes. Unfortunately this additional class has the same name (Person) but is referred to as the Person metaclass.
The Person metaclass holds the class methods for the Person class.

This can be very confusing because the same name is being used over and over again. Going back to our name variable, we can say the following:

The name object is an instance of the NSString class.
NSString is an object instantiated from the NSString metaclass.
If you send the length message to the name object, Objective-C will go look for that method in the NSString class.
If you send the stringWithFormat: message to the NSString object, Objective-C will go look for that method in the NSString metaclass.

Here’s another diagram:

This time the blue box represents the NSString object. Here’s the creation of the name object again:

NSString *name =
  [NSString stringWithFormat:@"%@", @"Peter"];

When the stringWithFormat: message is sent to the NSString object Objective-C will:

Ask the NSString object for it’s class, which is the NSString metaclass.
It will then start searching the NSString metaclass for the stringWithFormat: method. If it doesn’t find the method there it will continue up the inheritance hierarchy.

Naturally, since the NSString class inherits from the NSObject class, the NSString metaclass inherits from the NSObject metaclass.

Wacky NSObject inheritance

If you’ve made it this far you can rest assured that you know enough about the Objective-C object model to know a lot more about what’s going on in your code at run time. There’s just one more thing that you may want to know: Why does it look like the NSObject metaclass inherits from the NSObject class? Because it does.

Consider the following:

The NSObject class contains methods for its instantiated objects.
The NSObject metaclass contains methods for the NSObject class (i.e. NSObject class methods).
The NSObject metaclass inherits from the (non-meta) NSObject class.
Therefore, any instance methods defined for NSObject will also be in the search path as class methods for any Objective-C class.

It’s a bit wacky, but if you look at the diagram above it should start to make sense.

C++11 move semantics and rvalue references

Tue, 15 Nov 2011 00:00:00 -0700

I stopped following the C++0x standardization process a long time ago, so when C++11 was announced I delighted in some of the developments that I had missed. Then I saw “move semantics” and thought to myself WTF? Then I found this stackoverflow answer and now I’m good. There’s also this section in the C++11 FAQ.

Remote pair programming in a terminal

Mon, 14 Nov 2011 00:00:00 -0700

Pair programming is becoming more popular due to the good old terminal. Instead of having to sit next to your coworker who had an onion sandwich for lunch you can use terminal sharing from across the office or the country. And with services like Ruby Pair that connect people who want to pair program, knowing how to use a terminal multiplexer is worth a few minutes of your time.

The venerable choice for a terminal multiplexer is GNU Screen. Dalibor Nasevic has written an article with instructions on how to configure SSH and Screen to allow remote pair programming. In my opinion, a huge downside to using Screen is having to setuid the process to enable multiuser mode. A much better option is tmux.

If you’re looking for a more modern terminal multiplexer that is much easier to configure, tmux is the answer. Joe Moore has an article about using tmux with Vim for pair programming, but it’s really just a few simple commands:

# Host
tmux -S /tmp/pairprog

# Client
ssh pair-host
tmux -S /tmp/pairprog attach

Remote pair programming with Screen or tmux is also a great excuse to polish your Vim or Emacs skills.

Fun stuff for Friday Nov. 11

Fri, 11 Nov 2011 00:00:00 -0700

Not only is it Friday, but here in the US it’s veterans day, which means some of you might not even be working. That reminds me, I better go out and take the mail out of my mailbox since the post isn’t running today.

Here are some fun things to look at over the weekend:

ZeroMQ is an older project that I’ve just stumbled upon. It provides an API to just about any programming language for doing distributed work using the actor model. The cool thing is, the same API works locally between threads and processes but can scale up to processes on different machines.
I had to rub my eyes and read the headline again, but it’s true, Microsoft contributed patches to Samba.
For those of you who haven’t jumped ship to a different compiler Anders Ahlström has compiled a list of the most useful GCC options and extensions.
Did you know that you can configure Git to pull from one URL, but push to another? It’s perfect for when you have a submodule configured on a public read-only URL, but want to push to a private read-write URL.
Videos to watch: my kids are going to love watching this super slow motion video of a bouncing water droplet, but they won’t appreciate this watching old school video game deaths like I do (via kottke).

Those Objective-C "@" Directives

Thu, 10 Nov 2011 00:00:00 -0700

Steffen Itterheim has compiled a complete list of those Objective-C @ compiler directives:

I haven’t been able to find a list of all Objective-C @ compiler directives in one place. We all know the keywords like @interface and @implementation but others like @dynamic and @encode are lesser known, and possibly even much less understood.

You might also like this really nice Objective-C cheat sheet from Ray Wenderlich.

The basics of good programming style

Wed, 09 Nov 2011 00:00:00 -0700

Isaac Taylor has written an article that we’ve all probably read before: the dos and don’ts of programming style.

Striving to write good code should be the goal of every Software Developer. Writing code that is easily maintainable, robust, simple, and makes sense is no easy task. It takes years of practice and uncountable hours of time to become a good Programmer, and the job is never really done.

Some of his advice is pretty standard (e.g. choose meaningful variable names) but this one ruffled my feathers a bit: “Comment frequently”. If the article’s comments are any indication, others feel the same way. A comment from reader MarchVeg sums it up:

Personally I don’t agree with comments, they are another thing to maintain as the code changes over the years.

I find that comments tend to be the most brittle part of source code. I would change Isaac’s recommendations for comments to:

Comment independent units of code such as classes and functions as a form of documentation and a reminder to maintainers. These comments are more likely to be relevant over time since function maintainers are free to change how they implement a certain task but shouldn’t change a function so it does something entirely different than calling code expects.
If you find yourself needing to comment particularly complicated lines of code try rewriting it first. This is where tests can be very helpful. It’s alright if the code increases in size as long as it becomes more clear and simple.

I suppose my second point also goes against Isaac’s recommendation that you shouldn’t “write 4 lines of code when 2 will do”, which I also disagree with. 4 lines of simple idiomatic code always beat 2 lines of terse obfuscated code. I’m sure what Isaac meant to say was “don’t write overly verbose code when succinct code will do.”

Update: As it turns out, the actually author of the article above isn’t Ilias Tsagklis, it’s Isaac Taylor. Sorry about that Isaac!

A closer look at Redis

Tue, 08 Nov 2011 00:00:00 -0700

If you’re not familiar with Redis it’s a key-value store that gained popularity as a replacement for memcached due to its ability to store and manipulate structured data (e.g. increment an integer, get the union of two sets, etc.). Redis is worth checking out if you need a high performance, in-memory, key-value store with persistence.

The fifteen minute introduction is particularly good:

It’s not always trivial to grasp how this data types work and what to use in order to solve a given problem from the command reference, so this document is a crash course to Redis data types and their most used patterns. [sic]

Armed with data type information from the introduction you may want to take a look at Karl Seguin’s Zero to Master article:

More than once, I’ve said that learning Redis is the most efficient way a programmer can spend 30 minutes. This is a testament to both how useful Redis is and how easy it is to learn. But, is it true, can you really learn, and even master, Redis in 30 minutes?

I found the article informative yet wanting. There’s a part 2 but I didn’t find that it added anything useful.

Managing debt buildup while developing

Mon, 07 Nov 2011 00:00:00 -0700

Technical debt is one of those things that every developer knew about before it had a proper name, but after Ward Cunningham coined the term we had a tool to drive conversation and encourage decision makers to do the right thing. It also gave us the ability to evaluate how much technical debt was acceptable and even sometimes necessary.

In the real world, completely avoiding technical debt isn’t feasible, but keeping it as low as possible is. Chris Hartjes wants you to also consider what he calls infrastructure debt:

Infrastructure debt is debt that you build up because you have not been paying attention to the process of creating the environments your application will exist in and have not been paying attention to the process of how your code gets from development into production. In my opinion, infrastructure debt is much more difficult to “pay off” than technical debt. Why? It is often very difficult for people to understand that it even exists.

It’s easy to see that there are all sorts of ways to accumulate debt while writing software. I’ve seen over and over again how stakeholders create debt in their products, seemingly without even realizing it, and without any help from the development team. That sort of debt is very hard for developers to pay back.

It’s not impossible though and with more examples of how debt builds and how to pay it down developers can educate themselves and the stakeholders.

Fun stuff for Friday Nov. 4

Fri, 04 Nov 2011 00:00:00 -0700

It’s Friday again and here are some geeky things to cuddle up with as the workday comes to an end and you start to dream of the weekend:

The Star Wars Alphabet. Can you figure out all the letters?
With git-jump coming in the next release of Git Emacs users might want to take advantage of it.
Videos to watch: a helicopter-like flying sphere and a cute little bicycle riding robot (I love watching it stop).
I’m sneaking this not-too-geeky video in here as well: a crazy Bollywood car chase.

As always, enjoy your Friday and have a great weekend.

Self-guided Ruby on Rails 3.1 Tutorial

Fri, 04 Nov 2011 00:00:00 -0700

One of the complaints about Ruby on Rails that I hear most often is its complexity, despite all the video tutorials showing how easy it is to build a simple application in 5 minutes. In reality, Rails was a huge paradigm shift for web application development and even with its complexity it’s a great platform.

I’ve spent the better part of the last 6 years writing applications using Ruby on Rails and people often ask me for my perspective on how to bootstrap an application and best practices for using Rails. I also see a lot of people getting lost in the noise of scaffolding and complicated tutorials.

That’s why I decided to write an example application using Ruby on Rails 3.1. The application is on Github to make it easy to get to, and takes advantage of source code control features like branches so you can follow along as the application gains functionality.

I hope you enjoy this self-guided tutorial for Ruby on Rails 3.1.

The longevity of OS APIs vs. web APIs

Fri, 04 Nov 2011 00:00:00 -0700

Nick Bradbury makes a good point about the longevity of operating system APIs compared to the more brittle APIs of the web:

You might think you’re immune to this problem if you only integrate with APIs created by large players such as Twitter, Facebook and Google. But in recent years we’ve seen Twitter switch to a new authentication system, Facebook deprecate FBML, and Google discontinue several APIs. All of these changes have broken, or will break, existing apps.

Global Day of Coderetreat

Thu, 03 Nov 2011 00:00:00 -0700

Here in Colorado we’ve had a pretty regular helping of Coderetreats. And now everyone is going to do it on the same day:

Global Day of Coderetreat is a world-wide event celebrating passion and software craftsmanship. We will spend the day practicing the craft of software development using the [coderetreat format] coderetreats. You can find out more information about the day on the coderetreat blog.

Learning the fundamentals of Scala

Wed, 02 Nov 2011 00:00:00 -0700

It seems to me that Scala has been popping up all over the place in the last couple of years and I’m starting to see it in more and more job descriptions. If you’re new to Scala then Craig Tataryn has a very well written introduction to the fundamentals of Scala:

What was the biggest mistake I made when I first started learning Scala? I jumped in without looking. I felt I could just “pickup” the language, like a C# developer might pickup Java. So the point I’m trying to make is to start off in Scala by understanding some core concepts. If you do, the veil of the Matrix is lifted and you start to free your mind of the syntax which you aren’t yet used to.

I’m a little unnerved by the recent trend to implement new languages on top of the JVM. Is the JVM really that good?

Giving JavaScript another chance

Tue, 01 Nov 2011 00:00:00 -0700

JavaScript has been the subject of much public ridicule, and not just for its name. People have even written alternatives to JavaScript that translate into JavaScript so they can run in existing browsers. Personally, I think JavaScript has some obvious shortcomings but for the most part is a pretty usable language.

Here are some recent JavaScript related articles that have come across my radar that are worth checking out:

And don’t forget about the seminal book: JavaScript: The Good Parts.

More on code coverage metrics

Mon, 31 Oct 2011 00:00:00 -0700

After bashing code coverage metrics I came across an article from Naresh Jain about being seduced by code coverage numbers and gaming the system:

I once worked for a manager (a.k.a Scrum Master) who insisted that we should have at least 85% code coverage on our project. If we did not meet the targeted coverage numbers this sprint, then the whole team would face the consequences during the upcoming performance appraisals. In spite of trying to talk to her, she insisted. I would like to believe that the manager had good intent and was trying to encourage developers to write clean, tested code.

My thoughts on code coverage have been expressed before, but Kevin Rutkowski does a good job so I’ll use his words:

The bottom line is that tests can easily be written that exercise code without actually testing anything. So, when a team sets a specific goal of 100% Code Coverage and has a tight deadline, it is likely that a large number of those tests will be low quality.

Fun stuff for Friday

Fri, 28 Oct 2011 00:00:00 -0700

I love Friday. Everything seems to be slightly better and everyone tends to have a better attitude. Here in Colorado today is even suppose to be the warmest day of the week! The following tidbits are some fun things to share with your geek friends instead of pretending to work.

Commit Logs From Last Night has some outrageous commit messages and What the Commit helps you pick a random one for your next one character typo commit.
Twilio is pretty interesting and with their recent launch in the UK Richard Taylor has written an awesomely simple tutorial that uses Sinatra to create a call tree. Even cooler, Mike Szczys uses iPhone’s Siri to voice control his house by hacking the Arduino to receive text messages via Twilio.
Title Scream is an interesting site that catalogs the start screens for 8- and 16-bit video games. The title screens are even animated. Makes me want to start the NES emulator.
Apparently HarperCollins came across a fairly large collection of unpublished artwork from J.R.R. Tolkien while preparing the 75th anniversary edition of The Hobbit. The artwork is being published in a new book (via GeekDad).

As always, enjoy your Friday and have a great weekend.

Garbage collection/reference counting in C++11

Thu, 27 Oct 2011 00:00:00 -0700

There are plenty of people who criticize C++ for not having a standard garbage collector. Herb Sutter explains why C++ doesn’t support automatic mark-compact garbage collection but does have a formal ABI for mark-sweep style collectors and supports reference counting through the new unique_ptr, shared_ptr, and weak_ptr types. The article’s comments help shed some light on a very important C++ viewpoint: deterministic destructors are super helpful.

The C++11 FAQ maintained by Bjarne Stroustrup contains more information about the new automatic pointer types.

Don't do your own date/time calculations

Wed, 26 Oct 2011 00:00:00 -0700

Landon Dyer has a nice rant on date/time calculation over on his blog:

This is a class of problem that everyone knows is trivial. “Of course I know how to do this,” you probably think to yourself, “This is trivial, I can just code this thing up and have an early lunch.” Trivial is the most dangerous word in all of computerdom, it’s a misprounciation of “Evil”. Your blood should run cold if you hear it spoken out loud, because the Gods will be listening carefully and taking notes on how to screw you if they ever hear you say it.

The comments make this article even more interesting with everyone comparing their date/time war stories.

Keep learning and make mistakes

Mon, 24 Oct 2011 00:00:00 -0700

After reading a Wired article about why some people learn faster than others, Davy Brion wrote an article putting the study in the context of software development and why we need to continually challenge ourselves:

It also kinda confirms what I’ve seen from fellow developers ever since I graduated and started working. Every good developer I’ve met isn’t afraid to make mistakes and sees mistakes as learning opportunities. These people routinely challenge themselves by learning something new. They also keep an eye on what other people are doing to get better and they actively try to learn from people they consider to be better than them.

Davy’s experiences resonate with my own. I’m always tinkering with something new. From the outside I’m sure it looks like I have a very short attention span, and that’s often what I tell people.

Fun stuff for Friday

Fri, 21 Oct 2011 00:00:00 -0700

Here’s some fun stuff to help boost the enjoyment of your Friday.

Remember those noisy 3.5" floppy drives? Watch dueling drives play the Imperial March, and read how it was done.
You’ve probably seen the video of the levitating superconductor. This article explains how it works (with the video embedded so you can watch it a few more times).
You can now pre-order The Geek Dad Book for Aspiring Mad Scientists which promises to be a great way for you to share your geekiness with your kids.
If you missed it back in August, NPR compiled a list of the Top 100 Science-Fiction, Fantasy Books.

Enjoy your Friday and have a great weekend.

Using regular expressions in C++11

Thu, 20 Oct 2011 00:00:00 -0700

One of the major new features added to the recently released C++11 is support for regular expressions. Sadly GCC doesn’t have a working regex implementation yet, but clang and Visual Studio do. If you’re looking for an introduction then the Solarian Programmer has a two part tutorial to get you going:

Validating the user input was always a problem in C++, especially when the user is supposed to enter a number. Each programmer was forced to write his own error checking functions for numerical input validation, basically it was no uniform syntax for checking the correctness of the user input. Enter C++11 and regular expression!

Start with part 1 and finish off with part 2.

Write HTML/XML faster with Zen Coding

Thu, 20 Oct 2011 00:00:00 -0700

If you find yourself writing a lot of opening and closing tags by hand you really should check out Zen Coding:

Zen Coding is an editor plugin for high-speed HTML, XML, XSL (or any other structured code format) coding and editing. The core of this plugin is a powerful abbreviation engine which allows you to expand expressions–similar to CSS selectors–into HTML code.

A very large variety of text editors are supported by the plug-in, with alternative implementations for editors like Emacs and Vim.

Still looking for a language to learn this year?

Wed, 19 Oct 2011 00:00:00 -0700

It’s hard to believe the year is wrapping up, a constant reminder being all the Christmas decorations on sale (next to the Halloween decorations, which is weird). If you’re looking for something to do while taking all that copious holiday vacation time why not learn a new programming language? It sure beats listening to family members argue over politics.

To get you started Michael Fogus has written a couple of articles he calls Perlis Languages:

A Perlis Language is a programming language that I believe will shake one’s views on software development to the core.

The first article contains an overview of 9 programming languages, with the second article covering an additional 14 (with brief mentions of another 12).

Sharing your private Git repositories

Tue, 18 Oct 2011 00:00:00 -0700

When I learned about branching and merging in Git a few years ago I was hooked. Then the honeymoon came to an end and I realized that sharing a private repository with someone required giving them SSH access. Ouch! Fortunately GitHub came along and solved the problem well enough that the idea faded into the back of my mind.

Then one day I needed to share a private repository with someone just for fun and didn’t think that paying GitHub was justified. I was pleasantly surprised to see how many people have solved the private repository access issue.

If you’re looking to host your own private Git repository this list of resources should get you started:

GitLab is a new open source project with features similar to GitHub. It has a nice looking user interface and repository access control via SSH public keys.
Gitolite is another open source project for administering a centralized Git server with fine-grained access control. It doesn’t have a user interface so it’s purely for access control. A similar project is Gitosis.
If you want something even simpler Richard Taylor has written an article with instructions for setting up SSH to control access to Git.

Challenge your text editor fu

Mon, 17 Oct 2011 00:00:00 -0700

Ilya Grigorik has put together an excellent site called VimGolf where developers host challenges to see who can complete a task in Vim with the fewest keystrokes. Here’s an example challenge:

Reformat long lines. Rearrange this ruby method call to put each parameter on its own line. Could become a useful macro.

You are given an input file and an output file with the goal to provide the minimum number of keystrokes to convert the input file to the output file.

Vim not your cup of tea? Tim Visher goes through the VimGolf challenges with Emacs and posts the results on Vimeo. His videos are great to watch if you want to improve your Emacs skills.

Reactions to Dart, 17k line Hello World

Fri, 14 Oct 2011 00:00:00 -0700

If you’ve been following all the chatter after Google’s Dart announcement you know that reactions have been… mixed? No, not really. Besides the occasional and misguided praise for Dart as a JavaScript killer, the majority of blog articles and comments are critical of Dart and Google.

Starting with a bit of humor, if you haven’t seen the 8-line Dart version of Hello World translated into 17k lines of JavaScript it’s a fun read. Of course the generated JavaScript from the Dart translator includes all of the Dart libraries and isn’t optimized, so it’s not an entirely fair look at Dart, but it is funny. Optimizing the JavaScript doesn’t help much either. If you’re looking for a better technical analysis of Dart, Rafaël Garcia-Suarez wrote a good article on the subject.

The criticism directed specifically at Google deals with Dart being developed behind closed doors, with its open source status being more for marketing than anything. Brendan Eich (creator of JavaScript) has this to say about Google and Dart:

Innovating in the open and proposing early and often to standards bodies are fine. Delayed-open-washing and increasing piles of proprietary (open-source doesn’t matter) single-vendor code, which implements features that are not ever proposed for standardization, are not.

Programming tips from a photographer

Thu, 13 Oct 2011 00:00:00 -0700

I’ve always rejected the term “Software Engineer” because I’ve met very few people who write software like an engineer would. To me software development ebbs and flows between science and art, probably like any creative endeavor.

Certainly I’m not the first person to think this way. People much smarter than me, Paul Graham for example, have already written about this. That’s why I pay attention when people in other creative fields give advice.

Chase Jarvis is one of those people, and he recently wrote 10 Things Every Creative Person Must Learn. It could have been written for programmers, any of this sound familiar?

Simple is good. Almost every photo that is bad has too much information. Outside of technical basics, the number one reason that most photos fail is because there is no clear subject. Often this is the case with design, film, fashion, you name it. Remove clutter, remove distraction.

One of my pet peeves is unnecessarily complex code. Simple code always beats clever code, with fewer opportunities for bugs.

Subversion 1.7 released, goodbye .svn pollution

Wed, 12 Oct 2011 00:00:00 -0700

The Subversion team has been making steady improvements to some of Subversion’s most criticized features, namely branching and merging. With the release of 1.7 Subversion finally takes care of all those darn .svn directories:

A key feature of the changes introduced in Subversion 1.7 is the centralization of working copy metadata storage into a single location. Instead of a .svn directory in every directory in the working copy, Subversion 1.7 working copies have just one .svn directory–in the root of the working copy. This directory includes (among other things) an SQLite-backed database which contains all of the metadata Subversion needs for that working copy.

Besides the release notes, Jessica Thornsby’s Top New Features in 1.7 article is a great place to get information along with her What’s New video.

C++11 is officially released, goodbye C++0x

Tue, 11 Oct 2011 00:00:00 -0700

The ISO/IEC C++ committee made the announcement that the latest version of the official C++ standard (C++11) is done, the first major change to C++ since 1998. The announcement quotes Herb Sutter:

…and now with C++11’s improvements that incorporate many of the best features of managed languages, modern C++ code is as clean and safe as code written other modern languages, as well as fast with performance by default and full access to the underlying system whenever you need it.

However you feel about C++, there’s something to be said about being an ISO standard and having a body of passionate people to guide a language through the tricky waters of backwards compatibility. I remember installing a maintenance release of an interpreted language in order to fix a bug and discovered later that this micro version included a syntax change that wasn’t backwards compatible and broke all my production code. That was the beginning of the end for Perl for me.

So what’s new, and what changed in C++11? Actually, the list of changes is pretty long, including a new lambda syntax, regular expressions, threading, and finally a resolution to the closing right angle parsing ambiguity. Danny Kalev wrote a great article back in June that covers some of the major changes in C++11. There’s also a compiler compatibility chart to get you started.

Google's language for people who hate JavaScript

Tue, 11 Oct 2011 00:00:00 -0700

Poor JavaScript, first you get stuck with a terrible name, then years of horribly busted implementations and web pages contaminated with unrecognizable code, and now everyone hates your syntax.

If you too are a JavaScript hater and CoffeeScript leaves a sour taste in your mouth, and you happen to love Java syntax, Google gives you another option: Dart.

For client-site execution (in the web browser) Dart, like CoffeeScript, is translated to JavaScript first. But unlike CoffeeScript, Dart also comes with a virtual machine for server-side execution (without translation to JavaScript), which the Dart team says will allow you to reduce context switching by allowing you to write your entire application in a single language.

Google is also proposing a new mime type “application/dart” with the hope that browsers will adopt execution of Dart directly without the need to translate to JavaScript first. It will be interesting to see if Google has enough pull and Dart has enough of a following to actually get browser makers to implement/include a Dart interpreter.

After spending some time digging through the rather large source tree for Dart, it’s not clear to me how the DOM library plans to deal with JavaScript compatibility issues between different browsers (IE I’m giving you the evil eye). CoffeeScript on the other hand has an interesting position because it’s a superset of JavaScript which means you can continue using your favorite JavaScript framework (e.g. jQuery, Prototype, etc.).

Better analysis from a Java perspective, with syntax examples, can be found in Axel Rauschmayer’s article on Dart from yesterday.

Unix toolbox: sed and awk

Mon, 10 Oct 2011 00:00:00 -0700

Chances are good that you’re developing software on a platform that supports the Unix command line, even Microsoft platform developers have access to the Unix toolbox. There’s a lot of power and convenience to be found on the command line that can’t be replicated in a GUI.

If you’re manipulating text, which is what we do all day long, and you’re not taking advantage of the Unix command line, you’re really missing out on some huge productivity benefits. But getting started with tools such as sed and awk can be intimidating and the manual pages aren’t very helpful for a beginner either.

Here are some resources to get you going:

Bruce Barnett maintains a rather complete introduction to sed with a PDF cheat sheet.
If you like Bruce’s sed tutorial be sure to check out his equally good awk tutorial.
Peteris Krumins wrote an article about sed oneliners and a book about awk.
Joshua Thijssen has some practical examples of using awk to mine data from an Apache log file.
And there’s the sed & awk book if you’re looking for a complete text with examples.

NewtonScript and other prototype-based languages

Fri, 07 Oct 2011 00:00:00 -0700

With news this week that Steve Jobs had passed away I took a moment to reflect on my obsession with Apple products over the years and was reminded of my passion for the Newton Message Pad. Beyond being a super cool gadget the MessagePad introduced me to prototype-based languages through NewtonScript. Strangely enough, it was Steve Jobs who canceled the Newton project in 1998 after his return to Apple. I wasn’t too happy with him at the time.

In 1997, when I got my first MessagePad, I was splitting my time between C++ and Perl. Trying to learn NewtonScript twisted my brain and the utter lack of proper NewtonScript debugging tools didn’t make anything easier on me. After I got the hang of things, however, I really started to enjoy myself. It seemed like I could write sophisticated GUI applications with very little code, and absolutely no design skill.

NewtonScript was heavily influenced by Self, another prototype-based language. Self also influenced another prototype-based language that has become very popular, JavaScript.

What’s so different about prototype languages? It might help to consider that prototype-based languages are also referred to as classless object oriented languages. Class-based languages use classes to distribute behavior to objects through types and inheritance, whereas prototype-based languages do everything at the object level, on a per-object basis.

If you’re interested in playing with a prototype-based language NewtonScript probably isn’t the way to go, but one of these more active languages should pique your interest:

Io: a tiny pure OO language
Lua: a popular embeddable scripting language
And of course there’s Self

If you haven’t learned a new language this year maybe it should be a prototype-based one.

Playing with Google Go

Thu, 06 Oct 2011 00:00:00 -0700

If you’re curious about Google’s Go language but haven’t moved past “hello world” you might be interested in playing with the new interactive introduction to Go, A Tour of Go:

The tour is interactive. Click the Compile button now (or type Shift-Enter) to compile and run the program on a remote server. The result is displayed below the code.

These example programs demonstrate different aspects of Go. The programs in the tour are meant to be starting points for your own experimentation.

There’s also a long list of blog articles looking at the Go syntax. Jon Cooper recently wrote an article exploring the syntax of Go and Ruby by comparing a script written in both. Personally, I’m a huge fan of Ruby syntax so I still have a way to go in order to I warm up to the Go syntax.

Ruby Core Team: Dude, Ruby 1.8 is dead, really

Thu, 06 Oct 2011 00:00:00 -0700

I don’t know if I’ve let time get away from me or I’m just not as cool as I used to be but I’m still using Ruby 1.8 most of the time. The last couple of years has seen a big push towards 1.9 and now the Ruby Core Team is getting serious about leaving 1.8 in the past:

Ruby’s core developers have been actively working on their new version, 1.9, and they are about to release new 1.9.3. I have been using 1.9 for years and now I cannot go back to the days without it. Rich features. Faster execution. Rubygems integrated. Rails works perfectly. I cannot but say it is totally wonderful. Everyone please, use 1.9.

If you haven’t made the transition to 1.9, here are a few links to get you moving:

Bruce Williams’ Migrating to Ruby 1.9 presentation
A list of 10 things that David Black wants you to keep in mind
The Ruby 1.9 Walkthrough video ($16) by Peter Cooper

I guess I better start updating that Rails 1.2.6 app I have running in production.

Don't write CPU intensive code in JS, duh

Tue, 04 Oct 2011 00:00:00 -0700

If you haven’t been following the flame war that ensued after Ted Dziuba wrote his now famous Node.js is Cancer article you’re missing out on some good fun. Ted’s main point is that event-based frameworks, specifically Node.js, suffer from a poor implementation that allows the main event loop to block if a handler blocks.

Tony Arcieri wrote a great follow up covering the fact that some people totally missed Ted’s point and even tried to solve the example problem that Ted used in his article to demonstrate the weakness in Node.js.

The Node.js blog didn’t directly respond to Ted’s criticism but instead decided to say “don’t do that” in a roundabout way:

Using Node does not mean that you have to write a video encoding algorithm in JavaScript (a language without even 64 bit integers) and crunch away in the main server event loop. The suggested approach is to separate the I/O bound task of receiving uploads and serving downloads from the compute bound task of video encoding. In the case of video encoding this is accomplished by forking out to ffmpeg. Node provides advanced means of asynchronously controlling subprocesses for work like this.

Disclaimer: Using JavaScript server-side seems gross to me.

The diminishing returns of testing

Mon, 03 Oct 2011 00:00:00 -0700

I cringe when I hear people talking about the need to reach 100% test coverage because the return on investment after making it to that point is pretty low. The true value in testing lies in the validation that critical code functions properly and continues to do so after features are added to the system.

An article on the BitRoar blog has a lot more to say on this subject:

I’m not saying TDD is a bad thing, but there are more tests than unit tests, and there are more ways to verify software than testing. If you choose a methodology without comparing it against alternatives, if you claim that it works all the time without evaluating the results, than you’re not doing engineering - you’re practicing a religion.

The article picks on TDD specifically but I think the premise applies to all types of testing.

The path to being a better developer

Mon, 03 Oct 2011 00:00:00 -0700

There are plenty of blog articles about being a better software developer like this one by Dodgy Coder or this one by Josh Tynjala:

Not entirely sure how to go from code monkey to code ninja? I don’t have all the answers, of course, but each of these suggestions has personally helped me improve my career and discover other talented people that were definitely worth meeting.

When it comes to becoming a better software developer the gold standard for books is The Pragmatic Programmer by Andrew Hunt and David Thomas:

Straight from the programming trenches, The Pragmatic Programmer: From Journeyman to Master cuts through the increasing specialization and technicalities of modern software development to examine the core process—what do you do, as an individual and as a team, if you want to create software that’s easy to work with and good for your users.

If you haven’t read The Pragmatic Programmer then you’re really missing out. Personally, it was a huge inspiration for me to really step up my game as a developer. In fact, my copy of the book is fairly shabby and looks as if it had been found in a cave near Israel.

Replacing master with another Git branch

Fri, 30 Sep 2011 00:00:00 -0700

If you’re like Dave O’Neill and find yourself working on a Git branch that has diverged from master to the point that a merge would make Linus Torvalds cry, why not just replace master? Dave provides step by step instructions in this article:

Over time, master diverged significantly from the current maintenance branch. Recently, we decided to rebase our development work on the maintenance branch, but there were just too many ugly conflicts to resolve. As we didn’t want to keep all of the code on that branch anyway, the easiest solution was to rename our 2.63-branch to master and resume development there, cherry-picking some of the useful things from the development branch.

Google releases JS Test

Thu, 29 Sep 2011 00:00:00 -0700

While there’s certainly no shortage of testing frameworks for JavaScript, the vast majority of them require you to run your tests inside a web browser. If you’d rather test your JavaScript outside of a browser (and don’t want to use something heavy like RhinoUnit) Google’s newly released JS Test framework might be the ticket.

From the announcement:

Google JS Test is a JavaScript unit testing framework that runs on the V8 JavaScript Engine, the same open source project that is responsible for Google Chrome’s super-fast JS execution speed. Google JS Test is used internally by several Google projects, and we’re pleased to announce that it has been released as an open source project.

Keep in mind that since JS Test runs inside Google’s V8 your JavaScript won’t have access to the DOM. Google recommends you refactor your JavaScript to use as little DOM as possible then mock out the remaining DOM pieces using JS Test’s mocking facilities.

Understanding Ruby Singleton Classes

Mon, 15 Sep 2008 00:00:00 -0700

If you learned object oriented programming from one of the more static languages such as C++ or Java, the dynamic nature of Ruby may seem magical and elusive. After running into the syntax dedicated to meta-programming you may have been left scratching your head or at least wondering what’s happening behind the scenes. Singleton classes, not to be confused with the singleton design pattern, can easily be placed into this head scratching category.

The name itself is confusing leading people to create alternative names such as: object-specific classes, anonymous classes, virtual classes, and eigenclasses. Anonymous class is my favorite, but since the source code to the Ruby interpreter uses the term singleton that’s what I’m going to stick with.

Truth be told, singleton classes aren’t really that difficult to understand. In this excerpt from my Accelerated Ruby workshop, I remove the mystical fog surrounding the singleton class, and even demonstrate how they can be useful in your day-to-day Ruby adventures.

Method Dispatching

Before we dive into the mysterious singleton underground, let’s step back for a quick refresher on dynamic dispatch in Ruby. With each method call, the Ruby interpreter needs to examine the inheritance hierarchy of the receiver in order to find the method that will be executed.

foobar = Array.new
foobar.class # => Array
foobar.size  # => 0

In the last statement above, the size method was called on the foobar object. You can also say that the foobar object was the receiver of the size message. It’s the class of the receiver where the interpreter will start its search for the size method, which in this case is the Array class.

If the Array class doesn’t contain the size method, the search will continue up the hierarchy until it’s found. If the method isn’t found at the top of the hierarchy, the search is aborted, and the receiver is sent the method_missing message. Luckily for us the Array class does indeed have a size method and in the example above, it returned zero.

The diagram below illustrates how an object is connected to its class, and how a class is connected to its parent class, also known as its super class.

Enter the Singleton Class

Instead of working our way through some wordy and complicated description of what a singleton class is, let’s just dive right in and see what it does.

foobar = Array.new

def foobar.size
  "Hello World!"
end

foobar.size  # => "Hello World!"
foobar.class # => Array

bizbat = Array.new
bizbat.size  # => 0

As in the previous example, a new instance of the Array class is created and safely tucked away into a variable named foobar. Then, without warning, we have a funny looking method definition.

Ruby, being the flexible language that it is, allows you to explicitly specify a receiver when defining methods. That is, we can tell Ruby what object will acquire the new method. This syntax shouldn’t look too unfamiliar, since it’s similar to the syntax used when creating class methods. However, in the case above, we’re adding a method to the foobar object.

That’s right, that method definition above creates a method that only exists for a single object, not for a class of objects. As you can see from the example code above, other objects of the Array class do not have this new size method, they’re still using the one defined in the Array class.

That cheeky little foobar object has now been transformed to something slightly different than what it was before. How is this possible? What is it that makes this foobar object so special that it gets to have methods that no other object has? If you said “singleton class” you’d be correct. When you add a method to a specific object Ruby inserts a new anonymous class into the inheritance hierarchy as a container to hold these types of methods.

This new singleton class in the foobar inheritance hierarchy has some special properties. As mentioned before, the singleton class is anonymous, it has no name and is not accessible through a constant like other classes. You are able, however, to access the singleton class using some trickery but you can’t instantiate a new object from it.

Object instantiation is prevented by the interpreter for any class that has a special singleton flag set on it. This internal flag is also used when you call the class method on an object. You would expect that the singleton class is returned from that method call, but in fact the interpreter skips over it and returns Array instead.

Another very interesting side effect of the singleton class is that you can use the super method from within your singleton method. Looking back to the code above, we could have called the super method from inside the singleton method. In that case, we would be calling the size method from the Array class.

Speaking of side effects, if you’re curious whether or not an object has a singleton class, there is an introspective method called singleton_methods that you can use to get a list of names for all of the singleton methods on an object.

Finally, a word of caution. Once you’ve created a singleton class for an object you can no longer use Marshal.dump on that object. The marshal library doesn’t support objects with singleton classes:

>> Marshal.dump(foobar)
TypeError: singleton can't be dumped
        from (irb):6:in `dump'
        from (irb):6

Since some of the distributed programming libraries use the marshal calls to move objects around machines you might run into the error message above. And now you know what it means.

More Ways to Skin a Singleton

It shouldn’t be surprising that Ruby has several ways to create a singleton class for an object. Below you’ll find no less than three additional techniques.

Methods From Modules

When you use the extend method on an object to add methods to it from a module, those methods are placed into a singleton class.

module Foo
  def foo
    "Hello World!"
  end
end

foobar = []
foobar.extend(Foo)
foobar.singleton_methods # => ["foo"]

Opening the Singleton Class Directly

Here comes the funny syntax that everyone has been waiting for. The code below tends to confuse people when they see it for the first time but it’s pretty useful and fairly straightforward.

foobar = []

class << foobar
  def foo
    "Hello World!"
  end
end

foobar.singleton_methods # => ["foo"]

Anytime you see a strange looking class definition where the class keyword is followed by two less than symbols, you can be sure that a singleton class is being opened for the object to the right of those symbols.

In this example, the singleton class for the foobar object is being opened. As you probably already know, Ruby allows you to reopen a class at any point adding methods and doing anything you could have done in the original class definition. As with the rest of the examples in this section a foo method is being added to the foobar singleton class.

Evaluating Code in the Context of an Object

If you’ve made it this far it shouldn’t be shocking to see a singleton class created when you define a method inside an instance_eval call.

foobar = []

foobar.instance_eval <<EOT
  def foo
    "Hello World!"
  end
EOT

foobar.singleton_methods # => ["foo"]

Practical Uses of Singleton Classes

Before you chalk this all up to useless black magic let’s tie everything off with some practical examples. Believe it or not, you probably create singleton classes all the time.

Class Methods

While some object oriented languages have class structures that support both instance methods and class methods (sometimes called static methods), Ruby only supports instance methods. If Ruby only supports instances methods where do all those class methods you’ve been creating end up? Why, the singleton class of course.

This is possible because Ruby classes are actually objects instantiated from the Class class. Their names are constants that point to this object instantiated from the Class class. While this object holds the instance methods for objects instantiated from it, its so-called class methods are kept on a singleton class.

class Foo

  def self.one () 1 end

  class << self
    def two () 2 end
  end

  def three () 3 end

  self.singleton_methods # => ["two", "one"]
  self.class             # => Class
  self                   # => Foo
end

Two of the three methods defined in the code above are class methods, and therefore go into a singleton class. Left as an exercise for the reader is the inheritance hierarchy for the object that the Foo constant references.

Test Mocking

Mocking is a popular testing technique that allows you to stub out method calls for an object or class, forcing them to return a specific value or ensuring that they are called a specific number of times. While there are several good mocking libraries available for Ruby, wouldn’t it be nice to know how they work?

In the example below there is a Foo class with two instance methods. The available? method is dependent on the results of the status method. What do you do if you need to verify that the available? method works correctly given the varying results which the status method could return? Mocking and singleton classes to the rescue.

require 'test/unit'

class Foo

  def available?
    status == 1
  end

  private

  def status
    rand(1)
  end
end

class FooTest < Test::Unit::TestCase
  def setup
    @foo = Foo.new
  end

  def test_available_with_status_1
    def @foo.status () 1 end
    assert(@foo.available?)
  end

  def test_available_with_status_0
    def @foo.status () 0 end
    assert(!@foo.available?)
  end
end

Conclusion

Understanding the advanced aspects of the Ruby programming language need not be difficult. Hopefully singleton classes make a lot more sense to you now and even seem somewhat useful. You may have missed it but hiding in the text above is also a rather shallow examination of the object system that Ruby employs.

I recommend that you continue your journey in that direction. Hint: see if your local library has a book on Smalltalk.

Behind the code: project planning

Thu, 24 Jul 2008 00:00:00 -0700

In this new series of articles, Behind the Code, I’ll be looking at the tools and processes that I’ve put into place to make me a more efficient and effective developer. I’m hoping to share my experiences, and while doing that, refine the way I work using your feedback.

Today, I’ll be talking about project planning, the process of breaking a project into tasks, and estimating the effort to complete those tasks. The artifacts generated during this process can then be used to create a proposal for a client, cost estimations for your boss, or even the start of your project documentation.

File Formats and Tools Matter

If there’s one thing I’ve definitely learned the hard way, it’s that the tools I use today aren’t necessarily the tools I’ll be using tomorrow. A better tool might come along, or I might be forced to use another operating system where my preferred tools aren’t available.

I avoid binary proprietary file formats like used car salesmen. It’s so important to me that it’s my first rule:

Rule 1: Use plain text files when possible.

No matter which platform you’re using, you can read and work with text files. If you happen to use a platform with good command line tools (anything Unix based), you have a powerful system available for finding, searching, and transforming text files.

Text files rock, and what’s the best tool to edit them? Being a developer, using a text editor shouldn’t be unfamiliar to you. As a matter of fact, it should be the tool that you’re most comfortable and efficient with. If there’s only one thing you can call yourself an expert in, it should be in using your text editor.

Tools for Project Planning

There are several tools you can use to do project planning, all while working with nothing more than plain text files. My tool of choice is Emacs, using the amazing Org-Mode package.

The tools that you pick will depend completely on you. In this case, the process is slightly more important than the tools. Don’t get me wrong though, a good tool can make a big difference in your workflow.

I like Org-Mode because it provides a lot of functionally, yet at the end of the day everything is just plain text. I can easily keep my project files in a revision control system, and use tools like Ruby and grep.

Breaking a Project Into Features and Tasks

The first step for planning a project is breaking it into smaller pieces. I like to create a shallow hierarchy, where I break a project into features, and then take those features and break them down into tasks.

Features are usually the bigger chunks that your client or boss would know by name. Since features are big chucks, you can’t estimate them correctly, or think clearly about their dependencies without breaking them down further.

By breaking a feature down into tasks, you virtually walk though the implementation of that feature, thinking about dependencies and potential problems that you’ll have along the way.

While breaking a project into features and tasks, don’t think about time estimates, only process. Work through the steps you would actually perform to implement a feature in your mind, recording them in your project plan as tasks.

Below is screenshot from Emacs, showing the outline I’ve created for a fictional social networking application. Org-Mode allows me to fold the outline at any level. Here I’m viewing just the outline headings that show two features, broken down into tasks.

While I’m creating the feature and task breakdown, I leave myself little notes just in case the headings are too vague and a few weeks later I’m not sure what the hell I was talking about.

Below is a fully expanded version of the outline from above, so you can see the notes. Later, these notes can be expanded and used as project documentation or a client proposal.

In Org-Mode, the tab key is used to expand or fold an outline. This makes it very easy to see just the part of the project plan that you’re interested in.

Estimating the Tasks

As I mentioned before, a feature by itself is too vague to estimate. But now that you’ve broken each of the features down into tasks, you should have enough information to estimate the effort of actually implementing those features.

I’m not going to dive too deeply into the art of estimating in this article. That said, put your estimating cap on and let’s get to it.

The best way to estimate your tasks is to begin with the first task, and work through them one at a time. I prefer to break my tasks down into hours and minutes, usually in 30-minute increments.

If you come across a task that will take more than a workday to complete, you need to break the task down into more tasks. It’s important that your tasks are granular enough, I personally don’t like my tasks to take more than 5-hours. Along the same lines, any task that takes less than 30-minutes, however, should be reviewed to ensure it has been thought through enough.

Below is a screenshot showing estimates assigned to each task. I like to switch Org-Mode into column view. Org-Mode makes it easy to see summations of task estimates using its handy column view mode, which turns your hierarchical outline into a table.

In this screenshot, I’ve assigned estimates to each task. The features now show the total estimates for their respective tasks, leaving the top headline (the project name) with a total estimate for the project. Org-Mode automatically summed the time estimates for me, I only assigned estimates to the tasks themselves (in purple below).

Presenting the Project Plan

With your task estimates completed, you’ve collected enough information to present a proposal to a client, or a project plan to your boss.

If you’ve been using a software tool to create your project plan, it probably has an export feature. Org-Mode can export your document to HTML, LaTeX, and even a cleaned up ASCII text file.

Even if you’re not using a tool with an export feature, you’re a software developer right? Write a script that will transform your text file to something useful for your situation.

As an example of a client proposal, I’ve exported the project file into a cleaner PDF file for my fictional client. If this were a real proposal, I would add additional notes, along with details about the cost of the implementation.

Using the Project Plan During Implementation

Why do all the work of creating a proposal, only to let it collect dust? When it comes time to implement the features that you created a spec for, why not turn them into a to-do list?

Using the Org-Mode ability to manage to-do lists, I turned the estimated tasks into to-do items. With Org-Mode I can “clock in” to a task, which starts a timer. When a task is complete, I mark it as being done, and then I can compare the time I spent working to my estimates.

Below is a screenshot where I’ve completely expanded a finished task so that you can see the meta-data that Org-Mode was adding to it when I used the timer on a task and then marked it as complete. You can also see how a headline can be turned into a to-do item.

It’s much more useful to switch back into column view for the comparisons between my estimates and the clocked time. The below screenshot shows my outline back in column view, this time also showing clock time summaries.

This is a good example of how Org-Mode can collect the meta-data in your text file and then present it to you in a more efficient way. You can even edit the data in this table-like view, and Org-Mode will do the right thing.

In addition to column view, Org-Mode has several ways to generate reports from your data. I especially like the agenda view.

With the agenda view, you can see which tasks you’ve scheduled for specific days. I find it very useful to spend a little bit of time at the end of my day, and schedule which tasks I’ll be working on for the following day.

In the morning, I can review my agenda and start working with a clear idea of what I should be doing. This really helps with any tendency to stall or procrastinate during the first hours of the day.

Article Artifacts and Further Reading

I chose to show screenshots above so you could get a feel for how a good tool like Org-Mode can provide a nice interface for project planning. The nice thing about Org-Mode is that you’re just working with a text file, and if you open that file in any other text editor, it doesn’t look foreign.

Below are links to files I created while writing this article. Open the project plan in your favorite text editor and compare it to the screenshots above. Try opening it in Emacs and see what happens.

The Plain Text Org-Mode File (Project File)
A PDF Client Proposal

If you’re looking for more information about Org-Mode, the main web site is a great resource. There is a fantastic manual, and a very responsive mailing list if you have any questions.