Building an API with Netrc

If you track the progress of the Heroku client, you may have noticed that in the last few months we've switched the way that your credentials are stored over from a custom format in ~/.heroku to an older and more normalized storage standard, .netrc. This isn't an isolated event either, you may have noticed that GitHub has recently changed the recommended clone method on new repositories to https, which has the side-effect of bypassing your standard access with ~/.ssh/id_rsa. How do you get back to not being prompted for your credentials every time you push to the repository? Netrc.

.netrc is an old standard that dates all the way back to the days of FTP, that romantic wild west era of the Internet where the concept of "passive mode" kind of made sense. Its job is to store a user's credentials for accessing remote machines in a simple and consistent format:

machine brandur.org
  login [email protected]
  password my-very-secure-personal-password
machine mutelight.org
  login [email protected]
  password my-even-secure-password-with-a-number-on-the-end-7

Although originally intended for FTP, its use has since expanded to a other network clients including Git, Curl, and of course Heroku.

A common pattern that I've run into while building API's over the last few months is to protect APIs with HTTP basic authentication. This isn't necessarily the best solution in the long term, passing tokens provisioned with OAuth2 may be better, but it's a mechanism that can be set up quickly and easily.

Take this Sinatra app as an example:

# run with:
#   gem install sinatra
#   ruby -rubygems api.rb

require "sinatra"

set :port, 5000

helpers do
  def auth
    @auth ||= Rack::Auth::Basic::Request.new(request.env)
  end

  def auth_credentials
    auth.provided? && auth.basic? ? auth.credentials : nil
  end

  def authorized?
    auth_credentials == [ "", "my-secret-api-key" ]
  end

  def authorized!
    halt 401, "Forbidden" unless authorized?
  end
end

put "/private" do
  authorized!
  200
end

After running it, we can test our new API with Curl:

curl -i -u ":my-secret-api-key" -X PUT http://localhost:5000/private

HTTP/1.1 200 OK
X-Frame-Options: sameorigin
X-XSS-Protection: 1; mode=block
Content-Type: text/html;charset=utf-8
Content-Length: 0
Connection: keep-alive
Server: thin 1.3.1 codename Triple Espresso

Now here's the interesting part. Add the following lines to your .netrc:

machine localhost
  password my-secret-api-key

Try the same Curl command again but using the -n (for --netrc) flag:

curl -i -n -X PUT http://localhost:5000/private

VoilĂ ! The speed of being able to run ad-hoc queries against an API you're building rather than drudging up your API key every time turns out to be a huge win practically, and it's a pattern that I now use regularly during development.

A limitation that's hinted at above is that you can only have a single entry for localhost. Generally, I find that this isn't a huge problem because most of the APIs I want to hit are deployed in a staging or production environment with a named URL.

Heroku

Now onto a nice real-world example. Are you a Heroku user? Have you updated your Gem since February 2012? If the answer to both these questions is yes!, try this from a console:

curl -n https://api.heroku.com/apps

Security

A glaring problem with .netrc is that it keeps a large number of your extremely confidential credentials out in the open in plain text. Presumably, the file is chmod'ed to 600 and you're using full-disk encryption, but that's still probably not enough (say someone happens to find your computer unlocked).

The netrc gem used by the Heroku client will try to find a GnuPG encrypted file at ~/.netrc.gpg before falling back to the plain text version. Although this convention is far from a standard, it's still recommended security practice.

Posted on June 20, 2012 from San Francisco

Newest Articles