Building an API with Netrc
If you track the progress of the Heroku client, you may have noticed that in the last few months we've switched the way that your credentials are stored over from a custom format in
~/.heroku to an older and more normalized storage standard,
.netrc. This isn't an isolated event either, you may have noticed that GitHub has recently changed the recommended clone method on new repositories to https, which has the side-effect of bypassing your standard access with
~/.ssh/id_rsa. How do you get back to not being prompted for your credentials every time you push to the repository? Netrc.
.netrc is an old standard that dates all the way back to the days of FTP, that romantic wild west era of the Internet where the concept of "passive mode" kind of made sense. Its job is to store a user's credentials for accessing remote machines in a simple and consistent format:
machine brandur.org login [email protected] password my-very-secure-personal-password machine mutelight.org login [email protected] password my-even-secure-password-with-a-number-on-the-end-7
Although originally intended for FTP, its use has since expanded to a other network clients including Git, Curl, and of course Heroku.
A common pattern that I've run into while building API's over the last few months is to protect APIs with HTTP basic authentication. This isn't necessarily the best solution in the long term, passing tokens provisioned with OAuth2 may be better, but it's a mechanism that can be set up quickly and easily.
Take this Sinatra app as an example:
# run with: # gem install sinatra # ruby -rubygems api.rb require "sinatra" set :port, 5000 helpers do def auth @auth ||= Rack::Auth::Basic::Request.new(request.env) end def auth_credentials auth.provided? && auth.basic? ? auth.credentials : nil end def authorized? auth_credentials == [ "", "my-secret-api-key" ] end def authorized! halt 401, "Forbidden" unless authorized? end end put "/private" do authorized! 200 end
After running it, we can test our new API with Curl:
curl -i -u ":my-secret-api-key" -X PUT http://localhost:5000/private HTTP/1.1 200 OK X-Frame-Options: sameorigin X-XSS-Protection: 1; mode=block Content-Type: text/html;charset=utf-8 Content-Length: 0 Connection: keep-alive Server: thin 1.3.1 codename Triple Espresso
Now here's the interesting part. Add the following lines to your
machine localhost password my-secret-api-key
Try the same Curl command again but using the
curl -i -n -X PUT http://localhost:5000/private
Voilà! The speed of being able to run ad-hoc queries against an API you're building rather than drudging up your API key every time turns out to be a huge win practically, and it's a pattern that I now use regularly during development.
A limitation that's hinted at above is that you can only have a single entry for
localhost. Generally, I find that this isn't a huge problem because most of the APIs I want to hit are deployed in a staging or production environment with a named URL.
Now onto a nice real-world example. Are you a Heroku user? Have you updated your Gem since February 2012? If the answer to both these questions is yes!, try this from a console:
curl -n https://api.heroku.com/apps
A glaring problem with
.netrc is that it keeps a large number of your extremely confidential credentials out in the open in plain text. Presumably, the file is
600 and you're using full-disk encryption, but that's still probably not enough (say someone happens to find your computer unlocked).
The netrc gem used by the Heroku client will try to find a GnuPG encrypted file at
~/.netrc.gpg before falling back to the plain text version. Although this convention is far from a standard, it's still recommended security practice.