Writing a Ruby on Rails application to fetch data from Basecamp

Introduction

This is the first in what I hope will be a series of posts following my attempts to build a little Ruby on Rails web app that will grab data from Basecamp using the Basecamp API. The aim is to develop an app that will produce a "what I have done this week" report using the data from the Basecamp To-Do lists.

The app will be based on Rails 2.3.5 and Ruby 1.8.7. I realise that these aren't the latest versions, but they are the standard versions available on the two platforms I use (Mac OS 10.6.5 and Ubuntu 10.10). Later on I might look at upgrading to Rails 3.

Update: I have now upgraded to Rails 3 and have updated this post with changes the required for Rails 3.

You don't need to know Rails to read this, but I am assuming you have some knowledge of Ruby. If you don't know Ruby there are some tutorials at http://www.ruby-lang.org.

A word of warning for Mac users, we will be using the Terminal app. It is possible to develop Rails applications using XCode, and Apple have a good tutorial on this at http://developer.apple.com/Tools/developonrailsleopard.html, but you'll still have to do the first step (creating the basic file and directory structure) from Terminal.

I also haven't said anything about what text editor to use. I'll leave that up to you. On Mac you can use XCode, I also like Scite, and a lot of Ruby/Rails people recommend TextMate.

Getting Started

OK, so let's get started. The first thing we need to do is install Ruby and Rails. If you're using Mac OS X 10.6 (i.e. Snow Leopard), then it's easy. There's nothing to do as the versions I'm using are already installed. On Ubuntu 10.10 all you have to do is run the following commands:

sudo apt-get install ruby
sudo apt-get install rails
sudo gem install -v 2.3.5 rails
sudo apt-get install sqlite3
sudo apt-get install libsqlite3-dev
sudo gem install sqlite3

Rails 3 Update: To update OS X to Rails 3 run the following commands:

sudo gem update --system
sudo gem install -v '>3.0.0' rails

Ubuntu is a little more tricky, as the rubygems version supplied with 10.10 is too old to install Rails 3, and it has been modified so you can't do a gem update --system. You will need to install the rubygems version from rubygems.org and then run:

sudo gem install -v '>3.0.0' rails
sudo apt-get install sqlite3
sudo apt-get install libsqlite3-dev

Note that we aren't installing the sqlite3 gem at this stage. We'll get to that later.

If you're using Windows, sorry, I don't have any instructions on how to install Rails on Windows as I've never done it. I'm sure there's plenty of information out there on the web if you search for it.

The next step is to create the basic file and directory structure for our application. Just cd to the directory you will be doing your development in and run the following command:

rails basecampreport

Rails 3 Update: This command has changed in Rails 3 (I've also change the app name to bcr - less typing):

rails new bcr

This will create a subdirectory called basecampreport (feel free to use a shorter/snazzier name instead of "basecampreport"). Have a look in there and you'll see a whole lot of subdirectories. Some of these are:

  • app: This will contain much of the application code. We'll be spending a lot of time here.
  • config: Overall configuration information for the application.
  • db: Database configuration, although we won't be using a database for this app.
  • lib: Libraries of code which will be shared by other parts of the application. We'll be putting the Basecamp API Ruby wrapper in here.
  • public: General webby stuff like stylesheets and images.
  • script: Some scripts for running and doing other things with your app.

Try running the following command:

script/server

Rails 3 Update: This command has changed to:

rails server

Now open your web browser and go to http://localhost:3000. If everything is working you'll see a Rails Welcome Aboard website. You can now feel that you've accomplished something and be very proud of yourself.

Installing the Basecamp wrapper

The next step is to install the Basecamp wrapper. This is a bit of Ruby code that provides an interface to the Basecamp API. You will need to download the basecamp.rb file from https://github.com/anibalcucco/basecamp-wrapper and put it in the lib directory.

There's also a Ruby gem called xml-simple that the library needs but you may not have installed. You can just download and install this by running

sudo gem install xml-simple

Rails 3 Update: The Basecamp wrapper is now available as a gem. In Rails 3 you configure the gems required in a file called Gemfile in the top level application directory. Add the following lines to Gemfile:

gem 'basecamp' gem 'xml-simple'

Now run the command bundle install to install the required gems. As the default Gemfile contains sqlite3, the sqlite3 gem will also get installed at this point.

Let's Start Coding

OK, time to finally start writing some code. To start off with we're just going to have a single web page that lists the names of all your projects in Basecamp.

You've probably heard a lot about Rails being a MVC, or Model-View-Controller framework. In fact if you look in your app directory you'll see subdirectories called controllers, models and views. So what does that actually mean?

At a very basic level, the model manages the application's data, e.g. data in a database, the controller contains the application logic, and the view is responsible for interacting with the user, e.g. by displaying a web page.

We're only going to have controllers and views, as the data is coming from the Basecamp API, not a database (although I guess the interaction with the Basecamp API could be considered a model).

Projects Controller

Create a new file under the app/controllers directory called projects_controller.rb and copy the following to it:

class ProjectsController < ApplicationController

def index
 Basecamp.establish_connection!('myhost.basecamphq.com', 'api_token', 'X', true)
 @projects = Basecamp::Project.find(:all)
 end

end

There are a few things in that establish_connection call that you'll have to change to match your Basecamp setup.

First, myhost.basecamphq.com should be changed to the website you use to access Basecamp.

Next, you'll need to put your own API token in where it says api_token (you can find this under "Authentication tokens" on the "My Info" page in Basecamp).

And finally if you use http rather than https to access Basecamp you'll need to change that last true to false.

So what is this code doing?

The establish_connection is setting up the connection to the Basecamp API.

The next line is fetching all your projects from Basecamp and storing them as an array in a variable called @projects. If you're familiar with Ruby you'll know that the @ on @projects means that it's an instance variable, i.e. a variable which is accessible to all methods in the object. In Rails, if we set an instance variable in the controller, it will be available to be used in the view.

Projects View

To create the projects view, create a folder called projects inder the app/views directory. We are going to create a couple of files under here.

The first file is index.html.erb and will contain the following:

<table>
 <tr>
 <th>Project Name</th>
 </tr>

 <%= render :partial => 'project', :collection => @projects %>
</table>

The other file is called _project.html.erb. Don't forget the leading underscore, that's not a typo and it's important. This file contains:

<tr>
 <td><%=h project.name %></td>
</tr>

Templates

The .html.erb files are called templates. These will be processed by Rails to generate .html files which are then sent back to your web browser.

You can see that these templates contain some standard HTML tags like <table> and <td>, and then some funny looking things like <%= ... %> that have angle brackets like HTML, but aren't HTML.

I'll assume that you already know about HTML (if you don't, http://www.w3schools.com is a good place to learn) and am just going to talk about those <%= ... %> tags.

Basically what happens is that everything between the <%= and the %> is executed as Ruby code and the output is put back in the resulting .html file where the <%= ... %> was. So <%=h project.name %> is just going to put the value of project.name (i.e. the name property of the project object) in the web page.

That h after the equals sign is shorthand for html_escape. Without this, any HTML in project.name would actually get processed by your web browser, which might mess up your nice web page. In fact if someone can get a <script> tag with some Javascript into project.name, they might be able to get your browser to run their Javascript and do nasty things.

You should get into the habit of putting that h in front of any data which comes from outside your application or which you can't 100% trust. In fact Rails 3 does the html_escape by default, you have to say if you don't want it. But for now we're using Rails 2, so keep putting those h's in.

Rails 3 Update: You no longer need the h in Rails3, it's done by default.

Partials

Our web page is going to display all your projects in a table. If you look at index.html.erb you can see that it has the basic outline of the table, i.e. the <table> tags and the heading, but not the actual table rows. Instead it has this:

<%= render :partial => 'project', :collection => @projects %>

This is telling Rails to take the output of another template (called a partial template, or partial for short) and put it in this template. The partial in this case is called project, which rails will expect to be in a file called _project.html.erb (told you that underscore was important, that's what tells Rails that this file is a partial).

The :collection => @projects says to render the partial using the @projects collection. Remember from our controller that @projects is an array containing all our projects. The :collection argument means that the partial will actually be called once for each member of the array.

If you look at the partial you can see that it just has code for displaying a single row in the projects table, but as it will be called once for each project, the result is a table with a row for each project.

Each time the partial is called, the local variable project will be set to the array element that we want to display. The name of that local variable comes from the name of the partial, i.e. project, not from the name of the array @projects.

Time to Run It

The time has now come to run our application. In your browser go to http://localhost:3000/projects. You should get back a table with a list of all your projects from Basecamp.

So what just happened then. To explain it I'm first going to go over a couple of concepts in Rails, the first being controllers and actions, and the second called "convention over configuration".

Rails 3 Update: In Rails 3 you'll also need to add the following line to config/routes.rb:

resources :projects

Controllers and Actions

We've already seen controllers. In our example we have a projects controller. Within a controller we have one or more actions. In our projects controller we have an action called index.

The code for the index action within the projects controller is contained in the index method in the ProjectsController class. You can see this if you go back and look at the code in projects_controller.rb.

Convention over Configuration

Convention over configuration means that Rails has certain default ways of doing things and, unless we tell it otherwise, it will do these things without us having to explicitly code them.

An example of this is that after calling a controller action, the default is to display the matching view. We can see this in our projects controller, there is no code in there to call the view, Rails just does it for us by default. If we wanted to we could put some code in the controller to tell it to do something other than display that view.

So What Did Just Happen?

We told our browser to go get http://localhost:3000/projects. From this, Rails knows to run the index action in the projects controller.

The projects part obviously comes from the URL, but what about the index part. This is our first bit of convention over configuration. When the URL is http://hostname/XXXX, the default is for Rails to call the index action in the XXXX controller.

So now we've run our index action, which has fetched all our projects and put them in the @projects variable. The next bit of convention over configuration is the one we have already covered, i.e. calling the matching view after running the controller action. In this case the matching view is index.html.erb in the app/views directory.

What Next?

You might be already thinking that hard coding our API token into our code isn't a good idea. First of all it's a security issue, as anyone who has access to the code can find out our API token, and secondly we'd have to modify our code each time a different user needed to use it.

In my next blog post I'll show how to implement a login form to prompt for the user's Basecamp username and password, use these to fetch the API token, and then use that fetched token to call the Basecamp API.