Send your emails using Postmark

Written by Corey on April 12, 2010 | Comments Off

Wildbit recently launched a great new email delivery service named Postmark.  Once I read what it was about I knew that we had to integrate in to beyond the whiteboard.

First a little background.

Sending emails from your web application is one of those things that all the programming guides show you how to set up and get going in a few easy steps.  However, they never point out the problems that you might come across while trying to send out emails.

When launched the beta version of beyond the whiteboard we setup Gmail to be our smarthost for sending emails.  That was pretty easy to get going and worked out pretty well, there where copies of the sent emails in that Gmail account and emails where sent out.  However, there where a couple of things which I didn’t really like.  First off Gmail, replaces the from field with the user you are sending email through.  For example, if your Gmail user is ‘mycoolapp’, that’s what the from will be, even if you in your application set the from to be something else like ’support’.  A minor little issue, but we wanted some emails to be from the application and others be from individuals which just wasn’t possible.  Lastly, Gmail limits the number of emails that you can send through it in a given period.  This was the big killer for us, we hit limit and didn’t know it right away.  That cause us some headaches because there really wasn’t a good way to know if we where close to the limit or what.

After the few Gmail incidents I finally broke down and decided to set up an SMTP server to send out our application emails.  I really didn’t want to because, well email is kind of scary to me.  The server configurations are very complex and they have warnings everywhere that if you make a mistake every spammer in the world will use your server.  Then if that happens every ISP will hate you and little kids will point and laugh when you go to the mall.  Despite all that I did manage to get a configuration going and we didn’t let any evil spammers or anything bad like that.  Everything appeared to be going well, or so I thought.

You see, I setup our SMTP to only send out emails, the receiving part was still handled by Gmail.  I hadn’t keeping tabs on the Gmail inbox of the address we where sending from in some time.  When I did happen to remember to look in that account, several months later, I discovered that EarthLink and AT&T had decided that we where spammers and blocked all emails from our server.  What a drag that was, I had no clue how long that had been going on, and it wasn’t a very good feeling to be labeled a spammer, even if it was from an automated system.  They have methods to get you removed from list and I filled out their forms, but it happened at least one other time.  That’s when I decided that there had to be something better out there and that’s when I found Postmark.

Why Postmark is good.

Right when I read the description of the Postmark service I signed up, it was an easy decision for me.  Here’s how it works, you post your email to Postmark through their HTTP API and then they send your email to it’s destination.  See super simple, but wait there’s more, they have a nice overview with a graph of how many emails you’ve sent, how many bounced, and how many where marked as spam.  You can then dig deeper and see which emails bounced, why they bounce, and there a way to reactivate those emails if you think there was a mistake.

The integration was also pretty easy, they have published a Ruby gem that hooks right in to ActionMailer.  After a couple lines of configuration you can just use ActionMailer just like before and instead of sending emails via SMTP, they are sent to Postmark for delivery.  I also really liked the way the set it up by changing ActionMailer’s delivery method.  Why is that good?  Well, different ActionMailer subclasses can have different delivery methods, which is completely different from the SMTP settings which are shared across all instances.  That makes it possible to have emails sent to users got through Postmark, while system emails, like error reports can go through SMTP.  Which is exactly how we have it set up today.

I’m really happy with the service, now I know exactly how many emails to users we are sending out each day and how many are bouncing back.  Even though Postmark just launched a few weeks ago I know that the folks at Wildbit are going to continue to make it better because they use it with their other web applications. Personally I’m really looking forward to when they get some sort of webhooks going for bounced emails.  I would like to inform our users that we couldn’t send them an email at the address they have given us.

All of that totally justifies the cost of the Postmark service, which is currently $1.50 for 1,000 emails.  If you are building a web application and are sending out transactional emails to your users, I would really encourage you to investigate a service like Postmark.  Why worry about emails when you should be worrying about your application’s main features.

Using Yaml to stream lots of data

Written by Ben on September 24, 2009 | Comments Off

Most people just use Yaml for configuration. The prime example is the database.yml file found in almost every single Rails app. But here’s a way to take advantage of Yaml in Ruby as a way to serialize and transfer lots of separate records from one place to another.

The standard yaml library found in Ruby is all you need. First, to serialize the target set of records. For our example, we’re just going to stream to a file, but one can use any IO stream. And instead of streaming an indeterminate number of records, we’re going to bound this process with a fixed number of records.

require ‘yaml’
file = File.open(‘test.yml’, ‘w’)
for x in 0..100
hash = {
‘id’ => x,
‘field1′ => “somedata”,
‘field2′ => “someother data”,
‘option’ => true
}
file.write hash.to_yaml
end
file.close

Notice that all we needed to do is call the #to_yaml method to do the serialization. It also handles adds the Yaml document start character sequence automatically (i.e. “—”, three dashes alone on a new line).

To read that stream of data:

File.open(‘test.yml’, ‘r’) do |io|
YAML.each_document(io) do |record|
# Do something here
puts record.inspect
end
end

Question: What’s the advantage here?

Answer: Constant memory footprint. We only read in as much as we need to handle the next Yaml document (object, hash). So if you’re processing a 2 Gig file, the process won’t try to load the entire file into memory at once. Or you may not actually know how many records will come over the wire.

Some Uses:

• Mass dumping data from one system to another where the source system lacks write permissions, or even network access, to the target system.
• Data Archival in a common format. Yaml is accessible by tools without the need for a SQL database… if one archived everything in SQL dumps. And this lets you keep rows clustered in manageable sized files.
• Stream real-time updates from one system to another. Either HTTP post a batch document stream to a Webhook, or just read and write on raw TCP sockets.

Installing Mysql with MacPorts for Rails on Leopard

Written by Moe on September 19, 2008 | 5 Comments

I just spent a hour going through this so you won’t have to. I thought I installed mysql using mac ports but I kept getting this error.

Errno::ENOENT (No such file or directory - /tmp/mysql.sock):

Below are the three steps you need to get Mysql running on Leopard for MacPorts.

sudo port install mysql5 +server
sudo launchctl load -w /Library/LaunchDaemons/org.macports.mysql5.plist
sudo ln -s /opt/local/var/run/mysql5/mysqld.sock /tmp/mysql.sock

Authentication and Authorization in Rails

Written by bill on September 16, 2008 | 4 Comments

Some say Rails is “missing” a lot of things you might expect to find in full-featured web development framework, but it doesn’t matter – what’s it’s NOT missing is a plugin system which allows you to add any functionality you need by pulling a few bits of code from other authors into your site. What I’ll be using in this example are the restful-authentication plugin for authentication and the role_requirement plugin for authorization. Both of these are hosted on github, which hosts loads of Rails plugins along with other open projects. As the name implies, they use git for their repositories, so you should install git to grab these plugins.

Setting up authentication

First, you’ll need to set up authentication. In the vendor/plugins folder of your project, run:

git clone git://github.com/technoweenie/restful-authentication.git restful_authentication

This will grab a copy of the restful_authentication plugin; you don’t need to mess with any of the code in the plugin itself. go back to your project’s root and run:

script/generate authenticated user sessions
rake db:migrate

This will set up the user model for you and insert the users table in your database. You can add arguments to the generate script such as –include-activation –aasm to enable activation emails but we’re not going to cover all of that right now.

Now, you’ll have two new controllers in your application, sessions_controller.rb and users_controller.rb. Go to each of these files and remove or comment out the line that says ‘include AuthenticatedSystem’, and copy this line to the top of the application controller instead, right at the beginning of the class definition:

class ApplicationController < ActionController::Base
  include AuthenticatedSystem

and so on. The generate script should have also added these lines to routes.rb:

map.logout '/logout', :controller => 'sessions', :action => 'destroy'
  map.login '/login', :controller => 'sessions', :action => 'new'
  map.register '/register', :controller => 'users', :action => 'create'
  map.signup '/signup', :controller => 'users', :action => 'new'
  map.resources :users

  map.resource :session

These give you some prettier URLs rather than, for example, /users/new to sign up and /sessions/new to login. Generally you want the first thing a user sees to be the login page, so if you want to you can make that the default by adding

map.root :controller => "sessions", :action => "new"

to your routes. You’ll also need to remove or rename index.html in the public folder.

At this point you have a basic authentication system, which is great considering how easy it is to set up, but alone it’s pretty useless. You’ll notice, no matter if you’re logged in or not, you still have full access to your app. So why did we even bother? Because now, you can add authorization to lock down actions based on roles you set up.

Setting up authorization

There are a few different ways to do this; if you want a very, very granular authorization system you can install padlock authorization which allows you to set roles per each object in your application. We decided this was probably overkill for our latest project, but I may touch on it in a later blog if we decide to use it after all. We’ll be using the aforementioned role_requirement plugin.

Back to github with us! Head back to your /vendor/plugins folder and run:

git clone git://github.com/timcharper/role_requirement.git role_requirement

Go back up to your application’s root, and run:

script/generate roles Role User
rake db:migrate

Now, you’ll have to make some manual database changes. You need to add one or more roles to the roles table, and if you have any users, assign them initial roles, if you want them to have roles, in the roles_users table. You can, of course, just add a new controller and view to make all this changeable from your application, but you’ll probably still be setting up one admin user by hand to start things off when you go live.

Now you can go about editing your controllers to make each one accept and reject your roles. For example, say I have a simple model with with a TV show, which can have one starting date. To allow only administrators to make changes, you can set up your Shows controllers like so:

class ShowsController < ApplicationController
  require_role "ADMIN", :for_all_except => [:index, :show]
  def index
    @shows = Show.find(:all)

(The rest is standard Rails boilerplate)

A later post will probably deal with setting/changing roles within your application.

Rails Observer Field

Written by Moe on September 11, 2008 | 3 Comments

The Rails observer field is an easy way to add an ajax select menu to your site. Below is an example to get you going.

Problem

Let’s say you have a list of users on your site and want to filter them by a group. In the example we will be using gym members.

View

<%= select "gym", :gym_id, Gym.find(:all).collect { |p| [p.name, p.id]}, {:selected => params[:gym]}  %>
<%= observe_field "gym_gym_id", :url => { :action => :members_by_gym , 
                                          :controller => :members},
                                          :update => "member_list", 
                                          :with => "'gym='+ escape($('gym_gym_id').value)" %>
<div id="member_list">
   <% for member in @members %>
      <%= link_to member.name, member %><br/>
    <% end %>
</div>

The first thing we do is build the select form by using the Rails Select Helper. The id of the select tag is generated by rails, “gym_gym_id”. We need to pass the id name as the first parameter to the observe field. The url parameter is pretty straightforward. The update parameter of the observer field designates which element will be updated. The “:with” parameter adds a parameter to the form named “gym”.

Post

Parameters: {"contoller"=>"members", "action"=>"members_by_gym", "gym"=>"1", "controller"=>"members"}

The “gym” parameter posted here is from the observer_field tag.

Controller

def members_by_gym
     @members = Member.find_all_by_gym_id(params[:gym])
     render :layout => false
  end

It’s important that you add the render :layout => false or bad things will happen. We still have one more thing to do. We have our controller being hit via ajax when the select menu changes but we need to make the html that will be updated on the page. In the view/members folder we need to make a new page called members_by_gym.html.erb

members_by_gym.html.erb

<% for member in @members %>
     <%= link_to member.name, member %><br/>
<% end %>

This code will be interpreted and update the “member_list” element on the page. That’s it. Rails Ajax magic.

Google Charts in Rails, gchartrb

Written by Brian on September 8, 2008 | 3 Comments

This week I was playing around with Google Charts, a wonderful API for creating charts via a URL. While I developed a helper class to create charts more easily in Rails, I did a little research and realized the RubyGem gchartrb has a great API for creating these charts. Here’s a quick tutorial on using gchartrb from the ground up.

Installing gchartrb
To install gchartrb, simply use the command gem install gchartrb from your command prompt. Check out these instructions for additional help and to access the gchartrb packages (Downloadable here).

Quick Examples
Now that we have our gem installed, we can create graphs right away. First, remember to require the gem at the top of your code. Notice how the gchartrb gem is actually called “google_chart”:

1 require 'rubygems'
2 require 'google_chart'

There are a couple ways that you can create a chart. You can instantiate the graph and then add attributes to it:

1 lc = GoogleChart::LineChart.new("400x200", "My Results", false)

Alternatively, you can create the graph and add the attributes within a block:

1 GoogleChart::LineChart.new("400x200", "My Results", false) do |lc|
2 # Put lc data here
3 # ...
4 puts lc.to_url
5 end

I personally prefer the latter method, as the creation of the graph and its attributes are kept together in a modular fashion.

Here are some examples of line, bar, and pie charts:

Line Chart

1 lc = GoogleChart::LineChart.new("400x200", "My Results", false)
2 lc.data "Line green", [3,5,1,9,0,2], '00ff00'
3 lc.data "Line red", [2,4,0,6,9,3], 'ff0000'
4 lc.axis :y, :range => [0,10], :font_size => 10, :alignment => :center
5 lc.show_legend = false
6 lc.shape_marker :circle, :color => '0000ff', :data_set_index => 0, :data_point_index => -1, :pixel_size => 10
7 puts lc.to_url

Notice that each lc.data call will create a new data line for you. The second parameter accepts a integer or float array, which can be anything you like. Lets say you have a WeighIn model that has weight as an attribute and belongs to some Person. If a person weighs himself/herself once a month for a year, then we can collect that data and show your trend via a line graph. Collecting that data and using it would look like this:

# In the controller Person
def show
@person = Person.find(params[:id])
weigh_ins = @person.weigh_ins.collect(&:weight)

lc = GoogleChart::LineChart.new("#{width}x200", "My Results", false)
lc.data "Weigh ins", weigh_ins, green
# etc, etc...
puts lc.to_url # Or @line_graph = lc.to_url
end

Another useful function, lc.axis, can be used to specify your own data range labels, but it is not required to create a chart (default ranges and labels will be used). The last function, lc.to_url, takes the information specified for the line chart and puts it into the Google charts url.

Bar Chart

1 bar_1_data = [350,110,700]
2 bar_2_data = [200,800,50]
3 color_1 = 'c53711'
4 color_2 = '0000ff'
5 names_array = ["Bob","Stella","Foo"]
6 GoogleChart::BarChart.new("600x300", "Horizontal Bar Graph", :horizontal, false) do |bc|
7 bc.data "FirstResultBar", bar_1_data, color_1
8 bc.data "SecondResultBar", bar_2_data, color_2
9 bc.axis :y, :labels => names_array, :font_size => 10
10 bc.axis :x, :range => [0,1000]
11 bc.show_legend = false
12 bc.stacked = true
13 bc.data_encoding = :extended
14 bc.shape_marker :circle, :color => '00ff00', :data_set_index => 0, :data_point_index => -1, :pixel_size => 10
15 puts bc.to_url
16 end

An interesting attribute to pay attention to is bc.stacked, which gives you the ability to stack bar data, or to create separate bars together. When bc.stacked = false, the chart above is what is shown.

If bc.stacked = true, the same data will look like this:

Also, if you wanted to use this code in one of your controllers, and then display the graph in a template (view) file, you would set the graph’s url in the controller: @graph = bc.to_url Then within the view, you can simple show the graph as an image like this:

<!-- From within the view file --> 
<img src="<%= @graph %>"/>

Pie Chart

1 GoogleChart::PieChart.new('320x200', "Pie Chart",false) do |pc| 
2 pc.data "Apples", 40 
3 pc.data "Banana", 20 
4 pc.data "Peach", 30 
5 pc.data "Orange", 60 
6 puts "\nPie Chart" 
7 puts pc.to_url 
8 # Pie Chart with no labels 
9 pc.show_labels = false 
10 puts "\nPie Chart (with no labels)" 
11 puts pc.to_url 
12 end

This last example was taken from gchartrb’s readme file, which has an example for each kind of chart you can create (including Venn diagrams, Scatter charts, XY Line charts, etc). Notice that the data does not need to be within 0 to 100 (like percents on the pie chart) – the data will be added together and scaled appropriately by gchartrb. If you want to make the chart 3d, either set the 3rd parameter on PieChart.new to true, or you can set the attribute manually: pc.is_3d = true

Useful parameters

Google charts, by default, works with the data range between 0 and 100. Most of the time, your values will not work with this same scale. Instead of converting your data to the default scale, you can specify the data range with the :range parameter in bc.axis. So if you data was between 1 and 12 on the x_axis (like months in a year), then you could specify that range like so:
lc.axis :x, :range => [1,12], :labels => month_names_array

If you’re working with large data sets/numbers, then you may have chart data that is slightly off the mark. The default encoding that gchartrb uses is called Simple Encoding, which only has 62 points of resolution. We can easily expand this resolution (to 4096 points of resolution) by using Extended encoding, which a chart can select by changing the data_encoding attribute with :extended. For a line graph lc, it would look like this: lc.data_encoding = :extended. See Chart data for an explanation for all the encodings Google charts uses.

gchartrb subtlities

While gchartrb provides a great API for create charts in Rails, there are some subtle details about gcharts that you should be aware of. First of all, all charts derive from the base class GoogleChart::Base – check it out to see all the methods and attributes that apply to all charts. Secondly, gchartrb encodes all of the data values in some_chart.data from numeric form into a string representation of the data. This conversion doesn’t effect the final output of your chart, and helps make the chart url shorter in length. Again, if your working with large values, make sure the use the extended coding by using lc.data_encoding :extended.

Other Resources
Now that you have a taste of Google Charts with gchartrb, check out the API documentation here. If gchartrb doesn’t suit your tastes, you can go directly to the Google Charts API – it is thorough and self-explanatory. Ajaxian shows a number of quick examples of Google Charts here, and Matthew Bass gives a solid introduction to Google Charts and a few examples with gchartrb here. Lastly, here is a great link for 50 Cool Things You Can Do with Google Charts.

Rails Select Helper

Written by Moe on September 2, 2008 | 4 Comments

One of the major benefits of Ruby on Rails is rapid development. Something that we love at BadPopcorn. Rails has great helper methods to minimize code and bugs. One of my favorites is the select helper. There are several different ways to make a select tag, but I am going to demonstrate the one I like best. Let’s say you are creating a new “gym member”. He will have a name and belong to a gym.

View

1 <%= form_for(@member) do |f| %>
2   <%= f.text_field :name %>
3   <%= select :member,:gym_id,Gym.find(:all).collect{|p| [p.name, p.id]}%>
4   <%= f.submit "Create" %>
5 <% end %>

Line 1 builds the form. Rails uses the @member variable to generate the action,id and name of the form.

Notice the “do | f |” code in the form deceleration (line 1). You can now use the local variable f to bind attributes of member to input variables. Line 2 of the code generates the below html. The name of the input variable is a Rails convention.

<input id="member_name" name="member[name]" size="30" type="text" />

Line 3 is the Rails select tag helper. The select tag generates the html select tag cleanly. The “name” of the select is dynamically created by using the first two parameters of the helper function “:member” and “:name”. The options are generated by the third parameter.

<select id="member_gym_id" name="member[gym_id]">
   <option value="1">Get Ripped Here</option>
   <option value="2">Hot Girls Welcome</option>
</select> 

Line 4 creates the html submit button.

Also, Rails will throw an exception if you attempt to bind to a member attribute that does not exist. For example, <%= f.text_field :height %> will throw the exception: undefined method `height' for #. Changing the “gym_id” parameter in the select helper will cause the same type of exception. This will save you from making dumb mistakes time and time again.

So what’s posted?
This is where the rails form helper tags come in useful. An organized list of Member was passed as a post parameter to the controller.

Parameters: {"commit"=>"Create", "member"=>{"name"=>"moe", "gym_id"=>"1"},
"authenticity_token"=>"52646c5a18d11c6009b5597af3bf798942a59c0a",
"action"=>"create", "controller"=>"members"}

Controller
When the form is posted it will hit the “create” function in the members controller by default. You can change where it will post to if necessary.

1  def create
2    @member = Member.new(params[:member])
3    respond_to do |format|
4      if @member.save
5        flash[:notice] = 'Member was successfully created.'
6        format.html { redirect_to(@member) }
7      end
8    end
9  end

Line 2 makes a new instance of Member and passes the post parameter member list in the constructor. This returns a member object with the name and gym_id set properly. Line 4 saves the new member in the database.

Member Create (0.000386)   INSERT INTO members ("name", "updated_at",
"gym_id", "created_at") VALUES('moe', '2008-09-02 02:00:35', 1,
'2008-09-02 02:00:35')

The same type exception(s), discussed earlier, will occur here. For example, an exception will be thrown if there is a “height” attribute in the members post parameter.

How we saved time (the short list)

1. Initially we didn’t have to pass much to the view. Just an instance of Member.
2. We were able to use the Rails html helper to build the correct form quickly.
3. Rails conventions made it easy to know where the post would be directed to within the controller.
4. In two lines we created a new Member instance, updated the values with the posted parameters and updated the database.
5. It’s also worth noting that the Rails error checking, when using the helper tags, saves us a lot of debugging time.

What’s Next?
Input validation. Making sure the user isn’t inputting dumb data. For example, in the above code the user could use undesirable characters in their name. Yup, that’s super simple too.