You can now use Embedit with your Facebook apps! Embedit now provides FBML code for pretty much all it’s services (I’m working on the last couple)

You can find the entire list here.

The new Ruby wrapper looks like this

require 'open-uri'
require 'net/http'
require 'rexml/document'
require 'cgi'
$KCODE = "u"
 
class EmbeditRuby
 
  attr_reader :title, :format, :url, :html
 
  def initialize(url, size={})
    url = CGI.escape(url)
    data = REXML::Document.new(open("http://embedit.me/home/index.xml/?url=#{url}&height=#{size[:height]}&width=#{size[:width]}"))
    @title = REXML::XPath.first(data, "//title").text
    @format = REXML::XPath.first(data, "//format").text
    @html = REXML::XPath.first(data, "//html").text   # Service will provide a way to change size, this may be providing user with regex pattern to the sizes, so the wrapper can change it
    @url = REXML::XPath.first(data, "//url").text
    @fbml = REXML::XPath.first(data, "//fbml").text
    @valid = REXML::XPath.first(data, "//valid").text
  end
 
  def valid?
    @valid
  end
 
end

Lovely, so now in you Facebook app, all you need to do is

EmbeditRuby.new('url_goes_here', :width => 400).fbml

Wow, couldn’t get much easier.

So I wrote a post last night about how I was getting around the url breakage problem. Well thanks to James (author of Carbon Diet, Has My Gem Built Yet? and s.andwi.ch) I’ve got a better way but also the ‘proper’ way of fixing this url problem.

I’ve updated the wrapper to now use

require 'cgi'
 
CGI.escape(url)

So now wrappers must all escape the url they are submitting. 

The new wrapper looks like this

require 'open-uri'
require 'net/http'
require 'rexml/document'
require 'cgi'
$KCODE = "u"
 
class EmbeditRuby
 
  attr_reader :title, :format, :url, :html
 
  def initialize(url, size={})
    url = CGI.escape(url)
    data = REXML::Document.new(open("http://embedit.me/home/index.xml/?url=#{url}&height=#{size[:height]}&width=#{size[:width]}"))
    @title = REXML::XPath.first(data, "//title").text
    @format = REXML::XPath.first(data, "//format").text
    @html = REXML::XPath.first(data, "//html").text   # Service will provide a way to change size, this may be providing user with regex pattern to the sizes, so the wrapper can change it
    @url = REXML::XPath.first(data, "//url").text
    @valid = REXML::XPath.first(data, "//valid").text
  end
 
  def valid?
    @valid
  end
 
end

All fixed :)

Now this is something I should have probably noted while originally creating Embedit, but the problem only recently showed up while adding the Rock You service.

I’ll keep this short as it’s rather important if your using Embedit and I strongly suggest you update your old wrapper.

How Embedit used to work

http://embedit.me/?url=http://uk.youtube.com/watch?v=zuI9ZOu18Gc

You see that second “?”, yer, well that breaks it. It’s used to break parameters from the main url, but obviously this is two urls in one.

So to fix this I needed a way to *encrypt* (I use this word lightly) the url, so that Embedit can decode it and then process the url. Considering Embedit is good because its SIMPLE, I really needed an *encryption* that was stupidly simple, that anyone can implement. So I went with ASCII ;) Urls are now required to be sent to Embedit with each character turn into ASCII and separated by “-”. Here’s the code I added to my Embedit Ruby wrapper.

  def encrypt(old_url)
    new_url = ""
    old_url.each_byte {|d| new_url << d.to_s << '-'}
    return new_url
  end

But I like it more as an extension to String class, but whatever floats your boat.

class String
  def ascii_encrypt
    new_url = ""
    self.each_byte {|d| new_url << d.to_s << '-'}
    return new_url
  end
 
  def ascii_decode
    new_string = ""
    self.split(/-/).each {|d| new_string << d.to_i.chr}
    return new_string
  end
end

If anyone has a better solution to the whole encrypt/decrypt thing please give me a shout in the comments. Remember it needs to be SIMPLE and ACCESSIBLE to as many programmers and languages as possible.

Also, in the pipeline is a Embedit Facebook app. It’s my first Facebook app, but I am surprised how simple it is to get something usable up. I’ll make a post when the app goes live.

For those that don’t know, Embedit is my oEmbed on steroids. It takes a bunch of services that allow you to embed different media types (video, image, audio) and turns them all into one amazingly simple API. 

We released Green Thing to the public today and since this morning Embedit has processed around 13,000 requests. Thats roughly 22 requests a minute, which in all honesty isn’t that much however these do come in peaks.

Old users have been able to use Green Thing since Saturday, and yesterday Embedit had some growing pains. The tubes were getting clogged and Green Thing was timing out waiting for Embedit to reply. Shit! 

In all honesty I haven’t had any past experience in scaling, so what a great time to get started and I thought I’d share it here so I can get feedback on what I did, what I did wrong and I’ll ask questions on the way if anyone has an answer please leave a comment or send me a email :)

The Old Setup

Embedit is written in Ruby with Merb. It ran on one Mongrel instance and Nginx with a MySql DB.

merb -e production -d -p 4001

I would guess that the problem here was running on just one Mongrel and with many requests coming in from just one page load over at GT it didn’t like it one bit.

The New Setup

Ok, so I want to spread the load and I also want to be able to sleep at night knowing that something is going to look after my dear Embedit while I’m dreaming about rabbits. 

So, I went from 1 instance to 3

merb -e production -d -p 4001 -c 3

Then I changed my Nginx config to use the extra instances. This is something I’m a bit unsure of (location part)

 
 
worker_processes  1;
 
 
 
events {
 
    worker_connections  1024;
 
}
 
 
 
http {
 
    include       mime.types;
 
    default_type  application/octet-stream;
 
    sendfile        on;
 
 
 
    keepalive_timeout  65;
 
 
 
  upstream embedit {
 
    server 127.0.0.1:4001;
 
    server 127.0.0.1:4002;
 
    server 127.0.0.1:4003;
 
  }
 
 
 
    server {
 
        listen       80;
 
        server_name  208.78.103.199;
 
 
 
        location / {
 
            proxy_pass http://127.0.0.1:4001;
 
        }
 
 

So basically adding upsteam and the ports Nginx does all the rest. I am however a bit confused about the location part, will my it make use of all instances? I’ll go do some reading now :) 

Next part is to use something to monitor Embedit. I’ve tried God but just kept getting errors, so with the short time span I went for Monit. 

However we are first going to create an init script for our Merb instances

I got this info from here. 

Here is my script. You should create one for each of your instances.

Find the init.d folder at 

/etc/init.d

 
 
#!/bin/bash
 
 
 
case "$1" in
 
 
 
  start)
 
    echo -n "Starting Merb for Embedit:"
 
    cd ~/../usr/local/www/embedit/current/    # Path to the app
 
    sudo merb -p 4001 -d -e production   # Change the port to whichever one you use
 
    echo
 
    ;;
 
 
 
  stop)
 
    echo -n "Stopping Merb for App1:"
 
    cd ~/../usr/local/www/embedit/current/
 
    sudo merb -K 4001  # Again, change the port number here
 
    echo
 
    ;;
 
 
 
  restart)
 
    $0 stop
 
    $0 start
 
    ;;
 
 
 
  *)
 
  echo "usage: $0 [start|stop|restart]"
 
esac
 
 
 
exit 0

Ok, so now we have that, lets install Monit

wget http://www.tildeslash.com/monit/dist/monit-4.10.1.tar.gz
 
tar zxvf monit-x.y.z.tar.gz
 
cd monit-x.y.z
 
./configure
 
sudo make && make install

Now Monit looks for its config files in:

~/.monitrc
 
/etc/monitrc
 
./monitrc

So create yours

nano ~/.monitrc

You do need nano installed for that. It should open up a blank document. My Monit looks like this, it’s pretty self explanatory 

 
 
set daemon 10
 
check process embedit4001
 
        with pidfile "/~/../usr/local/www/embedit/current/log/merb.4001.pid"
 
        start = "/etc/init.d/embedit4001 start"
 
        stop = "/etc/init.d/embedit4001 stop"
 
        if cpu > 80% for 2 cycles then restart
 
        if totalmem > 200 MB for 2 cycles then restart
 
 
 
check process embedit4002
 
        with pidfile "/~/../usr/local/www/embedit/current/log/merb.4002.pid"
 
        start = "/etc/init.d/embedit4002 start"
 
        stop = "/etc/init.d/embedit4002 stop"
 
        if cpu > 80% for 2 cycles then restart
 
        if totalmem > 200 MB for 2 cycles then restart
 
 
 
check process embedit4003
 
        with pidfile "/~/../usr/local/www/embedit/current/log/merb.4003.pid"
 
        start = "/etc/init.d/embedit4003 start"
 
        stop = "/etc/init.d/embedit4003 stop"
 
        if cpu > 80% for 2 cycles then restart
 
        if totalmem > 200 MB for 2 cycles then restart
 
 

Basically, every 10 seconds Monit will check that each instance is still running. It will also check that if CPU usage is over 80% for 2 cycles (20 seconds) then it will restart that instance, the same goes for using over 200MB of memory. 

Monit has a huge amount of options, this is only the tip of the iceberg.

There you go! That is how Embedit is running now. It hasn’t had a single hiccup yet.

If you have any tips please leave them in the comments.

Phew it’s been busy week, but Green Thing V2 has now been officially launched. It’s awesome so you should go check it out!

For a quick update on how Embedit did with GT ’soft’ launch; everything went perfectly smoothly apart from a little bit of downtime around midnight last night, but this was either due to Slice Host or something getting stuck in the pipes, but considering I changed nothing I’m guessing it was on their end.

From Tuesday this week Embedit has already embedded over 2000 entities (I guess I could call them that) and is currently on 2865 embeds. I’ll let you all know how it goes with the big launch next Monday :)

I’ve also create a User Voice account for Embedit so that you can all add and vote on services/features you would like to see in Embedit. Add any site you want that has an embed-able entity. I guess to say it on one sentence, I want Embedit to become a platform for embedded stuff ;) So more the better

Anyway, check out the User Voice Here

Probably not really worth a full blog post but ah well…

Embedit now supports College Humor’s videos and images. I am also creating wrapper in Python for all you snake lovers out there, I shall keep you all updated on the progress of that.

We are also releasing Green Thing to a ton of users tomorrow to try it out, so I’ll probably make a post about how well/un-well Embedit held up.

On a side note - Have you every got bored of typing “git update <gem>”? Well my friend James has created an app to do the hard work for you, just head on over to http://hasmygembuiltyet.org to check it out.

Firstly a big shout-out goes to @bumi whose helped a ton with Embedit.

Embedit is now in version 0.1.1 (I made a silly mistake so had to update). We have a bunch of new services, the list of all these can be found over here. 

I have also released a gem for Embed.me which can be found over here. I am, however, having a really strange problem with ‘turning’ on the gem facilities on Github. When I tick the ruby gem box and reload the page, the box becomes un-ticked again. I get no error messages at all, and even when I tick the box it says my setting have been saved. So anyone with an idea of what might be happening please contact me. I have emailed Github but haven’t heard back yet. I want to get it out properly as quickly as possible, so you guys can start using the API :)

Tomorrow I shall be moving Green Thing over to using the embedit API. So will let you know how that goes.

I also wrote a quick benchmark test as I was asked about the performance of the gem over the web service. You would expect the web service to be slower, for obvious reasons, but each time it showed to be much quicker, sometimes close to half the time. If anyone would like to explain these happenings, please leave a comment :)