Projects
Embedit + Facebook
Sunday, November 2nd, 2008 | Projects | 2 Comments
You can now use Embedit with your Facebook apps! Embedit now provides FBML code for pretty much all it’s services (I’m working on the last couple)
You can find the entire list here.
The new Ruby wrapper looks like this
require 'open-uri' require 'net/http' require 'rexml/document' require 'cgi' $KCODE = "u" class EmbeditRuby attr_reader :title, :format, :url, :html def initialize(url, size={}) url = CGI.escape(url) data = REXML::Document.new(open("http://embedit.me/home/index.xml/?url=#{url}&height=#{size[:height]}&width=#{size[:width]}")) @title = REXML::XPath.first(data, "//title").text @format = REXML::XPath.first(data, "//format").text @html = REXML::XPath.first(data, "//html").text # Service will provide a way to change size, this may be providing user with regex pattern to the sizes, so the wrapper can change it @url = REXML::XPath.first(data, "//url").text @fbml = REXML::XPath.first(data, "//fbml").text @valid = REXML::XPath.first(data, "//valid").text end def valid? @valid end end
Lovely, so now in you Facebook app, all you need to do is
EmbeditRuby.new('url_goes_here', :width => 400).fbml
Wow, couldn’t get much easier.
Major Changes To Embedit && Facebook App
Friday, October 31st, 2008 | Projects | No Comments
Now this is something I should have probably noted while originally creating Embedit, but the problem only recently showed up while adding the Rock You service.
I’ll keep this short as it’s rather important if your using Embedit and I strongly suggest you update your old wrapper.
How Embedit used to work
http://embedit.me/?url=http://uk.youtube.com/watch?v=zuI9ZOu18Gc
You see that second “?”, yer, well that breaks it. It’s used to break parameters from the main url, but obviously this is two urls in one.
So to fix this I needed a way to *encrypt* (I use this word lightly) the url, so that Embedit can decode it and then process the url. Considering Embedit is good because its SIMPLE, I really needed an *encryption* that was stupidly simple, that anyone can implement. So I went with ASCII ;) Urls are now required to be sent to Embedit with each character turn into ASCII and separated by “-”. Here’s the code I added to my Embedit Ruby wrapper.
def encrypt(old_url) new_url = "" old_url.each_byte {|d| new_url << d.to_s << '-'} return new_url end
But I like it more as an extension to String class, but whatever floats your boat.
class String def ascii_encrypt new_url = "" self.each_byte {|d| new_url << d.to_s << '-'} return new_url end def ascii_decode new_string = "" self.split(/-/).each {|d| new_string << d.to_i.chr} return new_string end end
If anyone has a better solution to the whole encrypt/decrypt thing please give me a shout in the comments. Remember it needs to be SIMPLE and ACCESSIBLE to as many programmers and languages as possible.
Also, in the pipeline is a Embedit Facebook app. It’s my first Facebook app, but I am surprised how simple it is to get something usable up. I’ll make a post when the app goes live.
What’s Running Embedit?
Tuesday, October 21st, 2008 | Projects | No Comments
For those that don’t know, Embedit is my oEmbed on steroids. It takes a bunch of services that allow you to embed different media types (video, image, audio) and turns them all into one amazingly simple API.
We released Green Thing to the public today and since this morning Embedit has processed around 13,000 requests. Thats roughly 22 requests a minute, which in all honesty isn’t that much however these do come in peaks.
Old users have been able to use Green Thing since Saturday, and yesterday Embedit had some growing pains. The tubes were getting clogged and Green Thing was timing out waiting for Embedit to reply. Shit!
In all honesty I haven’t had any past experience in scaling, so what a great time to get started and I thought I’d share it here so I can get feedback on what I did, what I did wrong and I’ll ask questions on the way if anyone has an answer please leave a comment or send me a email :)
The Old Setup
Embedit is written in Ruby with Merb. It ran on one Mongrel instance and Nginx with a MySql DB.
merb -e production -d -p 4001
I would guess that the problem here was running on just one Mongrel and with many requests coming in from just one page load over at GT it didn’t like it one bit.
The New Setup
Ok, so I want to spread the load and I also want to be able to sleep at night knowing that something is going to look after my dear Embedit while I’m dreaming about rabbits.
So, I went from 1 instance to 3
merb -e production -d -p 4001 -c 3
Then I changed my Nginx config to use the extra instances. This is something I’m a bit unsure of (location part)
worker_processes 1; events { worker_connections 1024; } http { include mime.types; default_type application/octet-stream; sendfile on; keepalive_timeout 65; upstream embedit { server 127.0.0.1:4001; server 127.0.0.1:4002; server 127.0.0.1:4003; } server { listen 80; server_name 208.78.103.199; location / { proxy_pass http://127.0.0.1:4001; }
So basically adding upsteam and the ports Nginx does all the rest. I am however a bit confused about the location part, will my it make use of all instances? I’ll go do some reading now :)
Next part is to use something to monitor Embedit. I’ve tried God but just kept getting errors, so with the short time span I went for Monit.
However we are first going to create an init script for our Merb instances
I got this info from here.
Here is my script. You should create one for each of your instances.
Find the init.d folder at
/etc/init.d
#!/bin/bash case "$1" in start) echo -n "Starting Merb for Embedit:" cd ~/../usr/local/www/embedit/current/ # Path to the app sudo merb -p 4001 -d -e production # Change the port to whichever one you use echo ;; stop) echo -n "Stopping Merb for App1:" cd ~/../usr/local/www/embedit/current/ sudo merb -K 4001 # Again, change the port number here echo ;; restart) $0 stop $0 start ;; *) echo "usage: $0 [start|stop|restart]" esac exit 0
Ok, so now we have that, lets install Monit
wget http://www.tildeslash.com/monit/dist/monit-4.10.1.tar.gz tar zxvf monit-x.y.z.tar.gz cd monit-x.y.z ./configure sudo make && make install
Now Monit looks for its config files in:
~/.monitrc /etc/monitrc ./monitrc
So create yours
nano ~/.monitrc
You do need nano installed for that. It should open up a blank document. My Monit looks like this, it’s pretty self explanatory
set daemon 10 check process embedit4001 with pidfile "/~/../usr/local/www/embedit/current/log/merb.4001.pid" start = "/etc/init.d/embedit4001 start" stop = "/etc/init.d/embedit4001 stop" if cpu > 80% for 2 cycles then restart if totalmem > 200 MB for 2 cycles then restart check process embedit4002 with pidfile "/~/../usr/local/www/embedit/current/log/merb.4002.pid" start = "/etc/init.d/embedit4002 start" stop = "/etc/init.d/embedit4002 stop" if cpu > 80% for 2 cycles then restart if totalmem > 200 MB for 2 cycles then restart check process embedit4003 with pidfile "/~/../usr/local/www/embedit/current/log/merb.4003.pid" start = "/etc/init.d/embedit4003 start" stop = "/etc/init.d/embedit4003 stop" if cpu > 80% for 2 cycles then restart if totalmem > 200 MB for 2 cycles then restart
Basically, every 10 seconds Monit will check that each instance is still running. It will also check that if CPU usage is over 80% for 2 cycles (20 seconds) then it will restart that instance, the same goes for using over 200MB of memory.
Monit has a huge amount of options, this is only the tip of the iceberg.
There you go! That is how Embedit is running now. It hasn’t had a single hiccup yet.
If you have any tips please leave them in the comments.
College Humor Comes To Embed
Monday, October 13th, 2008 | Projects | No Comments
Probably not really worth a full blog post but ah well…
Embedit now supports College Humor’s videos and images. I am also creating wrapper in Python for all you snake lovers out there, I shall keep you all updated on the progress of that.
We are also releasing Green Thing to a ton of users tomorrow to try it out, so I’ll probably make a post about how well/un-well Embedit held up.
On a side note - Have you every got bored of typing “git update <gem>”? Well my friend James has created an app to do the hard work for you, just head on over to http://hasmygembuiltyet.org to check it out.
Embedit.me
Tuesday, September 30th, 2008 | Projects | No Comments
Considering Embedit was initially created to make our lives easier I’ve had a great response back from people using the gem. This just goes to show that if you find something annoying or too complicated for it’s own good, then its extremely likely that others have the same problem.
However the gem, being a gem, provided one problem; it is quite isolated and only really useful to Ruby programmers. So I thought it was time to show the other communities a little love.
After receiving a great contribution from @Bumi I thought it was time to move Embedit over to a web service. The new Embedit provides an extremely easy to use API that spits out xml (JSON coming very soon). It also has a web interface so you can have a play around.
Find it @ http://embedit.me
I’m working (almost finished) on a wrapper in Ruby, which you’ll be able to find over at GitHub
More services are in the TODO, along with json and calls to allow you to specify height and width of the html (this should be up tonight).
Enjoy!
Panda+MySql Capistrano Recipe
Sunday, September 21st, 2008 | Projects | 4 Comments
I’ve been spending the last fortnight moving Panda over to use MySql and integrating it with The Green Thing V2 site. We decided to host Panda on our own server and not on Amazon’s EC2 service, this choice was mainly to do with cost, which has be talked about on the Panda mailing list.
If you decide to use Panda on EC2, New Bamboo (creators of Panda) created a AMI (Amazon Machine Image) which means installing all the required libraries and what-not, is all taken care of.
So to cut a long story short
- Installing everything that is required makes me sad.
- While installing everything to make Panda run, I created a Capistrano Recipe.
- This means that, as long as its maintained, I won’t have to go through that horrible process again.
- It’s on my Github account.
It defiantly needs some tidying up, but it gets the job done. There are still some things that require setting up - e.g. Nginx configuration, but I’ll add that later on after the huge amount of busyness has settled down.
Embedit!
Wednesday, August 27th, 2008 | Projects | No Comments
We are currently well into the re-make of the Green Thing site.
V2 is defiantly going to kick V1’s ass. Everything has been re-thought, re-designed and re-coded. Don’t think of it as a ‘better’ V1 because it’s not, it’s completely new.
Here’s a little sneak peak:
But anyway, onto the actual reason I started this post (I just wanted to give some background info). As you can see from the sneak peak we use embedded videos (along with lots of other embed-able media). Handling lots of different API’s and lots of different embed code and what-not was just going to get ugly.
Originally we were going to just use oEmbed supported services. OEmbed is awesome, but it just cut the API-calling-work on several services. We would still be using the oEmbed API + all the others.
The project was originally a very simple Ruby wrapper for the oEmbed API called oEmbed-with-ruby, but for reasons I’ve just stated, it wasn’t enough.
So it got a name change, oEmbed-with-ruby is now called Embedit. At the moment it is at v0.0.3. It’s being heavily developed, as its one of the main backbones on V2, well, when it comes to media side anyway.
Services that are currently supported include all oEmbed services and You Tube.
Code Stuff
It’s installable as a RubyGem.
To Install -
gem sources -a http://gems.github.com sudo gem install reddavis-embedit
To Use -
require 'rubygems' require 'embedit' media = Embedit::Media.new('http://www.youtube.com/watch?v=j3TOT1lnVTA') media.title #--- Title of the media media.format #--- 'video' media.url #--- url to the video media.html #--- This can be the embed-able video from services like youtube, or an image tag for Flickr, or audio player or whatever.
That’s basically it! Nice and simple. I’m currently adding validations so validating that and URL can be used with Embedit will be just as easy as what you’ve just seen.
The code is available at GitHub
If you need to get hold of me grab me on Twitter or send me an email - [reddavis] [@] [gmail.com]
Is My Tube Late - Full-time
Tuesday, August 12th, 2008 | Projects | No Comments
Is My Tube Late is now full-time. Yey! I’ve converted my old Mac Mini into a server running Ubuntu Server Edition, everything went smoothly, which was cool.
I have also bought the http://ismytubelate.com domain name. In the future this will point to an actual page, but for now it just points to my blog post about imtl.
Is My Tube Late?
Friday, August 8th, 2008 | Projects, Uncategorized | 7 Comments
After much inspiration from:
I’m please to announce Is My Tube Late?
In short…There are 12 tube lines, each now has its own Twitter account. I have also created an ‘all tubes’ account that keeps up to date with all the 12 lines.
The status of each line is checked every 5 minutes and if there is a change in status a message is sent to the respective Twitter account and of course, the all tubes accounts.
If you wish you receive these updates on your phone, all you have to do is:
- Sign up to Twitter.
- Go to the Twitter device setting and set-up your mobile to receive update.
- Go to a lines Twitter page
- Click follow and then select ’status updates’ to on
Due to furniture maneuver (the server will have to be shut down) the service won’t be up 100% until 9th August, but then should be pretty reliable.
Here is the list of all the lines and their Twitter accounts
If there are any problems feel free to contact me @ reddavis [at] gmail [.] com.
The code will be released on GitHub as soon I tidy it up a bit.
Enjoy!
FeedBurner
Search
My Twitter
- @Bumi thanks! Yer I got the idea off your delicious log :) Will write a blog post about it after I fix these things on GT
- fuck its 3am
- my quick little project for tonight http://twitterembedstream.com/ i'll post about it tomorrow
- please ignore, just testing something ;) http://www.vimeo.com/1981896
- my internet is sooooooo slow this evening
- some big changes being pushed to gt right now.
- lunch time!
- hahah http://tinyurl.com/65jw6f
- wooo party http://tinyurl.com/5zzpav
- UGh, our staging server is so slow. It's kinda lost any use
- @codyo Check out http://embedit.me I wrote it. Its like a massive embedding api
- @matclayton You should check out http://embedit.me for your embed stuff