LakTEK A Sri Lankan, A Rubyist and A Web Dude

The Best Role Model of Our Time

“Who is the biggest role model of your life?” My answer to that question would be Muttiah Muralitharan. I know that answer would confuse most of you. You would expect a geek like me to name some one like Linus Torlvards, Yukihiro Matsumoto or Sergy Brin and Larry Page as a role model. But how come a cricketer be my role model??

In an era where vanity role models are hyped to the top by mass media, Murali stands out from the rest by his own feet. He is great not only for his phenomenal performances in the cricket field, but for his character. None of us could ever emulate his unique bowling action. But there are certain things that we can try to emulate from Murali’s character.

A Human!

It was somewhere in 1998. As a kid, I had rather unusual hobby of collecting cricket statistics. Those days, I didn’t know the existence of Cricinfo or didn’t even had a computer. I use to record all the scorecards of the matches played during that time in an exercise book. However, my record collection was never complete, I missed lot of score cards of the old matches. Then my uncle tipped me, that the Sri Lankan Cricket Board has a library, where they have all old Wisdens and “The Cricketer” magazines. I could use it to collect the missing match records. After lot of persuasion, I was able to convince him to take me to this place.

Inside the Cricket Board, we had to go pass a gym to get to the library. I saw a very familiar face inside the gym. It was Murali! He was there with another cricketer (as I remember it was Ravindra Pushpakkumara). That was the first time I saw an international cricketer in real life. Murali also saw me and waved. Then I tried to go inside the gym, to get his autograph. The instructor of the gym was there. He told me not to disturb the players and didn’t allow me to go inside. As I was turning back in disappointment, the most surprising thing happened. Murali came to the door and signed my autograph book!

This was the period, where Murali was called for chucking for the second time in Australia and he was preparing to undergo medical tests to prove he’s legibility. So while he signed my autograph, I told him how angry I’m with Australians for the unjust happened to him. Returning my autograph back, with a bright smile in his face Murali said in Sinhalese “owa ohoma tamai malli…”(These stuff happens).

I couldn’t believe how humble and down to earth this person was. He was ready to go out of his way to make some random, pesky kid happy. He could still afford to smile genuinely and take things lightly amidst of all the trouble he has been experiencing at that time. Even today, when I reminisce this incident, it feels like a dream.

A Geek!

For me Murali is a geek. He’s not a geek who uses Linux as his primary OS, lurks in IRC or hacks micro-controllers. But his passion and obsession to the game of cricket, makes him a geek in that field.

He’s not only a geek, but a hacker. He changed the face of off-spin bowling. When off-spinner was about to go extinct from the game of cricket, Murali came and made it more challenging. He forked the `doosra` from Saqlain Mustaq and hacked it into a more lethal weapon. By being different from the rest, he created controversy.

Also, Murali only focused on doing what he can do best. He didn’t have to ride Lamborghinis, have affairs with bollywood actresses (but he’s got a beautiful wife ) or get into politics to keep him in the limelight. He made the world talk about him and respect him by doing what he can really do - bowling.

A Workaholic!

When Murali first won his test cap for the Sri Lanka team, it was not the professional, winning outfit you find today. At that time, Sri Lanka was ranked only ahead of Zimbabwe and playing for the national team was not even considered as a profession. Due to the political instability of the country during that time, there were no certainty of the tours and there were no policies for player selection. The future was gloomy and had lots of risk involved. I would say the situation was analogous to working for a startup in the corporate world.

He could have easily stayed in Kandy looking after his family business. Instead Murali took the challenge and came to Colombo to join the national team. It wasn’t an easy start. His brilliance was not a one night wonder. As the stats show it took 27 test matches and 3 years to complete his first 100 wickets. During that time, he bowled full days, without much support from the other end and tasted heavy defeats.

Murali persisted and perceived harder. As his performances improved, so did the Sri Lankan team’s winning ratios. However, he focused not on his personal feats, but on his team’s victory. He had no problems playing under different captains, even juniors to him like Mahela and Sanga. He delivered his best in all circumstances. He never let his personal ego to hinder his duty.

In the last 18 years, he had been working like a horse. As of record, he bowled 33% of all overs Sri Lanka bowled at that time. He had always made himself available for national duty over other more lucrative engagements, such as country cricket in England.

How many of us can have such dedication and commitment to our duty? How many of us would complain if we have to repeat the same old boring job? Murali was no such a person. When he was on field he seems to be enjoying the every moment of it. That should be the secret mantra of Murali’s success. That’s why I call him the best role model of our era.

Hail Murali!

(Photo credit: Wikimedia Foundation - http://upload.wikimedia.org/wikipedia/commons/d/d4/MuralitharanBust2004IMG.JPG)

Handy Git commands that saves my day

There are 3 essential weapons every developer should have in their armory. A text editor, a terminal and a source code management system. Picking powerful, flexible and personalized tools will make your workflows more productive. As a developer, I use Vim for text editing, bash as the terminal and Git for source code management.

Out of those, Git turns out to be the most fascinating piece of software to me. It’s more than a SCM system. It represents a paradigm shift on the way we code. It’s decentralized nature, gives freedom to experiment and innovate, without having worry about others’ work. It brings sharing and collaboration of work to a new level. It’s like the democracy in coding!

Basic understanding of pull-commit-push cycle of Git may be sufficient for most daily ethos. However, there are plethora of other options in it which deserves some time for comprehension. Here are some of such commands, which I found useful and use in my regular workflows.

git-rebase

When I first started to use Git, my workflow used to be like this:

  git pull origin master
  git checkout -b branch_for_new_feature
  git status
  git commit -am "commit message"
  <--cycle from step 2-4, until my work is complete-->
  git checkout master
  git merge branch_for_new_feature
  git push origin

However, on many occasions when I try to push the changes to remote server(origin), I will end up with the following error:

  ! [rejected] master -> master (non-fast forward) error: failed to push some refs to 'ssh://user@server:/repo.git'

This is because my colleagues have also pushed to the remote server, while I was working on the new feature. At this point, I could simply do a `git pull` to update my local repo, which would merge the remote changes with the current HEAD. On most cases, this leads to a chain of conflicts, which requires manual resolution. Due to my laziness (or lack of concentration), I often end up deleting what was on the HEAD and replace it with the upstream changes. Mid way, I realize I was actually deleting the new updates on the HEAD that were supposed to be pushed to the remote server. From that point onwards, cleaning up the mess involves pulling out my hair and lot of swearing!

Actually, there is a smarter way to do this. That is to use git-rebase. When you do a rebase, it saves all commits in the current branch that are not available in the upstream branch to a staging area, and reset the current branch to upstream branch. Then those saved commits would be reapplied on top of the current branch, one by one. With this process, it ensures my newest changes would remain as the newest.

The new workflow with rebasing would be:

  git pull origin master
  git checkout -b new-feature-branch
  git status
  git commit -am "commit message"
  <--cycle from step 2-4, until work is done-->
 
  git checkout master
  git pull origin master #update the master before merging the new changes
  git checkout new-feature-branch
  git rebase master #apply the new changes on top of current master 
 
  git checkout master
  git merge new-feature-branch
  git push origin

Though it seems to be longer than the previous workflow, it helps me to stay away from unnecessary conflicts. Even if they do occur, resolution of them are pretty straight-forward, as I know for sure what is the newest change.

git-cherry-pick

While working on a new-feature-branch, I encounter quick fixes that are independent from the new feature; thus can be applied to master independently. Delaying the release of these fixes till new-feature-branch gets merged to master seems unnecessary. On such cases git-cherry-pick comes in handy. As the name implies, you can pick exactly one commit (by the ref-id) and apply it to another branch. To avoid conflicts, those commits should be self-contained patches. If it depends on another commit, you will require to apply it first.

git-blame & git-show

Some days, you wake up to find some one has changed your pet hack! Rather than blaming the cat for eating the code; you can easily find out who is the real culprit, by running:

 git-blame application.js

It would return the author and last commit SHA for all lines in the file. You can even narrow down the output by specifying the starting and ending lines:

git blame -L 450,+10 application.js

If you want to do further investigation, such as why did the author actually made this change and what are the other patches he committed along with this, you can run:

git show last_commit_sha

git-bisect

With git-blame you were able to track down the issues that is visible to the naked eye. But most freaking bugs are spread out and harder to detect at a glance.

This is more common when you work as a team, each one would be working on modular sections and will have tests covering the code they write. Everything seems to be running smoothly, until you merge all modules together. Tests would start to fail and you are left with no clue what breaks the tests. On such instances, what we normally do is rollback the commits one by one, to find where it causes the trouble. But this can become a tedious process if you have large set of commits. In Git there’s a handy assistant for this; it is the git-bisect command.

When you specify a range of commits it will continuously chop them in halves, using binary-search until you get to the last known good commit. Typical workflow with `git-bisect` is as follows:

git bisect start
# your repo would be switched to a temporary 'bisect' branch at this point
 
# you mark the current HEAD of the repo as bad
git bisect bad 
 
# Then you set the last known good version of the repo
git bisect good version-before-merge
 
# This will start the bisecting process.
# Commits from last good version to current version will be chopped in half.
 
# Then you run your tests
rake test 
 
# Based on output of the test you mark the commit as good or bad
git bisect good/bad
 
# Git will chop the next subset automatically, and return for you to test
 
# Test and marking process, will continue until you end-up with a single commit,
# which is supposed to be the one which introduced the bug.
 
# When bisecting process is done; run:
git bisect reset
 
# You will be returned to your working branch.

git-format-patch/git-apply

Contributing to some open source projects is easy as sending a pull request via GitHub. But that’s not the case with all. Especially in large projects such as Rails, you are expected to submit a ticket to the bug tracker attaching the suggested patch. You have to use the command git-format-patch to create such a patch, which others can simply apply to their repositories for testing.

In the same way, if you want to test someone else’s patches, you need to use the command git-apply.

git submodule

Git submodules allows you to add, watch and update external repositories inside your main repository. It’s my preferred way of managing plug-ins and other vendor libraries in Ruby or Node.js projects.

To add a submodule to your project, run:

git submodule add repository_url local_path

When someone else takes a clone of your repo, they will need to run;

git submodule init
git submodule update

This will import the specified submodules to their environment. Deploying tools such as Capistrano has built-in support for git submodules, which will run the above two commands, after checking out the code.

git help command

Last, but not least I should remind you that Git has excellent documentation, which makes its learning process so easy. To learn the options and use cases of a certain command all you need to do is running:

git help command

Apart from the default man pages, there are enough resources on the web on Git including the freely available books: Pro Git and Git Community Book

If you have any other interesting tips on using Git, please feel free to share them in the comments.

Are you creating software to impress one person?

Ever wondered why we have so much of crappy and bloated software? The root cause they are built just to impress one person.

This is a widespread disease in software industry. At the academic level, you will find students writing software to impress their mentors and get the required credit. Then you get the developers working for large software firms, who are only concerned on how to get the nod of their pointy-headed bosses. Freelancers are only worried of getting the sign-off from their pesky clients. You may expect startups to their stuff out of passion. But in reality, most startups which runs on funding are building software just to impress their VCs. This system is plain wrong!

There are no software intended to be used by one person. In most cases, the person who is getting impressed is not the actual end-user of the software. He may not have a clue of what end-user really wants.

If you are a developer, think of the wider audience, who’d be actually using your stuff. Don’t ignore them! Try to impress those people at the end of the day.

Real-time Collaborative Editing with Web Sockets, Node.js & Redis

Few months ago, I mentioned I’m developing a real-time collaborative code editor (codenamed as Realie) for my individual research project in the university. Since then I did couple of posts on the design decisions and on technologies I experimented for the project. After some excessive hacking, today I’ve got something tangible to share with you.

Currently, I have implemented the essentials for real-time collaboration including ability watch who else is editing the file, view others edits, chat with the other collaborators and replay how the edits were done. You may think this is more or less similar to what Etherpad had - yes, it is! However, this is only the first part of the project and the final goal would be to extend this to a collaborative code editor (with syntax highlighting, SCM integration).

Web Sockets

The major difference of Realie from other Real-time collaborative editors (i.e. Etherpad, Google Docs & Wave) is it uses web sockets for communication between client and server. Web Sockets perfectly suit for cases like this, where we need to have asynchronous, full-duplex connections. Compared to alternatives such as long-polling or comet, web sockets are really efficient and reliable.

In traditional HTTP messages, every message needs to be sent with HTTP headers. With web sockets, once a handshake is done between client and server, messages can be sent back and forth with a minimal overhead. This greatly reduces the bandwidth usage, thus improves the performance. Since there is an open connection, server can reliably send updates to client as soon as they become available (no client polling is required). All this makes the app truly real-time.

As of now, only browser to implement web sockets is Google Chrome. However, I hope other browsers would soon catch up and Mozilla has already shown hints for support. There are also Flash based workarounds for other browsers. For now, I decided to stick with the standard web socket API.

Taking Diffs and Applying Patches

In case if you wonder, this is how the real-time collaboration is done:

1. when one user makes a change; a diff will be created for his change and sent to server.
2. Then the server posts this diff to other connected collaborators of the pad.
3. When a user receives a diff, his content will be patched with the update.

So both taking diffs and applying patches gets executed on the client side. Handling these two actions on browser was made trivial thanks to this comprehensive library written by Neil Fraser.

However, on some occasions these two actions needs to be executed concurrently. We know by default client-side scripts get executed in a single thread. This makes execution synchronous and slow. As a solution to this I tried using the Web Workers API in HTML5 (this is implemented in WebKit & Mozilla). Separate worker scripts were used for taking diffs and applying patches. The jobs were passed on to this worker scripts from the main script and the results were passed back to main script after execution was complete. Not only this made the things fast, but also more organized.

Node.js for Backend Servers

Initially, I started off the server implementation in Ruby (and Rails). Ruby happend to be my de-facto choice as it was my favorite language and I had enough competency with it. However, soon I was started feeling Ruby was not the ideal match for such asynchronous application. With EventMachine it was possible to take the things to a certain extent. Yet, most of the Ruby libraries were written in a synchronous manner (including Rails), which didn’t help the cause. As an alternative, I started to play with Node.js and soon felt `this is the tool for the job`. It brings the JavaScript’s familiar event-driven model to server, making things very flexible. On the other hand, Google’s V8 JavaScript engine turned out to be really fast. I decided to ditch the Ruby based implementation and fully use Node.js for the backend system.

Backend is consist of two parts. One for serving normal HTTP requests and other for web socket requests. For serving HTTP requests, I used Node.js based web framework called Express. It followed the same ideology as Sinatra, so it was very easy to adapt.

Web socket server was implemented based on the recently written web socket server module for Node.js by Micheil Smith. If you are interested to learn more about Node.js’ web socket server implementation please see my earlier post.

Message delivery with Redis Pub/Sub

On each pad, there are different types of messages that users send on different events. These messages needs to be propagated correctly to other users.

Mainly following messages needs to be sent:

• When a user joins a pad
• When a user leaves a pad
• When a user sends a diff
• When a user sends a chat message

For handling the message delivery, I used Redis’ newly introduced pub/sub implementation. Every time a user is connected (i.e. visits a pad) there would be two redis client instances initiated for him. One client is used for publishing his messages, while other will be used to listen to incoming messages from subscribed channels.

Redis as a persistent store

Not only for message handling, I also use Redis as the persistent data store of the application. As a key-value store Redis can provide fast in-memory data access. Also, it will write data to disk on a given interval (also, there is a much safer Append only mode, which will write every change to disk). This mechanism is invaluable for this sort of application where both fast access and integrity of the data matters.

Another advantage of using Redis is the support for different data types. In Realie, the snapshots are stored as strings. The diffs, chat messages and users for a pad are stored as lists.

There is a well written redis client for Node.js which makes the above tasks really simple.

Try it out!

I’m still in the process of setting up an online demo of the app. Meanwhile, you can checkout the code and try to run the app locally.

Here is the link to GitHub page of the project - http://github.com/laktek/realie

Please raise your ideas, suggestions and questions in the comments below. Also, let me know if you are interested to contribute to this project (this project is open source). Thanks for reading!

Apple is repeating the same mistakes from the past

In the 1980s, Apple jumped out to an early lead in personal computers, but then got selfish. Steve Jobs, a notorious control freak, just could not play well with others.

Along came Microsoft, with Windows, which was a knockoff of Apple’s operating system. Microsoft partnered with everyone and today has 90 percent market share, while Apple’s share lingers in the single digits.

Today the battlefield is mobile devices, and just as before, Apple jumped out to an early lead. And just as before, Jobs got selfish. He won’t support Flash, or any cross-platform tools—because he wants developers locked into his platform, and his App Store, where he collects a 30 percent commission.

Daniel Lyons (Newsweek) - http://blog.newsweek.com/blogs/techtonicshifts/archive/2010/05/20/sayonara-iphone-why-i-m-switching-to-android.aspx#

Clearly, Android is becoming the new Windows (or even better because it’s Open Source). Same as in 1980s, where Microsoft knocked off Apple with an OS that would run on any platform, today it appears Google would do the same for mobile market.

No matter how beautiful, people don’t like stay inside walled gardens. Apple doesn’t seem to learn this lesson.