$PATH for the pathless

With the rise in popularity of scripting languages and their adoption in the industry a lot of first time professional programmers (a.k.a fresh graduates) are being exposed to the Linux command line environment and some times Linux as a system itself might be new to them. So they tend to have difficulty in understanding how a program is actually being executed and/or what all is needed to make a program run. This coupled with the magic spewed around by certain tools like rvm / chruby for Ruby and nvm for Node.js (and their ilk) are just adding to the confusion of how things are executed in the first place. I am not at all saying that such tools are bad. I myself use them and they are very helpful. But if a person is using them without getting the basics right, he/she is going to have a mighty problem wrapping their head around how things are running..!

In this context, I have been explaining what the $PATH means and what is its purpose a few times of late. So instead of repeating myself every time I thought I would just document what I know about $PATH to help the pathless get on the right PATH (or the PATH OF RIGHTEOUSNESS if you will).

GEEK NOTE : This post is aimed at absolute newbies, folks who just got started with something like Ubuntu and are fiddling around with some of the cool new technologies like Node.js / Ruby / . So this post will  not go into the depths nor will it try to be GEEK ACCURATE, i.e. it will not deal with things like POSIX compliance or the behaviour on non-Linux systems, etc. For that matter this will to a large extent be specific to Ubuntu and most likely Bash. We will also not try to distinguish between login and non-login interactive shells. As the screenshots shown below indicate, the assumed environment is running the "Terminal" application in Ubuntu.

So, lets start with the basics and a few assumptions.

Assumptions :

1. We are primarily interested in understanding how programs are run in the Linux command line environment.
2. We will restrict ourselves to the "Terminal" program of the Ubuntu system as the environment. However all / most of the following might be applicable to all such similar programs on different flavours of Linux.

Basics :

Whenever you launch the "Terminal" program what you are doing is actually launching a "Shell". Like with any type of software in the Linux world, choices are abundant and there are a lot of shell programs. However the "Terminal" program in Ubuntu (as of this writing) uses the "Bash" shell and that is what we will stick to. So on launching "Terminal" you are effectively launching the bash shell.

The "Terminal" application in Ubuntu 16.04

What is a Shell?

In the simplest terms, a Shell is a program which helps you run several other programs. It is like this eternal waiter who just waits for you to tell him what program to run and when you give him a program to run, it just runs it, tells you what was the result of running that program and goes back to waiting. Broadly speaking this is all that the shell does : Wait - Run a program - Show results - Wait. Wash-Rinse-Repeat.

If you have used the Linux command line (a.k.a the shell), you might be thinking that the shell certainly does a lot more than this. You might say "I run so many commands on the shell and it allows me to do a variety of things like search of files, list the directories, check the date etc etc. What about all those functionalities??!". Well, almost all of those actions are nothing but the shell running some program or the other. Almost every "command" that you have run are independent programs provided with the Linux system by default. Whenever you "run a command", you are just telling the shell to search for that program on the system and run it. The standard commands that most Linux users are familiar with are stored in certain predefined standard directories / locations. On Ubuntu those locations are "/bin" or "/usr/bin". While "/bin" has the core system commands, "/usr/bin" has the commands added by other softwares that are installed on the system.

A listing of the /bin directory showing system program executable files. Notice the programs corresponding to the most commonly used commands like "cp", "ls", "mkdir" etc

While these are what could be called as "standard system programs", the shell is not limited to these. The shell can run just about any program (technically, any executable file) on the system. All you have to do specify the full path to the location of that file on the command line and the shell will execute it, reporting the result of that execution. So if you have a file in a deep directory structure as shown below :

The full path to "my_program" will be /home/srirang/work/path_learning/custom_programs/my_program

When you have such a setup, to run the file "my_program" you just have to type the full path to it and hit enter. Shell will take the file at that location and run it.

/home/srirang/work/path_learning/custom_programs/my_program ↵

There might be two questions now in your head :

1. For the standard commands I never specified the full path. I just specified the program name and shell ran it without any issues. How did that happen?
2. If I have to specify the full path for each and every time, it seems a mighty tedious way, especially for programs that I run very often. Is there no easier way?

The answer for both of the these  is the "PATH" environment variable.

You need not specify the full path to the executable program all the time. When you specify only the program name, shell has a mechanism to search for that program on the system and run it. Searching for a program in the entire system every time is massively inefficient. So the shell has a mechanism for us to tell the specific locations where the shell should search for a program whenever we do not specify the full path. The way to specify this is by putting all these locations in an environment variable called the "PATH".

Whenever a program name is specified with the full path, shell will take the directory locations specified in the "PATH" environment variable one by one and search for the program in each of those locations sequentially. So if you have one or more custom programs that you would be running frequently, then it is best to add the directory locations of those programs to the PATH variable.

Environment variables : Brief introduction :

As mentioned earlier PATH is an environment variable. This post will not go into the details of what environment variables are. For the sake of this discussion here are a few points to remember :

1. Environment variables are like the settings for the shell. The values of the environment variables are used by the shell to decide how it should act.
2. By convention all environment variables are specified using upper case alphabets, numbers and underscore.
3. You can have as many variables as you want, although the shell will use only the ones it has specified.
4. You can set the value of a environment variable like this :
   
   VARIABLE_NAME=value
   
   Examples:
   MY_VAR=1
   RUBY_VERSION=2.0.0
   LANG=en_IN
5. Very often to make sure that the value of the environment variable is persisted across multiple programs (and not limit it to a single program or command), the "export" keyword is used while setting the value of the variable. Examples :
   
   export MY_VAR=1
   export RUBY_VERSION=2.0.0
   export LANG=en_IN
6. You can read the value of a variable by prepending the $ symbol before the variable name like this :
   
   $VARIABLE_NAME
7. The value of a variable can be used to set the value of another variable by using the same $ symbol. Example :
   
   RUBY_DIRECTORY=/usr/lib/$RUBY_VERSION/
8. If you want to see the value of an environment variable use the "echo" command. See the image below :

Managing the PATH variable :

Here are a few characteristics of the PATH variable :

1. The value of the PATH variable is a string.
2. There can (and will) be multiple directory locations mentioned in the PATH variable.
3. The individual directory locations are separated by the : (colon) character.
4. Most common default PATH value is typically this :
   
   /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
5. To see the current value of PATH use echo $PATH. See the image below.
6. The most common way of adding a new directory location to the PATH is this :
   
   export PATH=$PATH:/the/new/directory/location

Once you  add a directory location to the PATH like this any executable program in that directory can be run on the command line directly by specifying just the program name.

In my system, I have installed Ruby to a particular directory and I want the ruby program to be available on the command line. So this is how my PATH variable looks :

Contents of the PATH variable on my Ubuntu system.

Few Gotchas :

1. Environment variables are local to each shell. So if you are running two terminal windows or tabs, the shells running in them will have their own set of environment variables. The value that you set in one shell is not available in the other shells.
2. The value stored in the environment variables is available until the shell process quits. The values are not persisted beyond the lifetime of the shell. Nothing like saving the environment variables to the hard disk.

This would imply that any PATH settings that you would need will have to be done every time you open up a new Terminal window or tab (a.k.a every time you launch a shell). That sounds awfully painful and tedious. Sure it is. But Linux has a solution for you.

Like many Linux programs, the Bash shell (and almost every other shell too) support a startup configuration file. This file is in your home directory with the name .bashrc (yes, the name starts with a dot). This file can contain any number of commands that you would run on the command line and all of those will be "executed" every time a Bash shell is launched (a.k.a every time you open a new Terminal window or tab).

The common practice is to put all the commands setting the values of the necessary variables into this .bashrc file. That way you are guaranteed that every time you open a new Terminal window or tab, all of these environment variables will be set to the appropriate values. This is why magic tools like "rvm" or "chruby" or "nvm" ask you to add some stuff to your .bashrc file.

This should give you an initial understanding of what the PATH variable is for and what someone means when they say "PATH is not setup properly" or "PATH needs to be setup". From this point on, you should be able to google around for additional information and make sense out of it. If something mentioned here is not clear, drop in your questions in the comments and I will try my best to respond with answers.

NAS and SAN explained -- with technical differences.

Acronyms and fancy buzz words (specifically computer science related ones) have always troubled me, at times making me very angry at the person using them and in many cases leaving me in a confused state eventually. So whenever I come across such acronyms/buzz words I try to dissect them and prepare a mental visual map that I will use every time the acronym is used in the future. The acronyms for this write up are NAS (Network Attached Storage) and SAN (Storage Area Network).

These might be very simple and obvious things for many people but I am sure I have lost quite a bit of my hair whenever someone mentioned these acronyms to me. So here is my attempt to decipher them.

First the basics. Both of these consist of two building blocks storage and network, or to put in a less naive manner both SAN and NAS allow applications on one machine to access data present on another machine. Okay, so why two names, why two acronyms? To answer that let me just take up these two building blocks separately.

In the simplest sense "Storage" means dealing with files stored on the hard disk attached to the system. We do that with the the APIs (or "methods" if you want to avoid the acronym) made available by the filesystem and libraries built using those methods. As an application programmer we almost never worry about how the files are actually stored on the disk. It is the responsibility of the filesystem, the kernel and the disk driver. The application always views the data stored on the disk in terms of files (used in a generic sense to refer to both files and directories) - more so as a stream of bytes. If we dig a little deeper we find that these disks are actually made available to the filesystem by the disk drivers as block devices - i.e. whenever they accept or return data they do it in quantum of blocks. A disk doesn't return you a single byte of data when you read from it. It always returns one or more blocks. From what I understand the size of a block these days is typically 4KB. Amount of data transferred to or from the disk is a multiple of this block size. Allocation of space for files is also made in terms of blocks, which some times leads to a file utilizing the last block partially (and that is why we see the difference in the actual file size and file size of disk entries).

That's about storage. To summarize; data is made available as files by filesystem software, but the device actually makes it available as blocks.

Network in the simplest sense is communication between two processes - running either on the same machine or on different machines. To simplify it further let's just limit to the case of communication between two processes on two different machines. Typically one of these two processes will be a server processes and the other a client process. The server process would be listening on a specified port to which the client can connect. The client can then send requests over the connection which the server will "serve", by sending back a suitable response. The format of the request and the response are specified before hand and the client and the server agree to conform to that specification. This conformance is what is called the "protocol" which the two processes (or in this case the two machines) are using for their communication. The client typically asks for some data and the server fetches it from some place and sends the requested data as response. The client doesn't know where the server is fetching the data from and the server doesn't know what the client is doing with the data. The protocol is all that matters to them.

That's network. No summary here.

Okay, so how do storage and network come together now?

In the storage example the data on the hard disk (referred to as "our hard disk" henceforth) was being accessed by the applications running on the same machine (referred to as the "host machine" henceforth). Now what if applications running on a different machine (referred to as the "new machine" henceforth) want to access the data on our hard disk? Let us call this requirement as "remote data access".

The traditional filesystem software is designed to interact with a disk that was made available to it on the local system by the disk driver and the driver is designed to handle a disk that is attached to this local system. For our "remote data access" either the filesystem software has to get smarter and start talking to the device available on our host machine or the disk driver has to become smarter and make the disk on our host machine available as a local device on the new machine. It is these two options that the two acronyms stand for. One acronym means a smarter filesystem software with the same old driver and another means a smarter driver with the same old filesystem. That's the difference between the two and the reason there are two names and two acronyms.. !

NAS - Network Attached Storage -- This one has a smarter filesystem and the same old driver. In our setup, the filesystem on the "new machine" knows that the disk is on the "host machine" and every time an application requests a file (either for reading or writing) it has to contact the "host machine" over network and retrieve the file. In other words the filesystem on the "new machine" makes a request to the "host machine" - making it a client process. To accept and respond to that request there must be a server process running on the "host machine". This server process fetches the requested file from the disk (using the old driver) and sends it back to the client. The client process, which is the filesystem software, in turn makes that file available to the application that requested it. We can see that the data on the server is made available to the client as a file. This is what defines NAS.

So for the filesystem software to get smart, it now needs two components - a client part used by the applications and the server part which handles the disk. There are quite a few such "smart filesystem software" out there. The most common in the UNIX/LINUX world is NFS - Network File System. The server part of NFS is named "nfsd". On the client side, the standard "mount" command is smart enough to mount drives with "nfs" filesystem type.

Note that here the filesystem software is aware that the disk (and hence the data) is on a remote machine. This is another defining trait of NAS.

More details are available here : http://nfs.sourceforge.net/ and here : https://help.ubuntu.com/8.04/serverguide/C/network-file-system.html

SAN - Storage Area Network -- This one has a smarter disk driver and the same old filesystem. The disk driver on the "new machine" lies to the OS and the filesystem software that there is a disk attached to the system locally. The OS and the filesystem software believe the driver and continue to use the fake disk that the driver provided. Whenever the disk driver is asked to fetch a block (not a file, a block), it in turn sends a request to the "host machine" and retrieves that block of data - thereby becoming the client process in the setup. Accordingly there will be a server process running on the "host machine" which accepts this request, fetches the corresponding block from the actual disk and sends it back to the client. The client, which is the smart disk driver in this case, in turn passes that data to the filesystem software and eventually to the application that requested the file data. It is evident here that the data on the server was made available to the client as "blocks" and not as files. This is what defines SAN.

Note that here the filesystem (and every other component apart from disk driver) is not aware that the disk (and the data) is on a remote machine. This is another defining trait of SAN.

A very common and popular appearance of SAN these days is in the various cloud offerings. For instance the Amazon cloud offering has a service named EBS - Elastic Block Storage, which makes network storage available as locally attached disk. We can have all the regular filesystems like ext4 or xfs on top of this EBS drive.

That's it. The two acronyms have been conquered... !

Analysis of the Duqu Trojan worm by Kaspersky Labs

This summary is not available. Please click here to view the post.

Shortcomings of aliased field or attribute names in Mongoid - Part 1

NOTE:

• The behavior and shortcomings explained below apply to Mongoid versions 2.4.0 (released on 5th Jan, 2012) and releases previous to that. A recent commit made on 10 Jan, 2012 fixes all these shortcomings.
• For those using the affected versions (all Rails 3.0 developers), this monkey patch will address the shortcomings.

In my previous post I wrote about getting a list of aliased field names. From that post it might be evident that dealing with aliased field names is not that straight forward in Mongoid. I am using Mongoid v2.2.4 which the latest version working with Rails 3.0. Mongoid v2.3 and later require ActiveModel 3.1 and hence Rails 3.1.

Anyways, aliased field names have these shortcomings :

1. Accessor methods are defined only with the aliased names and not the actual field names.
2. Dirty attribute tracking methods are not defined for the aliased names.
3. attr_protected, if used, should be used with both short and long forms of field names.

Writing about all three in a single post would result in an awfully long post. So I will put details about each of these in their own  posts, starting with the first one in this post.

Accessor methods are defined only with the aliased names and not the actual field names.

Consider the following model definition:

class User
  include Mongoid::Document

  field :fn, as: :first_name
  field :ln, as: :last_name
end

I would have expected the additional accessor methods names 'first_name', 'first_name=', 'last_name' and 'last_name='  to be simple wrapper methods which just forward the calls to the original accessor methods :- 'fn', 'fn=', 'ln' and 'ln='. But Mongoid just doesn't create the shorter form of the accessor methods at all.

user = User.new
user.respond_to?(:fn)         # Returns false
user.respond_to?(:ln)         # Returns false
user.respond_to?(:first_name) # Returns true
user.respond_to?(:last_name)  # Returns true

This doesn't appear like a problem at first sight because an application developer would use the long form of the methods in the application code. Trouble begins in the dirty tracking methods which use the actual attribute name and consequently the shorter form of field names. Take a look at these parts of Mongoid and ActiveModel:

• Definition of setter method for any attribute - Github link for v2.2.4

  define_method("#{meth}=") do |value|
    write_attribute(name, value)
  end
  

  Notice that the field name (i.e. the short form) is being passed to write_attribute, which eventually gets passed to ActiveModel's dirty attribute tracking method attribute_will_change!


• Definition of the ActiveModel method : attribute_will_change! -- Githib link for v3.0.11

  def attribute_will_change!(attr)
    begin
      value = __send__(attr)
      value = value.duplicable? ? value.clone : value
    rescue TypeError, NoMethodError
    end
  
    changed_attributes[attr] = value
  end
  

On line no : 3 the method with the same name as that of the attribute's short name is invoked with __send__. Since Mongoid doesn't define such methods this mostly results in NoMethodError which is caught and swallowed and nothing happens. This is comparatively harmless. But if at all a method with the same already exists, then that method gets called and a lot of unwanted things can happen. In the case of the User model above, the 'fn' just results in NoMethodError, where as the 'ln' field could result in any of the following methods :

Object.ln
FileUtils.ln
Rake::DSL.ln

That could result in pretty nasty errors about these ln methods and you wouldn't even know why these are being called!. Now whether it is a good practice to name your attributes in a way that clash with already defined methods is a totally different thing. But just remember that the cause of a weird error is probably aliasing.

A testimonial to one of the best mentors I have had

I have previously written about my internship at IBM in this post and this post. The first post makes it pretty clear that the internship was a very productive (very likely most productive) part of my engineering student life. The second post briefly talks about Gautham Pai as being my guru. Both are very much true and I am thankful to Gautham for having provided me the opportunity to be on the Eclifox team. Although I have expressed my gratitude to Gautham a few times, I never really wrote it down anywhere, neither in my blog nor on any social n/w site. A few months ago Gautham started his own company Jnaapti, which is a technical skill development company. Basically he is doing what he is very good at, i.e. bringing the best out of anyone willing to learn and succeed. As part of the company operations he conducts training sessions for various corporate clients and also mentors engineering students helping them understand their subjects better using some useful project as a means of teaching. He is experimenting with various educational methodologies and different ways to teach/mentor students remotely. I am very confident that his efforts are going to change the landscape of computer science engineering education vastly. Having been mentored and guided by Gautham at various points, I thought now would be a good time to pen down a testimonial and finally put that gratitude in words. So here it goes :

Every software engineer who has been in the industry, even for a small amount of time, surely knows the gap between academic teaching and the industry requirements and the initial uphill task of coping up when a fresh engineering graduate joins any company. It would not have been different for me, if not for Gautham's guidance as my senior at college and my mentor at IBM during my internship. With Gautham's mentoring, the internship was probably one of the most productive spans in my 4 years of engineering studies and also the one packed with maximum learning. Additionally it opened up a number of opportunities for me which I previously did not even know existed - like my participation in Google Summer code and later being with the Mozilla community for quite some time and many others.

Traits like general intelligence, theoretical understanding of the subjects and the ability to solve problems are undoubtedly necessary, but not sufficient. An engineer should be able to think not just about the next line of the code that he is going to write but also think about product that he is building or helping build. He should also know that any technology is just a tool to get the work done and not something that limits you. That way you just pick any new tool that you come across and find useful or necessary for the job. This also means you keep up with the latest happenings in the tech world via blogs, articles, mailing lists etc. Above all the zeal to do more, to come up with new ideas, to start executing those ideas and the persistence to see them through, in the course carefully managing a team as a leader, are what will make an engineer truly successful.

I, of course, did not realize or understand all this during my internship. These were not handed out to me in bulleted list of To-Dos. Rather it was all nicely baked into the project that I (with a few friends) carried out and I was set on the right path without any additional effort. More than that, all of this was demonstrated to us in practice by Gautham himself and some of it just rubbed off on me, making me a much better problem solver, much better product developer, much better ENGINEER, than I would have been otherwise. Now when I look back at my internship days and my days as an engineer after that, I clearly see the impact and how much it has helped. That's Gautham and his way of mentoring. Thank you Gautham for letting me be part of the Eclifox team and for your guidance till date and the future too. :). (For the readers : Eclifox was what we built during our internship - http://buzypi.in/2007/10/11/eclifox-bringing-eclipse-to-the-browser/  and I am very proud of it.)

 In case you are wondering where did all of this finally land me, here is my linkedIn profile. :)
Keep up the great work Gautham. Wish you all the success and happiness.

Transactions - both single node and distributed - are hardwired in Windows - since Win 95

Transactions or "Atomic Transactions" to be precise, are very well known to anyone who has worked with databases. With the recent advent of NoSQL databases and the CAP theorem being used/abused by anyone and everyone, words like "consistency" and "transactional model" have become run-of-the-mill jargon. But what is actually interesting is that the concept of transactions or transactional model goes beyond our typical RDBMS. Things get even more challenging when we try to achieve transactions in a distributed system. Because transactions inherently lock the resource(s)/data they are operating on until the transaction completes, those resources can become inaccessible altogether very easily in a distributed setup if one of the node fails or if there is some problem with the network or any such thing, there by increasing the complexity of implementing distributed transactions by many folds compared to transactions on a single node.

Today I was trying to figure out if there is a way to "simulate" (albeit it will be very crude) some sort of transactions in my application which uses MongoDB (which doesn't support transactions by design - to avoid the locking mentioned above, although ironically there is a global write lock..!!). Searching on the internet lead me to this blog of a Raven DB developer. The author there mentions that RavenDB supports both sharding and transactions, which means it has implemented distributed transaction support. At first read I was pretty impressed (this was the first time I had heard about RavenDB). Before I could ask the author about the implementation details I saw a comment in which the author had mentioned that they use DTC (which again was a new thing). Turns out DTC, Distributed Transaction Controller, is a service that is baked right in the Windows OS itself, that too dating back to the Windows 95 days (wow.. now I am impressed with Windows..!). Here is the MSDN article describing the service.

The MSDN article clearly explains the basics of distributed transactions and how it is modeled. What is worth noting is that, by abstracting out the code for carrying out distributed transactions as a service, multiple resource managers (like different databases, queue servers, file servers/managers, etc..) can all interact together in a single transaction. For example, lets say that you have web application where in a client request results in a job being picked up from a queue for processing and simultaneously you update the status of the job in a DB and also create a new file associated with the start of the job. Very evidently all the three resource managers and the web application itself can be (very likely will be) on different nodes. With something like DTC you can easily create a new transaction, send across a commit message and you will get a success notification only if all three actions were successful or else none of the actions go through. Of course, this is possible only if all the three resource managers involved here adhere to the Microsoft's DTC specification and provide the necessary interface to work with it.

The previous example might make DTC appear like this Jason Bourne kind of super dude who can take care of all the heavy lifting and also do it very efficiently. But remember even Bourne gets shot at and also loses his girl. So DTC is not fully immune to problems either. Here is one blog post titled "My beef with MSDTC and two phase commits". It is definitely worth reading. Note that my impression about DTC is purely based on reading the documentation. I have not written a single line of code using DTC.

Where does node.js figure in the typical web application stack?

I have been reading up about node.js for a little while now. Now having Javascript as one of my favorite languages I obviously love node.js. It's simply awesome. But it was not until very recently that I started to wonder where would this node.js fit in the typical three-tier web architecture, fondly know as "the stack". The three elements of the stack being Web server, app server and DB (DB is definitely out of question). Within the app server we typically have a language interpreter and a framework (like rails).

I initially thought of node.js as a JS interpreter (similar to the ruby interpreter) and can be used with FCGI. But then again node.js in turn uses the V8 JS engine. So it's not a language interpreter per say. Then I saw several node.js examples showing how easily web based applications can be created. So I started comparing it with other frameworks like Rails or Erlyweb. But no, its not that either. Sure there is a simple HTTP module in node.js but it's in no way anywhere close to these frameworks. So is it a web server then? Definitely not that either, considering the rich feature list of existing web servers like Apache or Nginx. So what is this node.js then?

From what I have understood till now, node.js is just a JS library (not like jQuery or Prototype which are meant to run in the browser context). node.js is more like a ECMA-Script library. If we treat JS as a generic programming language, I believe, we will see quite a few shortcomings, the most significant one being the lack of system i/o facility. I guess ECMAScript was designed to be run in a host environment and hence features like console i/o or file i/o or network i/o were not added. This makes it very hard for JS to be used outside the host environment. This is exactly what node.js provides.

node.js extends the ECMA script and provides these missing aspects which enable JS to be used on the server side for network programming. node.js provides file i/o, socket i/o, process handling, a mechanism for creating modules and specifying dependencies, several network oriented modules and so on. (The complete list is here).

So node.js is really a library, which adds capabilities, although these features are somewhat at a more basic level than the ones provided other libraries. For instance, look at the libxml library in C. The C language compilers come with their own standard library which provides mechanisms to do file i/o. But there is no out of the box provision to deal with XML files/documents. This capability is provided by the libxml library. So the libxml library allows programmers to do something more with C than what is provided by default. There are innumerable number of such libraries which add various types of capabilities.

In somewhat analogous way, node.js is also a library which adds a lot of new capabilities to the Javascript language, although, as stated earlier, these capabilities are much more basic in nature and in most cases are present in other languages as part of their standard offering.

• So node.js is not a new language and hence do not compare it with other languages like Ruby, Python, etc. Javascript is the language here.

• node.js is not a new web application framework. So do not compare it with Rails, Django, Sinatra etc. albeit, note that node.js was apparently developed as a means to write high performance client server programs. Consequently smart folks out there started working on web application frameworks based on node and there are a couple. I know about "Express" which AFAIK, is based on Sinatra and is gaining popularity. Now that is an item comparable to Rails and friends. Questions like will node.js replace rails are, technically speaking, absurd.

• node.js is not a server. Absolutely not. There are node.js based servers, just like Nginx and Apache are C based servers.

My GSoC 2007 Story

I always had this desire to do something big and noticeable in my life as an engineering student. Right from the day I was introduced to C programming I have enjoyed programming like anything and I was pretty sure that if at all I had to do that 'something big' it will be in terms of a major software project. This desire never actually got me to work on or do something. Meaning it never materialized. Around me, I saw many people(seniors and batch mates alike) who were pretty complacent. (There were indeed exceptions). So the desire gradually was subdued by fun, until 6th semester when a friend of mine, Ashwin(Setu), forced me into coming up with a technical paper. We did come up with one on search engines which was not really that technical and hence was not what I actually desired. Then almost at the same time a few people started asking me:
" Hey are you not participating in a coding competition called Google Summer of Code? Vikas GP(A very big open source freak and a real good coder - a big shot in the college)is doing that. He is getting a lot of money for that".

At that time I thought it was just another 2 days or 3 days contest for people with ultimate coding skills who can work out magic, because of which the money appeared inaccessible, and I did not even bother to find out what the program actually was. It was eventually forgotten.

Again in March 2007, few days before the mentoring organizations started registering for SoC 2007, a talk was organized by IBM on open source software, in which it was mentioned that some Google APIs are available out there on the net. I started checking it out at http://code.google.com/ and there I saw this link to Summer of Code and the last year's scenario instantly came back to me. So I started having a look at that as to what this program is all about. When I read the program details completely I was baffled at the magnitude of the program and its quality. (The $5000 really stunned me for some time). At that moment I decided that I need to get into this program and this will be the fulfillment of my long standing desire.

I started looking into various organizations where I can poke my nose and hope to get noticed and eventually be selected. There were very few avenues for me as my open source utilization was very less and  open sourcedevelopment experience was close to nil. The organizations that I shortlisted for me were:

1) KDE - I am big fan of that and I have been using only KDE (no GNOME) for a few years now.

The following because my internship at IBM dealt with these things.

2) Mozilla Foundation (MoFo)
3) Dojo Foundation
4) Eclipse

Of all these places, the only place where I knew a few developers and where I was also a little known was in the Mozilla Foundation and that was the strongest contender.

Then I talked to Vikas GP and told him about my plans to participate in this. He gave me this fantastic idea which he had used. The plan was to catch hold of a mentor outside the program realms and impress him with your idea/code, so that when the applications go for review, we will have already mentioned a mentor's name and that person will pitch in for us. If our mentor is well known and popular in the community then we will surely be selected. :). GP had done this by catching hold of a college senior, an identified GNU developer, whom he knew personally. Now there was only one guy whom I knew properly in the MoFo and that was Mook. Then again I did not have any ideas either and the ones listed on the brainstorming page of the Mozilla wiki were all Greek and Latin to me.

Still I started looking at the ideas proposed at the brainstorming page and there was an idea to implement metalink support for the Firefox Download Manager. I thought of taking it up. In that proposal I found a link to the Firefox feature request page. The list was obviously too long, and metalinks was also listed there. But two other things caught my attention (as I had felt a dire need for those):

1) Implementing download resume across different sessions
2) Reducing the memory consumption for Firefox.

I did not find any material for metalinks and memory reduction, but found a MDC page related to resumable downloads by a person named biesi. I read that and got a small idea as to what all things are involved in implementing resumable downloads.

With this I entered the #mozilla on irc.mozilla.org from where I was directed to #developers by one nice fellow. I put up this proposal of metalinks in #developers and got bad response. Only a couple of people replied and also told me that the actual person who had mentioned it was not known to them. I was surprised and so happy to see biesi amongst the people replying. I was like - "Oh man he is the author of that MDC page and here he is talking to me directly.."
Then with the metalinks idea failing to impress I proposed the download resume idea. This got some good response, but nobody was ready to mentor me. I was pestering biesi but he told me that he is not free and does not have time to mentor me. Then when I was searching for Mook or Mossop (whom I had pestered previously about extension development) amongst the nicknames, when dmose pinged me saying "brahamana, if you are going ahead with download resume then I can mentor you". I was only partly happy as I was not aware of the credibility and popularity of dmose (With all due regards my Master, now I do know), but at least I had someone who was interested in mentoring me. But I was still hoping for biesi, (he was the author of page on MDC man)

Then dmose asked me what were the requirements for a mentor and I told him that he needs to be a respected member of the mozilla developer community and at that point biesi told me that dmose suits that description very much. dmose then told me that he will talk to his boss, mconnor, The Firefox project lead. (Wow man..) I was taken aback. I was like, "Whom was I doubting??!!! This guy talks to Firefox project lead directly in person and he is speaking for me in front of him". I thought this was my Jackpot ;-). But again dmose told me that he still can't promise me anything and will have to find out with mconnor. Well that much was sufficient for me at that time and I was literally jumping in my room (It was around 4 a.m in the morning. Yes, as usual a night out).

Then the killer waiting period started. I pinged dmose the very next night(starting of the day for him). He told me that he has mentioned the idea to mconnor and he is awaiting his response. I was growing very impatient. The last date for submitting the application to google was approaching and there was no response or growth here. I was stuck with two options, submit the application without any reference and prior approval and compete with several others as a normal application or wait for the reply till the last moment and submit it with prior approval so that there will be no competition. But the problem with waiting was that last moment applications are given less value. So that was a risk again. Meanwhile I had read many blogs, previous year's applications and several articles on "How to write a proper SoC application" and even things on "How not to write an application" and "What the mentors actually look for in the application". So with all this gyan I had realized that application has to be sent quickly or it should have a solid reference.

I pinged dmose again few hours later and this time I pissed him off and got scolded. He told me to be a little patient and wait. I was afraid of screwing this up and hence did not bother him anymore. I just hung around at #developers and made my presence felt whenever dmose was active in some discussions. I was reading that MDC page again and again and trying figure out what those terms meant so that I can talk something sensible in the channel. In that course I had discovered quite a few resources and gained some know how on Mozilla and download resume feature. I got impatient again the next day and this time to be safe I emailed dmose. His reply seemed to be calm and I was positive that I am on the right track. He had written that there was no reply from mconnor yet. This process of hanging around at IRC for a major part of the day continued for another day also. Then 3 days after the initial proposal, when I was having a discussion with dmose, biesi and probably Gijs, dmose said this as a very matter of fact thing: "brahmana, BTW mconnor has agreed for the SoC idea and you can go ahead with the project". I was very happy and started immediately asking him what is to be done, so that I could start immediately and make my position still safer. He then asked me to send any of my previous projects so that he can have a look at my coding skills. I did not have any project expect for the two academic ones. The IBM internship carried a lot of weight but no code was available as it was a proprietary thing. But the code was essential for him. And for my bad luck I was not able to find my IDE for C on Linux project anywhere. The computer on which it was stored was formatted and it was gone. It was 7 a.m and I told dmose that the project is in a repository on the college server and that I will get it to him later. Luckily he agreed. Then I was checking my cd collections to find this project and instead found another project which I had submitted. It was 2D Graphics Editor, a paint brush kind of an application written in C. It was a very big one and also a good one, but the problem was that it was not entirely written by me. We(me and my project partner Vikas Patil) had flicked it from a senior. Since I had no other option I sent that one to dmose. I do not know what he actually thought. All that he told me was: "yes, its fine. But here at mozilla you will have to write a lot of comments". Well man, that was not a problem at all. And dmose confirmed my slot. (Thank you dmose very much).

And luckily google had extended the last date for application submission. I built up a nice application from various sources, put in Dan Mosedale (dmose) twice and in a prominent way and submitted it. I was a lot relieved and very happy that the desire is indeed going to be fulfilled and something solid will materialize.

UUUuUOOoooffff... That was one herculean task man and I finally did it.

I told GP and others that a MoCo employee has agreed to mentor me. I showed them Dan's name in about:credits and told them this is the guy who will be mentoring me. It was a matter of pride for me.

Then with Dan's name in the application I did not have any problem in getting selected. No questions asked, no comments. The only notification was when I was selected.

This whole experience was awesome.

Thank you Google for conducting this program.
Thank you Mozilla Community for the guidance.(and also fun)