Invalid credentails : OmniAuth + oAuth2 + Rails 4 encrypted cookie store + simultaneous requests

OmniAuth is a very well know gem in the Ruby/Rails world. Almost every Rails application out there is probably using it to authenticate with one of the various mechanisms it supports. OmniAuth is just awesome.!

I have been using OmniAuth for about 3 years now in various Rails projects and it has worked very well, although I have had to monkey patch it once or twice to allow me to exploit some of the Facebook features (like authenticated referrals). But all in all, it just works as advertised and you will have an authentication system in pretty much no time at all.

Yesterday, however, I was having quite a bit of hard time getting OmniAuth to do a simple "Login with Facebook" oAuth2 authorization. It was something that had worked seamlessly on innumerable previous occasions. But yesterday it just kept failing repeatedly, succeeding only once in a while. And it always failed with the same obscure error "Invalid Credentials" during the callback phase. (OmniAuth operates in three phases : Setup, Request and Callback. Apart from OmniAuth wiki on github, this is a good place to read about it : http://www.slideshare.net/mbleigh/omniauth-from-the-ground-up). The fact that the error message was not so helpful made this whole process a lot more frustrating. After some hunting on the inter-webs I found that the culprit could be a bad "state" parameter.

Wait, what is this "state" parameter?

Background : oAuth-2 CSRF protection

oAuth2 specifies the uses of a non-guessable secure random string to be used as a "state" parameter to prevent CSRF attacks on oAuth. More details here : http://tools.ietf.org/html/rfc6749#section-10.12. This came out almost a year back and many oAuth providers implement it already, including Facebook. OmniAuth also implemented it last year. Although not written with the best grammar, this article will tell you why this "state" parameter is needed and what happens without it.

To sum it up, oAuth client (our web application) creates a random string, stores it in an accessible place and also sends it to the oAuth provider (Ex : Facebook) as "state" during the request phase. Facebook will keep it, authenticate the end user, ask for permissions and when granted sends a callback to our web application by redirecting the user back to our website "with the state" as a query parameter. The client (our web application) compares the "state" sent by the provider and the one it had stored previously and proceeds only if they match. If they don't then there is no proof that the callback that our web application received is actually from the provider. It could be from some other attacker trying to trick our web application into thinking (s)he is someone else.

In case of OmniAuth oAuth2, this state parameter is stored as a property in the session with the key 'omniauth.state' during the request phase. The result of the request phase is a redirect to the provider's URL. The new session with the "state" stored in it will be set on the client's browser when it receives this redirect (302) response for the request /auth/:provider (This is the default OmniAuth route to initiate the request phase). After the provider (Facebook) authenticates the user and user authorizes our application, the provider makes a callback to our application by redirecting the user back to our web application at the callback URL /auth/:provider/callback along with the "state" as a query parameter. When this callback URL is requested by the browser, the previously stored session cookie containing the 'omniauth.state' property is also sent to our web application.

OmniAuth checks both of these and proceeds only if they match. If they don't match it raises the above mentioned "Invalid Credentials" error. (Yeah, I know, not really a helpful error message..!).

Ok, that is good to hear, but why will there be a mismatch?

A mismatch is possible only if the session cookie stored on the user's browser is changed such that the 'omniauth.state' property is removed from it or altered after the request phase has set it. This can happen if a second request to our web application was initiated while the request phase of oAuth was running and it completed after the request phase completed but before the callback phase started. Sounds complex? The diagram below illustrates it.

The diagram makes it clear as to when and how the 'omniauth.state' gets removed from the session leading to the error. However, apart from the timeline requirements (i.e. when requests start and end), there is another essential criteria for this error to occur :

The response of the "other simultaneous request" must set a new session cookie, overriding the existing one. If it does not explicitly specify a session cookie in the response headers, the client's browser will retain the existing cookie and 'omniauth.state' will be preserved in the session.

Now, from what I have observed, Rails (or one of Rack middlewares) has this nifty feature of not serializing the session and setting the session cookie in the response headers, if the session has not changed in the course of processing a request. So, in our case, if the intermediate simultaneous request does not make any changes to the session, Rails will not explicitly set the session cookie, there by preventing the loss of 'omniauth.state' property in the session.

Ok, then why will the session cookie change and lose the 'omniauth.state' property?

One obvious thing is that the "other simultaneous request" might change the session - either add or remove or edit any of the properties. There however is another player involved.

This is where the "Encrypted Cookie Store" of Rails 4 comes into picture. Prior to Rails 4, Rails did not encrypt its session cookie. It merely signed it and verified the signature when it had to de-serialize the session from the request cookie. Read how Rails 3 handles cookies for a detailed breakdown. Rails 4 goes one step ahead and encrypts the session data with AES-256 (along with the old signing mechanism. More details on that coming up in a new post). The implementation used is AES-256-CBC from OpenSSL. I am not a Cryptography expert, but the way I understand it, the property of AES is that it results in a different cipher text every time you run the encryption for the same message plain text. Or it could also be because the Rails encryption scheme initializes the encryptor with a random initialization vector every time it encrypts a session (Implementation here). Either ways, the session cookie contents are always new for every request even when the actual session object or session contents remain unchanged. As a result Rails will always set the session cookie in the response header for every request and result in the browser updating that cookie in its cookie store.

In our case, this results in the session being clobbered at the end of the "other simultaneous request" and we end up losing the 'omniauth.state' property and oAuth fails.

Umm.. ok, but when and how does this happen in real world, if at all it can?!

All these requirements/constraints described above, especially the timing constraints makes one wonder if this can really happen in the real world. Well, for starters it happened to me (hence this blog post..!!). I also tried to think of scenarios other than mine where this would happen. Here are a couple that I could think of :

Scenario - 1) FB Login is in a popup window and the "Simultaneous request" is a XHR - Ex : an analytics or tracking request.

Here is the flow :

1. User clicks on a "Login with FB" button on your website.
2. You popup the FB Login page in a new popup window. Request phase is initiated. But there is a small window of time before the redirect response for '/auth/facebook' is received and 'omniauth.state' is set.
3. During that small window of time, in the main window, you send an XHR to your web app to, may be, track the click on the "Login with FB" button. You might do this to just track usage or for some A/B testing or to build a funnel, etc. This request sends the session without the 'omniauth.state'.
4. While the XHR is in progress, the redirect from the request phase is complete and the session with 'omniauth.state' is set. The user now sees FB Login page loading and proceeds to login once it is loaded.
5. While the user is logging in to FB and approving our app, the XHR has completed and has come back with a session without 'omniauth.state'. This is stored by the browser now.
6. Once user logs in and approves your app, the callback state starts. But the session sent to your web app is now missing the 'omniauth.state'. 
7. oAuth fails.

How big a deal is this scenario?

If you are indeed making a XHR in the background, then this scenario needs to be taken care of. Since the "other simultaneous" request is automatically triggered every time, it is very likely that session will get clobbered.

How to solve this?

You can either first send the XHR and then in the response handler of that XHR, you can open the FB Login page in the popup. Also have a timeout just to make sure you don't wait for too long (or forever) before you receive a response for that XHR.

Alternatively, if you can push your tracking events in a queue stored in a cookie, you can do that and then open the FB Login page. Once the FB Login completes, you can pull that event out of the queue and send it. As a backup have a code that runs on every new page load to look for pending events from the queue in the cookie and send those events.

With HTML5 in place, its probably better to use the localstorage for the queue than the cookie. But again that needs user's permission. Your call.

Scenario - 2) FB Login is in the same window/tab but User has the website opened in two tabs.

Here is the flow :

1. User has your website opened in a browser tab - Tab-1
2. User opens a link on your website in a second tab - Tab-2 (Ctrl + Click or 'Open in a new tab' menu item). This request sends the session without 'omniauth.state'.
3. While that Tab-2 is loading, user clicks on "Login with FB" in Tab-1 initiating the request phase.
4.  If the request loading in Tab-2 is a little time consuming the redirect of the request phase of oAuth in Tab-1 completes before request in Tab-2, setting the session with 'omniauth.state'. After that FB Login page is shown and user proceeds to login and authorize.
5. While the user is logging in, the request in Tab-2 completes, but with a session that is missing 'omniauth.state'.
6. After logging in to FB, the callback phase is initiated with a redirect to your web app, but with a session that doesn't have 'omniauth.state'. 
7. oAuth fails. 

How big a deal is this scenario?

Not a big deal actually. In your web app, in the oAuth failure handle, you can just redirect the user back to /auth/facebook, redoing the whole process again and guess what - this time it will succeed and that too without the user having to do anything because the user is already logged in to FB and has also authorized your app. But just to be on the safer side, you would want to be careful about this loop going infinite (i.e. You start FB auth, it fails and the failure handler restarts the FB auth). Setting a cookie (different from the session cookie) with the attempt count should be good. If the attempt count crosses a certain limit, send the user back to homepage or show up an error page or show a lolcats video, c'mon be creative.

Ok, those are two scenarios that I could think of. I am not sure if there are more.

Can OmniAuth change something to solve this?

I believe so. If OmniAuth uses a different signed and/or encrypted cookie to store the state value instead of the session cookie none of this session clobbering would result in loss of the state value. OmniAuth is a Rack based app and relies on the Session middleware. I am not entirely sure, but it can probably use the Cookie middleware instead. Just set its own '_oa_state' cookie and use that during callback for verification.

Will you send a pull request making this change?

I am not sure. I first will hit the OmniAuth mailing list and find out what the wise folks there have to say about this. If it makes sense and nobody in the awesome Ruby community provides an instant patch, I will try and send a patch myself.

THE END
 
Ok, so that was the awesome ride through oAuth workings inside the OmniAuth gem. In the course I got to know quite a bit of Rails and also Ruby internals. Looking forward to writing posts about those too. Okay, okay.. fine. I will try and keep those posts short and not make them this long..!!

Till then, happy oAuthing. :-/ !

P.S : Security experts, excuse me if I have used "authentication" and "authorization" in the wrong places. I guess I have used it interchangeably as web applications typically do both with oAuth2.

Rails cookie handling -- serialization and format

A typical Rails cookie has this format : cookie-value--signature (the two dashes are literal). The "cookie-value" part is a url encoded, base64 encoded string of the binary dump (via Marshal.dump) of whatever was set in the session. The signature part is a HMAC-SHA1 digest, created using the cookie-value as the data and a secret key. This secret key is typically defined in [app-root]/config/initializers/secret_token.rb.

Let us try and reverse engineer a session cookie for a local app that I am running. I am using Devise for authentication, which in turn uses Warden. I use the Firecookie extension to Firebug to keep track of cookies. It is pretty handy.

Here is the session cookie set by Rails:

# Cookie as seen in Firebug
BAh7B0kiGXdhcmRlbi51c2VyLnVzZXIua2V5BjoGRVRbCEkiCVVzZXIGOwBGWwZvOhNCU09OOjpPYmplY3RJZAY6CkBkYXRhWxFpVGkvaQGsaQGwaRBpAdFpCGk9aQHtaQBpAGkGSSIiJDJhJDEwJEZseHh3c293Q29LcHhneWMxODR2b08GOwBUSSIPc2Vzc2lvbl9pZAY7AEYiJTUwNDdkOTMwNDNkNGEzOTA4YTkwN2U2MDY5OGRmOTdm--51f90f7176326f61636b89ee9a1fce2a4972d24f

As mentioned at the beginning it has two parts separated by two dashes (--).

The cookie value in this case is :

# The cookie-value part
BAh7B0kiGXdhcmRlbi51c2VyLnVzZXIua2V5BjoGRVRbCEkiCVVzZXIGOwBGWwZvOhNCU09OOjpPYmplY3RJZAY6CkBkYXRhWxFpVGkvaQGsaQGwaRBpAdFpCGk9aQHtaQBpAGkGSSIiJDJhJDEwJEZseHh3c293Q29LcHhneWMxODR2b08GOwBUSSIPc2Vzc2lvbl9pZAY7AEYiJTUwNDdkOTMwNDNkNGEzOTA4YTkwN2U2MDY5OGRmOTdm

The signature is :

51f90f7176326f61636b89ee9a1fce2a4972d24f

Whenever Rails gets a cookie it verifies that the cookie is not tampered with, by verifying that the HMAC-SHA1 signature of the cookie-value sent matches the signature sent. We can also do the verification ourselves here. Fire up irb and try the following :

$ irb

irb(main):003:0> cookie_str = "BAh7B0kiGXdhcmRlbi51c2VyLnVzZXIua2V5BjoGRVRbCEkiCVVzZXIGOwBGWwZvOhNCU09OOjpPYmplY3RJZAY6CkBkYXRhWxFpVGkvaQGsaQGwaRBpAdFpCGk9aQHtaQBpAGkGSSIiJDJhJDEwJEZseHh3c293Q29LcHhneWMxODR2b08GOwBUSSIPc2Vzc2lvbl9pZAY7AEYiJTUwNDdkOTMwNDNkNGEzOTA4YTkwN2U2MDY5OGRmOTdm"
=> "BAh7B0kiGXdhcmRlbi51c2VyLnVzZXIua2V5BjoGRVRbCEkiCVVzZXIGOwBGWwZvOhNCU09OOjpPYmplY3RJZAY6CkBkYXRhWxFpVGkvaQGsaQGwaRBpAdFpCGk9aQHtaQBpAGkGSSIiJDJhJDEwJEZseHh3c293Q29LcHhneWMxODR2b08GOwBUSSIPc2Vzc2lvbl9pZAY7AEYiJTUwNDdkOTMwNDNkNGEzOTA4YTkwN2U2MDY5OGRmOTdm"


# This cookie_secret comes from [app-root]/config/initializers/secret_token.rb. Obviously you need to keep this secret for your production apps.
irb(main):005:0> cookie_secret = '392cacbaac74af104375eb91324e254ba232424130e69022690aa98c1d0dfade159260588677e2859204298181385a83b923e58c4ef24bb3a40bdad9a41431b4'
=> "392cacbaac74af104375eb91324e254ba232424130e69022690aa98c1d0dfade159260588677e2859204298181385a83b923e58c4ef24bb3a40bdad9a41431b4"

irb(main):006:0> OpenSSL::HMAC.hexdigest(OpenSSL::Digest::SHA1.new, cookie_secret, cookie_str)
=> "51f90f7176326f61636b89ee9a1fce2a4972d24f"

As can be seen the HMAC-SHA1 hexdigest generated with the cookie-value matches the signature part of the cookie. Hence the cookie is not tampered with.

Now that the cookie authenticity is validated, let us see what information it holds.

Let us retrace the steps taken by Rails to generate this cookie value to get the value stored in the cookie. The steps taken by Rails are :

1. session_dump = Marshal.dump(session)
2. b64_encoded_session = Base64.encode64(session_dump)
3. final_cookie_value = url_encode(b64_encoded_session)

The reverse process would be :

1. url_decoded_cookie = CGI::unescape(cookie_value)
2. b64_decoded_session = Base64.decode64(url_decoded_cookie)
3. session = Marshal.load(b64_decoded_session)

And with a beautiful language like Ruby all these 3 steps can be done in one single line of code. Here it is :
(Btw, I need to require 'mongo' because one of the values contained here is of type BSON::ObjectId which is defined in the mongo gem. Without this Marshal.load will error out)

irb(main):001:0> require 'mongo'
=> true
irb(main):002:0> require 'cgi'
=> true
irb(main):003:0> cookie_str = "BAh7B0kiGXdhcmRlbi51c2VyLnVzZXIua2V5BjoGRVRbCEkiCVVzZXIGOwBGWwZvOhNCU09OOjpPYmplY3RJZAY6CkBkYXRhWxFpVGkvaQGsaQGwaRBpAdFpCGk9aQHtaQBpAGkGSSIiJDJhJDEwJEZseHh3c293Q29LcHhneWMxODR2b08GOwBUSSIPc2Vzc2lvbl9pZAY7AEYiJTUwNDdkOTMwNDNkNGEzOTA4YTkwN2U2MDY5OGRmOTdm"
=> "BAh7B0kiGXdhcmRlbi51c2VyLnVzZXIua2V5BjoGRVRbCEkiCVVzZXIGOwBGWwZvOhNCU09OOjpPYmplY3RJZAY6CkBkYXRhWxFpVGkvaQGsaQGwaRBpAdFpCGk9aQHtaQBpAGkGSSIiJDJhJDEwJEZseHh3c293Q29LcHhneWMxODR2b08GOwBUSSIPc2Vzc2lvbl9pZAY7AEYiJTUwNDdkOTMwNDNkNGEzOTA4YTkwN2U2MDY5OGRmOTdm"

# Reverse engineering the cookie to get the session object
irb(main):004:0> session = Marshal.load(Base64.decode64(CGI.unescape(cookie_str)))
=> {"warden.user.user.key"=>["User", [BSON::ObjectId('4f2aacb00bd10338ed000001')], "$2a$10$FlxxwsowCoKpxgyc184voO"], "session_id"=>"5047d93043d4a3908a907e60698df97f"}

This is the session data that the session cookie was holding. This data is subsequently used by Warden and Devise to fetch the user from the DB and do the authentication.

And that is how Rails handles cookies (at least how Rails 3.0.11 does. I am not sure if things have changed in later versions)

Analysis of the Duqu Trojan worm by Kaspersky Labs

This summary is not available. Please click here to view the post.

Getting the list of aliased key/attribute names from a Mongoid model

At some point today when I was writing some model specs for one of my Mongoid models, I required the list of all of the attribute/key names. Mongoid provides a handy "fields" method for this, which returns an hash of key names and Mongoid::Fields::Serializable object pairs. Getting the list of names from that was easy : Model.fields.keys.

This gives the list of the actual key names. The actual key names, in my case, are very short strings (1 to 3 characters) and I have long aliases for those in my models. What I eventually realized was that I wanted the list of  the longer aliased names. Looking around the mongoid code did not give me any direct method. Turns out that the aliased names result in nothing more than a few additional 'wrapper' methods (like the accessors, dirty methods, etc) and there is no table/hash kind of thing maintained anywhere to give the mapping between the actual key name and the aliased ones. So my current guess is that the list of these aliased names is not available directly anywhere.

So I came up with this hackish way of getting that list of aliased names.

p = Post.new
actual_field_names = p.fields.keys
all_field_names = p.methods.collect { |m| m.to_s.match(/_changed\?$/).try(:pre_match) }
                    .select { |e| e }
aliased_field_names = all_field_names - actual_field_names

As mentioned earlier, this is pretty hackish. If you know of a straight forward way, do let me know.

Note : I eventually found out that I did not actually need this list of aliased names. I did not use this in my project. Nevertheless it works just fine.

Transactions - both single node and distributed - are hardwired in Windows - since Win 95

Transactions or "Atomic Transactions" to be precise, are very well known to anyone who has worked with databases. With the recent advent of NoSQL databases and the CAP theorem being used/abused by anyone and everyone, words like "consistency" and "transactional model" have become run-of-the-mill jargon. But what is actually interesting is that the concept of transactions or transactional model goes beyond our typical RDBMS. Things get even more challenging when we try to achieve transactions in a distributed system. Because transactions inherently lock the resource(s)/data they are operating on until the transaction completes, those resources can become inaccessible altogether very easily in a distributed setup if one of the node fails or if there is some problem with the network or any such thing, there by increasing the complexity of implementing distributed transactions by many folds compared to transactions on a single node.

Today I was trying to figure out if there is a way to "simulate" (albeit it will be very crude) some sort of transactions in my application which uses MongoDB (which doesn't support transactions by design - to avoid the locking mentioned above, although ironically there is a global write lock..!!). Searching on the internet lead me to this blog of a Raven DB developer. The author there mentions that RavenDB supports both sharding and transactions, which means it has implemented distributed transaction support. At first read I was pretty impressed (this was the first time I had heard about RavenDB). Before I could ask the author about the implementation details I saw a comment in which the author had mentioned that they use DTC (which again was a new thing). Turns out DTC, Distributed Transaction Controller, is a service that is baked right in the Windows OS itself, that too dating back to the Windows 95 days (wow.. now I am impressed with Windows..!). Here is the MSDN article describing the service.

The MSDN article clearly explains the basics of distributed transactions and how it is modeled. What is worth noting is that, by abstracting out the code for carrying out distributed transactions as a service, multiple resource managers (like different databases, queue servers, file servers/managers, etc..) can all interact together in a single transaction. For example, lets say that you have web application where in a client request results in a job being picked up from a queue for processing and simultaneously you update the status of the job in a DB and also create a new file associated with the start of the job. Very evidently all the three resource managers and the web application itself can be (very likely will be) on different nodes. With something like DTC you can easily create a new transaction, send across a commit message and you will get a success notification only if all three actions were successful or else none of the actions go through. Of course, this is possible only if all the three resource managers involved here adhere to the Microsoft's DTC specification and provide the necessary interface to work with it.

The previous example might make DTC appear like this Jason Bourne kind of super dude who can take care of all the heavy lifting and also do it very efficiently. But remember even Bourne gets shot at and also loses his girl. So DTC is not fully immune to problems either. Here is one blog post titled "My beef with MSDTC and two phase commits". It is definitely worth reading. Note that my impression about DTC is purely based on reading the documentation. I have not written a single line of code using DTC.

Microsoft's Virtual Wifi adapter ( or virtual wifi card) -- cool technology

I wasn't aware of the very interesting research on Virtual Wifi Adapters that Microsoft guys have been carrying out. Apparently they have been doing it for quite some time now. What this research group is trying to do is basically allow us to have an unlimited number of "virtual" wireless cards on our computers, each connecting to a different wireless connection, and all of it using just a single physical card. That is some awesome stuff.. !

A couple of days ago I opened up the Dell Support Center tool on my laptop and it popped up a message saying a device on my system is in the disabled state. I was pretty startled to see that, as pretty much every device on my laptop is used by me daily. On clicking the message it told me that the disabled device is "Microsoft Viritual Wifi Miniport". That did not make any sense to me. I had absolutely no clue about this device.

Searching the internet led me to this Microsoft page (along with several others, of course) which gave me a fair idea of what this device might be, but nothing concrete. It was this fine article on istartedsomething.com that clearly explained what this is all about. In the same article the author tells us that Microsoft has been carrying out research in this regard for a few years now. But nothing was given to end users until Windows 7 baked in this wifi card virtualization natively. And not just that, all WiFi card providers are expected to add support for this virtualization in their drivers if they want their drivers to be properly signed digitally and recognized by Windows during installation. I say that is "Wicked cool".. :)

About the technology itself, it can be described as a way to make "software copies" of your Wireless card and use those copies to "connect to multiple networks simultaneously".  Although research prototypes can apparently create any number of virtual devices over the single actual hardware device, Windows 7 limits it to just one copy/virtual device.

This whole research is doubly fascinating.

First because the applications of this research work are very interesting. One such application is explained in the article mentioned above. It talks about being able to connect to an existing wireless access point with your laptop and at the same time making your laptop a wireless access point in itself. It means, if someone is far from the actual access point and your laptop happens to be closer, he/she can connect to your laptop instead and your laptop will forward their connections to the actual wireless access point. Of course, this can only happen when the two laptops involved are in the same security/trust group. I wouldn't go on and connect via some random stranger's laptop. It is like letting that person look at all the data coming in and going out of your computer over the internet (or network in general). Despite such caveats, this is very much a practical use case. May be you wouldn't use it to be a hop in the network (or more like a virtual signal booster), but you may use it to make P2P/direct connections with another laptop close by for sharing files instead of doing over the wireless LAN. Or, if access to wireless network is possible only after you authenticate via certificate (like in a corporate setup) and the certificates can be put only one of your laptops (the official company laptop), the connection sharing will indeed come in handy.

Secondly, and more importantly, the complexities associated with this are lot and that makes it all the more exciting. If we delve a little deeper into what this virtual adapter is and how it works, we will see that it is actually a piece of software sitting between the actual device driver and the rest of the network stack (i.e. all above the MAC layer in the OSI model). This little piece of software is supposed to appear as one or more "devices" to the OS and hence it invariably has to have its own device driver. That is the "Virtual WiFi Filter" driver or VWifi driver. This VWifi driver tells the OS that there are multiple wireless cards and the OS then allows the user to connect to different available wireless connections via these virtual cards. But note that all this time, there is only one physical card and hence at any given point in time that one physical card can be connected to (or can communicate with) only one wireless network. It is the job of the virtual adapter software to cycle over all the virtual wireless cards and service the network requests made through them using the one physical card, in a time shared manner all the while keeping it transparent to the user. Although it sounds very similar to the kernel's process scheduling which makes use of the single processor in a time shared manner, this is actually somewhat different because of the way wireless networks work.

Note that different wireless networks behave differently. They might be operating at different frequencies, they might be having different authentication/encryption schemes, the bandwidth might be different and probably many other factors that I can't think of right now. So every time the actual wireless card switches to a different connection, there can be a shift in all or some of the above mentioned attributes. The card might have to step up or down its operating frequency, change to a different encryption scheme and all of this on the fly. Now that is a lot of work to do and in fact all of this switching can just drag the network performance to the ground. This makes the design and implementation of the virtualization software pretty challenging. This and many other challenges/caveats are discussed in this Microsoft Research's FAQ page.

I have been very excited about this research work ever since I read it and have been meaning to try out writing some network code using the virtual adapter. Sadly I have zero experience in network programming in Windows and currently don't have enough time to read up all of that. I hope such a thing will come up in Linux some time soon, if it isn't already there.

Where does node.js figure in the typical web application stack?

I have been reading up about node.js for a little while now. Now having Javascript as one of my favorite languages I obviously love node.js. It's simply awesome. But it was not until very recently that I started to wonder where would this node.js fit in the typical three-tier web architecture, fondly know as "the stack". The three elements of the stack being Web server, app server and DB (DB is definitely out of question). Within the app server we typically have a language interpreter and a framework (like rails).

I initially thought of node.js as a JS interpreter (similar to the ruby interpreter) and can be used with FCGI. But then again node.js in turn uses the V8 JS engine. So it's not a language interpreter per say. Then I saw several node.js examples showing how easily web based applications can be created. So I started comparing it with other frameworks like Rails or Erlyweb. But no, its not that either. Sure there is a simple HTTP module in node.js but it's in no way anywhere close to these frameworks. So is it a web server then? Definitely not that either, considering the rich feature list of existing web servers like Apache or Nginx. So what is this node.js then?

From what I have understood till now, node.js is just a JS library (not like jQuery or Prototype which are meant to run in the browser context). node.js is more like a ECMA-Script library. If we treat JS as a generic programming language, I believe, we will see quite a few shortcomings, the most significant one being the lack of system i/o facility. I guess ECMAScript was designed to be run in a host environment and hence features like console i/o or file i/o or network i/o were not added. This makes it very hard for JS to be used outside the host environment. This is exactly what node.js provides.

node.js extends the ECMA script and provides these missing aspects which enable JS to be used on the server side for network programming. node.js provides file i/o, socket i/o, process handling, a mechanism for creating modules and specifying dependencies, several network oriented modules and so on. (The complete list is here).

So node.js is really a library, which adds capabilities, although these features are somewhat at a more basic level than the ones provided other libraries. For instance, look at the libxml library in C. The C language compilers come with their own standard library which provides mechanisms to do file i/o. But there is no out of the box provision to deal with XML files/documents. This capability is provided by the libxml library. So the libxml library allows programmers to do something more with C than what is provided by default. There are innumerable number of such libraries which add various types of capabilities.

In somewhat analogous way, node.js is also a library which adds a lot of new capabilities to the Javascript language, although, as stated earlier, these capabilities are much more basic in nature and in most cases are present in other languages as part of their standard offering.

• So node.js is not a new language and hence do not compare it with other languages like Ruby, Python, etc. Javascript is the language here.

• node.js is not a new web application framework. So do not compare it with Rails, Django, Sinatra etc. albeit, note that node.js was apparently developed as a means to write high performance client server programs. Consequently smart folks out there started working on web application frameworks based on node and there are a couple. I know about "Express" which AFAIK, is based on Sinatra and is gaining popularity. Now that is an item comparable to Rails and friends. Questions like will node.js replace rails are, technically speaking, absurd.

• node.js is not a server. Absolutely not. There are node.js based servers, just like Nginx and Apache are C based servers.

What is Cloud? -- Simple terms please

Cloud has been making a lot of noise and almost every tech (or tech related) person knows about it or at least heard of it. Now for those who have just heard about it but do not know what it means here is a quick definition from Dave Neilsen, the founder of Cloud-Camp. He says, "For something to be called cloud, it should have these properties :

• Hosted by someone else
• On-demand. Do not have to wait or call somebody to get it.
  
• Metered somehow. So you know exactly how much you are using and how much you are paying.
  
• Scalable, both ways - up and down as and when you require."
  

He goes on to say that Cloud could mean different things for different people. Here area few examples stating what cloud is for a particular person :

For an IT guy -- Infrastructure as Service
For a Web Developer -- Platform. Just dump your code and don't worry what runs it.
For a Business guy -- SaaS (Software as a Service)

That was pretty neat. Helps me answer the standard question "What the hell is this cloud thing?" in a sane manner. Earlier I could never figure out what a proper answer should be for this question, because there was so much to tell.

Here is my attempt to elaborate on above mentioned examples.

So cloud is basically having the infrastructure to do what you do hosted by someone else and having it totally scalable. For example, in the above list, for a web developer cloud is a platform where he can dump his code and expect it to run as he has designed it. He does not worry about the machines, the network connectivity, the bandwidth. He just pays for those in a metered manner. He scales his platform whenever he wants. He can increase his bandwidth quota, move to a better machine, increase the number of machines and all of this without calling the customer care or the sales guy. He will do it by logging into the cloud services website or he would have a script do this for him automatically, i.e if he is geek enough.

Similarly for a business man, it is software as a service. E-mail service would probably be a good example. The business man does not know what software runs the email system, he does not worry about what version of email server is running, what os it is running on, what DB it is using to store the emails, what protocols it is making use of. If the email contents are not that sensitive he would not even worry about the physical location of the servers storing these emails. He just buys the email software as a service and uses it. All that he probably worries about is how many email accounts are available to him/his company and how reliable/usable they are. At any point he can increase or decrease the number of accounts, once again without making a call.

That's cloud .

Note : I got this definition from one of the IBM developerWorks podcasts which is available here.

Oh, and remember, all this time every reference to Cloud meant "Cloud Computing", not just plain "cloud"

Mozilla @ SJCE : Static Analysis projects

It has been a very long time since I posted anything about the Mozilla related activities going on at my college SJCE, Mysore. That in no way means absence of any activity. In a previous post I mentioned that along with the attempt to introduce Mozilla as a standard course I was working to get the current final year students (who are enrolled under the larger VTU University) to start working on Mozilla, using their final semester project as a means. Well I am happy to say that this has materialized. 8 final semester students from CS expressed interest in working with the Mozilla community as part of their final year project and it's been a month nearly since they started their work. Here is a brief write up about that.

As is with most of the Computer Science students in India approaching the Mozilla community, these 8 students also wanted to do something related to compilers. The JS engine and static analysis are two projects in Mozilla which would come under the compiler banner. These 8 students wanted to work on something substantial which can be presented by two teams of 4 students each as their final semester project. So the bugs that they would be working on had to be related. This was possible only with static analysis as there are a lot of related tasks available. Also static analysis would be something new to the students and it would give them an opportunity to understand the internals of the compiler (GCC here) like the AST (Abstract Syntax Tree) and other representations of the code. Moreover the static analysis APIs are exposed in JS and hence the analysis scripts would be written in JS. That way students would learn JS also. Above all these students would be doing something genuinely new.

The students could not be asked to start working on the bugs directly. They were new to open source development, the tools used there like the bugzilla, using email as a formal medium of communication, the source control system (to add to the complexity Mozilla now uses a distributed RCS - Mercurial [hg]), using IRC, the linux development environment etc. It has been these things that the students have been learning for this part month or so. This learning has been in the form of accomplishing the tasks which form the prerequisites for the actual static analysis work. These are things like downloading gcc and mozilla sources from the ftp hosts and from the mercurial repository respectively, applying the mozilla specific patches to gcc for plugin support etc, etc... These are all listed here. Note that some things like installing all the dependency packages for building these applications from sources, learning to use the linux command line itself and others are not on that page but were new to these students nonetheless.

All the students have been putting in substantial effort and have picked up the traits of an open source hacker pretty soon. We have had a few IRC meetings and a lot of formal communications over emails. In parallel we were also working towards shortlisting 8 static analysis bugs. Based on the feasibility of the bug being completed by an amateur developer within a span of 2.5 months and based on the students' interest we finally decided on these 8 bugs :

1. Bug 525063 - Analysis to produce an error on uninitialized class members
   
2. Bug 500874 - Static analysis to find heap allocations that could be stack allocations
   
3. Bug 500866 - Warn about base classes with non-virtual destructors
   
4. Bug 500864 - Warn on passing large objects by value
   
5. Bug 528206 - Warn on unnecessary float->double conversion
   
6. Bug 526309 - Basic check for memory leaks
   
7. Bug 542364 - Create a static analysis script for detecting reentrancy on a function
   
8. Bug 500875 - Find literal strings/arrays/data that should be const static
   

These tasks are good, challenging and provide an opportunity to understand compilers very closely.

Currently the students have downloaded gcc, applied the patches, built it along with the dehydra plugin support and are ready to run static analysis on the mozilla code. They are now trying to run simple analysis scripts like listing all classes in mozilla code and all classes and their corresponding member functions. It is still quite a long way to go, but it has been a real good start. Let's wait and watch what great feats are in the pipeline.

I hope to keep this blog updated at the same pace at which the students are working.

Good luck to the students. :-)

Mozilla Developer Network (MDN) survey -- My inputs

I just finished the MDN survey and here is what I said in that last box which was put there for the people like us to pen down our rants. ;-)

• Project documentation needs improvement. It has improved and is improving, but a lot still needs to be done specifically about the oldest lines of code.
• I hear from some of the core developers that there are lots of hacks which make the code not entirely predictable. These need to be removed and replaced by proper, reliable code. Again the cleaning is going on, am just saying that it is really important so that there is some sort of SLA based on which people can develop applications.
• Consolidation of the content on MDC and MozEdu so that we can have a "The Mozilla Book", which any beginner can go through and dive into Mozilla related development -- either the platform or the browser or the add-ons or anything.
• Finally, making various Mozilla components available in the form of easily pluggable library modules and step by step guides telling us how to use them.

I do not know if any of this is useful to anyone else in the community, but for me, these appeared to be very important based on my association with Moziila for about 2.5 years now.

Incrementally building Mozilla/Firefox

Mozilla code base is really huge and has variety of files which are built in a variety of ways. It was sort of always confusing for me to figure out where all I should run make after I change any of the files. I generally asked on the IRC and someone just told me where to run make.

Today it was the same thing. But I also thought I would as well learn the logic to decide for myself the next time. Here is the chat transcript of NeilAway answering these questions.

The MDC page (https://developer.mozilla.org/en/Incremental_Build) has almost the same content for the native code. Neil here explains it for all the types of files involved.

Also to add to the following things running : "make check" from the objdir will run the automated tests.

• For xul/js/css/xbl it usually suffices to find the jar.mn (it may be in an ancestor folder) and make realchrome in the corresponding objdir
• For idl you're often looking at a full rebuild, depending on how widely it's used
• For .cpp and .h you obviously have to make in the folder itself, and then look for a corresponding build folder
• Except for uriloader where you use docshell/build and content, dom, editor and view use layout/build
• If you're building libxul or static then this is all wrong
• You don't look for a build folder, I think for libxul you build in toolkit/library and for static you build in browser/app

Changing the start up directory of command prompt -- The safe and simple way

By default, the command prompt window (cmd.exe) starts in a particular directory which depends on quite a few factors. Specifically, the env varibales %HOMEDIR% and %HOMEPATH% matter the most or these are the ones which ultimately decide the location. There are some registry values also, but they are not in the game by default. This leads to the command window to start up in something like C:\Documents and Settings\<username>\ which is not particularly useful as this location is rarely used for anything useful. It can get worse. In a corporate environment it might so happen, more often than not, that your home directory is set to a shared network drive using a group policy and the command prompt starts in that remove drive location.. ! The pain can be aggravated if you are on VPN or something similar.

I personally feel that command prompt, which is mainly programmers tool, should not start in %HOMEDIR% as there are rarely (mostly never) any programming related files are kept. Anyways, there are many ways to solve this problem.

The first of them and the most dangerous is fiddling with the registry. It is mentioned here : http://windowsxp.mvps.org/autoruncmd.htm
The problem with this is that it will screw up make based build environments which spawn multiple child shells (aka command prompts), because the command prompt will start in this changed default location instead of the location where the make had to run.

The next is fairly simple and also elegant. You can create a shortcut to the main exe (C:\Windows\system32\cmd.exe) and in the properties of the shortcut you can provide the directory to start in. This good for mouse users. However for those (most programmers) who start command prompt from the run dialog (by typing cmd) this will fail.

The third solution is to deal with the previous solutions shortcoming. You can just create a batch fail in any of the locations where the run dialog looks into. I prefer C:\Windows\systme32\. Put the following line in the batch file: C:\Windows\system32\cmd.exe /K "cd <path-to-dir>" . Just replace the <path-to-dir> with the path where you want the command to start. I generally put it as C:\.
This will start a command window and execute cd <path-to-folder> and will stay for further inputs. I have named this batch file as sh.bat (obviously to get a linux feel :P ). So now when I press the Windows + R key and type sh, I get the command prompt started in C:\.
Done.
And yes, this is totally safe and will not affect any other application using cmd.exe. :)

OpenSSL base64 filter BIO needs an EOL and memory BIO needs to know about EOF

I recently started working with the OpenSSL library to do some https stuff (sort of obviously). OpenSSL apart from having an implementation for the SSL encryption part, it also nifty algorithms for certificate handling and more importantly an abstract I/O layer implementation called BIO which probably stands for Basic I/O or Buffered I/O or something else. I do not know, I could not find it. Nevertheless, the items of interest here are the BIO_f_base64() -- The base64 encode/decode filter BIO and the BIO_s_mem() -- The memory BIO, which can hold data in a memory buffer.

The BIO man page (or its online version present here: http://www.openssl.org/docs/crypto/bio.html) give a nice introduction. For now just consider BIOs as black boxes from which you can read or write data. If the BIO is a filter BIO then the data will be processed whenever you read or write to it.

The name, BIO_f_base64, says it all about the functionality of this BIO. If you read from this BIO, then whatever data is being read is first base64 decoded and given to you. OTOH, if you write something to this BIO it will be base64 encoded and then written to the destination. These BIOs can be arranged in the form of chains to do a series of processing on the data that you are reading or writing, all by just a single call to read() or write(). Its all abstracted. Saves a lot of time.

I was trying to decode some base64 encoded data which I had in a buffer, a char [] to be precise. So if you read up about the BIOs it becomes obvious that you first have to create a memory BIO, which will hold the actual encoded data. Write the encoded data to the memory BIO. Then you chain that memory BIO with a base64 BIO and read from that chain. Any data that you read from the chain will actually come from the memory BIO, but before it reaches you it passes through the base64 BIO. So essentially you are reading from the base64 BIO. As mentioned in the earlier paragraph, when you read from a base64 BIO it decodes the data and gives it you. So the base64 encoded data present in the memory BIO is decoded and presented to you. That's it. base64 decoding is done in one simple read() call !

But there is small catch here. For some reason, which I have yet partially understood, base64 requires that the data it is handling be terminated with a new-line character always. If the data does have any newline character, meaning all your data is present in a single line then you have to explicitly tell that to the BIO by setting the appropriate flag. Here is what the man page says:

The flag BIO_FLAGS_BASE64_NO_NL can be set with BIO_set_flags() to encode the data all on one line or expect the data to be all on one line.

That's about the base64's EOL. Now the other BIO involved here,the memory BIO, is also an interesting guy. When the data it has gets over, it doesn't say "Hey, its over, stop it!". Instead it says "Dude, you got to wait for some more data to arrive. Hang on and keep trying". !!! This is very much suitable, probably when you using the BIO like a PIPE, where you keep pumping data from one end by acquiring it from somewhere and some other guy consumes that data. But in a situation like mine where the data is all fixed I simply want it to tell that the data is all over and I need to stop it. To do this again I will have to explicitly set an appropriate flag and here is what the man page says:

BIO_set_mem_eof_return() sets the behaviour of memory BIO b when it is empty. If the v is zero then an empty memory BIO will
return EOF (that is it will return zero and BIO_should_retry(b) will be false. If v is non zero then it will return v when it
is empty and it will set the read retry flag (that is BIO_read_retry(b) is true). To avoid ambiguity with a normal positive
return value v should be set to a negative value, typically -1.

And this same thing is explained very well here: http://www.openssl.org/support/faq.html#PROG15.

I thank Dr. Stephen N Herson of the OpenSSL project for helping me out in understanding this. Here is the mailing list posting that taught me this thing : http://groups.google.com/group/mailing.openssl.users/browse_thread/thread/f0fc310c1bc6ec65#

Happy BIOing. :-)

Getting the size of an already loaded page (from cache) in a Firefox extension.

Today this question came up in the IRC (moznet, #extdev). One of the add-on developers wanted to get the size of the page, either bytes or number of characters. The most obvious thing that came to my mind was progress listeners for definitive answers or the content length from the channel for not so critical scenario. But then he said he wants it for an already loaded page. And he further said that the information is already there somewhere as it is shown by the Page Info dialog (Right Click on a web page and select View Page Info). He was indeed right. Somebody in the code is already going through the trouble of calculating the data size and we can just re-use that. And I immediately started the quest to find that out.

As usual to figure out any browser component I opened up DOM Inspector. That tool is improving, which was against my earlier perception (Sorry Shawn Wilsher), though the highlighting part is still screwed up. Nevertheless, locating that particular label "Size" and the textbox in front of it containing the value was not difficult at all. I got the "id" of the textbox containing the size value. (Its "sizetext" :) ).

Next it was MXR (http://mxr.moziila.org/) in action. I did a text search for the id and got a bunch of results, one of which was pageInfo.js with this entry : line 489 -- setItemValue("sizetext", sizeText); . It is here. The very line made it apparent that it is the place where the value is being set and hence it is the place from where I can get to know how the value is being calculated.

Once I saw the code it was very clear and straight forward and pretty simple also. We have the URL. From the URL we get the cache entry for that URL. (Every cache entry has a key and that key is the URL - so neat). We try to get the cache entry from the HTTP Session first and if that fails we try FTP Session. The cache entry has the size as an attribute on itself, so its just getting that attribute value. DONE.

I am not sure how this will behave if we have disabled every type of cache. AFAIK, there will still be some in-memory cache as long as the page is still loaded. Probably good enough.

That was the end of a small but interesting quest. :-)

Getting Open Source/Mozilla in my college - SJCE - Part I

Earlier I had written about my first Mozilla Education Status Call in which I mentioned my interest to bring Open source software in general and Mozilla in specific to my college, SJCE. Well, good news, it did not stop at that blog post. Actually speaking it had not started with that blog post either. It has been a long standing wish of mine, even from my college days when I participated in the Google Summer of Code 2007. (More about  it here). Back then there were a lot of short-comings, both from my side and the institution's side to actually make this idea into reality. Nevertheless, past is past and no point in brooding about it now. The good thing is that now both I and my institution have overcome our short-comings and we have started working towards making that idea into a reality.

Now for some background aka story telling (which I like the most :) )

As stated earlier nothing happened about this idea when I was in my college. Then after passing out of the college and having worked in the industry for about 12-18 months it hit me, very hard, that I did not learn a lot of things during my life as an engineering student which otherwise would have helped me a lot in my professional life. Also I learnt that I was not competent enough as an engineering graduate as compared to some of my foreign counterparts and also those from some of the "famed premier" engineering institutions of the country. It was not just one thing or two, but I saw differences in many aspects, both theoretical and practical. Gradually it occurred to me that these two aspects are inter-related. Since we did not have proper practical experience and exposure to real world software development we never really appreciated the basic theoretical concepts of computer science, which formed our regular syllabus. Note that here I am saying "we" and not "I". This part of the story talks about the state of most of my classmates and that is the worst part. Nevertheless, the moral of the story is the good old philosophy of teaching that the theory and practice should go hand in hand.

Now there is another part to this story. Before I start with it let me tell you that whatever I am putting here is based on what I have perceived. I may be wrong, but I personally don't think so. And this is absolutely not about boasting about myself. So the other part of the story is that, my association with Open Source development communities, Mozilla to be specific, has greatly helped me in my professional life. I am not going to give examples, but it has really really helped me a lot. Also it has sort of put me ahead of several other capable classmates and most juniors of mine (with the difference being considerably more in the case of juniors). The only differentiating factor between me and them was my exposure to developing a real world application, Mozilla Firefox, and the various lessons that I have learnt by being a part of the global developer community. I am also certain that I could have been a much better computer engineer if I had started working with Mozilla at a much earlier stage, say 2nd year or early 3rd year of engineering and had dedicated more time to it. I still continue to learn a lot of general computer science and software development concepts (concepts not specific to Mozilla development) even now whenever I try to fix a Mozilla bug or even when I try to answer any query on the IRC, many a times even when I just observe few people conversing on the IRC.

Ok, enough of story telling. Now is the part for moral of the story. Here is what I inferred from these experiences:

1) Engineering students, specifically Computer Science engineering students, must get exposure to real world engineering (aka application development) to understand and appreciate the theoretical concepts they learn.
2) Open source software development communities provide a suitable environment for students to work with real world applications. Suitable in terms of - opportunities, cost, mentoring and certainly a few more good things also.

With these two points, it was clear to me that we badly needed open source education/exposure for students in my college. I knew that once this happened the possibilities were endless. Every time I heard/read about some of the classic Free Software implementations done at Universities abroad, I thought that our college can at least have several continuous contributors to currently existing open source projects, if not have creators of some totally new world class software projects. We could be having several different groups of students working of different types of software which operate at various levels (which translates to contributing to different open source software). Then they all could be interacting to help each other in troubleshooting problems. I thought of scenarios/discussions like this happening in the hostel corridors:

Student_1 and Student_2 are working on the Mozilla Download Manager (and here goes the conversation)

Student_1 : Hey, I want to test my new implementation for Mozilla Download Manager for HTTPS downloads. I am unable to configure my test server for HTTPS. You got any idea?
Student_2 : No dude, never done any server side stuff. Lets ask Student_3 from the Apache team.

Then Student_3 comes and sets up Apache for HTTPS within minutes (because that's day-to-day kind of stuff for him) and Student_1 continues testing his new implementation.

After some time:

Student_1 : Oh man, SSL handshake is taking too much time. I need to talk to Student_4, he knows the SSL library code base.
Student_2 : Yeah, I talked about that to Student_4. He is coming up with a patch to reduce the handshake time. It will probably be ready by tomorrow, I guess. Apparently it was a race condition causing the delays.

And so on.

Something like this is really possible. In fact many things much bigger than this are possible. But only if our students start working with and for open source communities.

So this set of thoughts made me work towards getting Open Source into my college. Now that's the background and the story. In the next part I will write about the first set of steps taken towards this, how many of them worked and how many were dead even before they started. And just FYI, the next post too will have some story telling (Obviously since this is just a record of my experiences and my (our) actions).

Vim -- Restoring cursor from previous session

I am sure every Vim user needs this. Its just so frustrating to open a source file to see the #includes (the first few lines) in the file when we actually will be editing many hundred lines later. Today specially I was juggling with several source files, adding something to the .h file and then come back, do something in the .cpp file and then again go to some .c file and so on. Every time I opened a file the cursor was at the first line and every time invariably I had to search for the function I was editing and cycle through the matches to reach it. So I set on a "Search Mission" - A mission to search for the appropriate .vimrc settings to make Vim remember the cursor position from the previous session.

I got a lot of links. In fact there is a separate Vim Tip - Vim Tip #80 for this. But it has so many code lines and I was a little wary to put all that in my .vimrc file. Continued search revealed me a simpler way. Just two lines solution and it is here : Some Mr.Gopinath's .vimrc file. Its very big, but the lines concerning me are:

" VimTip 80: Restore cursor to file position in previous editing session
" for unix/linux/solaris
set viminfo='10,\"100,:20,%,n~/.viminfo

" only for windows [give some path to store the line number info]
"set viminfo='10,\"100,:20,%,nc:\\Winnt\\_viminfo
au BufReadPost * if line("'\"") > 1 && line("'\"") <= line("$") | exe "normal! g`\"" | endif

Looks like he also picked this up from the same Vim Tip #80 but was smart enough to take out only the necessary part. Nevertheless, this works for me. Thank you Mr. Gopinath.

Happy Vimming :-)

Edit:

I read the Vim Tip #80 again and it made more sense this time. I picked up the last line for a user comment. Now the cursor is put back on the same column too.

Recursively rename files in windows

My friend Prasanna B P asked me to solve a bunch of problems that he was facing with his computer. One of them was fairly common and there was a very easy and straight forward brute force solution. But coming up with a "smart" was really interesting.

Essentially some malware had renamed all the movie files that he had to carry a .jpg extension. His initial solution was to associate those jpg files with a movie player like Mplayer123 or VLC and the file would be invariably played without any regard to the extension. But this made the genuined jpg pictures to be opened with the player.

The obvious solution was to rename all the files and change the extension. Manually doing it from the GUI is the brute force idea that I earlier mentioned. And it is a totally crappy one. Next I can rename all files in a directory from the command line. But here the movie files were in directories of their own and hence I would have to move to each directory manually and run the rename command. This makes it as good as the GUI approach. In fact the extra effort of typing the commands might make it worse.

The I searched the internet a little and got to know that the windows command shell supports a "FOR" statement which can be used to recursively traverse directories, amongst many other things it provides. Using that I found this command:

FOR /R %x IN (*.jpg) DO ren "%x" *.avi

from this website : http://stackoverflow.com/questions/210413/command-line-recursive-renamemove-in-windows

People used to Linux Shell scripting might think of this as a wierd syntax, but yeah thats the windows choice.

It was good to do some Windows Shell scripting too. :-)

My first Mozilla Education Status call

Mozilla is the only open source community which I have understood a little and also to which I have contributed a little. I always wanted to make my college a sort of Mozilla hub with several contributors and many feature developments happening out of my college. It actually started with an idea of a "Complete Open Source Hub", but it got reduced to just "Mozilla Hub" (either because of my laziness or because of lack of resources). Anyways I did not take any proactive steps towards that wish of mine, until recently when our college got the "Autonomous" status. That is when I realized that bringing open source software development in the course mainstream has become a lot easier as the power to form the syllabus and conduct the tests and examination rests with my college itself and not the University.

Just when I thought of doing something tangible, Mozilla came up with their "Mozilla in Education" program. This is a brilliant idea to drive open source into the student community and also give the students opportunities to work on real world applications. I was impressed by this program the very moment I read about it. I decided to present this idea to my HOD at the college and get him to start working offering Mozilla education to students in my college. I started reading more about it and today I also attended my first Mozilla Education status meeting conference call, which happens every week. I got a wealth of information.

This week we had Pascal F presenting to us about the "Design Challenge" program organized by the Mozilla Labs. He talked to us about the way in which they organized this program. It was all very inspiring. They had nearly 30-35 people from different parts of the world (literally). In this contest the students initially submitted mock-ups of their ideas, nearly 40 of them. Amongst those, 30 fully completed mock-ups were chosen for the second level, in which the students got mentoring by some the well known names in the Mozilla community. The mentoring consists of 10 Webinar sessions conducted using WebEx, over a period of 3 weeks. This is going on currently and will end at the end of this month. These mentoring sessions aim at converting those mock-ups into working prototypes. At the end, the best prototypes are given honors.

During the presentation Pascal mentioned some interesting things. Of the students from various countries, it was the students from so called "2nd World Countries" (like Romania, India, Argentina, etc.. ) who showed a lot more interest than their US counterparts. There was tremendous enthusiasm in them where as the students from US expected -- in his own words -- "being entertained" and "spoon-fed". Though this is an alarming thing when viewed from a global perspective, I was personally very happy that Indian students, my fellow country-men, have shown such dedication. Pascal also mentioned that they were so interested that they were up in the middle of the nights for the webinars. All in all, I am even more motivated to bring open source in general and mozilla in particular to my college.

I decided to take this program as an example and present it to my HOD this Saturday and try and make him accept this and similar programs as official ones and the projects done in such programs be considered for the completion of the course goals.

Thank you Pascal, thank you Mozilla. Lets hope to have a Mozilla India Center, at least an unofficial one at my college SJCE. :-)

An official full-fledged Firefox Add-ons dev guide

Having done a little work on Mozilla Firefox and extensions for it I have always felt the lack of an official, step by step document for extension development. I do accept that there are a lot of resources available on the internet. Innumerable number of blogs and tutorials explaining the process step by step. But most of them become outdated with newer versions of Firefox and the authors are not really keen about updating the info. That is why an official tutorial or guide from Mozilla itself would be the right thing. It will make sure the contents in the guide are up-to-date and worth reading for developing an extension for the currently available version of Firefox.

There are documents for extension development on MDC like the various links present at: https://developer.mozilla.org/en/Extensions but it was either scattered or pretty brief in most places though it did touch most parts of extension development. Nevertheless a wholesome "official" guide was needed and here it is now. I went through the guide and as the blog post says it is still in BETA with a lot of "TO-DO" tags in there. In spite of that it is very much usable. Do visit it and post back your feedback to make it a much better guide.

Add-ons Blog » Blog Archive » Firefox Add-ons Developer Guide (beta release) - Calling all Add-on Developers!

Lets hope we will have more and more quality extensions now. :-)

Changing directory in which command window opens

For some reason my command prompt always opened up on my mapped network drive. That was probably because the network drive was set as my home or something. It was the sys-admins work about which I did not have any clue.

Every time I had to change to my local disk drive and then navigate to the directory I wanted. Also the network drive caused a lot of delay when it was not available, making my PC freeze at times and there by making me very irritated. So I finally decided to get rid of the network drive. But I couldn't and I had to settle for changing the default location in which the command window opens. The link below has all the details for doing the needful. As usual with windows its a registry value :-)

HKEY_CURRENT_USER \ Software \ Microsoft \ Command Processor
              |
              |
              |____ Autorun =

How to change the default startup directory for Command Prompt?

BTW, It mentions that this setting will render the "Open Command Window Here" XP PowerToy. Nothing like that happened to me, at least not yet. So you can be assured that both of these will work in harmony.

Happy Commanding. ;-)