Mozilla @ SJCE -- Contributing to Mozilla informally - Final semester student projects

Along with the attempt to introduce Mozilla as a formal elective at SJCE I have been working on getting some development work started informally too. The current final year students are enrolled under the central university - VTU and hence cannot be offered any new subject. However they are expected to do a project as part of the course completion. I thought of using this to get students to do their final semester project with Mozilla. Also Mozilla labs organizes this program named Design Challenge where enthusiasts -- students, teachers, academicians, software developers, etc, are invited to submit innovative ideas to make the Firefox web browser a better software and the Internet in general a better place. This has received tremendous participation from the student community across the globe. The best part about this program is that this is not just a competition. The selected students are trained on various Mozilla related technologies by the very Mozilla developers who are developing those technologies. After such mentoring the students can start contributing code to Mozilla and products based on it. And not to mention the wealth of knowledge they stand to gain and how much of positive influence it will have with prospective employers or while applying for higher studies. This suited very well for the final year and also the pre-final year students.

So when I visited my college last Saturday (14-Nov-2009) to meet my HoD and the coordinating lecturer, related to the Mozilla elective to be offered, I decided to talk to final and pre-final year students and motivate a few of them to participate in the upcoming design challenge (Nov - 09 to Mar - 10) and also take up Mozilla work for final year projects. I talked to the coordinating lecturer (Shri P M Shivamurthy) and asked to him make an announcement regarding this and have the students assembled in one of the classrooms or the seminar hall. After going to the college I got to know that the pre-final year (5th sem) students would not be available as they have their internal assessment tests starting from Monday. HoD suggested that I should stay back till Monday evening and address the 5th sem students on Monday after that day's tests. That was not possible for me and I decided to visit the college again on the next Saturday for that and I would talk to the final year (7th sem) students for now. As a result I decided to stick to the final year project only. Things were set up in the Network Lab and there were about 30 students.

Standing in front of them I blabbered a bunch of things about Mozilla, Open Source software, how engineers graduating are not really industry ready and the fact that they do not have any experience on working with real world applications with huge code base and contributions from a large number of developers and finally how participating in Mozilla would help them fill that gap. I also told them the vast amount of options that Mozilla provides in terms of technologies and that they could find some work or the other which lies in their area of interest. At the end I asked if anyone had any questions and as expected nobody did. Then on asking how many would be willing to try something like this I saw something like 3 to 4 half-hands rising up in the air. This was certainly not a good sign. So I started with the "motivational" speech. "This will really help you guys to be ahead of students from other colleges. You will be industry ready where as other will require a lot more training and mentoring. This is all HoD pre-approved... and on and on and on" for a few more minutes. That really did the trick. After this I had about 10 - 12 hands, full ones. Quite satisfied I told them to get my contact details from PMS sir and contact me for any queries. Till now I have received emails from 6 students (one of them representing a project group of 4 students. so 9 students actually). I have sent them a couple of links to start reading. None of them have responded after that. But I am still hopeful.

A little later I was talking to some of the students offline and I got to know some facts which would have been very useful to me in positioning this Mozilla project idea in a much stronger way.

1) Campus recruitment is pretty bad this time. Only 6 students in Computer Science have got job offers, compared to a daily average of 20 - 25 students a couple of years back. --- I could have talked about how open source development experience will help them with jobs. It did help me.
2) Project teams (generally of 3 to 4 students) have already been formed and a guide (a member of the faculty) has been assigned. This has two effects:
    a) Some teams have already been given the project work, which is a small part of the guide's doctorate thesis/research. The guide will now not happily approve of students under him/her pursuing a difference project.  -- We could talk to HoD and reason out with the guide. I could have told the students that such a thing is very much possible.

    b) In a project team of 4, generally one or two students are the smart ones and others will be banking on them for the project to be completed. I had told them that in Mozilla it is generally individual contribution or a team of 2 at the max. The teams, like those mentioned earlier, cannot be divided as the dependent folks will get into a problematic position. -- I could have told them that Mozilla does not bother if the work done by one student is present by 4 as a team work. So let the team enroll for a Mozilla project. Either all or a few in the team will work. If its all of them each one will have a bug assigned or the bug will be assigned to one guy with all of them working on it. If it is just one or two of them then there are no issues.

    c) On a related not to the above two points, some students told me that they would like to do a Mozilla project in addition to an already assigned final semester project. This really delighted me. But it also was, sort of, a matter of concern, as it appeared to me that people were desperate to do something like this with the hope that it will add a line to their resume and help them get a job. I might be wrong and I wish and hope I am. Students doing open source development just out of pure interest and not part of any course requirements is the best thing. But let me see what it turns out to be.

3) I did not make an announcement about the design challenge because the mentoring classes for that goes on from Dec-09 to Feb - 10 and these guys have their exams in the second  half of December. But I later got to know that no mentoring classes will be held from approximately 21st Dec to 4th Jan because of the holiday season in the US. So I talked to a smaller number guys, those who stayed back to talk to me, about the design challenge and am hoping to have 1 to 3 ideas being submitted.

I am going to use these points during my next visit, this coming Saturday.

Mozilla @ SJCE -- Modern teaching methods are still a stigma and considered unreliable.

In my effort to get open source development into main stream academics at my engineering college SJCE, I have been working with the Mozilla Education team for some time now. With our college getting the autonomous status and also with great help from the MozEdu folks like Prof David Humphrey and Frank Hecker and after considerable persuasion (more about that in a different post) I could get an elective named "Learning Open Source Software with Mozilla" added to the curriculum of the 6th semester students. We (me and the members of the college faculty) decided to roll out this elective in the year of 2010. The next step for me, I thought, was to get at least one lecturer trained with the curriculum and in general get that lecturer involved with Mozilla development and practices. Also with Prof David making the videos of his lecture sessions available on the internet, freely for everyone's use, my plan was to give the lecturer a head start (w.r.t students) so that with the beginning of the next semester he could start teaching the students those parts which he has already learnt. In the mean time he himself can continue his learning by going through David's lecture sessions and other resources that would be available on the internet. I would be visiting the college on alternate weekends. I thought, may be I could take a couple of hours of classes on Saturday for both students and also the lecturer(s). I could have used my presence to answer the queries that the students and the lecturers had, or at least I could point them in the right direction. Apart from these fortnightly visits I planned to be in touch with the college folks continuously on the internet -- email, irc, skype, etc. I would be actively involved the first time this subject is taught. After that the lecturer would be considerably capable and also the subsequent batch of students would have their seniors to help them out. At that point the program will not greatly depend on me and will be sort of self-sufficient with people directly talking to the Mozilla developers and the community in general. As a bonus the students who studied this subject would get to carry out their final year project work with Mozilla, either in terms of some feature implementation or certain bug fixes or any such task. It appeared like a sound plan and I had even decided that we would try to get about 15 students for the first time and gradually increase the number.

Last Saturday (14-Nov-2009) I went to meet the HoD and the lecturer who was coordinating this from the college side to get things started. The meeting was a big disappointment. Our HoD made it absolutely clear that the this elective will be offered only if 50% of the students (which translates to about 70 - 80 students) take up the elective. So the idea of first starting with a small number so that coordinating things on the internet will be easier and all that was just blown away. The reason for this is apparently because there are not enough class rooms to teach more than 2 groups of students from the same semester.!! It has to be a 50-50 division between two electives. So though there are 5 or 6 electives available to the students, they actually have to choose from just 2 of them, based on the majority and not interest.

Well my plan was not killed completely, yet, as the ideas in it were sort of the perfect solution for "the lack of classrooms" problem. I put forth the rest of my idea saying the remote teaching and a lot of learning on an individual basis (by reading up the resources on the internet and interacting with the Mozilla devs) would virtually the necessity of a full blown class room teaching always. But the HoD flatly rejected this idea and said that he understood what I was suggesting but there are rules saying classes must be conducted for a fixed number of hours for any subject offered and it has to be the traditional way. Also the idea of training the lecturers in a, sort of, asynchronous manner was also not acceptable. He would want a training session to be conducted - typically a week to a month long session, may be with a certification at the end of the session. Moreover currently I have one lecturer ready to take this up but department mandates at least 2 or 3. Now I have an additional task of motivating at leat two more faculty members. For this I have to prepare a write up explaining what the lectures stand to learn/gain by taking up this new thing. After that if any of them express their interest in taking this up, I will have to train them and probably it has to be in the traditional way - not sure yet.

Another problem is the pace. The next semester will be starting some time in Feb or Mar 2010 and my HoD keeps saying "Lets go slowly at first and see. If not in 2010 we will offer this in 2011" !!.. :-( . I hope we can get this thing started in 2010 itself. Another year of idle waiting might just terminate the interest that I currently have.

All in all, the wall between open source and my college is appearing to be more and more thick. I intend to meet the HoD the coming Saturday again and try to convince him to give his approval for the "internet based learning" approach. Lets see how it works out.

Apart from this I talked to a bunch of final year students about carrying out their final year project with Mozilla and also about participation the upcoming design challenge. More about that in another post.

Mozilla Developer Network (MDN) survey -- My inputs

I just finished the MDN survey and here is what I said in that last box which was put there for the people like us to pen down our rants. ;-)

• Project documentation needs improvement. It has improved and is improving, but a lot still needs to be done specifically about the oldest lines of code.
• I hear from some of the core developers that there are lots of hacks which make the code not entirely predictable. These need to be removed and replaced by proper, reliable code. Again the cleaning is going on, am just saying that it is really important so that there is some sort of SLA based on which people can develop applications.
• Consolidation of the content on MDC and MozEdu so that we can have a "The Mozilla Book", which any beginner can go through and dive into Mozilla related development -- either the platform or the browser or the add-ons or anything.
• Finally, making various Mozilla components available in the form of easily pluggable library modules and step by step guides telling us how to use them.

I do not know if any of this is useful to anyone else in the community, but for me, these appeared to be very important based on my association with Moziila for about 2.5 years now.

Incrementally building Mozilla/Firefox

Mozilla code base is really huge and has variety of files which are built in a variety of ways. It was sort of always confusing for me to figure out where all I should run make after I change any of the files. I generally asked on the IRC and someone just told me where to run make.

Today it was the same thing. But I also thought I would as well learn the logic to decide for myself the next time. Here is the chat transcript of NeilAway answering these questions.

The MDC page (https://developer.mozilla.org/en/Incremental_Build) has almost the same content for the native code. Neil here explains it for all the types of files involved.

Also to add to the following things running : "make check" from the objdir will run the automated tests.

• For xul/js/css/xbl it usually suffices to find the jar.mn (it may be in an ancestor folder) and make realchrome in the corresponding objdir
• For idl you're often looking at a full rebuild, depending on how widely it's used
• For .cpp and .h you obviously have to make in the folder itself, and then look for a corresponding build folder
• Except for uriloader where you use docshell/build and content, dom, editor and view use layout/build
• If you're building libxul or static then this is all wrong
• You don't look for a build folder, I think for libxul you build in toolkit/library and for static you build in browser/app

Changing the start up directory of command prompt -- The safe and simple way

By default, the command prompt window (cmd.exe) starts in a particular directory which depends on quite a few factors. Specifically, the env varibales %HOMEDIR% and %HOMEPATH% matter the most or these are the ones which ultimately decide the location. There are some registry values also, but they are not in the game by default. This leads to the command window to start up in something like C:\Documents and Settings\<username>\ which is not particularly useful as this location is rarely used for anything useful. It can get worse. In a corporate environment it might so happen, more often than not, that your home directory is set to a shared network drive using a group policy and the command prompt starts in that remove drive location.. ! The pain can be aggravated if you are on VPN or something similar.

I personally feel that command prompt, which is mainly programmers tool, should not start in %HOMEDIR% as there are rarely (mostly never) any programming related files are kept. Anyways, there are many ways to solve this problem.

The first of them and the most dangerous is fiddling with the registry. It is mentioned here : http://windowsxp.mvps.org/autoruncmd.htm
The problem with this is that it will screw up make based build environments which spawn multiple child shells (aka command prompts), because the command prompt will start in this changed default location instead of the location where the make had to run.

The next is fairly simple and also elegant. You can create a shortcut to the main exe (C:\Windows\system32\cmd.exe) and in the properties of the shortcut you can provide the directory to start in. This good for mouse users. However for those (most programmers) who start command prompt from the run dialog (by typing cmd) this will fail.

The third solution is to deal with the previous solutions shortcoming. You can just create a batch fail in any of the locations where the run dialog looks into. I prefer C:\Windows\systme32\. Put the following line in the batch file: C:\Windows\system32\cmd.exe /K "cd <path-to-dir>" . Just replace the <path-to-dir> with the path where you want the command to start. I generally put it as C:\.
This will start a command window and execute cd <path-to-folder> and will stay for further inputs. I have named this batch file as sh.bat (obviously to get a linux feel :P ). So now when I press the Windows + R key and type sh, I get the command prompt started in C:\.
Done.
And yes, this is totally safe and will not affect any other application using cmd.exe. :)

OpenSSL base64 filter BIO needs an EOL and memory BIO needs to know about EOF

I recently started working with the OpenSSL library to do some https stuff (sort of obviously). OpenSSL apart from having an implementation for the SSL encryption part, it also nifty algorithms for certificate handling and more importantly an abstract I/O layer implementation called BIO which probably stands for Basic I/O or Buffered I/O or something else. I do not know, I could not find it. Nevertheless, the items of interest here are the BIO_f_base64() -- The base64 encode/decode filter BIO and the BIO_s_mem() -- The memory BIO, which can hold data in a memory buffer.

The BIO man page (or its online version present here: http://www.openssl.org/docs/crypto/bio.html) give a nice introduction. For now just consider BIOs as black boxes from which you can read or write data. If the BIO is a filter BIO then the data will be processed whenever you read or write to it.

The name, BIO_f_base64, says it all about the functionality of this BIO. If you read from this BIO, then whatever data is being read is first base64 decoded and given to you. OTOH, if you write something to this BIO it will be base64 encoded and then written to the destination. These BIOs can be arranged in the form of chains to do a series of processing on the data that you are reading or writing, all by just a single call to read() or write(). Its all abstracted. Saves a lot of time.

I was trying to decode some base64 encoded data which I had in a buffer, a char [] to be precise. So if you read up about the BIOs it becomes obvious that you first have to create a memory BIO, which will hold the actual encoded data. Write the encoded data to the memory BIO. Then you chain that memory BIO with a base64 BIO and read from that chain. Any data that you read from the chain will actually come from the memory BIO, but before it reaches you it passes through the base64 BIO. So essentially you are reading from the base64 BIO. As mentioned in the earlier paragraph, when you read from a base64 BIO it decodes the data and gives it you. So the base64 encoded data present in the memory BIO is decoded and presented to you. That's it. base64 decoding is done in one simple read() call !

But there is small catch here. For some reason, which I have yet partially understood, base64 requires that the data it is handling be terminated with a new-line character always. If the data does have any newline character, meaning all your data is present in a single line then you have to explicitly tell that to the BIO by setting the appropriate flag. Here is what the man page says:

The flag BIO_FLAGS_BASE64_NO_NL can be set with BIO_set_flags() to encode the data all on one line or expect the data to be all on one line.

That's about the base64's EOL. Now the other BIO involved here,the memory BIO, is also an interesting guy. When the data it has gets over, it doesn't say "Hey, its over, stop it!". Instead it says "Dude, you got to wait for some more data to arrive. Hang on and keep trying". !!! This is very much suitable, probably when you using the BIO like a PIPE, where you keep pumping data from one end by acquiring it from somewhere and some other guy consumes that data. But in a situation like mine where the data is all fixed I simply want it to tell that the data is all over and I need to stop it. To do this again I will have to explicitly set an appropriate flag and here is what the man page says:

BIO_set_mem_eof_return() sets the behaviour of memory BIO b when it is empty. If the v is zero then an empty memory BIO will
return EOF (that is it will return zero and BIO_should_retry(b) will be false. If v is non zero then it will return v when it
is empty and it will set the read retry flag (that is BIO_read_retry(b) is true). To avoid ambiguity with a normal positive
return value v should be set to a negative value, typically -1.

And this same thing is explained very well here: http://www.openssl.org/support/faq.html#PROG15.

I thank Dr. Stephen N Herson of the OpenSSL project for helping me out in understanding this. Here is the mailing list posting that taught me this thing : http://groups.google.com/group/mailing.openssl.users/browse_thread/thread/f0fc310c1bc6ec65#

Happy BIOing. :-)

Should Indian farmer think of his family's hunger or the country's?

This is the type of condition the typical Indian farmer currently is in. When I say typical Indian farmer, I am referring to those lean hard working people who sweat out the whole day and yet just have a hand to mouth existence, that too if they are lucky. I do not have to substantiate the last part of my previous statement as the state of our farmer is well known. No matter how hard the farmer tries, how hard he struggles, the first person to gain from that is the middle-man, with whom the farmers are compelled to park their annual produce. The reason for that is another story in itself, probably some other time. Anyways, focusing on the topic of this post, let me try and tell you why I think farmers today are faced with this question.

We all know Indian population is growing faster than any other thing in the country with an unbeatable consistency. This directly corresponds to heavily increased demand & consumption of food. Good news is that the country's agriculture is able to meet this rising demand pretty well, thanks to the technology. With improved variety of seeds and fertilizers, increased proliferation of irrigation, some education to farmers, better knowledge of weather behavior and so and so forth, the throughput per acre of land has increased, in almost every variety of cultivation. So the farmer now has more goods to sell every year and get more revenue. But sadly this is has not transformed into increased income to the farmer yet because of the ever increasing input costs. The seeds have improved, but they cost more, a lot more. The fertilizers are better, they reduce the time to grow, but they really push the farmer deep into the pit of debt. Its the same story with irrigation equipments, modern cultivation methodologies, etc.. (Government is trying various policies and subsidies, but still the costs are higher and have increased substantially). As a result the farmer is still poor and leads a hand to mouth existence. Not only that, he now has to invest a higher amount initially (which most farmes do by taking a loan from the middle-man) and hence is taking a bigger risk, because in case of a failed crop he has a much bigger debt to repay. The trade off / inequality / tragedy is clear here. The farmer is the one who is providing the country's ever increasing population with the food supply, by putting on more risk on himself and yet he earns the same as he has been earning. All that he can think of doing is to have proper meals for himself and his family, in other words - bare survival. If he starts thinking of proper education for his children, or a proper house in place of the generations old dilapidated building, he invariably has to go for a loan again. In such a situation it is but natural for the farmer to think that whether the extra risk that he is taking every year is really required? If we put ourselves in his position and think about the question the answer stands out clearly as NO. And fortunately or unfortunately this is what the Indian farmer is thinking now.

He is thinking of securing his home first. He is not thinking about the national crisis it may cause and he can't be expected to think about that either. But the government has started to see this. It also has realized that if the farmer goes his way the country will soon be facing a huge food crisis and huge imports will be inevitable. I am not sure if imports is the right word here, it might as well become begging. With us (our country) in a such needy state all our policies will be influenced either by the one who will be lending us the food or the west (Europe and US) in general. That might bring a stop to the magnificent growth picture being painted everywhere in the country now. The government knows this all and it obviously can't let this happen because anything like this will instantaneously put them out of voters' favor. So the government has already started taking measures to counter this. Currently it is not very aggressive in its approaches. It is trying to convince the farmers to grow more and work towards increasing the yield instead of just thinking of survival. Though it does not appear plausible, we might just see a rule mandating a minimum yield/unit of land coming from the government. Or to be farmer friendly we might just see government going in for huge amounts of subsidies. I do not know how the government will handle this but I think the best way is to make arrangements for farmers to get a fair value for their produce, which means getting rid of middle-men. That is a very hard thing as every activity of the farmer is linked to the middle-man and hence there has to be an alternate system in place to replace the middle-men. Lets hope that the government will think of something that will feed both the country and the farmer well and equally.

Return value from system() is not reliable

Recently I happen to use this nifty utility available on the Linux platform to perform some maintenance work, a bit of housekeeping, before my application starts up. The need for this is not really important. Essentially, the environment needed to be configured for my application to start up and start functioning properly. This, ideally, should have been done by a configuration/setup script, probably written in Python or a similar programming language, which are meant for such tasks. But unfortunately I did have that privilege and I had to do every bit of it from my C-program. The task were simple and very regular, like clearing a workspace directory, setting appropriate permissions and the like. The initial thought was to use the dirent family of functions aka OS system calls to read the filesystem and modify it programatically. But doing that whole thing was a big PITA. Hence I took the easy route and simply used the system() function, which will execute shell commands.

The problem with this easy approach is that tracking the operation's success is really hard. The system() man page says that the function will return the actual value returned by the command that we pass to it to be executed. But sadly this is not how things are, at least on the Linux 2.6 machine on which I am developing and running my code. The return value from this system() function is highly unreliable. In fact the man page also puts it in there, but in a very subtle way. Here is a quote from the man page:

     The <b>system</b>() function returns the exit status of the shell as returned by
     <b><a href="http://www.manpagez.com/man/2/waitpid/">waitpid(2)</a></b>, or -1 if an error occurred when invoking <b><a href="http://www.manpagez.com/man/2/fork/">fork(2)</a></b> or
     <b><a href="http://www.manpagez.com/man/2/waitpid/">waitpid(2)</a></b>.  A return value of 127 means the execution of the shell
     failed.

I am not fully clear with the process semantics, but from my observations when I execute a command using this system() function the command would have executed successfully, as in, the corresponding operation would have been completed, and yet the return value would be -1, telling me that the execution of the command has failed somewhere but it does not tell me where. For example, if it is a command to clear a directory and create some other directory structure there, all these tasks would be completed. The old directory would be gone and the new ones created. I see that when I just navigate to that location from the command line, but system() would have returned -1. I initially was checking the return value to handle the failures and was taking some fail safe steps. But all that was happening even when it was all good. The logs repeatedly told me that the operations were failing where as it was all good actually.

The reason for this is not known to me. It probably lies in the quote from the man page that I have put above. As it says -1 can be returned for any of the errors, be it error from fork or wait. Now if the error was from wait, which I am guessing is the case, it makes a little sense. I think the fork and exec went through properly and the command performed the required action without any error. But later the wait failed and the system() could not collect the exit status and hence returned -1. This is the only thing that I can think of. Nevertheless bottom line is, Return value from system() is not reliable.

Getting the size of an already loaded page (from cache) in a Firefox extension.

Today this question came up in the IRC (moznet, #extdev). One of the add-on developers wanted to get the size of the page, either bytes or number of characters. The most obvious thing that came to my mind was progress listeners for definitive answers or the content length from the channel for not so critical scenario. But then he said he wants it for an already loaded page. And he further said that the information is already there somewhere as it is shown by the Page Info dialog (Right Click on a web page and select View Page Info). He was indeed right. Somebody in the code is already going through the trouble of calculating the data size and we can just re-use that. And I immediately started the quest to find that out.

As usual to figure out any browser component I opened up DOM Inspector. That tool is improving, which was against my earlier perception (Sorry Shawn Wilsher), though the highlighting part is still screwed up. Nevertheless, locating that particular label "Size" and the textbox in front of it containing the value was not difficult at all. I got the "id" of the textbox containing the size value. (Its "sizetext" :) ).

Next it was MXR (http://mxr.moziila.org/) in action. I did a text search for the id and got a bunch of results, one of which was pageInfo.js with this entry : line 489 -- setItemValue("sizetext", sizeText); . It is here. The very line made it apparent that it is the place where the value is being set and hence it is the place from where I can get to know how the value is being calculated.

Once I saw the code it was very clear and straight forward and pretty simple also. We have the URL. From the URL we get the cache entry for that URL. (Every cache entry has a key and that key is the URL - so neat). We try to get the cache entry from the HTTP Session first and if that fails we try FTP Session. The cache entry has the size as an attribute on itself, so its just getting that attribute value. DONE.

I am not sure how this will behave if we have disabled every type of cache. AFAIK, there will still be some in-memory cache as long as the page is still loaded. Probably good enough.

That was the end of a small but interesting quest. :-)

Getting Open Source/Mozilla in my college - SJCE - Part I

Earlier I had written about my first Mozilla Education Status Call in which I mentioned my interest to bring Open source software in general and Mozilla in specific to my college, SJCE. Well, good news, it did not stop at that blog post. Actually speaking it had not started with that blog post either. It has been a long standing wish of mine, even from my college days when I participated in the Google Summer of Code 2007. (More about  it here). Back then there were a lot of short-comings, both from my side and the institution's side to actually make this idea into reality. Nevertheless, past is past and no point in brooding about it now. The good thing is that now both I and my institution have overcome our short-comings and we have started working towards making that idea into a reality.

Now for some background aka story telling (which I like the most :) )

As stated earlier nothing happened about this idea when I was in my college. Then after passing out of the college and having worked in the industry for about 12-18 months it hit me, very hard, that I did not learn a lot of things during my life as an engineering student which otherwise would have helped me a lot in my professional life. Also I learnt that I was not competent enough as an engineering graduate as compared to some of my foreign counterparts and also those from some of the "famed premier" engineering institutions of the country. It was not just one thing or two, but I saw differences in many aspects, both theoretical and practical. Gradually it occurred to me that these two aspects are inter-related. Since we did not have proper practical experience and exposure to real world software development we never really appreciated the basic theoretical concepts of computer science, which formed our regular syllabus. Note that here I am saying "we" and not "I". This part of the story talks about the state of most of my classmates and that is the worst part. Nevertheless, the moral of the story is the good old philosophy of teaching that the theory and practice should go hand in hand.

Now there is another part to this story. Before I start with it let me tell you that whatever I am putting here is based on what I have perceived. I may be wrong, but I personally don't think so. And this is absolutely not about boasting about myself. So the other part of the story is that, my association with Open Source development communities, Mozilla to be specific, has greatly helped me in my professional life. I am not going to give examples, but it has really really helped me a lot. Also it has sort of put me ahead of several other capable classmates and most juniors of mine (with the difference being considerably more in the case of juniors). The only differentiating factor between me and them was my exposure to developing a real world application, Mozilla Firefox, and the various lessons that I have learnt by being a part of the global developer community. I am also certain that I could have been a much better computer engineer if I had started working with Mozilla at a much earlier stage, say 2nd year or early 3rd year of engineering and had dedicated more time to it. I still continue to learn a lot of general computer science and software development concepts (concepts not specific to Mozilla development) even now whenever I try to fix a Mozilla bug or even when I try to answer any query on the IRC, many a times even when I just observe few people conversing on the IRC.

Ok, enough of story telling. Now is the part for moral of the story. Here is what I inferred from these experiences:

1) Engineering students, specifically Computer Science engineering students, must get exposure to real world engineering (aka application development) to understand and appreciate the theoretical concepts they learn.
2) Open source software development communities provide a suitable environment for students to work with real world applications. Suitable in terms of - opportunities, cost, mentoring and certainly a few more good things also.

With these two points, it was clear to me that we badly needed open source education/exposure for students in my college. I knew that once this happened the possibilities were endless. Every time I heard/read about some of the classic Free Software implementations done at Universities abroad, I thought that our college can at least have several continuous contributors to currently existing open source projects, if not have creators of some totally new world class software projects. We could be having several different groups of students working of different types of software which operate at various levels (which translates to contributing to different open source software). Then they all could be interacting to help each other in troubleshooting problems. I thought of scenarios/discussions like this happening in the hostel corridors:

Student_1 and Student_2 are working on the Mozilla Download Manager (and here goes the conversation)

Student_1 : Hey, I want to test my new implementation for Mozilla Download Manager for HTTPS downloads. I am unable to configure my test server for HTTPS. You got any idea?
Student_2 : No dude, never done any server side stuff. Lets ask Student_3 from the Apache team.

Then Student_3 comes and sets up Apache for HTTPS within minutes (because that's day-to-day kind of stuff for him) and Student_1 continues testing his new implementation.

After some time:

Student_1 : Oh man, SSL handshake is taking too much time. I need to talk to Student_4, he knows the SSL library code base.
Student_2 : Yeah, I talked about that to Student_4. He is coming up with a patch to reduce the handshake time. It will probably be ready by tomorrow, I guess. Apparently it was a race condition causing the delays.

And so on.

Something like this is really possible. In fact many things much bigger than this are possible. But only if our students start working with and for open source communities.

So this set of thoughts made me work towards getting Open Source into my college. Now that's the background and the story. In the next part I will write about the first set of steps taken towards this, how many of them worked and how many were dead even before they started. And just FYI, the next post too will have some story telling (Obviously since this is just a record of my experiences and my (our) actions).

Vim -- Restoring cursor from previous session

I am sure every Vim user needs this. Its just so frustrating to open a source file to see the #includes (the first few lines) in the file when we actually will be editing many hundred lines later. Today specially I was juggling with several source files, adding something to the .h file and then come back, do something in the .cpp file and then again go to some .c file and so on. Every time I opened a file the cursor was at the first line and every time invariably I had to search for the function I was editing and cycle through the matches to reach it. So I set on a "Search Mission" - A mission to search for the appropriate .vimrc settings to make Vim remember the cursor position from the previous session.

I got a lot of links. In fact there is a separate Vim Tip - Vim Tip #80 for this. But it has so many code lines and I was a little wary to put all that in my .vimrc file. Continued search revealed me a simpler way. Just two lines solution and it is here : Some Mr.Gopinath's .vimrc file. Its very big, but the lines concerning me are:

" VimTip 80: Restore cursor to file position in previous editing session
" for unix/linux/solaris
set viminfo='10,\"100,:20,%,n~/.viminfo

" only for windows [give some path to store the line number info]
"set viminfo='10,\"100,:20,%,nc:\\Winnt\\_viminfo
au BufReadPost * if line("'\"") > 1 && line("'\"") <= line("$") | exe "normal! g`\"" | endif

Looks like he also picked this up from the same Vim Tip #80 but was smart enough to take out only the necessary part. Nevertheless, this works for me. Thank you Mr. Gopinath.

Happy Vimming :-)

Edit:

I read the Vim Tip #80 again and it made more sense this time. I picked up the last line for a user comment. Now the cursor is put back on the same column too.

Recursively rename files in windows

My friend Prasanna B P asked me to solve a bunch of problems that he was facing with his computer. One of them was fairly common and there was a very easy and straight forward brute force solution. But coming up with a "smart" was really interesting.

Essentially some malware had renamed all the movie files that he had to carry a .jpg extension. His initial solution was to associate those jpg files with a movie player like Mplayer123 or VLC and the file would be invariably played without any regard to the extension. But this made the genuined jpg pictures to be opened with the player.

The obvious solution was to rename all the files and change the extension. Manually doing it from the GUI is the brute force idea that I earlier mentioned. And it is a totally crappy one. Next I can rename all files in a directory from the command line. But here the movie files were in directories of their own and hence I would have to move to each directory manually and run the rename command. This makes it as good as the GUI approach. In fact the extra effort of typing the commands might make it worse.

The I searched the internet a little and got to know that the windows command shell supports a "FOR" statement which can be used to recursively traverse directories, amongst many other things it provides. Using that I found this command:

FOR /R %x IN (*.jpg) DO ren "%x" *.avi

from this website : http://stackoverflow.com/questions/210413/command-line-recursive-renamemove-in-windows

People used to Linux Shell scripting might think of this as a wierd syntax, but yeah thats the windows choice.

It was good to do some Windows Shell scripting too. :-)

Profiling (timing) the firefox build process

Its been nearly 2 years since I am building Mozilla Firefox myself on my machines - various machines of varying capacity.

Initially it was a desktop having Intel Pentium 4, Single Core (obviously), 2.6Ghz, 256MB RAM, running Slackware 11. It probably used to take about 1.5 to 2 hours (I do not remember it now). I never profiled that at that time. Getting a mozilla build itself was a big achievement for me.

After that it was another desktop having Intel Core 2 Duo, Dual Core (obviously, again), 2.4 GHz, 2GB RAM, running Windows XP. This generally took about 45 minutes. AFAI Remember, I had several other programs running when firefox was building.

And this discussion of amount of time taken to build firefox came up a few times in IRC and I myself had this wish to time the build process. Off late, that is from about last week this wish became very strong and today finally I did time it, that too on two machines, my laptop and my desktop. This blog post is the result of the these two profiling tasks. Here are the results.

Note: By profiling I did not do any complicated or intricate. I just used the "time" utility which tells how much time the command takes.

1) On my laptop. --- Build was the only application running apart from the services.
Specs:
IBM Thinkpad T60p. (The one which heats up a lot)
Intel Centrino Duo, T2600 @ 2.16GHz, (Dual Core)
2GB RAM
Windows XP Pro SP2.

The results shown by the "time" utility are:

real 36m33.476s
user 4m17.776s
sys 4m36.271s

The build was done in the MingW shell that the mozilla build system provides. I am not sure to what extent these are reliable, but the real time is pretty much acceptable.

Edit:

Today I ran the build command from the history and hence the 'time" prefix got in automatically and the build was timed again. Surprisingly today's times are way away from the last one. Here are the times:

real 74m26.484s
user 9m53.015s
sys 8m18.365s

Well this time a lot of other apps were running. Firefox (2 instances), Chatzilla, Outlook, Several command windows, Komodo Edit (which again is like another firefox), notepad and a couple of explorer windows. So the time being doubled is not a surprise. Guess this just gives a perspective. :)

2) On my desktop --- Apart from the build process, FF with Chatzilla was running and several instances of bash (Terminals) were running.
Specs:
DELL Optiplex 755 (Sleak, powerful, sometimes fragile)
Intel Core 2 Quad, Q6600 @ 2.4Ghz
4GB RAM
Ubuntu, Gutsy Gibbon

And the results are:

real 20m40.497s
user 19m4.320s
sys 1m17.885s


The two sets of times are a little confusing. But real is all that matters as that is how long I have to wait for the build to be ready.

If you are planning a build then this info might help you plan things accordingly.

Happy building.

Camouflage in full action -- Totally mind boggling

Even the makers of the bond movies did not think of anything as crazy as this. This was a real stunner for me.

Making firefox use a little lesser memory

A lot people tell me that firefox uses a lot of memory and it grows like anything when used for a long duration. From the discussions developers and some blog posts/forums it appears the prior to FF3 it was memory leaks which formed a major part of this. Apart from that one of the important features "bfcache - Back/Forward Cache" which, arguably, makes back and forward navigation very fast, uses a lot of memory. In on the pages I read apparently it can be up to 4MB per page. This cache essentially keeps the parsed HTML of the mark-up in the memory along with the Javascript state. Sometimes, or more like most of the times, this can very taxing. You might be the kind of person who does not really uses the Back-Forward navigation and in case you do it you are OK with a slight delay (which may not percievable with a good conenction). If indeed you are that kind then disabling this "bfcache" will probably make your firefox eat lesser memory.

The setting/preference that controls this behavior is : browser.sessionhistory.max_total_viewers

By default it is -1, which is no limit. Setting this to 0 (zero) will disable this cache completely. Essentially you are telling Firefox to not to store the state of any document in the memory. If you want to have this feature with a saner limit you can set it to an integer representing the number of pages you want to be saved in memory.

Like any other pref not exposed through the Options dialog this has to be edited by visiting the page about:config. Open this in a tab and key in (or copy and paste) the preference string. Double click it to edit.

May be you can now lose lesser hair as your other applications will run smoothly. :P

Hari Om.

April 2009 - Appraisal

This was making rounds as email forward. Just could not laughing for a loooooong time..

You and Your BOSS

Undo "Always display images" switch in Gmail

Gmail has this nifty and very useful feature to avoid mostly annoying images being displayed in our incoming emails. Whenever we get an email containing an embedded image, it is not displayed by default. Instead there appear two links at the top of the email saying:

"Display images below"
"Always display images from <this sender's email>"

The first one is a one time thing and the second one is a setting for that particular email id.

I once clicked on the second link accidentally and had unwanted images cluttering my email view. I looked for this thing to be turned off in the settings page, but found nothing. I had a very hard time undoing this accidental setting. Finally I discovered the solution also accidentally. The switch to undo this setting is "hidden" under the "Show Details" link for the email header.

The right end of the very first line of the email, which contains the sender's name/email id", has this "Show Details" link. This shows the email header with several details. At the end is the link saying something like this:

"Do not display images from this sender".

Just use this switch to go back to the good old times of emails without images. :)

My first Mozilla Education Status call

Mozilla is the only open source community which I have understood a little and also to which I have contributed a little. I always wanted to make my college a sort of Mozilla hub with several contributors and many feature developments happening out of my college. It actually started with an idea of a "Complete Open Source Hub", but it got reduced to just "Mozilla Hub" (either because of my laziness or because of lack of resources). Anyways I did not take any proactive steps towards that wish of mine, until recently when our college got the "Autonomous" status. That is when I realized that bringing open source software development in the course mainstream has become a lot easier as the power to form the syllabus and conduct the tests and examination rests with my college itself and not the University.

Just when I thought of doing something tangible, Mozilla came up with their "Mozilla in Education" program. This is a brilliant idea to drive open source into the student community and also give the students opportunities to work on real world applications. I was impressed by this program the very moment I read about it. I decided to present this idea to my HOD at the college and get him to start working offering Mozilla education to students in my college. I started reading more about it and today I also attended my first Mozilla Education status meeting conference call, which happens every week. I got a wealth of information.

This week we had Pascal F presenting to us about the "Design Challenge" program organized by the Mozilla Labs. He talked to us about the way in which they organized this program. It was all very inspiring. They had nearly 30-35 people from different parts of the world (literally). In this contest the students initially submitted mock-ups of their ideas, nearly 40 of them. Amongst those, 30 fully completed mock-ups were chosen for the second level, in which the students got mentoring by some the well known names in the Mozilla community. The mentoring consists of 10 Webinar sessions conducted using WebEx, over a period of 3 weeks. This is going on currently and will end at the end of this month. These mentoring sessions aim at converting those mock-ups into working prototypes. At the end, the best prototypes are given honors.

During the presentation Pascal mentioned some interesting things. Of the students from various countries, it was the students from so called "2nd World Countries" (like Romania, India, Argentina, etc.. ) who showed a lot more interest than their US counterparts. There was tremendous enthusiasm in them where as the students from US expected -- in his own words -- "being entertained" and "spoon-fed". Though this is an alarming thing when viewed from a global perspective, I was personally very happy that Indian students, my fellow country-men, have shown such dedication. Pascal also mentioned that they were so interested that they were up in the middle of the nights for the webinars. All in all, I am even more motivated to bring open source in general and mozilla in particular to my college.

I decided to take this program as an example and present it to my HOD this Saturday and try and make him accept this and similar programs as official ones and the projects done in such programs be considered for the completion of the course goals.

Thank you Pascal, thank you Mozilla. Lets hope to have a Mozilla India Center, at least an unofficial one at my college SJCE. :-)

An official full-fledged Firefox Add-ons dev guide

Having done a little work on Mozilla Firefox and extensions for it I have always felt the lack of an official, step by step document for extension development. I do accept that there are a lot of resources available on the internet. Innumerable number of blogs and tutorials explaining the process step by step. But most of them become outdated with newer versions of Firefox and the authors are not really keen about updating the info. That is why an official tutorial or guide from Mozilla itself would be the right thing. It will make sure the contents in the guide are up-to-date and worth reading for developing an extension for the currently available version of Firefox.

There are documents for extension development on MDC like the various links present at: https://developer.mozilla.org/en/Extensions but it was either scattered or pretty brief in most places though it did touch most parts of extension development. Nevertheless a wholesome "official" guide was needed and here it is now. I went through the guide and as the blog post says it is still in BETA with a lot of "TO-DO" tags in there. In spite of that it is very much usable. Do visit it and post back your feedback to make it a much better guide.

Add-ons Blog » Blog Archive » Firefox Add-ons Developer Guide (beta release) - Calling all Add-on Developers!

Lets hope we will have more and more quality extensions now. :-)

Browsers are undergoing continuous innovation.

It was sometime since I blogged about anything. I have had several things on mind and many of them are presents as drafts. But this one really caught my attention and I felt I should put in my thoughts about this.

Until a couple of years, people rarely looked beyond Internet Explorer for their browsing experience, though they kept cursing it a lot. After that came the Mozilla Firefox web browser with its pack of addons allowing users to actually customize for their needs. They could actually make the browser do what they wanted it to do and not just set some options. The browser wars had started again. At least I started reading about browsers and started following up on things happening with these browsers. Opera and Safari and a few other, actually used, browsers were not really "news" as such.
This was dying off. Firefox had created a sizable chunk of user base and was pretty stable with it. Except for few traditional enhancements like memory optimizations, bug fixes, etc.. nothing big was happening. It was a lot silent.

Then came the next wave with Google announcing the release of its browser with a nice, easy to understand "comic" book. It came with a whole new paradigm for building browsers with the "process-per-tab" concept. It was really innovative. Though discussions about this happened in other browser communities also, Google Chrome was the first one to implement it. Also it boasted of its super-fast V8 JavaScript Engine and also the browser UI, which gave more screen real estate to content than chrome. This was a real big thing and made the browser wave go much higher than what it had ever been. With the Google branding a vast majority of internet users rushed to have a sneak peak at Google Chrome or may be try it or even keep it as their regular browser. There was a lot of noise about this and people indeed listened. Mozilla and IE people were not silent and did responded very well. Mozilla came up with its ultra-fast "Tracemonkey" JS engine, implementing the trace trees and there by making it much faster than V8. IE8 also has the "process-per-tab" and "private browsing" features first presented by Chrome. But like any other sound this too dampened a little after it was created and browsers were back in the silent phase doing traditional improvements and bug fixes. At least that's how I percieved the situation.

[Edit 06-Mar-09 : Shawn Wilsher suggest that Tracemonkey had appeared before Chrome made the pubic appearance. He certainly knows these things better than me and I believe he is right. But still I wanted to keep my original post as it is and instead I put this separate edit note. :) ]

But I was clearly, totally wrong. People have realized that internet is the place to be in the future and thats where a large part of our life will be. With browser being the main and central interface for people to use that internet, it makes a real good sense to make this browser as robust and reliable as possible. New things keep coming on the internet, both and bad, the latter being more often, and the browser has to keep up with all of it. What we thought yesterday as being a good design apporach might just look senseless tomorrow. There just seems no end to it and researchers appear all ready for it. I am coming across so many innovations happening in the browser domain. This article : Researchers Say Gazelle Browser Offers Better Security -- Campus Technology -- gives us an idea about how much effort the scientists are putting in making our internet lives better. The article is about Gazelle, but it also mentions about another experimental browser OP.

This browser Gazelle, uses the Trident rendering engine (used by IE) but builds upon a OS based process architecture where websites form the processes and they communicate by passing messages like IPC (Inter-process communication).

I am not yet sure how this will all work out. Probably the websites need to built with some intelligence so that it can do the so called IPC when it actually has to. But the evoltion in browsers is for sure. They will not be same as they are now. Several things are going on along the UI front. When these things mature and come togther you can have the scenes of your favourite sci-fi movie scenes right in front of you everyday.

Lets just hope these come soon enough for people like me to enjoy, not when I am all grey hair and toothless.

Hari Om.

Attitude -- Totally enviable

This is how knowledgeable you should be and also equally confident.

Changing directory in which command window opens

For some reason my command prompt always opened up on my mapped network drive. That was probably because the network drive was set as my home or something. It was the sys-admins work about which I did not have any clue.

Every time I had to change to my local disk drive and then navigate to the directory I wanted. Also the network drive caused a lot of delay when it was not available, making my PC freeze at times and there by making me very irritated. So I finally decided to get rid of the network drive. But I couldn't and I had to settle for changing the default location in which the command window opens. The link below has all the details for doing the needful. As usual with windows its a registry value :-)

HKEY_CURRENT_USER \ Software \ Microsoft \ Command Processor
              |
              |
              |____ Autorun =

How to change the default startup directory for Command Prompt?

BTW, It mentions that this setting will render the "Open Command Window Here" XP PowerToy. Nothing like that happened to me, at least not yet. So you can be assured that both of these will work in harmony.

Happy Commanding. ;-)

Image for a statusbar panel in a firefox extension

I was working on a firefox extension which, like many other extensions, was adding a small button to the status bar. Doing this is very clearly explained here in : this MDC tutorial

I followed the instructions properly but to my surprise I did not see the image on the status bar, but just a square box in the defualt chrome color. I tried forums/web search and the IRC too, but no help. Then I read the statusbarpanel reference page - statusbarpanel - MDC -- completely and it appears that for the image to show up the statusbarpanel has to be of a specific class called : "statusbarpanel-iconic". I just added that and it worked. :-)

Here is my code

<statusbar id="status-bar">
     <statusbarpanel id="graph_status_bar_panel"
              image="chrome://myext/skin/chart_status_bar.png"
              class="statusbarpanel-iconic" onclick="toggleGraphPanel(event)"
              tooltiptext="View Graph Panel" />
</statusbar>

Pictorial representation of blog

I came across this post by Shawn (sdwilsh) and instantly wanted to know what I am concentrating on in my blogs. And here is what my blog has to say:

Here is the link where you can get one for your blog : http://www.wordle.net/

Happy picturing. :-)

Documentation appears by itself -- from the code.

Beautifully Documented Code at Toolness

I am slightly into web and web related development and as part of that I come across various JavaScript (JS) libraries which  claim to make my life a lot easier and many of them actually do that. But several times there is a big hurdle that I have to cross before actually things become easy. Its understanding how to use that library. This sometimes turns out to be a pain, often bigger than the one if I had done everything by myself without any external library.

This is probably all going to change. The lack of documentation might just vanish and all the necessary documentation might start appearing from the code itself. And the steps for building this, collating things and publishing on website are taken care of. You just need to annotate your code with the right tags and the JavaScript linked at the top (Written by Atul) will do the documentation generation and will also display it alongside code.

This is particularly useful for libraries providing various APIs as the users can see a function in the raw code and understand its meaning and purpose and the arguments it expects and its return value. All in once place at one time.

This is really cool. Go check out the tool and give the world a better library to use :-)

Happy (auto)Documenting

The best blonde joke ever

I never really blog about jokes, and never for sure on my tech blog. But this one is the best ever blonde joke. Its simple awesome.