$PATH for the pathless

With the rise in popularity of scripting languages and their adoption in the industry a lot of first time professional programmers (a.k.a fresh graduates) are being exposed to the Linux command line environment and some times Linux as a system itself might be new to them. So they tend to have difficulty in understanding how a program is actually being executed and/or what all is needed to make a program run. This coupled with the magic spewed around by certain tools like rvm / chruby for Ruby and nvm for Node.js (and their ilk) are just adding to the confusion of how things are executed in the first place. I am not at all saying that such tools are bad. I myself use them and they are very helpful. But if a person is using them without getting the basics right, he/she is going to have a mighty problem wrapping their head around how things are running..!

In this context, I have been explaining what the $PATH means and what is its purpose a few times of late. So instead of repeating myself every time I thought I would just document what I know about $PATH to help the pathless get on the right PATH (or the PATH OF RIGHTEOUSNESS if you will).

GEEK NOTE : This post is aimed at absolute newbies, folks who just got started with something like Ubuntu and are fiddling around with some of the cool new technologies like Node.js / Ruby / . So this post will  not go into the depths nor will it try to be GEEK ACCURATE, i.e. it will not deal with things like POSIX compliance or the behaviour on non-Linux systems, etc. For that matter this will to a large extent be specific to Ubuntu and most likely Bash. We will also not try to distinguish between login and non-login interactive shells. As the screenshots shown below indicate, the assumed environment is running the "Terminal" application in Ubuntu.

So, lets start with the basics and a few assumptions.

Assumptions :

1. We are primarily interested in understanding how programs are run in the Linux command line environment.
2. We will restrict ourselves to the "Terminal" program of the Ubuntu system as the environment. However all / most of the following might be applicable to all such similar programs on different flavours of Linux.

Basics :

Whenever you launch the "Terminal" program what you are doing is actually launching a "Shell". Like with any type of software in the Linux world, choices are abundant and there are a lot of shell programs. However the "Terminal" program in Ubuntu (as of this writing) uses the "Bash" shell and that is what we will stick to. So on launching "Terminal" you are effectively launching the bash shell.

The "Terminal" application in Ubuntu 16.04

What is a Shell?

In the simplest terms, a Shell is a program which helps you run several other programs. It is like this eternal waiter who just waits for you to tell him what program to run and when you give him a program to run, it just runs it, tells you what was the result of running that program and goes back to waiting. Broadly speaking this is all that the shell does : Wait - Run a program - Show results - Wait. Wash-Rinse-Repeat.

If you have used the Linux command line (a.k.a the shell), you might be thinking that the shell certainly does a lot more than this. You might say "I run so many commands on the shell and it allows me to do a variety of things like search of files, list the directories, check the date etc etc. What about all those functionalities??!". Well, almost all of those actions are nothing but the shell running some program or the other. Almost every "command" that you have run are independent programs provided with the Linux system by default. Whenever you "run a command", you are just telling the shell to search for that program on the system and run it. The standard commands that most Linux users are familiar with are stored in certain predefined standard directories / locations. On Ubuntu those locations are "/bin" or "/usr/bin". While "/bin" has the core system commands, "/usr/bin" has the commands added by other softwares that are installed on the system.

A listing of the /bin directory showing system program executable files. Notice the programs corresponding to the most commonly used commands like "cp", "ls", "mkdir" etc

While these are what could be called as "standard system programs", the shell is not limited to these. The shell can run just about any program (technically, any executable file) on the system. All you have to do specify the full path to the location of that file on the command line and the shell will execute it, reporting the result of that execution. So if you have a file in a deep directory structure as shown below :

The full path to "my_program" will be /home/srirang/work/path_learning/custom_programs/my_program

When you have such a setup, to run the file "my_program" you just have to type the full path to it and hit enter. Shell will take the file at that location and run it.

/home/srirang/work/path_learning/custom_programs/my_program ↵

There might be two questions now in your head :

1. For the standard commands I never specified the full path. I just specified the program name and shell ran it without any issues. How did that happen?
2. If I have to specify the full path for each and every time, it seems a mighty tedious way, especially for programs that I run very often. Is there no easier way?

The answer for both of the these  is the "PATH" environment variable.

You need not specify the full path to the executable program all the time. When you specify only the program name, shell has a mechanism to search for that program on the system and run it. Searching for a program in the entire system every time is massively inefficient. So the shell has a mechanism for us to tell the specific locations where the shell should search for a program whenever we do not specify the full path. The way to specify this is by putting all these locations in an environment variable called the "PATH".

Whenever a program name is specified with the full path, shell will take the directory locations specified in the "PATH" environment variable one by one and search for the program in each of those locations sequentially. So if you have one or more custom programs that you would be running frequently, then it is best to add the directory locations of those programs to the PATH variable.

Environment variables : Brief introduction :

As mentioned earlier PATH is an environment variable. This post will not go into the details of what environment variables are. For the sake of this discussion here are a few points to remember :

1. Environment variables are like the settings for the shell. The values of the environment variables are used by the shell to decide how it should act.
2. By convention all environment variables are specified using upper case alphabets, numbers and underscore.
3. You can have as many variables as you want, although the shell will use only the ones it has specified.
4. You can set the value of a environment variable like this :
   
   VARIABLE_NAME=value
   
   Examples:
   MY_VAR=1
   RUBY_VERSION=2.0.0
   LANG=en_IN
5. Very often to make sure that the value of the environment variable is persisted across multiple programs (and not limit it to a single program or command), the "export" keyword is used while setting the value of the variable. Examples :
   
   export MY_VAR=1
   export RUBY_VERSION=2.0.0
   export LANG=en_IN
6. You can read the value of a variable by prepending the $ symbol before the variable name like this :
   
   $VARIABLE_NAME
7. The value of a variable can be used to set the value of another variable by using the same $ symbol. Example :
   
   RUBY_DIRECTORY=/usr/lib/$RUBY_VERSION/
8. If you want to see the value of an environment variable use the "echo" command. See the image below :

Managing the PATH variable :

Here are a few characteristics of the PATH variable :

1. The value of the PATH variable is a string.
2. There can (and will) be multiple directory locations mentioned in the PATH variable.
3. The individual directory locations are separated by the : (colon) character.
4. Most common default PATH value is typically this :
   
   /usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
5. To see the current value of PATH use echo $PATH. See the image below.
6. The most common way of adding a new directory location to the PATH is this :
   
   export PATH=$PATH:/the/new/directory/location

Once you  add a directory location to the PATH like this any executable program in that directory can be run on the command line directly by specifying just the program name.

In my system, I have installed Ruby to a particular directory and I want the ruby program to be available on the command line. So this is how my PATH variable looks :

Contents of the PATH variable on my Ubuntu system.

Few Gotchas :

1. Environment variables are local to each shell. So if you are running two terminal windows or tabs, the shells running in them will have their own set of environment variables. The value that you set in one shell is not available in the other shells.
2. The value stored in the environment variables is available until the shell process quits. The values are not persisted beyond the lifetime of the shell. Nothing like saving the environment variables to the hard disk.

This would imply that any PATH settings that you would need will have to be done every time you open up a new Terminal window or tab (a.k.a every time you launch a shell). That sounds awfully painful and tedious. Sure it is. But Linux has a solution for you.

Like many Linux programs, the Bash shell (and almost every other shell too) support a startup configuration file. This file is in your home directory with the name .bashrc (yes, the name starts with a dot). This file can contain any number of commands that you would run on the command line and all of those will be "executed" every time a Bash shell is launched (a.k.a every time you open a new Terminal window or tab).

The common practice is to put all the commands setting the values of the necessary variables into this .bashrc file. That way you are guaranteed that every time you open a new Terminal window or tab, all of these environment variables will be set to the appropriate values. This is why magic tools like "rvm" or "chruby" or "nvm" ask you to add some stuff to your .bashrc file.

This should give you an initial understanding of what the PATH variable is for and what someone means when they say "PATH is not setup properly" or "PATH needs to be setup". From this point on, you should be able to google around for additional information and make sense out of it. If something mentioned here is not clear, drop in your questions in the comments and I will try my best to respond with answers.

PAUS, Mideval history, pre-islamic middle east, religion - A saga of Wikipedia addiction

I came across the word "PAUS" today in an email and I did not know what it meant. The email did not have much context either. As usual I googled for "define: PAUS" and the smart boy Google showed me the Wikipedia link for some Norwegian patrician family from Oslo named PAUS. I was very sure that the "PAUS" word I had come across had nothing to do with a medieval family from Oslo. Nevertheless being a big Wikipedia addict, I bravely caressed my laptop touchpad, moved the mouse pointer to the Wikipedia link and clicked on it.. ! And the Saga begins.. !

The Paus family has some semi-interesting history, not very intriguing or aything, but still there is something to read. The one piece of information that got my attention, apart from the awesome estates they own, is one of their coat of arms : They call it "Crane in its vigilance". Intrigued by the "vigilance" word I click on that link to see if I will get to read some nice history or myth about some super vigilant crane. I was wondering if it had anything to do with this Crane :

The link took me to this page...!!! It was such a "Duh" movement..! :(

But Wikipedia being Wikipedia, never ceases to amaze me. After glancing over the page reading a little here and there about the bird I came across this section about Cranes in Mythology and Symbolism. Here I found that in pre-islamic Arabia the three chief goddesses were referred to as the "the three exalted cranes". Being a religion history buff I immediately opened the three Wikipedia pages to the three goddesses. I did not read all of it but merely glanced over. I found this very interesting modern depiction of the three goddesses.

                                                                                 Source : Wikipedia

What immediately caught my attention was that the temples of all three goddesses were raided and idols destroyed when Muhammad and his army seized Mecca at the very beginning of the Islamic period. It was also mentioned that a lot about these idols is written in the "Book of Idols". Apparently this book was instrumental in molding the Islamic belief that idol worship is sin. Anyways, religious beliefs apart, the page on this book mentioned about the "founding of Kabba" and I did not know that. As you would have guessed, next I started reading about Kabba.

Contrary to what I though Kabba existed much before Muhammad. There were apparently 360 idols before Muhammad seized Mecca and destroyed them. Amongst several other things, one very interesting incident from history caught my attention. It was the attack on Kabba and the stealth of the black stone by a sect of people called Qarmatians. After this the ruling caliph had to get the stone back by paying a ransom. One can imagine how much of a tremor this would have created in the Muslim world back then. Now I had to know who these Qarmatians were and why they attacked Kabba. Turns out that Qarmatians are a sect of Shia Muslims itself. Their history is pretty interesting. Apparently the basis of forming this sect was their belief that one of their imam was the Mahdi. Who or what is a Mahdi you ask? Exactly the question that came to my mind and I opened the page on Mahdi.

So Mahdi is this prophesied redeemer of Islam who would come back to earth, sent by god himself, and who would cleanse the world and uphold Islam. Pretty standard fare. Almost every religion has such prophecy I believe. Well so far 4 prominent people have claimed to be the Mahdi (very surely amongst several other not-so-prominent ones). Guess what? Out of the 4, 2 have been from India.. !! 50% share man... wohooo..! We really are a country of god-men, religion no bar.. ! :)

One of the two men was "Mirza Ghulam Ahmad" founder of the Ahmadiyya movement in India. This one instantly rang a bell in my head. I had heard and read a little about the Ahmadiyya sect. I immediately started reading about this person. His life history was somewhat interesting not very much though. He had some pretty radical thoughts about interpretation of Islam and Jihad. Like any other god-man he had debates with several religious people, wrote a bunch of books. Apart from that the Wikipedia page on him did not offer anything interesting enough to click and go to a new page.

And at that point this particular Wikipedia journey came to and end.

However, after all this I still did not know what the "PAUS" in the email meant. Surely my work has nothing to do with a medieval Norwegian family.. ! So I went back to the email to try and get more context. Digging through some of the older emails from the same person I finally found what he was referring to.

Here comes the grand ending : ;-)

After all this it was revealed to me (probably by divine grace) that, the sender of the email was referring to nothing but this :

YES.. !! He was referring to PAVS (P - A - V - S).. Pavs as in Vada Pav, Misal Pav, Samosa Pav, Pav Bhaji, etc etc.. !

He had misspelled it. He had put in a "U" in place of "V" and that made me wade through Wikipedia for about 45 minutes and then another 30 minutes writing this blog post.. !!

All hail the PAU.. !

There you go. That's the root of this entire saga. In honor of this epic saga, I am going to eat Pav... oh sorry PAU BHAJI today.

Happy PAUing to you too...! :-) 

Layman's analysis of an ULIP : Bajaj Allianz Future Gain

Recently a friend of mine asked me whether he should choose to invest in a Mutual Fund or this ULIP named Bajaj Allianz Future Gain. The ULIP salesman have apparently been pestering him a lot to invest in that scheme while I had strongly suggested him to not go with any ULIP at all. In my books ULIPs have a blanket ban. My friend Prabhakar Kudva and Dhirendra Kumar of valueresearchonline.com have given me enough information to realize that combining investment and insurance is not a good idea and ULIPs in India are definitely very bad investment options for almost all.

valueresearchonline.com had been a regular critic of ULIPs back  in 2000s when ULIPs had a free run without any check. In those days ULIPs charged insane charges like 25% to 30% and have literally scammed investors. After a lot of noise was made about this, IRDA finally woke up and put in some nominal rules to regulate this kind of practice. It however left a huge holes in the policy for these insurance companies to exploit. Insurance companies are not so closely watched as the Mutual Fund companies are watched by SEBI and transparency requirements are also lower for insurance companies. As a result ULIPs become very risky options for investors because of the exploitation possibility.

Anyways that was the history. Despite all this gyaan, my friend however wanted me to take a specific look at this particular ULIP - Bajaj Allianz Future Gain.

So I tried to do a quantitative analysis of the same to demonstrate how bad an investment option it is.

NOTE : Professionally I am an engineer and finance is just a hobby for me. So calculations below are as per my understanding and hence should not be taken as serious financial advise.

Bajan Allianz Future Gain plan facts (Source : http://www.bajajallianz.com/Corp/content/endowment/future-gain-web.pdf)

Minimum premium : 25,000/- Yearly
Minimum sum assured : 2,50,000/- (10 * annual premium)
Minimum Policy term : 10 years
Minimum Premium paying term : 5 years
Lock in period : 5 years

Premium allocation charge :
1st year : 5.5.% of Annual premium = 1,375/-
2nd - 5th year : 3.75% of Annual premium = 937.5/- per year. (3,750/- for 4 years)

Average yearly charge = (1375 + 3750) / 5 = 1025

Yearly charges so far = 1,025/-

Policy administration charge :
33.33/- per month increasing at 5% p.a. every month
Assuming this 5% is compunded every month at the end of 5 years this value will be 42.77.
So for the sake of simplification let us take the average of starting and ending value as the monthly dedudction

Average monthly dedudction = (33.33 + 42.77) / 2 = 38.05 =~ 38/-

Yearly policy administration charges = 38 * 12 = 456/-

Yearly charges so far = 1,481/

Mortality charge :
For a person aged 30, charge = 1.34 per thousand sum at risk (Deducted at monthly anniversary a.k.a every month)

Sum at risk = Maximum of [death benefit – regular premium fund value - top up premium fund value, zero]
Death benefit = 2,50,500/-
Regular Premium Fund value will keep changing depending the fund performance.

At beginning for first year = (First premium - charges) = 25,000 - 1481 = 23,519
So sum at risk = 2,50,000 - 23,519 = 226481/-
Mortality charge = 227 * 1.34 = 304

Yearly charges so far = 1,785/-

NOTE : The mortality charges will decrease as you pay more premium and also as the fund gains more value. But to keep things simple taking an average value is acceptable for this calculation and to illustrate the point that ULIPs are not really good options.

So for a yearly premium of 1,785/- you get a life cover of 2,50,000/- for 5 years (assuming you do not want to pay a premium after the mandatory 5 years)

Now consider the Anmol Jeevan - II (822) term plan from LIC.
For a yearly premium of 1,617/- you get a life cover of 7,00,000/- for 5 years (Source : https://www.licindia.in/premium_calculator.htm)

So wouldn't it make sense to separate out investment and insurance? For insurance take a plain term plan and for investment choose a decent ELSS scheme. This way your investment will perform much better and you will also have a much better insurance life cover.

AJAX for beginners - what it is and how it evolved

Today's websites or web pages are very complex structures involving several components. It can be daunting for a beginner to understand it all at once. To ease the process let's break things down and see how the various pieces fit together.

At the core of web pages is the HTML. It is the language that browsers understand and it is what web developers use to showcase rich content. Without HTML all our web pages would have been simple text files. Now I assume that you are aware of HTML and how it is used in web pages. Another critical piece that works closely with HTML is CSS, but we can skip that one for now.

Plain HTML web pages are pretty dull. They are not really interactive. What they mainly have apart from text are :

1. Some formatting information which tells the browser how to show the contents.
2. Some media content like images or videos.
3. A bunch of hyperlinks which lead to other similar web pages.
4. In slightly evolved pages there would be forms which allowed a user to send data back to the servers.

It is the third one which helped form this "web" of pages which we call the Internet. How do these links work? Pretty simple actually. Each link typically points to a web page on a server. Whenever a user clicked on a link, the browser would fetch that page, remove the existing page and show the newly downloaded one. Some times the link could point back to the same page in which case clicking on it would just "refresh" the existing page. We all know about the refresh/reload button in the browsers don't we?

There are a few things to note here.

1. These basic web pages were dull in the sense that they were not interactive. They did not respond to user actions except for clicks on links, in which case the existing web page goes away and a fresh page is loaded.
2. When a link was clicked the browser would decide to send a request to the server. The web page itself did not have much control on when the request should be made.
3. When the browser was making a request in response to a link click, there was pretty much nothing you could do with the web page until the request completed. Every request to the server blocked the entire page and if there were multiple requests that had to be made it had to be sequential, one after the other. A fancy technical word to describe this would be "synchronous".
4. An obvious observation is that the data that came from server in response to request from a browser was HTML, as in the data and information on how to display that data were sent back from the server all the time.

While the fourth point appears as not such a big problem, the first three are definitely limitations.

To work around the first problem browser developers came up with JavaScript. Browser folks told web developers that they can make their web pages interactive and responsive to user actions by writing code to handle those interactions in this new language called JavaScript and including it in the web pages. Web developers could now write different pieces of code to handle different user actions. The browser provided a few functions (aka APIs) in JavaScript which the web developers could use to make changes to the web page. This API is called DOM (Document Object Model). Thus web pages became interactive and responsive.

The second and third limitations still existed however. The only way to request data from server was through links or in case of evolved pages through forms. Also the web page was one single unit. Whenever something had to change the whole page had to be reloaded. Imagine something like that with your gmail page. How would it be if the page refreshed every time you received a chat message or worse every time one of your contacts logged in or out of chat? That would be disastrous. The fate would be similar of something like a sports website in which the small corner showing the score needs to be updated a lot more frequently than the rest.

To solve this some people thought of iframes as a solution. Iframe is a HTML tag inside which you can put another HTML page. Now with this arrangement the inner page could be reloaded without affecting the outer page. So the sports website could now put the score part in an iframe and refresh it without affecting the rest of the page. Similarly gmail could break up its page into multiple iframes (It actually does this in reality, but it also does a lot more than this).

With iframes the "synchronous" problem was somewhat solved. Multiple requests could now be made in parallel from a single web page. However the method of making the request largely remained the same, using links or forms. It was either not seamless or almost impossible to make requests in any other manner, like say whenever user typed something (like Google search). Now let's see what all capabilities we have on top of the basic web pages.

1. Web pages can respond to user actions. Some code can be executed on various user actions made on different parts of the page.
2. This code can modify the web page using DOM.
3. Browsers are now capable of making multiple requests in parallel without blocking the web page.

The browser folks now combined these abilities thereby allowing web developers to make requests to servers in response to user actions using JavaScript. These requests could be made in parallel without blocking the page i.e. Asynchronously. Now when the response came back the same JavaScript code could add the HTML received in the response to the existing page using DOM.

The developers went a step ahead and said "why receive HTML every time? In most cases the formatting or presentation information is already there in the web page. What is needed is only new data or content (For example, in the sports website example, all that is needed is the updated score. Information on how to show that score is already present in the web page). So instead of fetching HTML every time let's just fetch data to make things much faster and efficient". And back then the most popular way of representing data in a structured manner was XML. So developers started receiving data in XML format from the server and using that to update the web page. XML was so popular that browsers added support for automatically parsing XML data received in response to these requests.

This approach of making requests and updating parts of web pages became very popular and was much widely used. Since this was predominantly used to make requests over HTTP (The protocol of the web) to fetch XML data this technology was called "XMLHttpRequest" (Internet explorer called it something different, it probably still does).

What all does this XHR offer?

1. Making requests "Asynchronously"
2. Making requests from "JavaScript"
3. Making requests for "XML"

Put together it forms "AJAX - Asynchronous JavaScript and XML".

Notes :

1. Several things have been simplified here as this is aimed at people new to web development and Ajax in particular. This should nevertheless give a good starting point.
2. References to XML should be read as " something other than HTML". Nowadays XML has been largely replaced by JSON because of several advantages.
3. XMLHttpRequest does not necessarily mean making asynchronous request. It supports making synchronous requests also. But I don't know if there is a good reason to do it. Making a sync xhr will block the entire page until that request is complete. Really don't do that.

As citizens, we are the system

This post is in response a comment on quora ( http://www.quora.com/What-are-the-biggest-misconceptions-that-Indians-have-about-India/answer/Prashant-Bhattacharji/comment/2765540) and a couple more comments from the same user in that same thread.

===================================================

Dear Abhijeet Choudhury,

A lot of what you have said here is true, about the politicians, thugs and the plight of poor people in India. However I think there are some misconceptions. First of all here is my inference about who you are based on your comment and quora profile (Ignore if my inference is wrong ) :

1) You are an IT engineer (mostly at TCS) from a middle class or upper middle class family who has been in an urban place for most part of your life, there by lacking touch with the ground reality, especially what happens in rural and semi-urban places.
2) You probably had been to USA for work and were impressed by the much efficient system and people there
3) You probably follow Hollywood and US news/comedy etc and have a very positive view of USA

Now the misconceptions

1) USA is definitely advanced than India in several aspects. But remember that
 - They got independence centuries before we did. Their situation was also pretty bad even till the first half of 20th century. The poor people like the mine labourers, construction workers suffered very badly. (Read these : Truck system   Sixteen Tons ) It was mainly during and after the second world war that USA super charged and surged ahead, primarily by accepting people from across the world and creating an ecosystem where innovation thrived. (That is something we definitely have to learn)
 - their population and population density was and is much lesser than that of India
 - they sort of (not entirely) had a clean slate to start with while we had a much older history/legacy to deal with. The older it is the more time it takes to change.

None of these are reasons for us to be complacent with the current state of affairs. Singapore is a perfect example of how development can be achieved in short time also. I am merely saying that a comparison of current India and current USA is not fair. We are definitely progressing. May not be at rate which is impressive to you, may not be at the level the west is, nevertheless we are moving forward and there is ample proof of this. Just look around you and compare the 90s with the current. Has there been no positive change at all? After 67 years of independence we are definitely ahead of where USA was after 67 years of their independence. This is of course because we have leapfrogged in various fields. And that is precisely why we can catch up with the developed world. Let us be positive and give ourselves some more time. Because of our huge mass and very old history, the inertia is very high. Change will be slow but it will happen. It already has started (and no this is not a reference to Modi coming to power).

2) You "almost all common people in USA have a lavish life" - Who are these common people you are referring to? The middle class and upper middle? Your USA counterparts? Fact check : These people are not "almost all" by any statistic. It is true that the urban middle class people are having a decent life in the USA. But so is the urban middle class in India. We sure have more friction when we deal with certain archaic govt institutions but on a day-to-day basis the urban middle class has a fairly smooth life (except may be traffic woes..!). The poor and jobless, even in USA, have a lot of problems. The number of homeless people is increasing like anything. The wealth distribution is very skewed there too. Well qualified people are spending nights in buses because they can afford an overnight bus journey but cannot afford to rent a home. (Read : The dark side of Silicon Valley - Telegraph). I also belong to this same urban middle class category and my life is pretty smooth. May not be lavish but definitely not struggling. So is the case with almost all the people in my circles. You cannot compare the picture of a poor farmer or a construction worker in India with the picture of an IT guy in USA. The blue collar workers suffer in USA too (Read : 'Being homeless is better than working for Amazon')

3) You talk about loan to poor people. Here are some things you should know :
 - Govt provides "interest free" agriculture loans to farmers. This facility is being used by many farmers (albeit being misused also).
 - Co-operative societies provide loan to the poor at 3% interest rate (which is less than the interest your bank gives on your SB account)
 - Micro finance institutions exist through out the country (and several other neighbouring countries) which are helping workers and entrepreneurs at the rural, grass-root levels. These are people who sell fruits and vegetables on hand pulled carts, people who put up small mobile recharge shops, etc.
 - Govt runs the rural employment guarantee scheme which is analogous to the "Unemployment compensation" in USA. Again, this is also being misused but this has indeed prevented a lot of migration of poor people from rural to urban places there by reducing the "slums" that we see in all cities.

There are other several steps being taken by the govt to help uplift the poor. Sure the execution is not the best and efficient. There are a lot of leaks. But it is not summing up to a zero. There is a positive effect and we are progressing. It can definitely be a lot better, but that won't happen with negative attitude. We have to keep a positive outlook and be ready to get our hands dirty. Otherwise we will just be like those people sitting in a comfort chair in front of tv watching Sachin get out and then say "Arre square cut maarna tha yaar..!".

4) In your another comment you talk about traffic violations. You say people do it because they see politicians and other big people doing it and also because police extort money from people. I fail to recognize the correlation. If a politician is breaking traffic rules, what is stopping you from following the same? I see so many people jump signals while I always make a point to stop when it is a red. People behind me honk like crazy, some switch lanes and point fingers at me when they overtake me and jump the signal. I still don't budge. It is not just me. I have seen several other people do the same. If you are genuinely interested in creating a lawful ecosystem, then start following the rules. You cannot say that I won't do it until every one else starts doing it. That mentality won't get us anywhere. As Gandhiji said "Be the change that you wish to see in the world".

That brings me to my final and most important point : In all your posts you are just pointing fingers at others : other people, the system, the politicians, the police and so on. You compare all of these in India and USA. You declare that all of this is better in USA. But you fail to recognize what contributed to making all of this better in USA. Did it magically happen? Of course not. The people of the country toiled hard to reach there. People like you and me.

There was a time in USA when child labourers in coal mines was a common place. There was a time when farmers in USA paid lawyers with grains and nuts because they did not have money. There was a time in USA when racial discrimination was so strong that the African-American folks were as afraid of just talking as were the shudras in India when untouchability was at its height. The USA too had a lot of shortcomings. But they worked to overcome all of that and fullfil their dreams, albeit in their own ways. There was several altercations and fights. People had to fight for their rights, their safety, for development. They still do. (Remember "Occupy Wallstreet", Ferguson shooting?). It only through such fights that we weed out the bad. Not by complaining.

You might say "What could a regular corporate employee like me who works 9 to 6 for 5 days a week do in this matter?". The simple answer is that there is not easy way out. If we want to bring about a change we should be ready to face hardships. To start with pledge to abide by rules everywhere you interface with government. Getting your driving license? Do not go through an agent or pay any bribe to the RTO officer. Purchased a piece of land? Do not under-quote the value to save tax and pay a bribe to the sub-registrar. Is there a pot hole in the road in your locality? Go notify that to your ward corporator. If it doesn't happen in one complaint, do it many times. You sure will be spending quite a bit of time, at least initially. Please don't think of it as a waste of time. That would just be a short sighted view. Eventually they will give in and do the work. There are a lot of things that each and every one of us can do as citizens to improve the situation. Complaining is definitely not one of them. And pessimistic attitude definitely doesn't help one bit.

Not until too long ago the Americans looked up to Europe like you look up to the USA, picturing Europe to be that much more developed and sophisticated place. That in no way reduced their national pride. Everything they did, everything they said was about USA and people of USA. Majority of the people aiming to be the best cared about being the best in USA. They did not look for a stamp of approval from some other country. Their growth and development period was characterized by indigenous innovation and not by simply importing ideas and technologies from the already developed world. That is what we should be doing too. Be proud about our nation while accepting our shortcomings and working towards overcoming them.

All said and done, please do note that India is not at all a failed democracy. Compared to a host of oppressive regimes around the world we have freedom of speech to a vast degree. Except for a few isolated incidents like the one you mention (two mumbai girls being arrested) we have a lot of freedom of speech and it is improving. The 66A of IT Act (under which the girls were arrested) has been challenged in the court and the court has already opined that the legislation is very vague and ripe for misuse. The hearing is still on and soon we will see the legislation being revamped to be meaningful, if not scrapped altogether (Read : Sec 66A of IT act lacks guidelines, arrests made over social media posts prone to abuse: SC).

Things will change. They will improve. They are just waiting for each of one of us to do our part. Let's not shy away from confronting the problems and work on solving them instead of just complaining.

My response to request for feedback from AAP (Aam Aadmi Party)

Today I got an email asking me for feedback on AAP (Aam Admi Party). The first question was

What would make you volunteer in future? What stopped you from volunteering this time? *

Give us as much detail as you'd like

I took the sub-text literally and a little seriously and wrote a really detailed answer. I thought I might as well put it up here and share with the world (or the one or two who actually read my blog.. :P ). Anyways here it goes :

Not interested in volunteering for AAP.

AAP as a party has acted in the most irresponsible, selfish and foolhardy manner ever since it formed the government in Delhi. People of Delhi whole heartedly supported AAP with the hope that they would get some real stable governance for the next 5 years. Instead what they got was utter chaos.

I expected changes like corruption free governance, no undue fiscal burden on the state, public works development, fulfillment of election promises for once, but all I got to see was a couple of filmy style sacking of corrupt officers, some agitation and finally a resignation just after 49 days. If that was not irresponsible then what is?

And what do you do after resigning? You forget Delhi altogether and try to become the prime minister of India. If this isn't selfish then what is it?

Prime minister of India, no less. Are you kidding me? You could not run the state of Delhi for two months and you expect me to hand over the reins of the country to you for the next 5 years?

Alright lets assume that you genuinely felt that contesting the Lok Sabha elections was a key thing and something that you had do. What did you do? Again like the typical filmy style you go out like an underdog hero and challenge the most popular leader, in a place which is not even your home turf. If this is not foolhardiness then what is it? How much would it have mattered whom you win against. Wouldn't you be able to serve the people better being an MP from somewhere in Delhi (your home turf) instead of creating all this drama of challenging Mr. Modi from Varanasi. Look at how Dr. Jayaprakash Narayan (http://en.wikipedia.org/wiki/Jayaprakash_Narayan_(Lok_Satta) ) has done it. His party was never in power anywhere, but that never deterred him from becoming one of the best performing MLAs. I believe you wanted to show that you are not afraid of facing anyone in elections and you have the peoples' mandate to contest and win against Mr. Modi. But who cares about that Mr. Kejriwal? Sure you might have put up a good fight, but what does it account to in the end? All that I see is that you were trying to be a bollywood hero in the Indian politics, who wants to the take on the big bad guy (again as declared by yourself). You now lost your chance to voice the opinion of the people you represent in the LokSabha.

Then comes the ultimate flip-flop you did with the Delhi assembly. Until LokSabha election results were announced you kept hounding the Lt. Governor to dissolve the assembly. You even approached the Supreme Court for this. Then on one fine Tuesday you go and meet the Lt. Governor and make an announcement to the public that you have requested the Lt. Governor to NOT dissolve the assembly but give you another chance and that you will ask people about forming the government again and then decide. And the next day (on Wednesday) you make an yet another U-turn announcing that you have asked the Lt. Governor to dissolve the assembly and you will now ask people about contesting again in the elections.

How do you think this reflects on you? You accuse every politician out there of corruption, double standards and every other accusation possible and you yourself act in such an irresponsible and opaque manner, saying contradictory things on a daily basis and on top of it claim to be "THE ONLY HONEST POLITICAL PARTY". How can I believe anything you say after you have done so much of circus?

All this drama eroded complete faith in the capability of AAP to form and run a government in any form, let alone provide better governance than the existing political parties.

Please note that I have talked only about your own actions and what I feel about those. I have not mentioned any of the other accusations made about AAP or Mr. Kejriwal, like Mr. Kejriwal coming to Varanasi by train saying he has Rs. 500/- in his pocket and going back by flight, or the episode around Mr. Kejriwal's official residence as CM of Delhi, or the incident of Mr. Somnath Bharti and certain people of African origin or the mayraid images and posts circulated on the social network which, of course, you totally accuse as baseless and a defamation propaganda by the IT cell of BJP. I am not even making the comparison between Mr. Modi's Gujarat model and your achievements, because you say that its all an eyewash. So be it. That's why I have mentioned only the actions of AAP and it's leader Mr. Kejriwal.

Now a few times AAP has come out and said that it is a new political party with inexperienced people and hence they could have made some mistakes. It's good that you acknowledge this. But with this statement you can't ask people to hand over the country's administration to you. If you are new and inexperienced, then start with the basics. Gain that experience that you think is needed. I don't have to tell you that you need to start from the grass-root levels. Making noise in urban places, parading urban youth with the "Aam Aadmi" cap is one thing and working at village panchayat level, addressing rural problems is entirely different.

You do not have to be a MLA or MP to provide good governance. Those are not the only places where good politicians are needed. Those are not the only places where big time scams and plundering of public funds is done. Gram panchayat, taluk panchayat and Zilla Parishad are equally in need of good politicians and bureaucrats. If one were to accumulate the amount of money siphoned off in the name of NREGA through out the country then I am sure it will beat any of the other big scams that the previous government(s) have been accused of. The amount of pilfering that happens is insane. It easily runs into crores for each panchayat, every year. And I know this first hand.

Start at those places. Let people across the state and country know you, not just by your agitations, but by real work that you do, by some real meaningful change that you bring about. It's ok if you are unable to bring in a strong legislation like Jan Lokpal. You don't have to throw your hands up in the air in despair saying you were not given enough support. There are other innumerable things that you can and should do and those will have a big positive impact on people even without the Jan Lokpal. If you tell me that there is absolutely no machinery to fight corruption in our system currently then you are absolutely wrong. You do not have to put every single corrupt bureaucrat in jail or suspend them. It's sufficient if you make an example out of one or some of them in a few departments on a regular basis. The rest will automatically shy away from corruption. And you have enough ways to enforce such a thing.

Also, corruption is not the only problem plaguing our country. There are numerous other things that require attention. Being in a position of power allows you to address these other issues while you carefully maneuver the juggernaut that our system is. It doesn't have to be a head on collision. It's not a "Last man standing" kind of battle. I have personally seen government officials doing this in a very intelligent manner at various levels and I have seen it from very close quarters. It is possible. It is doable. Above all it brings out the desired positive changes and makes the lives of people better. It provides them what they really crave for. It's going to be difficult no doubt, but our constitution and our legal system provide so many options to deal with this juggernaut. This same government infrastructure which the corrupt are mis-utilizing can be turned around as a weapon against them.

No matter which legislation you bring in, we will not be a corruption free state overnight. It's going to be a long road and you need to be patient about it. We as a country are definitely in a much better position than we were a few decades back. So we can get better. Its just that we have a huge inertia owing to our big mass (aka population). Change is difficult. One is easily tempted about the idea of bringing about a revolution and making overnight changes. Personally, I feel the cost involved for such a revolution is very high and more importantly the desired result is not guaranteed. In comparison the slow and steady here will indeed be the winner.

You ask me what would make me a AAP volunteer in future. I say look beyond cities. Look at the vast majority of semi-urban and rural places. Go there. Find like minded people in those places. I am sure they are there. Empower them with all that you have. You have top class lawyers with you who can carry out a PIL right up in the supreme court. You have contacts in the press. You can mobilize man power. You can draw the attention of the nation and beyond. You know how to effectively use the RTI. You run NGOs which, despite the allegations of being funded by the CIA through various organizations, can do a lot of good. Do all of that. Become a party of the rural India. Become a household name as the best choice in panchayat politics and then I will probably become an AAP volunteer. Well technically you will not need me at that point and thats a fine contradiction to have. :).

I say probably because apart from the stuff that I have said above I am not really a fan of AAP party principles, which according to me are very close to communist ideas. So that's altogether a different matter and a commentary on that will have meaning only after the above mentioned things are addressed.

Oh and one more thing which I almost forgot. Please groom local leaders everywhere. Or rope in existing local leaders whose thought process aligns with yours. Personally a tie-up with Loksatta would have been really good. Nevertheless, do not be a typical "high command controlled" party. Be true to your vision and words and be truly federal even in your party structure and operation.

P.S : I might have come across as a staunch BJP supporter through out, which at the moment is true. In the recently concluded elections BJP under the leadership of Mr. Narendra Modi has been the best option for the country according to me and I am positive about India's future under the current government. Ideologically though, I think Loksatta is more appealing to me than AAP.

Invalid credentails : OmniAuth + oAuth2 + Rails 4 encrypted cookie store + simultaneous requests

OmniAuth is a very well know gem in the Ruby/Rails world. Almost every Rails application out there is probably using it to authenticate with one of the various mechanisms it supports. OmniAuth is just awesome.!

I have been using OmniAuth for about 3 years now in various Rails projects and it has worked very well, although I have had to monkey patch it once or twice to allow me to exploit some of the Facebook features (like authenticated referrals). But all in all, it just works as advertised and you will have an authentication system in pretty much no time at all.

Yesterday, however, I was having quite a bit of hard time getting OmniAuth to do a simple "Login with Facebook" oAuth2 authorization. It was something that had worked seamlessly on innumerable previous occasions. But yesterday it just kept failing repeatedly, succeeding only once in a while. And it always failed with the same obscure error "Invalid Credentials" during the callback phase. (OmniAuth operates in three phases : Setup, Request and Callback. Apart from OmniAuth wiki on github, this is a good place to read about it : http://www.slideshare.net/mbleigh/omniauth-from-the-ground-up). The fact that the error message was not so helpful made this whole process a lot more frustrating. After some hunting on the inter-webs I found that the culprit could be a bad "state" parameter.

Wait, what is this "state" parameter?

Background : oAuth-2 CSRF protection

oAuth2 specifies the uses of a non-guessable secure random string to be used as a "state" parameter to prevent CSRF attacks on oAuth. More details here : http://tools.ietf.org/html/rfc6749#section-10.12. This came out almost a year back and many oAuth providers implement it already, including Facebook. OmniAuth also implemented it last year. Although not written with the best grammar, this article will tell you why this "state" parameter is needed and what happens without it.

To sum it up, oAuth client (our web application) creates a random string, stores it in an accessible place and also sends it to the oAuth provider (Ex : Facebook) as "state" during the request phase. Facebook will keep it, authenticate the end user, ask for permissions and when granted sends a callback to our web application by redirecting the user back to our website "with the state" as a query parameter. The client (our web application) compares the "state" sent by the provider and the one it had stored previously and proceeds only if they match. If they don't then there is no proof that the callback that our web application received is actually from the provider. It could be from some other attacker trying to trick our web application into thinking (s)he is someone else.

In case of OmniAuth oAuth2, this state parameter is stored as a property in the session with the key 'omniauth.state' during the request phase. The result of the request phase is a redirect to the provider's URL. The new session with the "state" stored in it will be set on the client's browser when it receives this redirect (302) response for the request /auth/:provider (This is the default OmniAuth route to initiate the request phase). After the provider (Facebook) authenticates the user and user authorizes our application, the provider makes a callback to our application by redirecting the user back to our web application at the callback URL /auth/:provider/callback along with the "state" as a query parameter. When this callback URL is requested by the browser, the previously stored session cookie containing the 'omniauth.state' property is also sent to our web application.

OmniAuth checks both of these and proceeds only if they match. If they don't match it raises the above mentioned "Invalid Credentials" error. (Yeah, I know, not really a helpful error message..!).

Ok, that is good to hear, but why will there be a mismatch?

A mismatch is possible only if the session cookie stored on the user's browser is changed such that the 'omniauth.state' property is removed from it or altered after the request phase has set it. This can happen if a second request to our web application was initiated while the request phase of oAuth was running and it completed after the request phase completed but before the callback phase started. Sounds complex? The diagram below illustrates it.

The diagram makes it clear as to when and how the 'omniauth.state' gets removed from the session leading to the error. However, apart from the timeline requirements (i.e. when requests start and end), there is another essential criteria for this error to occur :

The response of the "other simultaneous request" must set a new session cookie, overriding the existing one. If it does not explicitly specify a session cookie in the response headers, the client's browser will retain the existing cookie and 'omniauth.state' will be preserved in the session.

Now, from what I have observed, Rails (or one of Rack middlewares) has this nifty feature of not serializing the session and setting the session cookie in the response headers, if the session has not changed in the course of processing a request. So, in our case, if the intermediate simultaneous request does not make any changes to the session, Rails will not explicitly set the session cookie, there by preventing the loss of 'omniauth.state' property in the session.

Ok, then why will the session cookie change and lose the 'omniauth.state' property?

One obvious thing is that the "other simultaneous request" might change the session - either add or remove or edit any of the properties. There however is another player involved.

This is where the "Encrypted Cookie Store" of Rails 4 comes into picture. Prior to Rails 4, Rails did not encrypt its session cookie. It merely signed it and verified the signature when it had to de-serialize the session from the request cookie. Read how Rails 3 handles cookies for a detailed breakdown. Rails 4 goes one step ahead and encrypts the session data with AES-256 (along with the old signing mechanism. More details on that coming up in a new post). The implementation used is AES-256-CBC from OpenSSL. I am not a Cryptography expert, but the way I understand it, the property of AES is that it results in a different cipher text every time you run the encryption for the same message plain text. Or it could also be because the Rails encryption scheme initializes the encryptor with a random initialization vector every time it encrypts a session (Implementation here). Either ways, the session cookie contents are always new for every request even when the actual session object or session contents remain unchanged. As a result Rails will always set the session cookie in the response header for every request and result in the browser updating that cookie in its cookie store.

In our case, this results in the session being clobbered at the end of the "other simultaneous request" and we end up losing the 'omniauth.state' property and oAuth fails.

Umm.. ok, but when and how does this happen in real world, if at all it can?!

All these requirements/constraints described above, especially the timing constraints makes one wonder if this can really happen in the real world. Well, for starters it happened to me (hence this blog post..!!). I also tried to think of scenarios other than mine where this would happen. Here are a couple that I could think of :

Scenario - 1) FB Login is in a popup window and the "Simultaneous request" is a XHR - Ex : an analytics or tracking request.

Here is the flow :

1. User clicks on a "Login with FB" button on your website.
2. You popup the FB Login page in a new popup window. Request phase is initiated. But there is a small window of time before the redirect response for '/auth/facebook' is received and 'omniauth.state' is set.
3. During that small window of time, in the main window, you send an XHR to your web app to, may be, track the click on the "Login with FB" button. You might do this to just track usage or for some A/B testing or to build a funnel, etc. This request sends the session without the 'omniauth.state'.
4. While the XHR is in progress, the redirect from the request phase is complete and the session with 'omniauth.state' is set. The user now sees FB Login page loading and proceeds to login once it is loaded.
5. While the user is logging in to FB and approving our app, the XHR has completed and has come back with a session without 'omniauth.state'. This is stored by the browser now.
6. Once user logs in and approves your app, the callback state starts. But the session sent to your web app is now missing the 'omniauth.state'. 
7. oAuth fails.

How big a deal is this scenario?

If you are indeed making a XHR in the background, then this scenario needs to be taken care of. Since the "other simultaneous" request is automatically triggered every time, it is very likely that session will get clobbered.

How to solve this?

You can either first send the XHR and then in the response handler of that XHR, you can open the FB Login page in the popup. Also have a timeout just to make sure you don't wait for too long (or forever) before you receive a response for that XHR.

Alternatively, if you can push your tracking events in a queue stored in a cookie, you can do that and then open the FB Login page. Once the FB Login completes, you can pull that event out of the queue and send it. As a backup have a code that runs on every new page load to look for pending events from the queue in the cookie and send those events.

With HTML5 in place, its probably better to use the localstorage for the queue than the cookie. But again that needs user's permission. Your call.

Scenario - 2) FB Login is in the same window/tab but User has the website opened in two tabs.

Here is the flow :

1. User has your website opened in a browser tab - Tab-1
2. User opens a link on your website in a second tab - Tab-2 (Ctrl + Click or 'Open in a new tab' menu item). This request sends the session without 'omniauth.state'.
3. While that Tab-2 is loading, user clicks on "Login with FB" in Tab-1 initiating the request phase.
4.  If the request loading in Tab-2 is a little time consuming the redirect of the request phase of oAuth in Tab-1 completes before request in Tab-2, setting the session with 'omniauth.state'. After that FB Login page is shown and user proceeds to login and authorize.
5. While the user is logging in, the request in Tab-2 completes, but with a session that is missing 'omniauth.state'.
6. After logging in to FB, the callback phase is initiated with a redirect to your web app, but with a session that doesn't have 'omniauth.state'. 
7. oAuth fails. 

How big a deal is this scenario?

Not a big deal actually. In your web app, in the oAuth failure handle, you can just redirect the user back to /auth/facebook, redoing the whole process again and guess what - this time it will succeed and that too without the user having to do anything because the user is already logged in to FB and has also authorized your app. But just to be on the safer side, you would want to be careful about this loop going infinite (i.e. You start FB auth, it fails and the failure handler restarts the FB auth). Setting a cookie (different from the session cookie) with the attempt count should be good. If the attempt count crosses a certain limit, send the user back to homepage or show up an error page or show a lolcats video, c'mon be creative.

Ok, those are two scenarios that I could think of. I am not sure if there are more.

Can OmniAuth change something to solve this?

I believe so. If OmniAuth uses a different signed and/or encrypted cookie to store the state value instead of the session cookie none of this session clobbering would result in loss of the state value. OmniAuth is a Rack based app and relies on the Session middleware. I am not entirely sure, but it can probably use the Cookie middleware instead. Just set its own '_oa_state' cookie and use that during callback for verification.

Will you send a pull request making this change?

I am not sure. I first will hit the OmniAuth mailing list and find out what the wise folks there have to say about this. If it makes sense and nobody in the awesome Ruby community provides an instant patch, I will try and send a patch myself.

THE END
 
Ok, so that was the awesome ride through oAuth workings inside the OmniAuth gem. In the course I got to know quite a bit of Rails and also Ruby internals. Looking forward to writing posts about those too. Okay, okay.. fine. I will try and keep those posts short and not make them this long..!!

Till then, happy oAuthing. :-/ !

P.S : Security experts, excuse me if I have used "authentication" and "authorization" in the wrong places. I guess I have used it interchangeably as web applications typically do both with oAuth2.

Creating Wildcard self-signed certificates with openssl with subjectAltName (SAN - Subject Alternate Name)

For the past few hours I have been trying to create a self-signed certificate for all the sub-domains for my staging setup using wildcard subdomain.

There are a lot of guides and tutorials on the internet out there which explain the process of creating a self-signed certificate using openssl with a good amount details. Further there are also certain guides to create a self-signed cert for wildcard domain. It's fairly easy. You just specify that your Common Name (CN) a.k.a FQDN is *.yourdomain.com while creating the certificate signing request (CSR).

This will take care of all of your sub-domains under yourdomain.com (like www.yourdomain.com or mail.yourdomain.com), however your top level bare domain (yourdomain.com) itself is not covered under this certificate. When you use a certificate generated by specifying *.yourdomain.com the browsers will throw up an error when you hit your server with the top level domain name https://yourdomain.com/.

To address this X.509 certificate standard allows for a type of extension named subjectAltName (http://en.wikipedia.org/wiki/SubjectAltName). Using this you can specify that there are a few other domain names for which this certificate is valid.

This requires specifying the use of this extension while generating the certificate request AND while signing the certificate. To do this you will have to add a few things to your openssl configuration file (typically /etc/ssl/openssl.cnf on a Ubuntu like machine). Alternatively you can copy the config file to another location, add these extension stuff there and then specify the new config file for all your openssl commands. The commands shown below assume the default config file at /etc/ssl/openssl.cnf was updated with extension details.

Here is one blog post which details the updates needed for the openssl config file. http://grevi.ch/blog/ssl-certificate-request-with-subject-alternative-names-san. It also has commands for generating the private key, converting the key to a format which does not ask for password (a.k.a unencrypted key), generating the certificate request (CSR) and finally signing the certificate. The steps mentioned there until the generation of certificate request are correct. It is the last step of signing the certificate that is missing one small piece of information because of which the extension mentioned above (subjectAltName) doesn't get added to the final certificate, despite they being present in the certificate request.

After a lot of searching on the internet, copy pasting the commands exactly, trying my luck at IRC (irc.freenode.net#openssl) the answer finally appeared to me in the man pages (duh..! RTFM dude..!!).  The man page for x509 (man x509) command of openssl has this little entry :

-extfile filename
           file containing certificate extensions to use. If not specified then no extensions are added to the certificate.

So turns out that just specifying the extensions in the openssl config file is not sufficient, but you must also specify the same file as the file containing the extensions to be included in the command line using the above -extfile option.

With that added, you will get a self-signed certificate for your wildcard subdomain which is also valid for your top level bare domain.

Changes made to /etc/ssl/openssl.cnf

Uncomment the req_extensions = v3_req

req_extensions = v3_req # The extensions to add to a certificate request

Add subjectAltName to v3_req section

[ v3_req ]

# Extensions to add to a certificate request

basicConstraints = CA:FALSE
keyUsage = nonRepudiation, digitalSignature, keyEncipherment
subjectAltName = @alt_names

Finally add the alternate names for which you want this certificate to be valid. It would be your toplevel bare domain

[alt_names]
DNS.1 = yourdomain.com

This last section [alt_names] will not be present in the file. You can add it right after the [ v3_req ] section.

Once config update is done, create your certificate.
Here are the commands that I used to create such a certificate :

Create a private key :

openssl genrsa -des3 -out ssl/staging/yourdomain.com.key 2048

This will ask you for a password. Key in a simple one and remember it.

Convert the private key to an unencrypted format

openssl rsa -in ssl/staging/yourdomain.com.key -out ssl/staging/yourdomain.com.key.rsa

This will ask you for a password. Key in the same thing that you used in the previous step.

Create the certificate signing request

openssl req -new -key ssl/staging/yourdomain.com.key.rsa -out ssl/staging/yourdomain.com.csr

This will ask you for a bunch of fields. Enter *.yourdomain.com when it asks for Common Name (or FQDN). The fields after that can be left blank by just hitting return (Enter) key.

Sign the certificate with extensions

openssl x509 -req -extensions v3_req -days 365 -in ssl/staging/yourdomain.com.csr -signkey ssl/staging/yourdomain.com.key.rsa -out ssl/staging/yourdomain.com.crt -extfile /etc/ssl/openssl.cnf

Note that here we specify the openssl config file as the file file containing extensions as that is where we have defined it. Probably we can put the extensions in a separate file too, but I haven't tried that. (Homework?! :P)

Thats it. Now you have a self-signed wildcard subdomain certificate which is valid for your top level domain too.

Despite this the first time you hit your SSL enabled website with the above generated certificate your browser will show the standard "Invalid Certificate - Untrusted" page. It is because your certificate is self-signed. You can view the errors in the "Technical Details" section.

Package installation error on Ubuntu because of MTS MBlaze package installed

A couple of weeks ago I was trying to get the MTS MBlaze data card to work with my Ubuntu machine (A VM actually). I was happy to see that the MTS guys had provided a .deb package to be used with their data card. Installing that .deb package installed something under the name of "crossplatformui" (very misleading btw). There was probably some problem during the installation, some error thrown, but I am unable to recall that now. Despite several attempts I was not able to use the data card and I finally gave up.

However, that was not the problem. It was what followed afterwards. Ever since that attempt, during every Ubuntu software update process I would encounter an error and would be presented with a huge error log. Every time this was the error :

make -C /lib/modules/3.2.0-38-generic/build M=/usr/local/bin/ztemtApp/zteusbserial/below2.6.27 modules

make[1]: Entering directory `/usr/src/linux-headers-3.2.0-38-generic'

  CC [M]  /usr/local/bin/ztemtApp/zteusbserial/below2.6.27/usb-serial.o

/usr/local/bin/ztemtApp/zteusbserial/below2.6.27/usb-serial.c:34:28: fatal error: linux/smp_lock.h: No such file or directory

compilation terminated.

make[2]: *** [/usr/local/bin/ztemtApp/zteusbserial/below2.6.27/usb-serial.o] Error 1

make[1]: *** [_module_/usr/local/bin/ztemtApp/zteusbserial/below2.6.27] Error 2

make[1]: Leaving directory `/usr/src/linux-headers-3.2.0-38-generic'

make: *** [modules] Error 2

dpkg: error processing crossplatformui (--configure):

 subprocess installed post-installation script returned error exit status 2

This happened during literally every update. My guess is that the "crossplatformui" package never got fully installed and was in a "to be installed" zombie state because of the error in compilation and the post-installation script. That way the package manager would try to install this and run its post-install script on every software update/installation and end up reporting a package installation error.

Looking at the compile commands it was clear from the "zte" string that this was related to the internet data card software. However there was no package with that name and as I said earlier the package name "crossplatformui" is very misleading! After a little bit of looking around I found out that the "crossplatformui" package is indeed the MTS MBlaze software which is causing this problem.

Immediate first instinct was to remove the package. But alas, their post-uninstall (post-remove) script also was buggy and resulted in an error. Repeated attempts to remove it or even "Completely Remove" it failed and the package just stayed there. Finally I inspected those post-install and post-uninstall scripts. Those are present at :

/var/lib/dpkg/info/crossplatformui.postinst
/var/lib/dpkg/info/crossplatformui.postrm

The "postrm" script was trying to kill a process, but that was failing. So I commented it out and tried to kill the process myself, but there was no such process. The next failure was because of a "-d" option being passed to the "rm" command. I could not find any documentation for such an option. So I just removed it. Finally at the end of the script I put a "exit 0" because the package manager was complaining that the post-uninstall script exited with a exit value of 1.

And boy finally it was uninstalled. However these scripts still lingered around. I once again had to go to the Synaptic Package Manager and mark it for "Complete Removal" and purge it away.

Now it appears that I have gotten rid of this package and all its files. Sadly there were no updates available to check if the error had gone away, but I am thinking (and hoping) it has. Will wait for the next update to find out.

Here is the update "postrm" script : https://gist.github.com/brahmana/5087790 . If you are seeing the same package installation error, replacing the "postrm" with this script should do the trick. The changes are on line no : 12,14 and 44

Presumed Innocent - by Scott Turow - A review

Last week my friend Ananth Prasad, returned a bunch of my books that were with him for quite some time. Amongst them was a book that I did not remember having read. The age of the paperback was apparent from the brown color that the once white pages of the book now donned. It was a novel named "Presumed Innocent" by the author "Scott Turow".

Neither the title nor the author's name ring any bell when I first saw it and that made me a little skeptical about the book. Nevertheless, I decided to try it out and I am very glad I did it. Although very old, it turns out this book was somewhat of a hit back then and has even been made into a movie starring Harission Ford. Additionally the author is a pretty successful one with his books being translated into 20 languages.. !

It was very clear from the cover page design and the title that this is about a murder. The author doesn't try to hide this fact and starts the story with the murder of a women. There is no back story, no suspense, nothing. A woman is murdered, that's it. The protagonist is introduced as the guy investigating the murder, on a special request from his boss. The pace of story telling was also OK. It was not super fast-paced like the modern military/CIA spy thrillers. But it was not boring either. I was hooked to the book for the initial few pages when the characters were introduced and the initial scene/plot is laid out. But after that the pace just drops dead and I felt like I am reading the script of one of the Hindi daily soaps that my mother, sisters and aunts watch (In case you did not know the story barely moves ahead in a week's duration in those soaps). IMHO, the initial 150 pages could probably be all dealt with in just about 40 to 50 pages. Personally I am not really a big fan of overly descriptive narrations of mundane things like walking in the woods where the walker notices the rustling of the leaves and imagines something in those sounds or the collection of vague thoughts that the protagonist has when he has hit a low point in his life, etc. If it is descriptive narration it has to be something superbly imaginative and way beyond my own imagination. That's why I find descriptive narrative appealing only when I am reading a fantasy novel. Nobody does it better than JRR Tolkien. :).

Anyways coming back to "Presumed Innocent", at the end of that boring, slow paced section it is revealed that the protagonist, a character named Rusty Sabich, who was investigating the murder is actually now the accused. And then things start to get interesting. As expected, Rusty goes to the most famous defense lawyer, who undoubtedly was his arch enemy while working as a public prosecutor. Anyways, once the court room drama starts and things start to unravel it gets really gripping. At the very beginning you are given a bunch of data - some well known facts and some speculations and some extrapolated facts at that point in time. With that, and having read a bunch of John Grisham legal thrillers, I decide upon one of the characters as the murderer. As the investigation and trial proceed and more things surface, I can't help but to suspect some other character. This goes on for most part of the rest of the book. But as I proceeded I noticed that I never suspected a character for a second time. It was almost like an elimination process, bringing the reader closer to the actual murderer. After all this running around, it is revealed that my initial suspect is indeed the murderer. So that kind of saves me the "Holy Shit... !!" moment and instead leaves me a "Aah.. Damn it... I was right initially" moment. Although the ending wasn't very spectacular the journey was pretty awesome. So I guess in this case the means justify the end.. :P

There are two things that I specifically liked about this book :
1) The crisp and clear explanations about the legal procedures and jargon. It was interesting to know. Kind of helped me imagine myself in the story as apart of the legal system. Before reading this I did not know that the Judge played a very important role in a trial in a USA court. I always thought the Jury was the most important and the judge was only there to oversee the trial.

2) The whole story, plot, dialogues, character presentation, everything appeared very close to reality and not at all very flamboyant. Although this has been made into a full length motion picture, I would prefer to see this as an episode of "Law and Order" (that currently airs on FOX-Crime).

The story telling is not continuous. The protagonist is not active 24x7. When its weekend and its a holiday in the court, the protagonist spends most of his time at home and his lawyer also takes a break. The story just resumes on Monday, from pretty much where it left off on the previous Friday evening. In fact after the first hearing, nothing much happens before the trial date. This is pretty much how every episode of "Law and Order" is presented. And I like it that way. Makes it appear realistic.

All in all, it was a good read. In fact very good if I discount the initial slow paced part. Now I am very eager to watch this movie. Apparently the movie was also very well received. Next weekend I guess. :)