Where are we headed? (Part III)

I finished my last post with an observation that the chaotic changes taking place around us are in fact, driven by three distinct forces - our ability to move information quickly over vast distances (networking), our ability to process tremendous amounts of information quickly (computing) and our ability to store tremendous amounts of information (storage). It is in the interplay of these potent forces that we can understand the bewildering changes taking place around us and where they are leading us. It is also important to understand that the development in these areas is not necessarily pointing in the same direction. We will discuss this as we go along.

Having said that, we must begin naturally from an examination of the state of the art in these areas. Lets start with storage.

In the desktop drives market, Seagate released a 750 GB model based on perpendicular storage technology in April of 2006. Perpendicular storage technology allows much higher amounts of information to be stored on the same disk area thus allowing much higher capacity disks. While these disks will still take some time to become mainstream, 160 GB disks are extremely common these days with 80 GB disks being the entry level across most segments. For recent entrants into the computer-user world, this might not seem like a lot. However, at a time not long ago (a time which even I can remember distinctly even though I'm only 22), 4 GB disks sounded like overkill to most people. Today, 4+ GB DVD disks are available for as low as Rs. 15 - and DVDs are old technology. The latest entrants fighting for supremacy are the Blu-Ray and High-Density DVD disks offering, at the higher end, storage capacities of 50 GBs. And even as these technologies struggle to get to the retail market, scientists around the world are trying to fit more and more data into disks of similar sizes. On the flash memory drive side, the latest entrant is a 16GB pen drive that you can carry around your neck - and this was in March of 2006. These are all products available in the retail end-user segment.

On the server side, the story is even more mind-boggling. Companies like SGI are offering storage capacities in excess of 400 terabytes (a terabyte is 1000 gigabytes) in a single system with access speeds of 2.5GB/s and higher. The internet archive is on its way to creating a machine capable of storing and managing 1 petabyte (which is a thousand thousand gigabytes). To put this in perspective, the entire text within the Library of Congress takes up only about 20 TB of space. However, several organisations around the world have space requirements running into several petabytes. And this requirement will only grow. We'll come back to this point again later.

Moving to networking. Again, several things are happening in the area of networking. A very visible effort taking place is the process of covering entire cities with wireless network access (for example, this is planned for the city of Pune). The idea is that wherever you are in the city, you should have access to the internet. Ubiquitous networking. And the hope is that this will spread to more and more cities until most populated places on the earth are network enabled. The other process is, of course, the increasing availability of wired broadband access in more and more cities. Broadband penetration is increasing rapidly and more and more people are getting on to the internet. In parallel, optical bandwidths have gone through the roof. In a recent breakthrough, researchers at NTT were able to pump 14 terabits of data in a second over a distance of 160 kilometers. With a still-in-research technology called "All Optical Networks", these speeds are set to be dwarfed. We'll discuss the possibilities opened up by such speeds in another post.

The story on the computing side is also very exciting. 4 GHz processors are common on desktop PCs today. This is almost 400 times faster than an average PC only 12 years ago (when I got my first one). 3D gaming freaks are pushing even this to its limits and it is almost necessary to have high-performance graphics cards to play the more and more closer-to-life 3D simulation games. The sheer computing power available to an ordinary user today is mind-boggling by even very recent standards. As if PCs were not enough, mobile devices are pushing the computing envelope very rapidly. The Nokia N-Gage phone sports a 104 MHz processor in a package weighing a total of 5 ounces (140 grams) and the Palm Treo 700p flaunts 312 MHz of processing power in a 111x58x23 mm package. Many of us reading this probably remember owning and using PCs (the famed Intel MMX 200 MHz machine) with less computing power. At the higher end of the spectrum, companies around the world are trying to smash through the 1 petaflop barrier. With brains like that of Dr. Narendra Karmarkar (creator of the famous Karmarkar's algorithm for optimization problems) at work on this problem at Computational Research Laboratories (CRL), such a machine may be available sooner than we expect, opening up tremendous opprotunities for research into biotechnology, cosmology, defence and a plethora of other fields. Even companies like Google, Microsoft, etc are building up huge computational capabilities to support the paradigm of software as a service. In this way, compute power is increasing at all ends - the mobile and embedded devices, PCs as well as compute-clusters available with corporates.

This increase in capabilities at both ends of the spectrum seem counter-intuitive to many. On the one hand, SaaS (software as a service) is gaining currency and it seems that all heavy-duty data processing will take place on centralized servers, thus obviating the need for high-performance systems at the user-end. On the other hand, more and more applications are being pushed onto mobile devices raising the requirements of low-power high-performance computing in small packages. In this tussle, what is the fate of software and the end-user? What will happen to the PC? Will my refrigerator actually speak to the supermarket? I'll try to examine these questions in posts to come.