free geoip Jayson's Blog -
A conduit to the voices inside my head.

Jayson's Blog

  • Resurrected

    For a while I've hosted this website from a virtualized web server running in my house. The hard disk that was housing the VHD failed a few months ago, and I just now got around to getting all the files I needed off of it. Self hosting is a labor of love (emphasis on the labor part), so I've decided to renew my services with the good folks over at ASPNix. I'm back up and running, and will start posting some new content soon, renewing my series of "Computing Explained In Layman's Terms" as well as exploring more design patterns. Also, if anyone needs a site hosted, I have 24 domains left and would be happy to host your site for free, or perhaps for an exchange of services. Stay tuned!

  • Happy Blog Birthday to Me: Ten Years of Writing (But Not So Much as of Late)

    Ten years ago, I started this website as a simple experiment with very straightforward goals:

    1. To see if I could write my own blogging engine (which I did, it was very simple…the earliest capture of my site is available here from April of '05). It was quickly overwhelmed with traffic, and I migrated it to Scott Watermasysk's excellent .Text platform, which was later folded into Community Server, which is what this site still runs to this day, even though it has been deprecated, and is now a closed source non-free offering from Telligent systems.
    2. To see if anyone would actually visit a personal weblog of my authoring. At the time, yes, it was a rousing success, but it is the opinion of this author that personal tech blogs are passé. Ten years ago, having a tech blog filed under your name was very popular. Now, I'm not so sure.

    I've maintained fairly steady traffic over the years, but sites like stackoverflow dominate the tech sector, and rightly so. Other popular programming sites have basically put personal tech bloggers out of business, and again, rightly so. Why have a huge amount of disparate information scattered across the web, when a central repository of like minded authors serves the greater good? I'll be honest, I really would love to have this site take off again like it did back in the day: It took me a long time to build up an audience. But the plus side is I've met some incredible .Net developers along the way, due to this blog. I won't do a roll call, you guys know who you are, but because of this site, I have a pretty nice .Net network, people I still talk to virtually on a fairly regular basis. Because of this blog, I was able to run my own consulting firm, based on building "web presences" for a couple of years, so overall I'm pleased with what this site has done for my professional life. But finding a new voice, breaking some new ground…it's been tough. Back in the day, if you built it, they would come. Not so much anymore though.

    I go back and forth: Take the site down, keep it going, let it stagnate, or fire it back up…ten years is a long time, and I have poured so much blood sweat and tears into this site, both building and customizing it, and in the content itself. It's just so funny when I think back to version 1 of this site, I literally hammered out the database design, backend coding, and frontend design in about a week. I set up the domain name as a dummy forwarder with FreeDNS, pointing to a tiny server I had in my condo at the time (it was an AMD 667mhz box with 512MB/RAM, running on a 1mb/sec broadband connection, it was 128kb/up though…pitiful). I had no clue what a trackback was, I just started writing random content. My early writing was awful, but I was just writing for the sake of writing, just putting up content to see what happened, and then visiting various programming sites posting links back to my various posts. Then Google spidered my site, and traffic jumped through the roof. My initial codebase crumbled under the load, especially given the constraints of my home server setup.

    I'd love to have the time and money to redesign my site, and take it in a different direction, but in the end, it's just a content driven site. I keep it up because I still get a fair amount of traffic, and every once in a while, I'll get a comment on an old post saying "thanks for this post, it helped me do my job today" or an old friend will find me, and want to reconnect. I'm still not sure how often I'll update this site. I was on a roll with my design pattern series, as well as explaining complicated computing concepts in everyday terms category, but as is usually the case, real life steps in, and little pet projects like this site have to take a back burner. The good news is, running this site is cheap. I'm back to hosting it on a home server (a very powerful quad core machine with 5mb/sec up), so all I have to do is pay my yearly DNS hosting fee, and forget about it. But, my little tech site holds a special place in my heart, so I doubt I'll ever take it down. And perhaps one day, I'll get around to a redesign, or a migration to a more modern CMS. We'll see. Happy 10th blog birthday to me though.

  • The Most Common Question I Answer on a Regular Basis: "How Do I Learn How to Write Code?"


    This is one of the most pervasive questions asked in my industry from folks who are not professional software developers, but perhaps have an interest in pursuing it as a hobby, or as a career: "So, how do I learn how to write code?" I'll start with my own experience as to how I got into this industry as a professional, will mention how some of my colleagues got started, and will try to wrap it all up with what I would have done differently and/or the ideal way to learn how to write software.

    A common misconception about writing code is that it's difficult. Writing code is a very abstract process, and is only as difficult as you choose to make it. There are an infinite number of ways to achieve a working piece of software, some of them right, many more wrong. It's only through years of making lots of mistakes that you learn how not to write code, but I think very few professionals no matter what their skill level produce code that is 100% correct, and even the word "correct" in the software business is extremely objective. It's a craft that requires discipline, and the ability and patience to understand that you will make mistakes, miss deadlines, and go down tangents that end up in complete disaster. If you don't like constantly learning and reinventing yourself every few months, learning how to write code is not for you. You also do not need to be highly technical (though it certainly helps): Some of the best developers I've worked with wouldn't know a CAT 5 cable from a USB cable, but they can hammer out quality code like there is no tomorrow. Also, most folks think you need to be a mathematical genius to write code. It certainly doesn't hurt, but I think the most fundamental knowledge anyone needs to have is at least a high level mastery of Boolean Algebra and Predicate Logic and their sub-disciplines. Neither of these are as scary as they sound, and can be learned in a couple of weeks by anyone who has completed high school level algebra.

    So how did I learn how to code? I actually learned the basics of coding long before I ever even considered making the transition into becoming a professional. I took Geometry when I was in middle school, and this was right around the time that programmable graphing calculators were making their mainstream debut (the TI* line from Texas Instruments). I was probably one of the few students who took the time to actually read the manual, and I discovered that the TI shipped with a rudimentary form of the BASIC programming language. I quickly picked it up and started writing simple programs to do geometric functions like finding the area of a circle, or the volume of a tetrahedron. By the time that class was over, I had written what would be in programming parlance a fairly sophisticated console application that presented the user with a list of function names that the user could choose from, then supply the necessary parameters needed for the calculations, and provide the correct answer. I ended up selling this app for 5 bucks a pop to anyone who was interested, and the teachers were none the wiser. Cheating? Perhaps, but just like any computer is, my TI was a tool to accomplish a specific set of tasks. As with any tool, if you can unlock its full potential, productivity goes up, and you needn't burden yourself with repetitive tasks. I actually learned more about geometry through programming the calculator than I did in class, because it forced me to really understand how these functions worked rather than just memorizing them. For those who bought my little program, probably not as much.

    A couple of years later, my family purchased our first computer, a Macintosh LC II. I won't bore casual readers with the specs of that machine, but it was paltry at best. But, I started pouring through computer journals, and of course the LC shipped with a BASIC compiler, as well as this new language called AppleScript, as well as the precursor to the modern day framework of the web in that "cards" were "linked" to one another via "hyper" constructs. It was similar to Access, or more accurately, FoxPro in that it was a simple easy to use all in one RAD (Rapid Application Development) environment. I wasn't creating anything terribly useful, I just knew I thoroughly enjoyed typing instructions into a machine, and getting results back. I had no idea what I was doing was even a path I could eventually pursue as a career path. To keep this story short, fast forward to college (my first round) where I was majoring in Mathematics, and decided to take a computer science 101 course. It was this class that would cement my decision to pursue writing code for a living, I was exposed to the granddaddy of all modern day programming languages, C. Bear in mind Java was still just a twinkle in some engineer's mind somewhere, and the architect of .Net (Anders Heljsberg) wasn't even working for Microsoft. C and C++ were pretty much it in the programming space, and were core staples in any computer science curriculum. The power of C is pretty infinite: It powers the entire phone system, the power grid, nuclear powerplants…C pretty much runs most of the mission critical systems on the planet. It's worth mentioning that when I say I know how to program in C, I do NOT know how to implement the national telephone switching infrastructure, nor do I know how to write an operating system like Windows (which is written in C). The rest was history. I started taking numerous technical courses, as many as I could afford at the time that covered all aspects of computers, and within a year I was gainfully employed and writing code for a living. I initially started out in Java, then switched to .Net/C# (which is what I do most of my work in)…but this brings me to my #1 most important bullet point on learning how to code:


    I cannot emphasize this enough. Sure, plenty of programmers are experts in one specific field or area, and that's all they do (and are quite successful at it). What makes a great programmer is someone who becomes a student of languages and frameworks themselves, and learns the pros and cons of each one. No one can ever master all of them (there are thousands of languages), but knowing most of the big ones, as well as studying the history of programming (new languages always build off of old ones) is hugely beneficial. It's only by knowing from whence we came can we figure out where we're going. But, baby steps first. Once you completely understand my above underlined statement, it's time to hit the books and start doing.

    I read. A lot. If I'm not writing code, I'm reading about code, or studying the history of coding, or thinking about how new(er) languages can help me out in projects. This industry changes so incredibly fast, it doesn't take much to fall behind the curve if you're not reading. Define "a lot of reading?" Before I ditched my physical books in lieu of replacing them all with their eBook counterparts, I had well over a hundred tomes on languages, programming theory, design patterns…each one averaging about a thousand pages, and I read them all cover to cover, hammered out every exercise in each one…repeatedly. This is in addition to the thousands of online journal articles, blog posts, forum threads, user groups, videos, podcasts…you get the point. In this industry, if you're not tryin', you're dyin'. The good news is that once you understand the fundamentals of programming, and learn the jargon/nomenclature, picking up new languages and frameworks is fairly trivial. I'll provide a list of the languages/frameworks I consider myself fluent in (e.g. I use them on a regular basis to get my work done), and follow up with a list that I think all programmers should know, or at least know about (as some of them are a bit arcane, but still fundamental in modern language design). In no particular order, I use the following on a regular basis (I will not provide wiki links for each, please use Google if you want more info for a specific item):

    • .Net/C#
    • C++
    • Java
    • JavaScript/ECMAScript
    • HTML/CSS
    • SQL
    • Ruby/Rails
    • Python/Django
    • XML (and all X* variants)

    These are the languages I believe every professional programmer on the planet should know at least at a high level (concepts and theory). If you are new to programming, first learn:

    • C (for learning about pointers, the stack, and memory management)
    • BASIC (learning flow control/logic in programming as ALL languages implement these constructs)
    • SQL (if you are new, this language will not make much sense, but knowing WHAT it does is essential)

    If you can become fluent in the above, you can pretty much learn any new language. However, learning a slew of languages does not a competent programmer make. Computer science students are notorious for the following: In most computer science regimens, students are subjected to a barrage of languages, and come graduation time throw all of these on their resume/CV without actually understanding why they were shown these languages, and what their strengths are. Languages all programmers should know by name, and be familiar with the theory behind them (read: at least know how these languages influence modern day languages. I consider the following languages the most important contributors to modern day programming:

    • Smalltalk (the entire concept of Object Oriented Programming owes itself to this one language, invented at Xerox-PARC, and the precursor to windows based OS's (Windows, Mac OS, *nix)). I personally think this one language has had the greatest impact of any language on programming, and computing as a whole.
    • Lisp (this language pretty much invented the idea of artificial intelligence). The concepts behind this language are incredible, and all modern day languages borrow something from Lisp.
    • Functional languages: Candidates are Haskell, OCaml, Scala (the hot new language du jour), and F#.
    • At least one compiled "enterprise level" language (Java, C#, C++)
    • At least one interpreted "enterprise level" language (Ruby, Python, Perl, JavaScript)

    Ok so you've done your research, and have learned at least one programming language. What's next? Build something…and this is always the 2nd question I get after the person has gone off and learned a language: "Well, what should I build?" This is perhaps the hardest obstacle for new programmers. I almost always say "find a solution to a problem you are having"…such as my case where I decided to write a program to help me with my geometry formulas. Almost all of my personal projects involve making something I do on a regular basis easier to do with a computer. Or, I come up with little brain teasers to solve with algorithms, and then test the performance of my code. What are some of the brainteasers I've come up with?

    • Write an algorithm that accepts a string as an argument, and then reverses the string without using any built in string manipulation libraries that are available for the programming language.
    • Write a program that accepts a string as input, then tests to see if the string is a palindrome (e.g., reads the same way forwards as backwards, regardless of whitespace or punctuation).
    • Calculate the Fibonacci sequence using only recursion.

    I'll provide my C# solution to the 2nd bullet from above, as well as a C++ solution that a buddy of mine wrote: How to figure out if a string is a palindrome:

       1: using System;
       3: namespace Palindrome
       4: {
       5:     public class PalindromeString
       6:     {
       7:         static char[] _firstHalf = null;
       8:         static char[] _secondHalf = null;
      10:         static string _stringToTest = null;
      11:         static int _halfwayMark = 0;
      13:         static PalindromeString() { }
      15:         static public bool IsPalindrome(string stringToTest, ref int numberOfIters)
      16:         {
      17:             ++numberOfIters;
      19:             if (_stringToTest != stringToTest)
      20:             {
      21:                 _stringToTest = stringToTest;
      22:                 char[] stringChars = stringToTest.ToCharArray();
      24:                 foreach (char character in stringChars)
      25:                 {
      26:                     if (!char.IsLetterOrDigit(character))
      27:                     {
      28:                         stringToTest = stringToTest.Replace(character.ToString(), string.Empty).ToLower();
      29:                     }
      30:                 }
      32:                 _halfwayMark = (stringToTest.Length / 2);
      33:                 _firstHalf = stringToTest.Substring(0, _halfwayMark).ToCharArray();
      34:                 _secondHalf = stringToTest.Substring(_halfwayMark).ToCharArray();
      36:                 Array.Reverse(_secondHalf);
      37:             }
      39:             for (int i = 0; i < _halfwayMark; i++)
      40:             {
      41:                 if (_firstHalf[i] != _secondHalf[i])
      42:                 {
      43:                     return false;
      44:                 }
      45:             }
      46:             return true;
      47:         }
      48:     }
      49: }

    In C++:

       1: #include <string>
       2: #include <iostream>
       3: #include <ctime>
       5: using namespace std;
       7: static bool is_palindrome( string t )
       8: {
       9:     const char *char_data = t.c_str();
      11:     for( const char *start = char_data, *stop = char_data + t.length() - 1; start < stop; ++start, --stop )
      12:     {
      13:         while( !isalpha(*start) )
      14:             ++start;
      15:         while( !isalpha(*stop) )
      16:             --stop;
      17:         if( tolower(*start) != tolower(*stop) )
      18:             return false;
      19:     }
      21:     return true;
      22: }
      24: static bool is_palindrome_sl( char *t, int length )
      25: {
      26:     for( const char *start = t, *stop = t + length - 1; start < stop; ++start, --stop )
      27:     {
      28:         while( !isalpha(*start) )
      29:             ++start;
      30:         while( !isalpha(*stop) )
      31:             --stop;
      32:         if( tolower(*start) != tolower(*stop) )
      33:             return false;
      34:     }
      36:     return true;
      37: }
      39: static bool is_palindrome_ss( char *start, char *stop )
      40: {
      41:     for( ; start < stop; ++start, --stop )
      42:     {
      43:         while( !isalpha(*start) )
      44:             ++start;
      45:         while( !isalpha(*stop) )
      46:             --stop;
      47:         if( tolower(*start) != tolower(*stop) )
      48:             return false;
      49:     }
      51:     return true;
      52: }
      54: static bool is_palindrome_f( string t )
      55: {
      56:     const char *char_data = t.c_str();
      58:     for( const char *start = char_data, *stop = char_data + t.length() - 1; start < stop; ++start, --stop )
      59:     {
      60:         if( tolower(*start) != tolower(*stop) )
      61:             return false;
      62:     }
      64:     return true;
      65: }
      67: static bool is_palindrome_fsl( char *t, int length )
      68: {
      69:     for( const char *start = t, *stop = t + length - 1; start < stop; ++start, --stop )
      70:     {
      71:         if( *start != *stop )
      72:             return false;
      73:     }
      75:     return true;
      76: }
      78: static bool is_palindrome_fss( char *start, char *stop )
      79: {
      80:     for( ; start < stop; ++start, --stop )
      81:     {
      82:         if( *start != *stop )
      83:             return false;
      84:     }
      86:     return true;
      87: }

    Once you've come up with a few short programs/algorithms to write, start writing! One thing I have done in the past is that I will attempt to write each of my algorithms in multiple languages, and then compare performance, how long it took to implement the algorithm in each language, and the ease of which it took to author. Repetition is key here, and once you begin to do the above over, and over, and over, you will begin to learn the intricacies of various languages and their strengths and weaknesses (something I do on a regular basis is "translate" applications from one language to another…note that this does not mean simply translating the syntax itself, you need to to accomplish the same functionality into the target language's best practices, and utilize any built in libraries to further simplify the solution).

    Like any other discipline, repetition is key. Practice, practice, practice. Write code for fun. As strange as this may sound, I view many common problems that I see on a daily basis, and I then start to think to myself: "How could I solve this via a computer program?"  It's said that math is the universal language that can be spoken by anyone. I think computer programming is almost on par with math. Computing is ubiquitous amongst all industries, ergo, learning to speak the language of computers is a universal language that can form a bridge between disparate systems and frameworks. Computers glue everything together, and knowing the underpinnings of how computers work at a fundamental level, at the code level, will empower anyone who uses a computer to get their job done to make the machine work for them, and not against them.

  • Finally Found a New Voice for This Blog: Repurposing and Mission Statement

    I've been struggling over the past couple of years to find a new purpose in keeping this site up and running. Over the past 2-3 years, most of my posts have been random in nature, and few and far between. I started this site back in 2003, so it's been almost 10 years since my first post. Back then I was just getting into the realm of professional IT (my first "real" IT job was in 2002), so naturally I started this off as a software development platform. .Net was brand spanking new, and good information was hard to come by, so I started writing about whatever I was learning about with the hopes that it would help others who were also trying to digest the big bad new world of .Net and C#/Asp.Net.

    Fast forward to the present: .Net is a mature platform, and there are hundreds of thousands of resources available on the web that are much more in depth than what I have time to write about. Plus, most of the bloggers that I "grew up with" in the blogging world have grown up and moved on in their blogging endeavors (and, we've all been using .Net for a decade+ now, and no longer read programming based blogs…to put it bluntly, we know our stuff now). Of course we all still struggle from time to time, and when MSFT drops new tech on us, we have to hit the books and learn.

    I was speaking with a friend of mine earlier today about computing, and without even knowing it, he summarized what the direction I want to take my writing towards: Empowering users to take control of their computers, and not feeling like their machines control them. My last couple of posts have been high level explanations of low level stuff, lots of metaphors and analogies to help explain computer fundamentals in a concise form that anyone can understand. I will still post code from time to time, but it will be more architectural in nature, but the vast majority will be breaking down what most folks perceive to be complex computing topics, when they actually are not.

    Computers are fundamental to our lives, and virtually every job utilizes computers, networks, and the internet in some way shape or form. Much as a plumber has to understand his tools, the materials he uses to get his job done, and the fundamentals of running his business, the same goes for anyone who uses a computer as their main tool to get their job done. By gaining a decent understanding of your machine, you can learn to work more efficiently, and make your computer work for you instead of against you. People think that computing, networking, and programming are these big mysteries better left alone, when in fact quite the opposite is true. Computers and how they work are eerily similar to how we operate as humans: Our interactions with others, running businesses, socializing, relationships, pretty much anything we do on a day to day basis can be analogized to computing in general.

    So, that's the direction I will attempt to take this blog towards. I want to empower users to utilize aspects of computers, networking, and programming that they may not even know exist. It'll be techie enough for hardcore users, but hopefully easy enough to understand for those who are simply curious as to what goes on inside the beige box, but perhaps aren't very technically inclined. I come from a software engineering background, so many posts will be related to programming concepts, operating system underpinnings (see my previous post on processes and threads), networking, and the internet. We'll see how it goes, but I think I've finally rediscovered my voice. Suggestions for topics and questions are welcome, if a topic gets enough requests I'll hammer out a post for it.

    Finally, this has an equal benefit for me. Sometimes I think we IT professionals take our knowledge for granted, and have forgotten just how hard it was to get into this field, and actually make a living at it. My roommates occasionally overhear me talking tech on the phone, and they swear I'm not speaking English (in all fairness, they are all engineers…they also speak what sounds like a foreign language to me when they are talking shop, but I find it fascinating). I believe that knowledge is useless if you can't give it back to others, and I know how much I learned from other seasoned professionals when I first started out. So we'll see where this goes while I fine tune the overall direction of this site. Feedback is more than welcome.

  • Why Are Modern Browsers Such Memory Hogs? A Short Primer on Processes, DLL's, and Threads

    I normally don't like to participate in the so called "browser wars" and that is not the intention of this post. But, I feel that the title poses a legitimate question: What is going on with modern browsers sucking down memory like tequila shots these days? As a web developer, I have a slew of browsers installed on my machines for testing:

    • Internet Explorer (unfortunately): Necessary evil, plus it's the only browser that supports Windows Integrated Authentication, which is mandatory on the business LAN…but in all honesty, it has gotten much better over the years.
    • Firefox: I'm a creature of habit, and have been using Firefox since the early betas, so over a decade. Plus, the plug-in support is phenomenal as are the built in web developer tools.
    • Opera: I'll be honest, I abhor Opera. Great idea (and I feel quite the opposite about their mobile browser on iPhone, it's fantastic, mainly because Opera routes web pages through their servers and compress them down to a fraction of their original size, super speedy).
    • Chrome: I'm neutral on Chrome. I love the V8 JavaScript engine…blazing fast, but I just don't really like Chrome in general. Of course the Google integration is nice, but I don't use that many consumer Google products to make the integration worthwhile.

    Another thing this post isn't meant to be is an all out comparison of different browsers as that is just too objective to even begin to approach. Really I just want to pose the question as to why these browsers are such memory hogs.

    A little history: When the internet came out, you had three choices of browsers: IE, Netscape, and AOL (which I believe was initially based on Netscape's Gecko engine, then later they licensed the Trident engine from Microsoft's IE). I used Netscape. Then, Netscape ceased to be, and a bunch of their developers went on to start writing Firefox as a completely free and open source solution based on the Gecko engine. 1.0 was released in late 2003, and it was an instant hit, namely due to the extensive library of plugins available. Plugins are now the industry norm, but back then it was a novel idea. Chrome wouldn't be released by Google for another 5 years (in beta) and the final version wasn't released until 2010. It quickly took off though, and depending on what browser stat site you visit, it's number one or two (I don't believe any of these sites by the way). Chrome runs on the WebKit engine, which powers Apple's Safari browser as well. One aspect Chrome really has going for it is the V8 JavaScript engine; it absolutely SMOKES the competition, mainly because it compiles JavaScript into machine code instead of bytecode.

    Back to Firefox: Since day one, it has always been known as a memory hog. In 2003, 4gb RAM was pretty much the standard maximum amount, unless you were willing to take the 64bit plunge with Windows XP 64. Windows 32bit can only address ~3.5gb of memory. This is due to Physical Address Extension limitations in the Windows Kernel, but that's another post. Regardless, Firefox with a few plugins installed could easily soar to half a gig of memory under prolonged use. When Firefox introduced tabs (another browser first), memory went through the roof. But, tabs were a nice addition rather than having to open up numerous instances of a browser. The downside is that if one tab decided to misbehave, it tore down the entire browser with it since they all ran in the same memory space as a single .exe process. Multiple instances of a browser each had it's own process space, thus if one instance acted up, it only took that one instance down with it.

    Then Google came out with Chrome a few years later, with a great idea (which in hindsight I'm shocked wasn't thought of sooner): Host each tab in its own process space, that way if one tab crashes, it won't take the browser down with it, and it can spin itself back up. To see this in action, if you have Chrome installed, fire up Task Manager and look for all the Chrome processes. There will be one per tab, and one per extension…each is sandboxed and can't interfere with any other process. Firefox would later take a similar approach, but only with tabs (more on this in a bit) in later releases of Firefox. Competition benefits the consumer, and to be honest, Chrome is the best thing that ever happened to Firefox, and vice-versa. Both browsers are extremely mature now, leaving IE in the dust.

    Sidenote: Why haven't I mentioned IE's memory consumption? Simple…there really isn't a way that I know of to actually figure out how much memory IE uses. A common misconception about IE is that it is much leaner than Chrome or Firefox, this can be gleaned by firing up IE, and looking at the memory consumption in Task Manager; it's usually a fraction of the other two. And it spins up much faster. But, both of these are misleading: IE is completely baked into Windows, and its Trident rendering engine is used by Windows in a bunch of different areas:

    • Windows Explorer uses Trident for rendering the folder views.
    • Windows' built in help uses Trident.
    • Office uses Trident
    • This list is already long enough…

    Point being, most of IE's memory consumption is part of other processes and can't be gauged very easily. Also, the reason it appears to spin up so fast is that the majority of IE's spinning up occurs during the Windows startup process: Most of IE is paged into memory by the time you get to the desktop. But I digress.

    For the less technically inclined that may stumble across this post, a brief explanation of what a process is, and why they are safe, as well as some caveats. When you open an executable such as IE, Chrome, or Firefox, Windows loads the process .exe into a dedicated space in memory, which is what you see in Task Manager. Processes are completely isolated from each other, this is enforced by Windows. Think of it like an apartment complex: There is a wall between each .exe, and each .exe occupies just one address space. Processes almost always load Dynamic Link Libraries (dll's, which I'm sure everyone is familiar with) into their own address space, which is why you don't see them listed in Task Manager (though numerous tools exist that allow you to see which dll's an exe has loaded, handy for debugging). Think of a dll as the furniture in your apartment; they are part of the apartment itself. If a single .exe process fails, it cannot take other processes down with it, they are isolated. I equate this to an apartment fire (loosely): If your neighbor's apartment catches fire, it should be contained by the firewalls between the other apartments, and is contained (theoretically at least). However, if your furniture catches fire, it's going to take the apartment down with it; if a dll misbehaves, it will take the entire .exe process it's loaded into down as well. There are ways to code around this, but it's trickery and easy to do wrong.

    The above is why all the major browser vendors have moved to the isolated .exe design paradigm. But, it comes at a cost. Dll's exist for a couple of reasons:

    • They are reusable chunks of code. Multiple applications can share a single dll's logic without having to have their own copy. This is why the Windows registry exists: If a DLL is registered properly, it gets a unique identifier (a GUID) called a CLSID (Class ID), which in turn can be looked up by a much friendlier text identifier called a ProgID. This is well beyond the scope of this post, just know that Windows exposes thousands of dll's that applications can use without having to reinvent the wheel.
    • Since they are loaded into the same memory address range as the .exe that needs it, they are fast. Orders of magnitude faster than cross process calls, or cross thread calls.
    • Dll's (if programmed correctly against the COM specification) are guaranteed to be usable by any other component on the machine, or even across machines. Again, beyond the scope of this article.

    The biggest limitation is the one I mentioned above, and if two applications want to use the same dll, each has its own copy loaded into memory, so cross dll communication between processes is very difficult to do, and frowned upon.

    Exe's are great in that they are isolated, but they come at a huge cost: Much higher overhead. I won't go into the details as to what Windows has to do to spin up a process, but it's very complicated. They use much more memory than dll's. And the biggest overhead is making cross process calls. If you look at Chrome's built in task manager (right click the title bar and choose Task Manager) you'll see something like the following (and if you really want to geek out, type chrome://memory-redirect/ in a new tab…lots of useful memory information). This is what mine looks like with one tab open (HootSuite) having only been open for 30 seconds or so. That's a lot of memory consumption. Start opening new tabs, and use it for a couple of hours, and it gets out of control fairly quickly. Yes, I have a lot of extensions installed, but I use them all. Seeing this is what prompted me to write this post actually, and I wouldn't have even noticed it had my computer not started paging to disk like crazy after opening Chrome to test a bit of code:

    Each entry has a corresponding entry in Task Manager, which means each is running in its own process space. The Browser process is like the leasing office at an apartment complex: It has "keys" to each of the child processes (apartments), but they don't have keys to the office. And of course, they don't have keys to each other. If the Browser process fails, Chrome crashes (though I've never seen this happen). If any of the other processes crash, only it goes down, rebuilds itself, then spins back up and tells Browser to load it back into the browser window (ideally at least). For the most part, this is a bulletproof design. But like I mentioned above, it's the cross process communication that makes performance suffer. Back to the apartment metaphor: If you want to move the furniture around in your apartment (the dll's) you just get up and move it, almost no effort required. But, if you have a leaky faucet and need the office (Browser process) to fix it, you have to put more effort into it by either calling them, physically going to the office, etc.…and it takes a lot more time to get accomplished: They have to put in a work order, get it scheduled, get the parts they need, etc. And tab processes further complicate the issue, view tabs as a group of apartments inside the overall complex itself. Plugins go through the tab process(es) to get to the Browser (usually), so that's 3 layers: Browser holds multiple Tabs, Tabs hold multiple Extensions…memory starts adding up quickly.

    This is the equivalent to a cross process call: For the child process to communicate with the coordinating process (Browser) takes a lot of time, and the coordinator is busy busy busy, so if it is busy handling another request, the callees have to wait (kind of…multi-threading solves a lot of these problems, but with a completely different set of ramifications that I won't even begin to get into…threading is an extremely difficult programming concept that I still struggle with from time to time, see the addendum at the end of this post for a further apartment analogy). These calls incur quite a bit of resource overhead: Processing cycles, memory consumption, and time.

    Ok enough of the history lesson and technical mumbo jumbo. And for the record, Firefox is memory heavy as well, but about half of Chrome with the equivalent tab open, and about twice as many extensions. It grows over time, but not as nearly as much. So, you'd think that over the years, these browsers would be lean mean rendering machines. But, memory consumption has gone up, not down. I have a few hypothesis about this:

    • Extensions, plain and simple. I have never written a browser extension, but most of them make heavy use of JavaScript, which is processed by the browser itself. And many extensions are authored by what I like to call "basement coders"…most of my extensions are written by seasoned professionals, but you never know. A badly written extension can send memory consumption through the roof.
    • Web pages themselves. Now that bandwidth is plentiful, browsers are maturing, and computers are much more powerful, the browsers do the bulk of the work for websites these days. I'm sure most readers of this blog have a Facebook account. Go to Facebook, right click anywhere in the page, and choose View Source. That is a lot of code for the browser to parse through for each request, and now with Ajax technologies, the browser is constantly doing work behind the scenes via JavaScript. All of this requires memory. Of course this is desirable, we get an extremely rich browsing experience, gone are the days of static HTML and boring horribly designed websites.
    • And finally (I keep mentioning this in other posts): Because they can. Memory is cheap, bandwidth is plentiful…that's no excuse in my opinion, but it is what it is.

    Don't get me wrong, I like Chrome, but I disagree with their architecture. Per tab processes is a no brainer, but a process per extension? I disagree with the design of that (but not the motivation behind it). Firefox uses the tab per process design, but lumps all of its extensions into one extra process, Yes, if one extension goes down, it'll take all the other extensions down with it, but Firefox will rebuild that process (which takes more time than one individual process) and recover nicely. Without taking down the offending tab, or the Firefox process itself. In Firefox, plugins do get their own process (not to be confused with extensions by the way, a Firefox plugin is something like Java, or Flash…but these must be isolated as they run with much higher privileges than extensions. Firefox does use the sandbox design for extensions, but they don't use process boundaries to do so, they have a homegrown solution (see addendum two for an explanation using the apartment analogy) to make sure extensions can't read other extension's data, and each extension belongs to one owner tab…think of this like the dll design in Windows, and the equivalent performance gains you get from using in-process loading. I just don't understand Google's implementation of per-process extension loading, but I'm assuming the smart folks over in Mountain View have a good reason to do so. But, at the expense of overall performance and resource consumption.

    Neither is a perfect solution, but memory on computers is still the scarcest resource on any machine, especially 32bit machines (such as the laptop I'm writing this post on). My desktop machine has 8 gigs, and my server has 16 gigs…and even that isn't enough these days. 4 gigs is like living in the stone age of computing, but I'm stuck with it on this machine. Chrome by itself can take up to 25% of that, so it's not an option for me. I wish both vendors would take a long hard look at how much the web itself has evolved since their respective browsers were released: It's a completely different web now, and web pages are consuming more and more memory. Be proactive in that regard. Also, get on top of the plugin developers about proper design (I'm not saying make the extension realm a police state like the iTunes App Store is, but at least I know the iPhone apps I install went through some sort of QA checks). And finally, just trim the fat from the browser itself. There is no reason why something as basic as a web browser needs to bring modern machines to their knees. Geeky addendum time.


    Addendum One: Threading.

    Threading is one of the holy grails of computing, and is also one of the most widely misunderstood and abused programming paradigms. Geek explanation first, then onwards with my apartment analogy to simplify it. A process can host multiple threads. A thread is a line of execution within a process: A thread carries out its set of logic independent from other concurrently running threads, though you can share data between threads. When you open a windows based application like Word, or a web browser, a process spins up (the .exe) and spawns a single main thread, usually the thread that hosts the window itself. A single threaded application only ever has one thread, and all lines of code execution run on this thread. This is undesirable, and I'm sure most of you have experienced why. When a thread executes an instruction set, it blocks all other requests until it's done (forcing other requests to queue up behind it). If you've ever clicked a button in an application, and the window freezes or says not responding, that's an example of thread blocking/queuing: The thread is busy, and when you click on it or try to drag the window around or minimize it, that request is queued until the thread can service the request.

    Single threaded apps are rare, and most modern programming frameworks support multithreading. But, writing a properly working multi-threaded application is not a trivial task. Multi-threading works like this: The main thread can spawn multiple worker threads at will to carry out different lines of code execution. This solves the queuing problem for the most part as an application can get more work done at the same time, and blocking/queuing is drastically decreased. Getting all of these threads to play together nicely is where it gets beyond complicated, even for modern programming frameworks.

    Let's take an extremely simplistic example before I move on to the apartment analogy. Suppose you have a very simple windows based application that allows you to calculate n number of primes and their values up to a specified number. The GUI is simple enough: a textbox that allows you to input the ceiling value, a button that says "Compute", and some sort of control that displays the values themselves. For small numbers like say, primes between 0 and 20, the result is instantaneous and will not block the main thread. But let's say you wanted the values of all primes between 0 and 1,000,000,000. This is going to take some time, no matter how tight your algorithm is. If this runs on a single thread, the application will be unusable until that routine has finished running. However, if you kick off the computation routine on a child thread, that thread will go do its work in the background, and the app is free to do other things, like find the factorial of a very large number, or do the Fibonacci sequence for 1,000,000 values (each of these on their own thread as well of course). But, these threads need to eventually give the result back to the main thread so it can display the value(s), so these threads must "merge" back into the main thread somehow, and they also have to let the main thread know when they are done. This can be accomplished in a couple of different ways:

    • Callbacks (also known as a semaphore): The child thread notifies the calling thread that it is finished and is ready to supply the result; it basically transfers control back to the calling thread, then terminates itself. The parent thread can either stop the work it is doing and process the result, it can continue its work and tell the child thread to wait until its done, or it can tell the child thread that it needs to wait for another child thread to finish its work first (in case results need to be processed in a specific order). This would be the equivalent of a child thread jumping up and down (just like real children) and saying "I'm done with my work, come check it out!"
    • Polling (also known as a spinlock): This is used in cases where the parent thread doesn't have much work to do, so it keeps polling child threads to see if they are done with their work. This model assumes the parent is done with its work and is waiting. It can either process the data as it comes in, or poll all child threads until they are all finished with their work, then move on. This is similar to an actual parent checking on a child who is supposed to be cleaning their room, or perhaps you've given multiple kids work to do (you, clean your room, and you do the dishes), and you can't run errands until they're done with their work.

    There are other paradigms, but these two cover most cases. Where threading can get really nasty are two conditions that can occur:

    • Deadlocks: Two threads try to access the same data at the same time, with each thread waiting for the other thread to then release the value. Both are contending over the same data. Modern frameworks can assign weights to different threads, and after a set amount of time, the higher weighted thread wins. This is the equivalent of kids fighting over toys, then the parent stepping in and picking who wins (the calling thread in this case).
    • Race Conditions: A thread reads a data value that it needs to do a computation with…right before it does the calculation, another thread changes the value of the data, and the first thread then computes the wrong calculation. Realize that computers are very fast, all of this happens on the order of nano-seconds. Race bugs are some of the hardest bugs to squash in programming, which is why the notion of locks exist: You can lock a data structure which basically says "while I'm doing this, no other threads can access my data"...this would be like a child coloring a picture, then gets distracted by the TV…his sibling comes in and decides he likes the color blue better, and colors over the existing picture. The TV show ends, and the other sibling comes back and sees that it has changed colors. This can be prevented by the first child locking his bedroom door while he goes to watch his TV show.

    Inter-child-thread communication is about as difficult as it sounds if we keep rolling with the child analogy. Two kids are pretty easy to maintain, especially if there are two parents in the house. But let's say you have 4 kids, or more…trying to get a set of tasks done on time, getting the kids to cooperate and share tasks (or even better, have them doing multiple tasks at one time, or switching tasks if they get bored with what they're doing), keeping them on task without them wondering off to watch TV, or get into fights with each other, and ideally they come and check in with parents when they are done so they can be told to do something else (see callbacks/polling above)…this is eerily similar to getting computing threads to play nice as well. Just like parenting, it is much more art than science. I won't go into detail here, but very strange stuff can happen when threads stop cooperating with each other.

    Back to explaining threading in general, and why threading is a good thing if done correctly. Continuing on the apartment analogy: Something breaks in your apartment, and you need the office to fix it for you. If only one person worked in the office, and had to take care of everything by themselves…well, not much would get done in a timely manner. You head off to the office to consult with what is sure to be the most miserable property manager in the world, and discover a line (queue) of other tenants. The manager can only process one request at a time, and you have to wait your turn. At some point, he has to leave the office to actually get the work done  The single thread (manager) can only tend to one process (apartment) at a time, and it's very time consuming as the tenant can't cook, use the bathroom, wash her hands until the water issue is fixed (the thread is blocked until the work is done).

    In a multithreaded design, there would be an office manager who directs multiple workers, each of whom can take service requests. The parent thread would be the manager, and the child threads would be the subordinates. If there is only one thread and it needs to process 8 requests, this is 4 times slower than 4 threads needing to process the same amount of work: 8 requests at 2 minutes each with one thread = 16 minutes. 8 requests at 2 minutes each with 4 threads = 4 minutes. Vastly simplified, but you get the point. The office manager then prioritizes (assigns a weight to) each work order based on severity, time to complete, and ideally gives the work best suited for each individual worker. Workers might need to cooperate on some jobs, and that can either go through the manager, or they can just call a co-worker. If the worker is busy, the calling worker can wait, or go do something else. Regardless, this is much more efficient provided the manager is good at what she does, and the workers are efficient.

    Almost every new programmer I've met or trained, upon discovering threading, their faces light up and I know exactly what they are thinking: I'll just throw a bunch of threads into an application, compute the largest known prime number known to mankind, and get published in all the journals. If threads were the solution to everything, then it wouldn't be nearly as difficult to implement as it is. Here are the caveats (and they are huge):

    • Too many threads create more work and processing bottlenecks than they are worth. Threads are great up to a point (if you've made it this far, do yourself a favor no matter what industry you are in and read the book entitled "The Mythical Man-Month"…it's geared towards the software industry, but can be applied to virtually anything you want to apply it to. When you write multi-threaded applications, it's up to the operating system to maintain the threads via the Thread Scheduler (also known as a context-switcher). In a nutshell, based on parameters you specify in your code, the thread scheduler then decides how much time to give each thread, maintains the threads themselves, takes care of merging, joining, data exchange, and which thread should run at what time and for how long of a duration. These are all called context switches, and can actually decrease performance if too many threads are used since the work gets too chunked up, and the context switches themselves aren't free, they incur overhead.
    • So, how many threads should I use then? Until relatively recently, consumer machines had a single processor, with a single core. Multithreading has been around since the early days of computing, but on single processor machines, it was an illusion. Apps seemed multithreaded, and OS's seemed to be able to run more than one task at a time, but what was happening under the hood is that the scheduler was simply switching between processes rapidly (again we're talking nanoseconds here, not perceptible to humans) to give the illusion of threads: This is called multitasking (and is similar to how we use that word in the human world, doing more than one thing at a time, and placing priorities on each task). To only give a processor a single task would be a huge waste of resources as computers are very good at one thing: Doing tasks extremely fast.
    • Multiprocessor systems are the de-facto standard now, as are hyper-threaded processors. My desktop machine has a dual core hyper-threaded processor, that's 2 physical cores, and 4 logical cores. My laptop is a quad core hyper-threaded setup, so 4 physical cores and 8 logical cores. One thread can be handled by one core, so an application could run 8 threads concurrently with no problems at all on my laptop, without the need to multitask. There is no hard fast rule, but generally speaking it shouldn't be any less than the number of logical cores on the user's machine. All of these environmental factors are discoverable at runtime as you won't know what configuration the user is running, and numerous libraries exist to ease the burden of threading. When you factor in multi-tasking, and thread pooling (and lots of testing) you can nail down a firm number. Most consumer apps won't need more than 2x the number of logical cores though.

    So why not just through a gazillion threads at a complex computation? Back to our apartment manager. She thinks to herself "well, I can get more work done if I hire more workers, and our tenants will be quite happy when their issues are addressed in a more timely manner." So, she goes out and hires a slew of new workers, let's say she hires 20 new workers to bring the total number up to 24. This creates several problems without really solving any:

    • If there isn't enough workload to keep each worker busy, she's losing money by paying them to do nothing. Same thing with unused threads, they are sitting around consuming computer resources with no benefit to the system (these kinds of factors weigh heavily into systems design by the way: Too many times I've seen managers go out and build a massive server with terabytes of storage, 100's of gigabytes of RAM, and a processor that's so fast it can travel forward in time and compute a result set before the user even knows they need it. The system is deployed, and the processors peak at 10% usage, the app only consumes a few gigs of memory, and the database barely tips the scales at maybe 20 gigs in size. All of those extra resources are sitting around doing nothing. This is especially common in the power-user consumer space. Go to any forum geared towards pseudo-computer geeks. You want resource usage to be at about 80% of capacity at all times in business systems, and consumers almost always buy more computer than they'll ever need, 95% of the time your machine is sitting there doing nothing at all).
    • The flipside is trying to manage all of these workers. Soon she finds herself in the managing workers business, and not the fixing/renting/marketing apartments business. She's swamped doling out work orders, figuring out scheduling, holding meetings, inter-staff conflicts, workers calling in sick, workers getting work orders wrong, ad nauseam…in the end, she suffers as do her tenants. Just like in the business world where trying to figure out the right balance of number of workers vs. tenant satisfaction is a balancing act, the same goes with threading and is the equivalent to wasted resources and context switching.

    All of these are vastly simplified, but should give a general idea as to how threads work, and the pitfalls that can crop up, and how it relates to the initial topic of web browsers and resource consumption.

    Addendum Two: Alternatives to processes

    This will be a much shorter addendum, and the apartment metaphor will be a bit more vague, but this topic is worth mentioning: There are alternatives to threading (though they do still need the threading infrastructure, what they don't need are separate processes to implement the isolation they need). As stated in the core of this article, modern browsers run different aspects of the browser in separate processes: They do this for isolation, security, and stability, but at the expense of cross boundary performance issues, memory consumption, and complexity. There are two alternatives in the Windows space to spawning individual processes, and both of them piggy back on the threading paradigm mentioned in addendum 1. As I've already been over threading, I'll compare these two to processes.

    We've already covered just how slow and expensive processes are, which begs the question "is that are only choice?" No. I know of two fairly well known alternatives (and no doubt there are more, but these are part of Windows itself so why reinvent the wheel). They are:

    • Fibers. Though now fairly obscure, this was Microsoft's attempt at implementing lightweight processes that exist within threads themselves. I'm the first to admit that I don't know much about Fibers, other than they attempted to implement the benefits of processes, without all the overhead. A fiber is like a process within a process (actually a process within a thread). I'll leave it at that, curious readers should consult Google for more information.
    • .Net has the notion of Application Domains. A few resources before I continue:
    • .Net AppDomains are similar to fibers in purpose, but different in several ways. First off, they can only be used the .Net framework, and languages that adhere to the CLI standard. Secondly, processes host threads, which host fibers, whereas a process hosts AppDomains, which hosts threads. As stated before numerous times, processes are very expensive. AppDomains are not, but come with all the benefits of processes, and AppDomains can host dll's as well. A short list of what AppDomains do:
      • Multiple threads can exist within a single application domain.
      • An application within a domain can be stopped without affecting the state of another domain in the same process.
      • A fault or exception in one domain does not affect an application in another domain or crash the entire process that hosts the domains.
      • Configuration information is part of a domain's scope, not the scope of the process.
      • Each domain can be assigned different security access levels.
      • Code in one domain cannot directly access code in another.
    • AppDomains can communicate with other AppDomains running in the same process while still guaranteeing the above bullets via a process called Marshaling, which is much faster and more efficient than a cross boundary process call. For even cheaper cross domain calls, you can create a ContextBound object which can be passed by reference (a managed code pointer) rather than the deep copy by value passing that Marshaling uses. I wrote an article on ContextBound objects 7 years ago if you'd like a more in depth discussion.

    So of course the logical question is "why not just use one of these two alternatives on Windows?" The first answer lies within the question itself: These two technologies are tied to Windows only. Chrome and Firefox are cross-platform: They run on Windows, Linux, and Mac OS X. As such, the majority of the code needs to be portable, otherwise it becomes a maintenance nightmare. Second: Fibers are largely deprecated (not officially, but no one uses them, and they are poorly documented), and you need to write your own scheduler. Finally, AppDomains are available only in .Net, and languages that fully implement the CLI. So why not just write a web browser in .Net? It's Windows only for starters (though Mono (the open source implementation of the the ECMA .Net standard) is supported on numerous platforms, but not fully baked yet). And, .Net is not suited to writing something as complicated as a web browser: Performance would be beyond awful. I am not downing .Net, this just wouldn't be its forte.

    My point is that I believe there are better solutions out there, or one that could be hand rolled and implemented within the browser itself. Throwing a bunch of processes at a problem is worse than over-threading in my opinion. With one of the paradigms above, you could have a single parent process that hosts multiple app domains, which in turn load the dll's they need to get their work done: If an app domain fails, it won't bring down other app domains or the parent process with it, all you lost are the threads and the child dll's, and app domains are very cheap to respawn. Implementing something similar to the above constructs could shave off some serious bloat from the browser and its extensions (unfortunately, we cannot control how much memory a web page consumes when it is rendered, that's up to the developers themselves).

    This is where my apartment analogy starts to break down a little bit (and I'm only going to use app domains), but I'll give it a shot. Our apartment manager is pulling her hair out trying to manage all these workers (threads) and needs help. Too much work to be done for tenants, but managing all these workers is bringing down the core businesses: Satisfying tenants, renting apartments, and marketing to would be tenants. This isn't going to turn out well, but she doesn't have the option of firing workers since there is plenty of work that needs to be done. She decides to add a buffer layer between her and these workers, and hires 4 supervisors to oversee 6 workers each. Each of these supervisors provides an isolation boundary between their department and the other ones. Our manager (the process) now manages 4 supervisors (the appdomains), who in turn manage 6 workers (threads) of their own. Of course, intradepartmental cooperation will need to occur, but instead of the workers just calling any co-worker to help them out, they have to go talk to their supervisor, who can then call another supervisor to see what his workers are doing, and get the resources they need scheduled (this is Marshaling between app domains). Also, if one supervisor's workers screw something up, it doesn't affect another supervisor's workers (isolation boundary implemented by the app domain), and workers need to go through their supervisor to get work scheduled from another department (thread pool). So here's our finalized workflow:

    • A tenant (a separate process) calls the office manager to get a work order created.
    • The manager creates work orders and starts assigning them to the supervisors (app domains). She doesn't need to be concerned with the details like scheduling, allocation, checking for completion of work…this is all handled by the supervisors now, and she can focus on the actual business end of her job.
    • The supervisor organizes her workers (threads) and puts them to work, checking in occasionally (polling) and waiting to obtain the results of the work order (callback).
    • She can then let the manager know what resources she has available and waits for more work to be assigned.
    • Individual workers have to go through their supervisor first if they need extra help getting a task completed (marshaling): They are oblivious to other threads in different app domains, as they should be. Their supervisor manages joint projects (thread merging and joining).

    So in the case of our fictitious apartment complex, more work gets done in a timely fashion, resources are utilized more efficiently, and each individual can focus on their own work without getting bogged down with tedious tasks which interfere with the business.

    Would this be difficult to implement initially in a web browser? Yes, but I believe this design could really benefit most applications, not just web browsers…browsers are just easy to pick on because A) everyone who has internet uses them and B) it provides a better definition of the chain of responsibility for components and subsystems.

  • On Microsoft and Backwards Compatibility: Windows 8 (and How to Make it More Usable)

    Note; If you don't feel like reading this entire post, at least read this: Start8 from Stardock Corp (30 day free trial, 5 bucks after that, and well worth it IMO): Get your start button back, disable hotspots, and boot directly to the desktop with that utility. Otherwise, read on.

    This post is not going to be a diatribe either for or against Windows 8 (though I will give a brief opinion towards the the end of this post), but rather on Microsoft's stance on backwards compatibility vs. other operating system vendors.

    Love or hate Microsoft, they bend over backwards to preserve backwards compatibility between OS releases. If you wanted to, you can still run DOS games on Windows 7. Code written on their old 16 bit systems will happily run on newer 32bit/64bit rigs. When they do a major OS release, they reach out to every vendor they can, and in many cases will help them patch their software packages to run on the new OS (why you might ask? Microsoft is nothing without 3rd party software vendors…people and corporations buy Windows so they can run the apps they need to be productive, and since Windows is ubiquitous on the desktop, everyone wins in the end). I can only think of a couple cases where Microsoft broke backwards compatibility in a major way (and there are probably more, but these are the ones I know of, and they are mainly developer-centric as that's what I do for a living):

    • The shift from 16bit to 32bit (they did NOT repeat this mistake during the move from 32bit to 64bit). The shift from Windows 3.x to Windows 95 and up was drastic to say the least, but they had no choice as the 16bit address space. If this doesn't make any sense, I'll put it like this: Using 16bit architecture allows the OS to address a maximum of 65MB of memory. Granted, back in the early 90's that was adequate, but not nearly enough towards the mid 90's. On a 32bit system, 4 gigs can be addressed (though due to limitations in XP/Vista/Win7, only 3.5 gigs is usable without some hacks), which these days still isn't enough for many business uses (I saturate 4 gigs easily on my rigs, and the 16 gigs on my main server isn't enough either). In the 64bit address space, you can theoretically address 16 exabytes (2^64), but due to the way chips operate, ~256 petabytes is the max. I don't see the move to 128bit coming any time in our lifetimes.
      • How did Microsoft not break backwards compatibility during the 32bit > 64bit shift? In techno jargon, it's called thunking. You can indeed run 16bit code on a 32bit machine via virtualization, but you cannot run 16bit code on a 64bit machine, it's just too complicated to get the mappings right. 64bit versions of Windows use a thunking layer called WoW (Windows on Windows). This guarantees that virtually all 32bit apps will run on a 64bit machine. For consumers, the ONLY reason to run a 64bit rig is for the extra memory support, it is not recommended to install native 64bit apps unless you have a very compelling reason to do so. Not sure if you are running 64bit? Fire up Task Manager and go to the processes tab: If you see an asterisk next to any of the processes, that indicates those apps are running in the 32bit (WoW) compatibility layer. Or, go to your system drive, and if you see a folder called Program Files (86), you are running 64bit.
    • Microsoft caused a huge furor amongst developers when they deprecated Visual Basic 6 in favor of Visual Basic .Net. This is probably the biggest break they've ever made as they provided no tools to port existing code, and VB 6 developers basically had to completely retool themselves for VB.Net as the two share nothing in common outside of a common syntax. Microsoft rolled the dice, and won though. The dust settled, the VB6 folks retooled, begrudgingly at times, and the programming space is MUCH better because of it. My friend Joel Spolsky disagrees though in his excellent article about how Microsoft lost the API war. The shift for me couldn't have come soon enough as I loathed VB6 and immediately jumped on C#.
    • Related to the above, Microsoft dropped ASP as their official framework for dynamic websites in favor of ASP.Net, though technically it's still supported by their web server, IIS. Who would want to still write classic ASP is beyond me, but there is probably some cranky programmer somewhere still clinging to their copy of Visual InterDev hammering out VBScript.
    • The move from the Windows 9x kernel to the NT kernel. This is by far the best thing Microsoft could have done for consumers (business users had the NT kernel from Windows NT 3.1 onwards). If you don't know what a kernel is, don't worry about it, just know that the NT kernel was the first fully 32bit kernel, the 9x kernels were 16bit/32bit hybrids, and were a total mess…remember all the BSOD's we used to get on 9x OS's?…NT is much more stable). At one point, MSFT was maintaining a slew of different kernels, which means they had to write apps targeting each kernel, each version of each kernel, and each version of apps like Office, a maintenance nightmare to say the least). Windows ME (arguably MSFT's worst product ever) was the last to use the 9x kernel, Windows XP (arguably one of their best consumer products ever) ditched the 9x kernel for NT. All OS's since are NT based, one kernel to rule them all (though now that RT tablets are out, they are back up to two kernels). Regardless, the jump to NT created a slew of problems for legacy application developers, but again in the end MSFT made the right move, and the consumer space is better for it.
    • The introduction of the ribbon in Office. I wouldn't necessarily call this a break in backwards compatibility as it was mainly a UX (User Experience) change, but it was met with heated resistance. I hate to say it, but menus are so 20th century at this point…cumbersome, ugly, and not intuitive. The biggest change for me was that they changed some common keyboard shortcuts in Outlook, but other than that, it only took me a few weeks to get used to the ribbon. Users better get used to it as this is now the standard UX on Windows 8.
    • There are other backwards compatibility breaks in the developer/business space, but those are beyond the scope and audience of this post.

    Which brings me to Windows 8 (which IMO is just Windows 7.1). Again this is not meant to be a review or even that much of an opinion on Windows 8 as there are many things I love about the new OS, and a handful of things I abhor. Plus, there are plenty of reviews from professionals on the web already. I'll start with the good:

    • Resource usage is incredible compared to Windows 7. This is no doubt due to Microsoft ditching the Aero interface (which I thought I would miss initially, but I actually like the new flatter interface). By resource usage, I mean memory consumption, processor usage, hard drive saturation, etc. I haven't done any serious benchmarking, but memory usage is roughly half that of my Windows 7 machines, and processor time seems much better utilized (it's a dual core overclocked to 3.8ghz so that could be part of it). My Windows machine just feels faster, which means it is. Startup time is phenomenal, about 60 seconds to desktop…err, Metro (and this was an upgrade, not a clean install; I bet a clean install would be even faster). It should be noted that my main workstation is 64bit, so that could be part of it as well, but it just feels crazy fast.
    • The new task manager is fantastic. Seems a little silly, but I almost always have an instance of task manager open. You have to see it to believe it; task manager has remained largely unchanged since the mid 90's. It expresses information in a much more logical intuitive manner with a refreshed GUI.
    • You can run XBox games on Windows 8 machines (I don't know if this is fully implemented yet, but this is pretty cool).

    And the bad, and these are huge for me and were almost deal breakers initially:

    • Windows Media Player no longer supports playing DVD's. Read that sentence again. I don't watch DVD's on my machine at home, but on long trips (planes, trains, etc.) I do. Granted 3rd party apps are of course still supported, but MSFT stated that the licensing costs for WMP codecs wasn't worth it for DVD support. Whatever.
    • Windows Media Center is now a paid add-on. My entire home theater is based around a WMC Virtual Machine which houses hundreds of digitized DVD's that I run through WMC.

    And finally, the biggie:

    METRO :-(

    First off I do applaud what Microsoft is attempting to do with Metro: Present a unified UX across all of their platforms, in this case Windows, WinPhone, and XBox, and as a byproduct of this, a unified framework for developers such as myself, the intention being that we can write one codebase in the Metro languages, then deploy it to any device that supports the Metro framework. In theory at least, I haven't tried it yet.

    I do realize that Metro is optimized for touch devices, and I have it installed on an old tablet…for touch, it's quite usable. But for the 95% of the population who don't have touch devices, the UX is miserable. One of the #1 rules of web design is no horizontal scrolling, ever. Guess what the Metro desktop, and almost all Metro style apps make you do? Scroll horizontally. For touch this is actually fine, but with keyboard/mouse, it's an awful experience. And, the Metro interface just looks silly on my 26" LCD (looks great on a phone and XBox though). The Windows store apps that I've played around with are dumbed down to the level of a kindergartener, and I haven't found a way to tile them so I can use more than one at a time (full screen only in Metro, really???). And, finding apps is a complete pain, you have to search for everything, which means you have to remember the name of the app you want. Bringing up the search "charm" (as it's called) does present a screen listing all of your apps…but it lists ALL of them with no way to collapse sections that I've found. It is nothing like the start menu that I've been using for almost 20 years, so it's a huge break in the UX department. Finding apps is an abysmal task at best, and fruitless at worst.

    What really sucks about the new Metro "start/search menu" if you can even call it that, is that you must use it. The start button is gone with no native way to get it back (more on this at the end of the post), which is a major UX no-no. Microsoft is shoving the new Metro interface down our throats. There is also no way to boot directly to the desktop, you have to go through Metro first (hint, WinKey-D will get you to the desktop from Metro). I can see MSFT helpdesk costs going through the roof with calls like "where is the start button" and "how do I find my dancing bunnies application"…it is beyond me why they didn't provide these two legacy options. Just give me my start button/desktop back.

    Once you are in desktop mode, all is still not so good. Each corner of the desktop brings up what Microsoft calls "Hot Spots"…when you drag your mouse to a corner, a little popup shows. Lower left is the hotspot for the Metro start menu (or just hit the Windows key), top left is a mini application switcher, and top/bottom right bring up the "charm bar" with the following options: Search, Share, Start, Devices and Settings. The hotspot locations for the charm bar are in the worst locations imaginable though. I work 95% of my time with a window maximized. Every time I want to exit out of a window (when I'm too lazy to alt-f4) via the X button…well, think about it: Where is the X button for a maximized window? Top left, so the charm bar pops up and even worse, it steals focus. It usually takes me a couple of attempts just to close a window. Bottom right corner is no better: When I don't feel like WinKey-D to minimize all open windows, I hit the minimize button which is located where? Bottom right corner. So again, attempting to do that brings up the charm bar, which steals focus. I have not found a way to natively disable hotspots, or at least move them to different corners.

    From the desktop, hitting the Windows key, as expected, brings up the new start menu. But, the transition is visually jarring from desktop to Metro. I live and die via the search box in the Windows 7 start menu: I can bring up apps, type in simple commands…I probably hit the start button (or WinKey) a hundred times a day. Hitting Ctrl-C or WinKey a hundred times a day on Windows 8 would probably give me a seizure, or at least a horrible headache by the end of the day. Yes, it's that jarring.

    Fortunately there are some workarounds. The first thing any Windows 8 power-user needs to do is read this article by Scott Hanselman on how to make Windows 8 much more usable via keyboard shortcuts: Windows 8 productivity. Again, I cannot stress this enough: Read it, memorize it, print it out and tack it to your cube wall, whatever it takes.

    And finally, the tool that has cemented my move to Windows 8, and solves all of the UX problems I mentioned above: Start8 from Stardock Corp (30 day free trial, 5 bucks after that, and well worth it IMO). I never install 3rd party GUI enhancements/add-ons/themes because the inevitably screw something up (plus I do a lot of design work, need the GUI in its default state). Stardock is a very reputable company who has been making GUI enhancements for Windows since…well, since Windows had a GUI. At the risk of this sounding like an advertisement, this little tool is incredible. You can:

    • Get your start button back (it is 100% identical to the Windows 7 start button)
    • Boot directly to the desktop and bypass Metro completely
    • Disable all the hotspots

    This tiny little tool makes Windows 8 completely usable. Is it silly to harp on about not having a start button? Yes. The start button is synonymous with Windows, it's the very definition of the Windows desktop. For people who make their living using Windows as their main tool, it'd be like the gov't making us switch to driving on the left hand side of the street, or keyboard makers switching to DVORAK instead of QWERTY, or mouse makers getting rid of right clicking. For me yes, it's on that scale, and for many other colleagues I know as well.

    For me as a software developer, the transition to Windows 8 isn't optional, it's a requirement for my job. Another break MSFT made is that you cannot develop Metro apps on any OS except Windows 8/ Windows Server 2012. There is no Metro runtime/emulator for Windows 7 (which I find odd), no doubt so they can push Windows 8 license sales; makes fiscal sense, but at the expense of pissing off developers, and whoever makes the budget for IT departments. Before I found that little utility, I dreaded hopping on my Windows 8 machine, now I actually look forward to it. The point is that one tiny little change like removing the start button, and thus undoing over 20 years of UX look and feel can have a huge impact. Always give your users an option to revert during a transitional phase as big as this, like they did with the Ribbon (in the first version of Office that had the ribbon, you could still hit the alt button to get the legacy menus back, this was removed in Office 2010…at least the provided a transitional option though), then remove it Windows 9 if need be.

    Overall, I like Windows 8, and I have high hopes for what MSFT is attempting to do with Metro. V1 (like most things Microsoft) will be slow to penetrate the market, but V2 should be better. I can see businesses that have employees out in the field adopting some tablet hardware and doing some Metro development in house for mobile-type apps, but cube dwellers will probably never see Windows 8 unless MSFT gives us some Group Policy options to boot to the desktop, and give us a native start button option. It would just be too costly to deploy and support, for absolutely zero gain. Even then, Windows 7 is good enough for businesses (and many are still on Vista, and even XP).

    The rest of this post will be largely academic in nature, so unless you're really curious about backwards compatibility and how it affects the main 3 software market segments, you can safely stop reading this article unless you're bored, curious, brave, or need something to help you sleep.

    Still here?

    A little more about backwards compatibility and what it means in both the IT realm, outward facing websites (like Twitter, Facebook, Google), and the ISV (Independent Software Vendor) realm. First, a differentiation between the two (and there is a huge difference). When I tell people I am a software developer, or a web developer, they immediately think I write games, or applications like Word or Photoshop, or operating systems like Windows, or websites like I listed above. A high level definition of each of the above:

    • ISV's are companies like Adobe, or Symantec, or AutoDesk…there are literally thousands of ISV's (other examples are business service companies like SAP, PeopleSoft…companies that large organizations hire to implement custom CRM/ERM/HR solutions, this end of the ISV business is crazy lucrative, but extremely difficult to run and I know nothing about them). Basically, they are companies that write and sell software via licenses for different OS's. Adobe is a classic example of an ISV: They write creativity software for Windows and OS X that you install on your machine, and they charge a fortune for it. They make their money via software licenses (Microsoft is both an OS vendor and an ISV: They license Office and numerous other desktop apps, but they are also a services company). ISV's usually target a very narrow vertical market: They do one thing, and they do it exceptionally well by cornering (or creating) a market.
    • Outward facing websites like Twitter, Google, Facebook, MySpace are a relatively new addition to the software space. Most people don't understand how these companies work, and how they are profitable. Hell, I barely understand their business model. It should be noted that these companies are NOT software companies, they are advertising based companies that just happen to use software/websites to generate advertising based revenues, which is how they pay the bills. That does not mean these firms don't have some of the most brilliant minds on the planet working for them. These companies have legions of Data Scientists, Software Scientists, Stat Scientists, Mathematicians, Physicists, Chemists…you name it, they hire the best minds for a couple of reasons, in this case I'll use Google as an example:
      • Google uses its search service to mine user data and patterns, plain and simple. It's not as creepy as it sounds, all ad based sites do this. But, Google is an ad company first and foremost, they just happen to use the internet as a platform to dish these ads out to consumers. They make all of their money based on firms selling ads to them, which they then serve up around the web. The reason they turn a profit in such a small margin based market segment is because they have the best scientists working for them, creating algorithms that eek out the absolute best and most comprehensive data mining algorithms on the planet.
      • Google uses off the shelf components in their data centers. Stuff you or I can pick up from any web retailer. This saves them a ton of money. Plus, they are one of the greenest companies on the planet (Google "Google Datacenter Green" and read some of the articles), and they are transparent about their green processes, they give it all back. Datacenters consume enormous amounts of energy, so if they can eek out a couple pennies per kilowatt, they save millions of dollars per year. Hence why they have physicists, chemists, environmental engineers, etc. They don't make any money from being green, but they save a ton of money, and they give all the information they've learned back to the community.
      • I'll touch on Facebook briefly as they do make money from ads, but FB cornered a relatively new space on the internet: Selling the FB platform as a service itself. I don't know the exact stats, and yes they do make the bulk of their money from ad sales, but FB is as much of a platform as it is a social site. Think about all the crap-ville games and other apps that we constantly get nagged about. FB makes a fortune from developers who want to peddle their wares via the FB platform (don't know what percentage they take in revenues). But they are still ad based.
    • And finally, what I do for a living: Information Technology (the software side, not infrastructure). This is the hardest of the 3 to explain. First off, IT departments do not make profits for a company, ever. If anything, they are probably the most expensive cost center of any company, hence why when layoffs roll around, we are the first to go. Information technology is not a new field (it's been around since business itself), it's just one that has seen enormous growth due to computing advances, and cost decreases. All companies have an IT department, or at least have a 3rd party vendor they use, it's a necessary evil as data and information storage/flow/dissemination/analysis is the core lifeblood of any business. Easier/smarter/faster is the mantra of IT. The "core" core of a business is its data. This is similar to the web companies listed about: How can we analyze our data to make better business decisions to become more profitable. At its heart, this is what IT is: Gathering data from customers via employees, sending it down the pipeline, scrubbing it, manipulating it, and storing it for later analysis. We just happen to use software as a tool to input and analyze this data. Yes, I write software and websites, but what I really do all day is streamline data flow and processes, gather business requirements and translate them into software, build new systems to gather and analyze information, build automated systems to communicate with other business systems…it's all about information and data. We help the company either make more money or lose less money. Sometimes, that means automating a knowledge worker's job, but that saves the company money. How do IT departments make money then (IT workers are paid quite well)? Of course we're given a budget from the company "kitty" as I like to call it, but we also bill other departments that we provide services for. They end up saving money in the end, but IT managers (good ones at least) are constantly drumming up business for their departments. This may sound odd to folks not familiar with the "business" end of IT, but in medium to large companies, most departments A) don't know what IT is B) don't know how/what/why they need processes streamlined (IT managers need to sell this) and C) if they do know about IT, have no idea how to get in touch with them. So we're almost like a mini consulting firm within a company, and we get paid off of selling our services to other departments. It is also (at times) extremely layered and beaurocratic. An IT dept. (at its most heavily layered) consists of the following roles:
      • The CIO: The big boss, shot caller, decision maker.
      • Dept. Director: Drives sales and drums up business for the cube dwellers.
      • Systems architect: In charge of overall systems design, application architecture (usually not a supervisory role though, but is usually the most Sr. person on the software team).
      • Database administrators: They hold the keys to the castle, the company's databases. Extremely well paid, and extremely high stress. If the database goes down, the company is hemorrhaging money until it's back up.
      • Software developers (my role): We coordinate with architects, project managers, business analysts, SME's (end users who are experts in their dept.'s), QA/testers, and technical writers. We wear a lot of different hats.
      • Project Managers: Usually not very technical, they drive the project schedule/deliverables/business meetings.
      • Business Analysts: Fairly technical, they help translate business speak into technical jargon (the good ones at least). They serve as a layer between the geeks and the business folks.
      • Technical Writers: In charge of writing technical manuals to help users learn the systems the developers write (these are some of the smartest people I've ever met by the way).
      • QA: These are the folks that developers go to (friendly) battles with day in and day out. They try to break our code, then tell us to fix it. They are the last step before a system is deployed.
      • Support Staff: The stuff we write will break at times, end users call these folks to help get it fixed.
      • Infrastructure Dept.: I have no idea what these guys actually do TBH, but the corporate LAN would not work without them. Very high stress role, and we constantly coordinate with them during deployments.

    That was a huge digression, but backwards compatibility affects each type of those segments differently. I briefly touch on how (I promise this will be brief) each type of business approaches backwards compatibility, and how it affects their bottom line. There are two types of backwards compatibility breaks:

    • Regressions, e.g. ones that are accidental. These are inevitable in any system, hence why QA exists. They can't catch everything though. Regressions must be fixed, hence why hotfixes and patches exist.
    • On purpose, such as discussed above (and businesses do this for one of the following reasons):
      1. Time to move on and ditch some legacy code or feature(s).
      2. Ditch an old feature in favor of a new one, usually to increase license sales (AutoDesk and Adobe are notorious for this to drum up new money). This isn't a bad thing per se, but it is annoying. In the case of AutoDesk, they will actually change file formats so that new files aren't compatible with old versions, and vice versa. Microsoft did this during the office 2010 release (moved from .doc to docx, xls to xlsx, etc.…but they maintained backwards compatibility).
      3. Scrap an old product completely and build a new one. Rare, but it happens.

    IT is notoriously lax when it comes to testing backwards compatibility for one main reason: Time is money. Departments want their solutions yesterday, and don't understand testing. This is changing, but slowly. The irony is that businesses have the most to lose via regressions or compatibility breaks: If a LOB (line of business)app breaks, you have users sitting on their hands doing nothing, losing money. The good thing is that repairs are usually fast since IT sits in the business itself, and patches can be rolled out swiftly. IT loses money as well as we don't bill for our own mistakes of course.

    ISV's: Might piss some customers off, but if the tool is essential to their job (Office, AutoCAD, Photoshop) users have no choice but to wait for a fix. They can usually go about their jobs though, not saying it's not a big deal, and patches are usually much slower to be released than IT, but ISV's have the upper hand.

    Outward facing websites: This is where it gets interesting. In the case of Google, Twitter, Facebook (all free, ad-based sites) these companies lose no money if a regression is introduces, or a feature changes/is scrapped (TimeLine anyone?)…users get what they pay for, you are at the mercy of each company and what they want to do. Granted they may introduce a regression on their backend systems and lose money, but we'll never know if they do. All we want to do is tweet, share kitten pictures, and post dumb status updates/play our games. Sure if we get the fail whale for more than 30 seconds we might start cursing, but in the end it doesn't matter. Of course there are service based websites out there that users do pay for, so if they go down or have regressions, it's a big deal, but we're still at the mercy of the company behind the service. Service based companies will usually pro-rate outages/errors from your bill if it is substantial enough.


    Finally I want to touch on a company that is notorious for just flat out disregarding backwards compatibility altogether, blatantly at times (and I'll also touch on why they can get away with it): Apple. It should be noted that I actually like Apple; their hardware is amazing, OS X is nice, and iOS is the most intuitive mobile OS I've ever used (and I've used a lot). A brief history of Apple, and my experience with their hardware/software.

    I grew up on Apple hardware. My first computing experience was in 2nd grade on an Apple II. It goes without saying I was hooked immediately. I graduated on to OS7 (our first home PC was a Mac LC II) in my early teens, got my own Mac when I left home (OS8 and OS9), then jumped ship for the PC world when I went to college. I've barely touched Macs since then.

    A misnomer about Apple that needs to be cleared up: They are not, nor have they ever been, a software company. They are a hardware company that bundles proprietary software with their hardware. Everything about Apple is proprietary (and I'm not complaining, but all the Apple kids love to rail on MSFT for being "closed"…Apple puts MSFT to shame in the closed off department): Their OS, the language they use to program their OS (objective-c), hardware specs/compatibility. And, they have made some of the largest most blatant compatibility breaks I've ever seen:

    • After OS 9 (and the return of Steve Jobs to Apple), they scrapped their entire OS codebase in favor of OS X, which is a Unix variant (BSD and the Mach kernel) but not open source. There was zero backwards compatibility provided by Apple, which meant you had to buy all of your applications all over again. That also meant that every single software vendor that targeted pre-OS X had to completely rewrite their applications from the ground up, none of the API's were preserved. And, they had to learn an entirely new programming language to boot: objective-c. This would be the equivalent of Microsoft scrapping the Win32 API, upon which every single software program runs off of.  If MSFT did this, retribution would be swift and severe…hence why they could never, ever do it. The user base wouldn't stand for it (more on that later). Apple certainly didn't make any friends during this process, but eventually the ISV's adjusted and settled in. They did provide a very thin compatibility layer called "Classic" (which was far from perfect) but this was short lived because…
    • A few years ago, Apple made the switch from their proprietary PowerPC chips to off the shelf Intel chips. Fiscally this made sense, much much cheaper in the end. However, the CPU is the heart of the computer: Changing chip architecture is the equivalent to open heart surgery. Reaction notes here, and this officially ended "Classic" support, and any software than ran on PowerPC chips. One good thing that came of this was the ability to run Windows in an emulator since Windows runs natively on Intel chips. Also, it turned out that the Intel chips outperformed the PowerPC chips by a huge margin. I do not think Apple could have written a compatibility layer for this transition, at least not one that performed well, but in the span of a couple years they had forced the major vendors to overhaul their apps…twice. Those firms have to recoup their development costs, at the expense of end users' pocketbooks.
    • Subsequent releases of OS X are notorious for breaking 3rd party software, and unlike Microsoft, Apple does not provide much assistance in patching those programs to run correctly, or providing backwards compatibility libraries in the OS itself. They just don't care. Also, for some releases, users had to buy all new hardware to run it. Yes I know they are in the business of making money, but to hard code hardware requirements in the OS itself is ridiculous. MSFT does this sometimes, but it's easy to bypass if you're willing to pay the performance penalty.
    • Cable interface changes. One thing about USB is that all 3 USB ports use the same cable, and are compatible with each other (and the specs are public). Apple recently came out with an upgrade to their proprietary firewire interface called thunderbolt. Want an adapter? Cough up 30 bucks. Apple/Intel are touting it's 10gb (little b)/s speed, which means absolutely nothing in the real world of computing, it's like the megapixel myth in digital photography. A computer is only as fast as it's slowest component, which will almost always be the disk drive. Even the fastest consumer solid state drives peak around 300MB/s, and spindle drives are less than 100mb/s at standard rotation speeds (5.4k and 7.2k). Not nearly enough to saturate thunderbolt's line speed. If you could go directly from memory to a thunderbolt device (and vice versa) it would make sense. Otherwise, it's rubbish and a cheap marketing tactic meant to sell cables and new peripherals.
    • And their most recent backwards compatibility gaffe: iPhone 5 has ditched the standard connector for a new smaller one (fine, it allows the phone to be thinner). This basically renders all your docking stations, connectors, etc. into paperweights, unless you are willing to cough up 30 bucks per cable, or just buy new peripherals. At least include a free adapter in the box Apple. If I were a tin foil hat toting type person, I'd swear Apple was in bed with the peripheral vendors and getting kickbacks.

    So to wrap this up (and I hate to use Microsoft vs. Apple as my example to follow), but I'll try to be as subjective as possible. Microsoft and Apple aren't really competitors except in the mobile space, this is purely one school of backwards compatibility vs. another, and why I think each company are polar opposites on how they go about it.

    • Microsoft does everything they can to ensure backwards compatibility in their software, up to and including helping vendors rewrite drivers and software to be compatible with new releases. But, why?
      • Simple: because they have to. I do honestly believe it is part of their company culture, but when stuff breaks, their customers are losing money (the overwhelming bulk of Microsoft's profits are from the business space, not consumer). Plus, businesses and IT now expect this from Microsoft, it's not an option. When MSFT has chosen to break backwards compatibility, they are extremely clairvoyant about it, they work with customers, and usually offer tools to bridge the gap (and they are almost always free).
      • Microsoft owns 95% of the computing space. If something breaks or regresses, it affects 10's of millions (if not more) of people. I do want to make something clear: I am not a Microsoft fanboy, though at times I may come off as one. They provide a good set of tools that allow me to get my job done, their software is easy to use, and let's face it, if you are in business, you have no choice but to use Microsoft products. What you choose to use at home is your choice.
      • In the end, it benefits Microsoft, and their vendor relationships. It's just good business practice to go the extra mile.
      • The downside is this leads to longer release cycles, increased cost of products, and bloated software with tons of legacy hacks in them.
    • Apple plays nilly-willy and loosey-goosey with backwards compatibility (I know that sounds pejorative, but this isn't a bad thing). Why?
      • Because they can, plain and simple, for two main reasons: Apple has something Microsoft will never have as much of: Customer loyalty. The Apple brand is up there with Disney, Coca-Cola, and Joe Camel (when he existed). Apple has built a legion of loyal customers (I'll be honest, I'm beyond loyal to the iPhone line and don't see myself switching pretty much ever). Their hardware is awesome, and since their hardware is a closed eco-system, it is guaranteed to work. Number two: Small market segment in the computing space (but they pretty much own the mobile space, this post is about computing though), generous numbers put it at 5%, in the consumer space, and less than 1% in the business space (basically non-existent). Small market share + all of them consumers + who are beyond loyal = can break backwards compatibility. They may bitch initially, but Apple knows they'll jeep coming back and dropping dough on their hardware.
      • Marketing. Apple is a marketing machine, and consumers are much more likely to react to marketing than businesses are, That's Apple's core market, and wow can they market the hell out of their hardware. I have met many technical Mac users, but the majority are not (and I'm not dissing Mac fans, these are just the facts) so when they see things like THUNDERBOLT, 10000000000gb/s TRANSFER, they buy into the hype and upgrade. Again there is nothing wrong with this, that's what marketing is all about, creating demand, make customers drool, and Apple is one of the best in the business, if not the best.
      • So what are the benefits of breaking backwards compatibility at will? Apple is one of the most innovative companies in the world because of it. They can play around, and stay bleeding edge without fear of much retribution. If vendors and consumers have hung in this long, with all the breaks they've done, they aren't going anywhere. That doesn't mean Apple gets free reign, but they know they can stretch their limits. The consumer (as long as they have the money) wins in the end. Microsoft will never have this capability since they have to kowtow mainly to the business space, who don't care about bleeding edge, they want rock solid results.

    I prefer the maintain backwards compatibility line of thinking, and in what I do for a living it's fairly easy to do since web standards are generally retro-active with one another. Regressions are impossible to avoid, and very costly in my business. In the end, you have to find a balance of what your customers expect, what are they willing to put up with, how can we still be productive and turn a profit at the same time, and is purposely breaking something worth it if we might lose customers because of it.

    I often wonder if Microsoft will ever pull an Apple and ditch the Win32 API. Sometimes I wish they would because it's ancient, written in C and Assembler, and probably has more cruft in it than my grandmother's basement. But I doubt it. Hardware keeps getting faster and cheaper, so they can just keep writing compatibility layers on top of it without worrying too much about performance hits. Will they ever ditch the NT kernel? Not anytime soon. Even the staunchest of Microsoft hating Computer Science students (who all come out of school praising *nix and how much MSFT sucks, then learn the business end of computing) will agree that the NT kernel is a pretty amazing feat of engineering. Regardless, backwards compatibility is an interesting topic. If you've made it this far, many thanks for reading.


    Addendum: A little trivia about the Microsoft/Apple relationship. Microsoft would not exist had it not been for Apple. Microsoft made their first applications for the Apple platform for many years before writing DOS. Had Apple not been around to sell licenses to, who knows what would have happened.

  • New Web Languages and Frameworks (and a Brief History of Web Programming): Pt. 1

    Software development has changed so much since I first got into the business over a decade ago, and lately, sometimes I feel like I’m becoming that 50 year old dinosaur who sits in the corner mumbling to himself about “the old days” of programming and how all the new technologies are ruining “proper'” software engineering. Ok not literally because unlike those guys (and we’ve ALL worked with folks like that) I love adapting and learning new technologies, but metaphorically…yup. I have my comfort zone which consists of Java and .Net, mainly on the Windows platform, with Windows centric middle-tier/backend plumbing:

    If I had to classify myself and what I like to develop, it would definitely be backend/middle-tier stuff. I am by no means the best front-end designer (WPF or Web stuff), but can get it built if a designer hands me the CSS and HTML framework/storyboards. A little background (queue harp music here);

    When I first got into the business, you were either a Java person, or a Microsoft person.Third party languages/frameworks were non-existent on the LAN (outside of Perl, which was the precursor to modern day frameworks via CGI (Common Gateway Interface)…if you’ve ever seen /cgi-bin/ in your URL, it was probably a compiled Perl script doing the dynamic processing. CGI is long dead, thank God. I wish Perl would suffer the same fate.

    I did a little of both initially. If you were a Java person, there were few frameworks outside of the Java libraries themselves, and if you were a Microsoft person, you used either Classic VB (v. 6 in my case), C++, or ASP if you dared to venture into this brand new thing called the web. At the time I thought I was in heaven: I loved C++ but it was way too complicated, so I went the VB route (which made ASP that much easier to pick up since VBScript is a subset of VB). Anybody remember Visual Studio 6? Visual InterDev (what a piece of crap IDE)? SQL Server 6.5? If you never worked with any of those, count yourself lucky.

    Then the web (HTTP/HTML) started to take off on the LAN. Java came out with some frameworks, mainly JSP. And, in 2001 Microsoft rolled the dice with this entirely new framework called .Net, with the promise that given a unified framework, anyone who could pick up the new languages (VB.Net, which had absolutely nothing to do with VB6…there were a TON of pissed off VB6 developers as VB.Net was 100% backwards incompatible with VB6, and holy crap, now VB developers had to learn this thing called Object Oriented Analysis and Design) could write apps that targeted the web AND the desktop, without having to write much Javascript or HTML by hand, the Asp.Net libraries would generate it all for you vie WebForms. For me it was a godsend. I immediately jumped off the VB6/ASP bandwagon, ditched VB entirely and learned C#.

    Initially this was great…just use the built in frameworks, hammer out some code, et voila…a website (back then, corporate LAN websites didn’t need to be pretty, just functional). For the most part (outside of god awful non-standards compliant HTML) it just worked. The LAN didn’t care about standards (though they most certainly do now), or dynamic GUI’s, or postbacks/page reloads on every request. Everyone was happy. The deployment teams had a centralized point to roll out updates (the web server), the support teams were happy since they didn’t have to troubleshoot hundreds of machines running custom windows apps, just the web server, and the end users were happy because the web is so much more intuitive and consistent than having to learn some home rolled windows app, which probably had zero consistency with other home grown windows apps.

    This worked great until about 2006…then the web blew the absolute hell up, and lots of independent developers started realizing how they could tame the web and make it do some pretty incredible stuff that no one expected possible. Browsers themselves became programming platforms in their own right via CSS and JavaScript (and other cool client side technologies). This was a double edged sword though: The benefit was that the web became a truly viable programming platform, with all the richness of client applications, without all the headaches involved that I mentioned above. The web became less about HTML and displaying information, and more about passing information around and letting the browser figure out what to do with it. The shift over the past 20-30 years has been pretty wild: We have gone from thin client (Mainframes and Greenscreens) where all the processing is done on a central server, and dumb terminals have to play round robin for processing time. Then thick clients came along: Client based applications where all the processing is done on the client itself (PC’s) with minimal processing on the server (probably just a database repository). Then web 1.0 came along, back to thin client architecture: The centralized web server does all the processing and spits out some HTML, all the client has to do is fire up a browser and render the HTML to the user. Now we’re going back to thick client (though it’s more of a hybrid) where the web server process chunks of data, does some processing, send out these chunks of data fragments to the client, and the browser can handle a ton of processing via the new JavaScript frameworks that exist (more on that in a bit). I expect this latest trend to continue and advance for some time. The downside? It is extremely complicated with all the layers involved, and web developers are swamped with all the disparate technologies, and how to glue them all together. Most people don’t realize just how complicated websites are these days. And just how much work goes into creating even a simple web 2.0 site. A typical web developer needs to at least be functional (if not an expert) in the following technologies (and this is a cursory list, no doubt there are plenty more that can be added to it). I’ll start from the backend and move up:

    • Database design: Some frameworks will take care of this for you via ORM’s, but this only works for small/medium size sites. Some developers used to devote their entire careers to just this alone…this is no longer the case. This includes:
      • Some dialect of SQL (Structured Query Language), a programming language based on Lambda Calculus. Just to make things interesting, each database vender has its own dialect of SQL, so it usually is not portable (even though an ANSI version does exist, vendors add their own extensions)
      • Database administration. This means learning hardware, RAID setup, roles and security, backups/restores, replication schemas, performance tuning, physical/logical partitioning of database files, production troubleshooting.
    • LAN technologies/operating systems/network theory. This is where things can get really hairy as most organizations have a dedicated infrastructure team, however developers are expected to have a pretty thorough understanding of the following infrastructure stuff:
      • Directory Services of some sort (AD/LDAP/RADIUS/etc.)
      • Domain Name System (DNS) and Subnetting
      • Dynamic Host Configuration Protocol (DHCP)
      • Basic network protocols: Transmission Control Protocol (TCP), Internet Protocol IP), and HTTP (which sits on top of TCP/IP…this protocol is what made web developers lives so miserable in the “old” days because it’s stateless, unlike TCP which is stateful), File Transfer Protocol (FTP), Simple Mail Transfer Protocol (SMTP)
      • Basic knowledge of networking hardware like routers, switches, VLAN’s, hubs, NICs
      • Client operating system configuration for the above protocols
    • Middle tier layer frameworks/technologies: This list alone could consume an entire post. The basics are:
      • Caching hardware: To offload client load on servers
      • Firewalls: These can wreak havoc on distributed systems if not configured properly
      • Web farm hardware/software: This is used to make multiple physical web servers act as one logical server, to offload server load amongst numerous machines. Big sites like Google, Amazon, eBay have thousands and thousands of servers around the world working together to server up web pages and crunch data as quickly and efficiently as possible. Getting these servers to talk to each other and act as one is probably the 8th wonder of the world.
      • Queuing hardware/software: The last thing a website wants is a dropped request/response, so this layer is in charge of “queuing” data if the servers are overloaded, and in the case of distributed systems, they guarantee delivery of a chunk of data, regardless of latency.
      • Web servers themselves: The big players are Apache and IIS, though there are hundreds to choose from. This is one of the few administration areas that developers actually own. An understanding of security, the HTTP protocol, caching, partitioning, configuration are a must. This is one area that drives developers absolutely crazy, but it’s a necessary evil as this is the heart and soul of a web developer’s application. Without a properly running web server, our apps are dead.
    • Though not necessarily a layer, the language and platform itself that will be used to write the application. Again this list could go on for pages and pages, but these are the major players (and the jist of the post, which I’ll go into at the end):
      • .Net: Two major web framework ship with .Net itself, and there are many others to choose from. Asp.Net WebForms, and Asp.Net MVC (Model View Controller). WebForms owns the LAN, MVC owns small to medium outward facing sites for the most part, but is catching on on the LAN (I prefer MVC).
      • Java: I’m a little dated, but JSP (Java Server Pages), Struts, Spring, and JSF (Java Server Faces) are the most popular.
      • Ruby on Rails: Ruby is the programming language, Rails is the framework. Ruby has been around for quite a while, Rails is fairly new to the scene though. It is a RAD (rapid application development) framework which mandates using the MVC pattern, and it took the web by storm when it was released. I’ve used it a bit in the past, and it is extremely powerful, very easy to learn, and it doesn’t take much work to get a fully functional site up and running, outside of learning ruby, and the rails framework. A quick list of sites that use Ruby on Rails:
        • Twitter. Yup. Twitter.
        • Yellow Pages
        • Hulu
        • github
        • Soundcloud
        • Groupon
        • and thousands more
      • Python/Django: A direct competitor to RoR, Python is the programming language, Django is the framework. Django is similar to Rails in that it mandates MVC (seeing a pattern here?), and again Python has been around for a while, Django is a fairly new framework. It also has taken the web by storm mainly due to its simplicity, and Python is drop dead simple to learn (if you are versed in any OO language, you can pick up the syntax in a couple of hours). Notable sites running Django:
        • Disqus (the service/site that powers my own site’s commenting system)
        • National Geographic
        • Washington Post
        • The Onion
        • Pinterest
        • Layar
        • NPR
        • New York Times
        • (lots of news sites as you can tell) and many more
      • PHP (which is a platform itself in its own right): This is the only popular web language that I know virtually nothing about, other than I really like its syntax as it's very C based. From what I’ve read, most of the popular frameworks for PHP pale in comparison to Rails and Django. What PHP really has going for it is turnkey CMS solutions, which is my next bullet.
      • Coldfusion (CFML): On its way out, if its not already dead. My first gig was in a CFML shop, and back in the day you couldn’t beat it. But, Adobe couldn’t keep up, and I haven’t seen a line of code in CFML in well over half a decade. Definitely on life support.
    • Just to further complicate web developer’s lives is that there is a whole slew of turnkey CMS/Blogging/Website solutions…many of them open source and free (you just have to learn the system, and pay for hosting), and a few proprietary and horrifically expensive. This is a layer that sits on top of the framework/language, and are useful if you don’t want to write your own solution, and is the last step before the content reaches the web browser. A quick list, the framework it’s built on (if at all) and the programming language:
      • Sharepoint, Microsoft Asp.Net, C#. This is for the LAN ONLY as it is tightly coupled to the Microsoft platform. A rudimentary SharePoint implementation will set you back 6 figures in licenses, which is why it is usually only used by large companies. It pays for itself though, and is a product I specialize in. (note: Microsoft does offer a stripped down free version called Foundation, which includes a free version of SQL Server called Express).
      • DotNetNuke: Microsoft Asp.Net, C# (was VB.Net for a long time). DNN itself is free, but to get any sort of functionality out of it, you have to buy add-ons and plugins, which can be very expensive. It is open source (but customize to your own heart’s content). Extremely customizable, fairly easy to use, and an extremely active developer community.
      • WordPress: PHP/LAMP (Linux Apache MySQL/PHP) Stack. Arguably the most popular blogging package on the web.
      • Drupal: PHP/LAMP. Probably my favorite CMS system due to the power of it. runs it, and it is probably the most widely used CMS (Content Management System) software used on the web.
      • Joomla!: PHP/Lamp. Starting to surpass Drupal as the most popular open source CMS solution.
      • Movable Type: Perl. Probably the only thing Perl is still good for.
      • Plone: Django/Python. List of sites here. Very powerful CMS framework.
      • Surprisingly, not many turnkey CMS’s targeting Rails/Ruby worth mentioning. The above systems account for > 90% of the popular vote.

    Quick interjection: My site runs on a CMS called Community Server (which is now defunct and no longer actively developed by Telligent Systems, who morphed it into a proprietary product called Telligent Community which is no longer open source). My site is showing its age, but it still works, and I've highly customized many of the subsystems. Moving to a new platform just doesn't make any sense.

    • If as a web developer you choose to go with a turnkey solution, job pretty much done (given you have a good understanding of the previous bullets) outside of finding the plug-ins/add-ons you need, doing some configuration in your CMS’s control panel, probably implement some custom JavaScript/CSS, design some fancy graphics, then deploy. If not, it’s time for the last step: Client side browser development technology choices and implementation. Again this is another bullet that could take up an entire post, and in this case, that’s exactly what I’m going to do in Part 2 of this article, which I’ll post later.

    These bullets encompass about half of what it takes to be a web developer. The other half (and by far the most important from the end user’s point of view since all they care about is how it looks, how easy it is to use, how responsive it is, and the overall “hook” and feel as I like to call it: You gotta get them to come back, and tell their friends about it. The end user doesn’t care about the above, but it’s just the tip of the iceberg. What most users don’t know is just how much work goes into creating a fully functional website. I’m not out to get them to care, but to simply A) let them know that it’s not just about typing up HTML pages anymore and B) hopefully get some sympathy from my fellow web developers as to just how swamped our brains are with ALL THIS STUFF!

    So in part 2, I’ll touch on all the various client side technologies that web developers are required to know (in addition to the above, and the underlying server side language which is what most programmers spend their lives trying to master just by itself), why these frameworks exist, the rise of web 2.0 (even though I loathe this term) and I’ll touch on NIH (Not Invented Here) syndrome, and what these new frameworks have done to change the face of web development: Even though they’ve eased the lives of web developers, they’ve forced us to rethink the entire way the web works. I’ll also briefly touch on many of the pitfalls that web developers still face since HTTP is stateless, and how various frameworks work to overcome this severe limitation of the web. And finally, I’ll briefly discuss alternative web technologies used to design browser based interfaces such as Flash, Silverlight, WPF/XAML and what I perceive as severe shortcomings of each of them.

  • How To: Compile ISO C ‘99 Code in Visual Studio

    This will be short how to on how to get ISO C ‘99 code to compile in Visual Studio 200x (in this case, Visual Studio 2012), since Microsoft only supports ANSI C ‘89 (and barely at that). Microsoft have made it very clear that they will not support C99, and instead urges users to move to Visual C++ instead. Ok that’s great…unless you need to write straight up C.

    Background: I’m brushing up on my ISO C for an upcoming interview I have next week. I haven’t written a line of C since the early 00’s, but fortunately it’s coming back fairly quickly (it’s a fairly simple language despite its reputation). Normally I’d fire up my favorite Linux distribution (in this case Ubuntu) and start hammering away in VIM or Emacs, but this is lightweight one off stuff that I’m doing. I have never used Visual Studio to write C (but plenty of C++ of course), but figured it’d be a no brainer. Immediately I started running into strange errors, mainly syntax errors that simply were absolutely false, they syntax was fine. Also, C89 doesn’t support the <stdbool.h> header, which isn’t a huge problem, just change the bools to ints and test for 0/1.

    But it was the syntax errors that were completely throwing me. Then I figured it out: C89 doesn’t support inline variable declaration; the variable must be defined and initialized before you can use it. Where this becomes really apparent is in ‘for’ loops. A standard for loop in C is simple enough:

       1: for(row = 0; row < SIZE; row++) /* Invalid in C89 */

    Instead you have to do the following

       1: int row = 0;
       2: for(row; row < SIZE; row++) /* Valid in C89 */

    Not a big deal, but for large nested loops it becomes extremely cumbersome. Plus all of the other features that are in C99 make it a no-brainer to use.

    I then started wondering what they other differences between 89/99 and found a laundry list of items (as an exercise to the reader, just Google it). I’ve always coded in C99 and didn’t realize how large the chasm was between the two. Unacceptable. I was about to ditch the whole Visual Studio endeavor and just hammer it out on a GCC compiler. But me being the stubborn developer that I am figured that had to be a solution. Since these are one-offs, I don’t need strict ANSI/ISO conformance, nor portability…I just need the code to compile and work, then I’ll throw it away.

    So I started poking around the Settings of the C++ project I had created to house the C code (in case you didn’t know, you can write standard C code in C++ projects since C is a subset of C++) and there it was under Project Properties –> Configuration Properties –> C/C++ –> Advanced –> Compile As: Choose Compile As C++ Code (/TP) from the dropdown. Problem solved (99% of the problems at least), the MSVC++ compiler supports C99. <stdbool.h> still isn’t supported, but this doesn’t matter since the C++ compiler recognizes bool as a valid keyword.

    If you truly need full C99/C11 support on Windows, you’ll have to install CygWin/MinGW, but it’d probably be easier to create a VM image of a Linux distro. Of course that all being said, it is so rare that anyone writes raw C code these days it probably won’t matter, but in my case it’s necessary as I’m brushing up on some of the fundamentals of programming. C is the granddaddy of ALL programming languages. Knowing the basics will make anyone a better developer.

  • Singleton Pattern Implementation in C#: One Global Object Instance, One Entry Point

    This post is going to explain how to implement the Singleton Pattern in .Net/C#. In most other programming languages/frameworks, global variables are allowed. In modern day programming this is extremely frowned upon, hence where this design pattern comes in handy: It provides a global object with only one point of reference. Once it is instantiated, all references to it are guaranteed to be to a single object instance which is available throughout the lifetime of the application.

    So, where would this be useful? Keeping an application-wide cache of objects to avoid expensive calls to non-local resources. .Net makes this fairly simple. Here’s the basic UML diagram for a singleton object:

    We create a static constructor (thus making sure only one object is created in the AppDomain) so the object can’t be created directly, a private backing store called instance of type of the singleton class, and a public accessor property (or method) that returns the private instance type. Your class must be declared as static (which implies sealed) otherwise subclasses can create new instances.

    I cannot repeat this enough: Global variables are evil. Global objects are not. .Net uses the singleton pattern for the classes contained in the System.Runtime.Caching namespace, and I suspect this pattern is used on top of the Flyweight pattern (another post for later) for String Interning within the .Net framework..In my experience, the singleton pattern comes in most handy for creating a global class encompassing a variety of collections…really, just a global cache of resources that are expensive to recreate on a regular basis (e.g. an object gets garbage collected, then has to be recreated on a regular basis). Usually this data should be largely static in nature, but using the classes in the Caching namespace, you can easily create a singleton object that maintains itself via callbacks, expirations, etc., which is something I routinely do in my applications: Create a global Cache object, write the plumbing for it to tidy itself up, keep a global instance…et voila, all of the tedious/expensive code to expensive resources takes care of itself. I’ll provide a further example of the caching pattern I’ve developed in a future post.

    Regardless, there are a couple of simple ways to implement this pattern in .Net. The first example is eager initialization, with a single lock check:and is thread safe:

       1: public sealed class Singleton
       2:  {
       3:      private static Singleton instance = null;
       4:      private static readonly object padlock = new object();
       6:      static Singleton()
       7:      {
       8:      }
      10:     public static Singleton Instance
      11:      {
      12:          get
      13:          {
      14:              lock (padlock)
      15:              {
      16:                  if (instance == null)
      17:                  {
      18:                      instance = new Singleton();
      19:                  }
      20:                  return instance;
      21:              }
      22:          }
      23:      }
      24:  }

    Note: Always create a separate threadlock object, don’t lock on the class instance itself as that’s just begging for a deadlock/race condition. Locking a static private object will prevent deadlocks since it will not allow new threads to access the object until the lock is released: Calling code can pulse/queue as much as they want, but until the lock is released, they have to wait.

    In .Net 4.0, it’s even easier to get thread safety and lazy instantiation via the Lazy<T> class:

       1: public sealed class Singleton
       2:  {
       3:      private static readonly Lazy<Singleton> lazy =
       4:          new Lazy<Singleton>(() => new Singleton());
       6:      public static Singleton Instance { get { return lazy.Value; } }
       8:     private Singleton()
       9:      {
      10:      }
      11:  }

    Very few lines of code, and we get thread safety, lazy initialization, and type safety. What’s the difference? It depends on what the object needs to acquire, and how expensive those resources are to acquire. I would lean towards lazy instantiation, mainly because I believe resources should be acquired as needed. A caveat though: code needs to be written to maintain the singleton’s references and memory usage. This tends to be a pattern that is overused in place of globals, and can be just as detrimental.

    Next post, a simple caching wrapper utilizing the singleton pattern.

  • Template Method Implementation in C#: An Easy Way to Make Your Code Hum Along

    A pet project I’ve been working on (on and off, time permitting) has been in need of some serious refactoring, so I finally sat down to eliminate some redundancy, and work on some design pattern work. A common problem in software design is making sure base class implementation is always called from overrides, but also deferring finer grained details to subclasses without changing the base class implementation, without having to always specify base.<implementation> in every single override in subclasses. Enter, the Template Method design pattern, which encompasses the polymorphic aspect of Object Oriented Design (which in my opinion is the most important of the 3 pillars: Encapsulation, Inheritance (aka Specialization) being the other two).

    The premise is simple, and here’s a very simple UML diagram that outlines the functionality:

    The base class specifies stubs (abstract methods, or protected virtual method implementations that all subclasses need base functionality for), and a public method that provides the “template” that must be followed. I’ll use a simple example for the sake of brevity. Let’s say you need a Data Access Object that connects to a database and fetches some data. The members common to all derived classes would be something like connecting to the database, then providing tidy cleanup code for the disconnect. Deriving classes would need to implement their own logic for fetching and processing the data, but all derived classes need to follow the same order of operations:

    1. Connect/Authenticate
    2. Fetch
    3. Process
    4. Disconnect/Cleanup

    This very simple example provides our template. So, our base class stub would look something like the following:

       1: abstract class DataAccessObject
       2:     {
       3:         protected string connectionString;
       4:         protected DataSet dataSet;
       6:         public virtual void Connect()
       7:         {
       8:             // Make sure mdb is on solution root directory
       9:             connectionString = 
      10:                 "provider=Microsoft.JET.OLEDB.4.0; " +
      11:                 "data source=..\\..\\..\\db1.mdb";
      12:         }
      14:         public abstract void Select();
      15:         public abstract void Process();
      17:         virtual public void Disconnect()
      18:         {
      19:             connectionString = "";
      20:         }
      22:         // The "Template Method" 
      24:         public void Run()
      25:         {
      26:             Connect();
      27:             Select();
      28:             Process();
      29:             Disconnect();
      30:         }
      31:     }

    Now we declare subclasses that implement the stub functionality of the abstract methods, Our base class will defer the fine grained details to the derived classes, but still guarantee that the core functionality is implemented. A simple class that selects a DataSet of Products from our fictitious database could look something like the following:

       1: class Products : DataAccessObject
       2:     {
       3:         public override void Select()
       4:         {
       5:             string sql = "select ProductName from Products";
       6:             OleDbDataAdapter dataAdapter = new OleDbDataAdapter(
       7:                 sql, connectionString);
       9:             dataSet = new DataSet();
      10:             dataAdapter.Fill(dataSet, "Products");
      11:         }
      13:         public override void Process()
      14:         {
      15:             Console.WriteLine("Products ---- ");
      16:             DataTable dataTable = dataSet.Tables["Products"];
      17:             foreach (DataRow row in dataTable.Rows)
      18:             {
      19:                 Console.WriteLine(row["ProductName"]);
      20:             }
      21:             Console.WriteLine();
      22:         }
      23:     }

    We could create as many subclasses as we need for various data fetching tasks, then from our main class, simply instantiate the specific subclasses and call the “Run()” method, which is a hook into our template. That would look like the following:

       1: static void Main()
       2:         {
       3:             DataAccessObject dao;
       5:             dao = new Products();
       6:             dao.Run();
       8:             // Wait for user
       9:             Console.Read();
      10:         }

    The beauty of this is that we are guaranteed the template behavior from all subclasses, and it makes our code much less brittle as we know that the inheritance chain will always return the desired implementation. If you combine this with Factory Method Pattern, you can start to harness some real power of polymorphic programming. But, that’s another post.

1 2 3 4 5 Next > ... Last »

Copyright © ::
External Content © :: Respective Authors

Terms of Service/Privacy Policy