I then exported the resource to disk to ensure that the cabinet archive contained the file I embedded into it.  When I went to open the .cab file, it didn’t seem to have any entries in it, and WinRar gave an error when trying to open the file.  Weird, I thought.  So, for fun, I opened the file in Notepad to only notice that it literally saved the text as it was shown in the hex editor. 

What a weird bug!

I ended up opening the exe in VS2008, exporting the resource from there and verifying it.  Here is what opening it in notepad looks like.

Prerequisites

When I started the installation, I was informed that I needed .net 3.5 SP1.  So I went and got it, rebooted my machine, then re-ran the installation.  Then I was told I needed Windows Installer 4.5.  Once again, I got the installation, rebooted, and started the installation.  A third time, during the installation, I was notified that I needed PowerShell!  Although the UI for showing me I needed PowerShell was much nicer, it was extremely annoying to have to run the installer three times and reboot my machine twice before I finally had all of the prerequisites.

Installing

My machine is running out of space on my C:\ drive, so I wanted to install on my D:\ drive.  So I change the drive and find the following errors:

What the hell are these errors? I mean, I realize that when I told the installer to create a directory it may not have the attributes the installer wants, but can’t the installer set those attributes?  And INSTANCESHAREDWOWDIR? Wow, that’s an ambiguous error.  So I decide to bite the bullet and install in the default directory.  I cancel out and low and behold!

The installation link has become disabled!  So now I have to close the entire installer and reopen it, how fun!  But wait, there’s more!  When I attempt to reopen the installer, I get another friendly error!

I assume it has something to do with the eight setup processes that are running, consuming over 250MB of RAM

I finally get to run the installer, I give up and let it try to install on the C:\ drive, so finally I am at “Server Configuration” where I am told to enter my service accounts.  Again, I get another error

For some reason it says I have invalid credentials, but let’s take a look at the install window. (My emphasis highlighted in the below picture)

Very nicely done SQL Server 2008.  Tell me to click help for more information… WHEN THERE IS NO HELP LINK.  Apparently I had to set up the account in the upper dialog, and not the lower one.

So now I’m just one step away from “ready to install”.  My nightmare will be over!  And guess what happens? You guessed it, ANOTHER ERROR!

Apparently, there was another prerequisite that I needed to install that it failed to tell me about!

In the end, I just ended up installing MySQL.

By the time anyone reads this, MSDN will probably have changed the link structure, so I’ll also explain it here:
If a user clicks a link within an HTML page viewed in MSHTML, MSHTML calls the HlinkNavigate function (after implementing an IHlink interface). If the host does not implement IHlinkFrame, this function launches a separate application to follow the hyperlink.

Wow, that was frustrating!

// Returns true if set, false if not set
bool ToDecimal(char* string, unsigned long* pdw)
{
    char* end = NULL;
    unsigned long dw = strtol(string, &end, 10);
    if ( dw == LONG_MAX || dw == LONG_MIN
     || (dw == 0 && wzEnd=string) )
    {
        return false;
    }
    *pdw = dw;
    return true;
}

Find the bug!

So what’s wrong with this code? Hint: many things are wrong (which is why I like it), but I’m speaking of what’s relevant to my post. Well, strtol happens to return a signed integer, not an unsigned integer.  What happens if –1 is passed in?  Well, strtol will return –1 (and on most machines it’s represented as 0xffffffff).  The compiler will then stick it into the unsigned long and away they go!

int main()
{
    //Sploits!
    unsigned long age = 25;
    unsigned long brosAge = 0;
    char* theInput = "-1000" //Nasty input
    if (ToDecimal(theInput, &brosAge))
    {
        printf("My age + my brothers age = %d",
		(age+brosAge)); //4294966320
    }
}

In my theoretical program, it’s hardly a problem, but what happens if I reserved a buffer, was doing pointer arithmetic, or something else that could cause serious damage?  This could have just become an entry point into a security hole.  So please, check your signs!

View Full Album

Overview

For those that use Linux, our installer is similar to a package manager. Each product team creates a package (usually installed via MSI), and a collection of packages is described in the catalog.

Some of the cooler features include the following:

• Completely data driven by the catalog and configuration.
• Full rule engine to exclude or include packages and products given a condition (os, service pack, specific software version installed, etc…)
• Robust downloading of packages.
• Transactional installation of packages and dependencies.
• Reducing the number of reboot’s necessary.

Catalog, Configuration, and Rule Processing

What I have worked on the most was the catalog, configuration and rule processing.  When the installer starts up, the first thing that it does is process the catalog and determine which products are applicable for installation or upgrade.  Rules are processed which determine weather or not a product can be installed.

Part of the catalog processing is to load the configuration.  This was the first thing that I worked on when I started.  If you look at the resources inside of the executable, you will notice items such as CONFIG0, CONFIG1, etc… Each of these configurations will be loaded at runtime in the order listed, with newer configuration files overriding the previous settings.  This allows us to easily change some settings for different languages or builds, without having to change anything in the code.  These may include reversing the text for right-to-left languages, choosing which products from the catalog to be displayed, or not having a checkbox checked by default due to privacy laws.

For the beta, users of Windows XP may see long startup times.  This is due to one of the products needing to find out if a certain hotfix is installed.  In order to check and see if these hotfixes are installed, we must make a WMI query, which for some reason or another is very, very slow on certain machines.  For the final release, the product was able to check for this information itself and we were able to remove the queries.

Part of our installer was to introduce a robust downloader that can download packages in the background and be resistant to network connectivity issues.  We also wanted to make sure that if we have partially downloaded a product, we can resume from where we left off.  Although the number of 56k users has dwindled in the US, there are still many people with slow connections, or pay per-kilobyte that we need to create a good user experience.

Transactional Installation

Part of the installation of the products includes a feature that allows us to install packages using transactions.  In the past (and in early internal releases of the installer), if a user had cancelled the installation, then we could only rollback the current package that is being installed.  So for Windows Live Messenger, if it installed a dependency and then started to install messenger, then the user cancelled it, we would only rollback the installation of messenger and leave the dependency.  With our transactional installer, we are able to have the rollback chain go as many packages back as we need to for the transaction.  This ensures that the user does not have incompatible dependencies after installation.

Reboot Reduction

Everybody hates to reboot their computer after installing a product.  Unfortunately in a lot of cases and the current state of Windows, we must live with the fact that reboots are necessary.  One of the items we worked on in this release was to reduce the number of people who needed to reboot their computer.  Through closing certain applications while installing the Windows Live products, we are able to prevent reboot’s.

One major change to this feature between the beta and the final release was the UI associated with it.  For the beta, the installer would check for files in use when the user selected the applications to install.  This may have lead to a long pause where the user would believe that the applications was frozen.

After checking for files in use, the installer did not give the option for users to leave their applications open, it only showed a list of applications it was going to close.

For the final release, we implemented a progress bar while checking for files in use (with the option to skip the check), and allowed the user to manually close the applications which have files in use.

Questions?

That’s about all that I can think of to cover.  What do you want to know about the installer? Any features you want to see implemented?

I’m pretty sure that I have settled on “The Engine” instead of saying the developer who wrote the code. What does everyone out there think?

Turns out that if you have custom resource types, the names of the resources in that type cannot be lower case.

As you see in this example, my resources begin with an upper case P.  Visual Studio will cry if you embed these resources with a lower case P.

Thank you, Visual Studio, for making my work week that much longer…

1. Download the AIR SDK
2. Unzip the AIR SDK
3. Download your favorite AIR application
4. Unzip the .AIR file (yes, unzip it)
5. /path/to/air_sdk/bin/adl -nodebug /path/to/air_application/META-INF/AIR/application.xml /path/to/air_application

For example, if you were to unzip the AIR sdk to ~/AIR-SDK and unzip Spaz to ~/spaz, the command for #5 would be

~/AIR-SDK/bin/adl -nodebug ~/spaz/META-INF/AIR/application.xml ~/spaz/

Good luck!

I want to give everyone a great big thanks for the valuable feedback and excellent encouragement!

2.0 Implementation

The MyEPICS 2.0 implementation contained one master database class (ME_DB) which contained generic CRUD functions, and each table contained its own class which would implement the specific CRUD functions for each table (ME_DBO). A sample way to read users with the name ‘bob’ would be the following:

$user=new ME_DBO_User();
$user->read(array('name'=>'bob'));

And to update a person’s last name to “foobar”, you would do the following:

$user->lastname="foobar";
$user->update();

About 90% of each of the ME_DBO classes were the same code, and there was even a script to create a DBO class given a SQL create statement. A major problem with this approach is that for every new table created, you needed to make sure you created a new ME_DBO file, and that the file really only contained such things as which columns were in the table and their default values. We would have about 2k lines of code between 30 files, of which only about 50 lines were different… YUCK! Over Christmas break, I decided that this was a major, major issue that needed to be worked out before MyEPICS 2.1.

MyEPICS 2.1 Implementation

With the 2.1 implementation, I took a step back and attempted to create two classes which would fix the pitfalls of the 2.0 implementation. What I ended up doing was using MBD2′s reverse engineering module in order to find out which fields were available and what their default values were. This reduced the need for a separate file for each table and removed about 1500 lines of code from the Active Record implementation. Now, instead of

$user=new ME_DBO_User();

You would do this instead

$user=ME_Db::factory('User');

This has the main database create a Dbo object with just the right properties that you need.

The only major gripe I have with the current implementation is that I don’t implement anything for relationships. If you need information from a relationship, then you have three options.

1. Cry
2. Write Manual SQL
3. Write many lines of code which does a lot of extra sql queries and is inefficient and crappy

I mainly want to create something so that the relationship table is transparent. A lot of the time, your relationship table doesn’t have anything of value in it, and it only exists for a many to many relationship.

SQL Command Functions

About

One such feature that I see in a lot of Active Record implementations is the ability to use functions named after SQL commands, such as:

$db=new Some_Active_Record_Class();
$db->select('firstname')->from('User')->where('id=5');

Mostly, I had disregarded this need in my Active Record implementation because it seemed like it was a lot of work for very little gain. Why would you need to allow such functionality when creating a simple sql statement would do?

Well, I think I had just stumbled upon the reason for its need.

The Problem

I have a class for a user and contains information pertaining to the user such as first name, last name, e-mail address etc… It has some very useful functions such as getUserWithRole($role) and getPriviliges($user). What I need to be able to do, is extend this information through one of the MyEPICS modules. Such an example is in my Roommate Finder application, which extends the user to give them traits. The user module does not know about the Roommate Finder application, nor should it have to. Now, here’s the problem. What I need to be able to do, is return a result set of all the users’ traits, where the user has a certain role. I already have the code to return all users with a certain role in the Users module, now I just need to attach the trait information to the output of it.

I have come up with a few different solutions that would plug this hole, but not fix the problem, with most of them being some variation of typing up a single sql statement to get me the information. What I don’t want to do, is getUsersWithRole(), then iterate through all of theses picking out the traits for each individual user. This would result in at least one sql statement per user, which is highly innefficient.

I want to be able to execute this functionality in one sql query, without having to tightly couple the User module with the Roommate Finder module, and without duplicating functionality already existing in the User module.

A Solution?

What I have been thinking about that would solve this problem would end up having me creating something like the database command functions. Then, what the getUserWithRoel() function would do, is instead of returning the results, it would return a Dbo object with the sql already added. So that if getUsersWithRole() would normally result in some query like “select role from roles where role=?”, then it would now return an object that represents this statement. What this would allow me to do is chain these functions together, then only execute the query at the end. For example:

//fetch results in an array
$dbo=$userModule->withRole('Tenant')->join('Traits')->fetchArray();

This has several features which I haven’t implemented. Easy relationships, chaining of functions, and the sql commands.

Is it bad that I am never satisfied with what I already have, and always want to refactor to make it better?