B# .NET Blog

While I'm in the mood of writing up those cookbook posts:

• The custom MSBuild task cookbook
• Easy Windows PowerShell cmdlet development and debugging

let's do another one. Custom actions are a powerful way to extend MSI-based installers with custom code. Maybe you've run into these before when you created for example a managed Windows Service and hooked it up in the setup project (essentially installutil-able components can be added to an installer by means of a custom action). In this post we'll write our own (dummy) custom action and show how to debug it nicely, something that seems to be a barrier for quite some developers to consider writing a custom action.

Step 1 - Create a class library project

Once more, Class Library is your friend:

Step 2 - Add references

Choose Add References from the context menu on the project node in Solution Explorer. In there, select System.Configuration.Install:

Step 3 - Plumbing the wires

Custom actions in managed code are wrapped in Installer-subclasses, so derive your class from Installer:

and import the System.Configuration.Install namespace. In order for the installer class to be picked up at runtime, you need to attribute it with RunInstaller(true):

This is the result:

Step 4 - Add functionality

In order to make the custom action do something, you need to override some methods of the base class. I've indicated the most common ones:

These correspond to the actions taken by MSI. Let's just override Install and Commit and play a little with state that's passed around (let's not go in depth, more information is available in MSDN):

Step 5 - Make it debuggable

Our goal is to attach a debugger but custom actions are launched somewhere in the middle of some MSI process. How can we allow for easy debugging? Here's a way to do it: instrument your code with some wait mechanisms to allow for attaching the debugger. A nice way is to use MessageBox. To do this, you'll need to add System.Windows.Forms to the references of the project (as in step 2) and import the namespace:

By wrapping these in DEBUG #if's we make sure the code doesn't make it to Release builds, which is good.

Step 6 - The setup project

Time to build the setup project. Add a new project to the solution and choose for a Setup Project:

The project will open in File System view. Go to the Application Folder, right-click in the right-hand side pane and choose Add Project Output:

Select Primary Output for the MyCustomAction project created above:

This will add the DLL file to the installation folder:

Now it's time to add the Custom Action to the installer (in technical terms to the to-be-built MSI database). Make sure the setup project is selected in Solution Explorer and select Custom Actions Editor from the toolbar:

Adding the actions is simple:

and select the Primary output from MyCustomAction from above:

Do the same for the Commit node.

Step 7 - Build and debug

That's it. Time for a test run. First, build the MyCustomAction project. Next, right-click the setup project node and choose Build. This is not done when building the solution (since it takes quite some time and you likely do it only sporadically in a bigger solution):

Next, right-click the project again and choose Install:

Here we go. Click your way through the installer and wait. On Vista you'll need to give UAC consent. After a while you'll see:

Don't click OK yet. Switch back to Visual Studio and choose Debug, Attach to Process:

You won't find the Attach debugger here dialog in the list at first sight, that's because the custom action is running under the context of the Installer Service in an msiexec.exe instance running under a different (service account) user. Mark 'Show processes from all users' to see it:

and click Attach. On Vista you might see the following when you didn't start Visual Studio elevated:

Elevation is needed because you're about to debug something with more privileges and rights (running under SYSTEM after all). Choose Restart under different credentials and accept the UAC prompt. Visual Studio will come back in the same project but you'll have to repeat the previous steps to attach the debugger. Notice the user name is now displayed in the dialog:

Set breakpoints on the instructions right below the #if DEBUG section:

and click OK on the dialog:

You'll see the breakpoint being hit:

Woohoo! Feel free to step though the (one) line(s) of the custom action and finally hit F5. Now the commit dialog appears and when dismissing it, we'll end up on the next breakpoint:

notice the state from the install phase was recovered:

Enjoy!