Building a Monster vs 2005 (or 2008) Deployment Project

.NET, Build, Deployment and Installers, Error, MSI, Microsoft, Visual Studio, Windows
Comments

So I don’t think I want to get into the details of why we have a solution with over 50,000 unchanging content files in it, but basically we were handed a legacy set of Java/Apache web products and were asked to migrate it to .NET with an aggressive deadline.  Anyway, we try to compress this thing up into an MSI file using a deployment project.  We were getting an error before it would start packaging the files that said "Not enough storage is available to complete this operation" and it would be on a different file every time.  It appears it means the process ran out of virtual memory.

When trying to figure out why, we tried a few things that would sometimes work, most notably, restarting the IDE and/or the whole machine.  Then we came upon the following article that informed us that Windows has a hard-coded 2 GB limit for any given process:  http://www.microsoft.com/whdc/system/platform/server/PAE/PAEmem.mspx

So here are the steps to up the process virtual memory limit to 3 GB and also to make the Visual Studio executable aware of this raised limit:

  1. I would first back up your boot.ini before making this edit.  It is located right in the root of your C: drive and it is a hidden, system file (so change your folder options to display hidden files).  You will need to add the /3GB switch to the end of the line with the OS that you will be using to build your deployment project, found under the [operating systems] section.  Again, refer to the above link for more information about where exactly to put the switch.
  2. It would be wise to also back up your devenv.exe (Visual Studio executable) before making this change as well.  It is located under the C:\Program Files\Microsoft Visual Studio 8\Common7\IDE (or Microsoft Visual Studio 9.0 for VS 2008) directory if you went with the default installation location.  You will need to open a Visual Studio 2005 Command Prompt (or 2008 works too), which loads some custom paths and environment variables different from a standard command prompt.  It’s found by going to Start Menu | All Programs | Microsoft Visual Studio 2005 | Visual Studio Tools.  When the command window comes up, change your directory to where the VS executable is (e.g., cd ..\Common7\IDE).  Next run the following command:  editbin.exe /LARGEADDRESSAWARE devenv.exe
  3. Now reboot the machine and try rebuilding the deployment project again.

If you have time, change the whole architecture of your program and development solution; otherwise, good luck.

FOLLOWUP (06/17/08):  You should check out the next blog post after this one about an error I got later using this fix.  We do have a guy’s laptop in our company that is stable using this fix, but I have tried without success to get 2 or 3 other machines to do the same.  It also might have something to do with domain user accounts vs. local user accounts.

FOLLOWUP (06/18/08):  I got it working!!!  The problem was being logged in with a domain user account for some reason.  I am really not sure why; there must be something different about using a domain user account (perhaps during the login process or some restriction set on our domain user accounts or policies).  I did test it without the changes involving the 3GB switch, but they are necessary as well.  In fact in my setup, I also didn’t put the box on the domain at all, just to be sure.  The box runs very stable as far as I can tell with these modifications to Windows and the Visual Studio executable.

Comments

Mike Murray
If you truly followed my instructions, you would have had backups of every change you made. That would make it very easy to revert your changes.

Also, I wouldn’t attempt to make these changes unless you really are having the EXACT same issue I described in the blog post. I had many serious issues if I did something different then the EXACT procedure I outlined above.

Now that you’ve made the changes (and are apparently regretting it), I think the most important change to be undone is to remove the /3GB switch in your boot file. I don’t think the modified Visual Studio executable is as crucial to the stability of your sysetm as that boot flag is.

If you have more specific questions, please elaborate better and I’ll try to help as much as I can. Good luck!
Rahul
How do I undo these changes?
Please Help!!

Rahul