Why is msbuild and link.exe "hanging" during a build?

We have a few C++ solutions and we run some build scripts using batch files that call msbuild.exe for each of the configurations in the solutions.

This had been working fine on 3 developer machines and one build machine, but then one of the projects started to hang when linking. This only happens on the newest machine which is a quad core, 2.8ghz I think. It runs on Windows Server 2003 and the others are on XP or Vista.

This happens consistently even if I change the order of builds in the bat file.

If I run the build from the IDE on that machine it does not hang.

Any ideas about what could possibly be causing this?

I am using Visual Studio 2008.

---
###Edit:

I see now that when it is hung the following are running:

- link.exe (2 instances) One with large memory usage and one with a small amount of memory usage.
- vcbuild.exe
- msbuild.exe
- vcbuildhelper.exe
- mspdbsrv.exe

---
###Edit:
The exe file exists and so does the pdb file.

The exe file is locked by some process, and I can't delete it or move it. I can delete the pdb file though.

I also have the problem if I just use VCBuild.exe.

I decided to try debugging the 2 link.exe processes and the mspdbsrv.exe processes.

When I attached the debugger/MSdev IDE to them I got a message box saying that the application was deadlocked and/or that "all threads have exited".

I guess I will have to check for a service pack for that msdev install on that machine.

---
###Edit:

In the debug.htm output file I get all sorts of stuff output after the link.exe command is generated.

However, for the release buildlog.htm the linke.exe line is the last line.

This is clearly a hang in the linker. Definitely a Microsoft bug.

I am now trying to figure out what the .rsp (linker response) file is.

When I issue:

>link.exe @c:\<project path>\Release\RSP00000535202392.rsp /NOLOGO /ERRORREPORT:QUEUE

That is the last line in the release build log. The debug one has lots more information after that.

Reinstalling a different version of Visual Studio did not solve the problem.

I will open an issue/ticket with Microsoft. I will post an answer if I can.

Are you using xcopy in your scripts? [This][1] suggests wrapping xcopy with cmd /c " .. " as a solution.

If that wasn't it, I'd recommend to narrow things down by only letting one cpu work (i.e. removing /maxcpucount) This would rule out any form of race condition between compilation processes.


[1]:

[To see links please register here]

Have you tried disabling incremental linking, or alternatively, always forcing a Rebuild All?

You could try this: Open the build dialog via
Menu -> Tools -> Options -> Projects and Solutions -> Build and Run

Here you can set "MSBuild project build output verbosity" to
"Diagnostic". Maybe this will deliver more information on what is
going wrong.

In the same dialog you can set "Maximum number of parallel project
builds" to 1. Maybe this works around the link.exe "hang".

mspdbsrv.exe is used to combine all debug info into one pdb file. The VS2005 version of mspdbsrv.exe is buggy, it might be that the VS2005 version has some of the same issues. Killing it before building is making a difference for some people. We're going to add it to our builds as well since we're regularly suffering from unknown PDB errors.

Whole-program optimization (/GL and /LTCG) and /MP don't mix -- the linker hangs. I raised this on [Connect][1].

The upshot is that it's a confirmed bug in VS2008; contact PSS if you want a hotfix; and the fix is included in VS2010.

If you can't wait that long, turn off /MP (slower compiles) or /LTCG (slower code).


![VS2012 screenshot for setting][2]


[1]:

[To see links please register here]

[2]:

I had a similar problem, but with Visual Studio 2010.

This is a project that had worked fine on another computer, but just not my new one.
The symptoms described matched the original Visual Studio 2008 Issue.

I was able to resolve the issue by installing the Visual Studio 2010 Service Pack 1 (SP1)

[To see links please register here]

- or just go to microsoft and search for "Visual Studio 2010 Service Pack 1"

I had run my windows "check for updates" and had thought I had installed all service packs, but apparently, I had not installed any Visual Studio Service Packs.

After installing the VS2010 SP1, I no longer had this issue.
I confirmed that I had installed VS2010 and the Service Pack 1 on other older computer with the working project a while back.