UPDATE: There is a workaround to what I have said here. See this post - a really cool way to build WP7 targeting different WP7 builds: http://justinangel.net/WindowsPhone7EmulatorAutomation#BlogPost=TFS2010WP7ContinuousIntegration
So the ultimate question is, how do I build my Windows Phone 7 Series apps on TFS and perhaps run unit tests against a Windows phone 7 series emulator. The short answer is you can't, well that's not entirely true. You can build your Windows phone 7 series apps on TFS so long as you install the developer tools on your build server.
Bear in mind when you install the developer tools you will not only get all the required build targets and Silverlight framework for Windows phone 7 series but also a VS2010 IDE as well. This will either be VS2010 Express edition if you don't already have VS2010 RTM installed on your server, or the tools will integrate into VS2010 RTM (if installed). Either way, you are going to get an IDE on your build server.
This might sound rather ugly and I have asked Microsoft for separation of the tools/framework etc from the IDE so ISVs/consultancies can be more flexible how they build their apps. But this is really not high on Microsofts priority list right now. The reason is simple. These tools are targeted toward consumers, so the majority or at least what Microsoft would like is the majority of people building apps/games for Windows phone 7 series are individual programmers whether professional or ameteur and not big companies. Long term this will change.
Instead, the priority is to deliver a really quick and easy way to get up and running building Silverlight or XNA apps for Windows phone. And Microsoft has done this. Simply go to http://developer.windowsphone.com/ and you can download a single package, run it and this will set your machine up ready to build and ship apps on the Windows phone platform. No other downloads are required.
The cool thing about this is, unlike Windows Mobile development, the tools are free. A great step forward.
So to recap, in order for more professional programmers to build Windows phone apps using a sophisticated continuous integration yada yada solution, you need to install the tools on your build server, but you will get the IDE as well.
I also mentioned unit testing. Currently the latest beta tools do not support unit testing on the emulator. Of course you can still write unit tests that execute on the desktop against your Silverlight for Windows phone apps. I have a sample of this over at http://wp7.codeplex.com/