Why I won't be adopting Visual Studio 2010 just yet

I'm probably the only person in the twitterverse that isn't excited about VS2010. I've given a few reasons, but in general I don't think I can adequately express my concerns in 140 characters, so in this post I will explain the reasons behind my choice, hopefully in a way that makes sense.

.NET 4.0

I want to be perfectly clear; I'm limiting this post to Visual Studio 2010, NOT the .NET 4.0 framework. I actually really like .NET 4.0 and I think that it has some great new stuff and some good fixes for old stuff. I know that WCF has done some cool things also and I'm still excited about where the whole System.Web.MVC thing is going. The problem I have is really limited to the Visual Studio IDE. I'm also not going to mention my raging hatred for TFS here.

Past versions

I've been really impressed with Visual Studio since VS97. In general, it's been fairly stable, it has some good features, and the debugger has saved me many times. I know that intellisense isn't always as intelligent as I would like, but overall it's been pretty great and has gotten better with time. I've always adopted fairly early on, also; I got VS 2008 from some sort of Visual Studio release event at MS (I forget what it was exactly). I have had a few complaints over the years (add reference took FOREVER in previous versions, vastly improved in VS2008, and then don't forget the dozens of unsorted new items you could add, all of which were fancy names for text file), but often these were with regards to how people were using Visual Studio (see my post about Visual Studio is not a build tool) and not really towards the product itself. I could leave it running for days and it would usually remain stable.

VS 2010- my history

So first of all, I'm not judging this just by the quality of the betas and of the RC build. Remember, I worked at Microsoft so I've actually been using VS2010 internally for a long time; I believe I started sometime back towards the end of 2008 actually and used it all through 2009, since I was working on .NET 4.0. My first impressions of NetFx for 4.0 were that the installer was pretty lousy (it didn't fully support win7 yet though) but otherwise it was pretty nice. However, my first impressions of Visual Studio were that it was slow and it crashed alot. No big deal, I thought, it's still very early in the product's lifecycle, I'll struggle with it for a while and see how things go.

So months go by and the product is not any faster. If anything, it crashes more frequently, and this is while I'm trying to do simple things like open a file or type, not anything difficult, although I had problems with the debugger as well. Beta 1 came out, and there was a lot of feedback that it was slow and crashed a lot. That's because it was and it did. That's ok, still early in the product's lifecycle, right? Surely they must be addressing performance issues.

Nope, still adding features. Next beta, product still slow and still crashes a lot. In fact, I think it was actually getting less stable towards the end of the year. In November and December, it would crash constantly when I would do simple, normal things like "find in files" or "right-click" or "type something" and I filed bugs. My personal favorite was when it crashed if a file was called "GlobalSuppressions.cs" (didn't matter if VS created the file or if I did, I think it was just the name). Ended up being something to do with corrupting the line endings. Oops. The beta didn't have that bug in it.

Of course, towards the end, there seemed to be a massive push to make the thing go faster and crash less, which of course resulted in the release date slipping and a public RC. Surely this is a good thing, right?

So why the hate?

Over the course of this product's life, I've seen it get less and less stable, up through the end of last year. Features were added, and some of them were good, but overall the damn thing just didn't run and cost me weeks of productivity fighting with it. This tells me several things:

1. This is a re-write of Visual Studio's IDE. In fact, it's written in WPF, which is not something that I would have described as "fast" or "performance oriented" in general, so I (and some others that I knew) had some doubts from the beginning. That on its own would be a pretty dumb reason to hate on a product but that brings us to my second point:
2. This thing has, from the very beginning, been unstable and slow. This does not inspire confidence in the underlying design of the software, particularly since it was a total rewrite and has gotten worse over time. Then, at the end, it magically speeds way up somehow. How did that happen?

Well, if I were working on a product for over a year and it was slow the whole time, and then in the last couple of months I had to make it faster, I would definitely not have time to go back and look at the whole design and refactor or re-work fundamental parts of the system. I would profile it, find a bottleneck, and then hack it into submission, rinse, and repeat. I don't think I would be able to change the design to something fundamentally more stable at that point, and knowing what I know about how software is developed at Microsoft, there's no way I would be allowed to alter parts of the system like that. Just looking back at my own experience, I feel like what best fits the evidence and my observations is that people started hacking away at it trying to fix the bugs, leaving it in a state where it works, but the underlying design and code is unstable and brittle. If I still had more hacking to do, I would try to buy time by pushing the release date back a bit.

I have also heard complaints and speculation that the use of WPF has caused problems. It is definitely true that WPF is somewhat untested technology, particularly within Microsoft, where VS 2010 (I believe) is the first major product to be built using that technology. Think of it as public dogfooding. However, I think trying to dogfood WPF by including it in a major product was a huge mistake, and other people (who will remain nameless but work at MS in divisions that aren't devdiv) have expressed similar concerns, and some of these people were in a better position than I to make those statements. My personal opinion is that WPF is a contributing factor but not the underlying cause of the performance problems.

Now, I have not looked at the code base and I can't say for certain that a bunch of people just started hacking away at something with a fundamentally unstable design, however, in my mind at least (and in the minds of others that I know), this is the scenario that best fits both the evidence available as well as what I know about how Microsoft works internally (at least in some orgs). I know that some of you will probably flame me and say that I'm being completely unreasonable and acting on something for which I have no evidence.

To that I would say: that's my point. I don't know exactly what happened with this product or why it was so slow and then suddenly got so much faster. I can't say for sure that it was a bunch of people just hacking away at the app at the last minute and I can't say with certainty that this made it brittle, unstable, or just poorly designed. What I can say is that, given what I have observed, I just don't trust it yet. I don't trust that it will increase my productivity. I don't trust that it will be stable and not corrupt my data. I don't trust that I will be able to use it without pain. I don't trust that I will be able to cleanly uninstall it if it ends up hurting more than it helps (heard plenty of uninstall complaints on Twitter and I know for a fact that the uninstall scenario is less of a priority at MS or at least it was when I was there).

I haven't lost hope for it; I certainly think there are some great features included in VS2010, but I think I'll wait for the service pack.

Visual Studio quirk- it works on my box

I encountered an interesting Visual Studio thing today. Someone sent me a bug with a repro. I ran it on my machine and started stepping through the internals of the framework to see what the issue is. It worked fine. Hmm, that was interesting, I wonder why? So I run it again and this time I don't step through it and it fails. Ok, that's strange. When I just execute the code, it fails, but if I step into it and don't do ANYTHING except step through the code, it succeeds. WTF?

Well, I had forgotten about something: Autos and locals. In Visual Studio, when debugging, the debugger creates watches on local variables as well as a few things it just watches automatically. In order to get the values of these, it has to evaluate them, and therein lies the problem: If evaluating any of those variables causes any side effects that don't occur during the normal running of the application, it can cause unexpected behavior. Here's an example:

Let's say that I have some state property on my object that is initialized to null. I have a method that depends on this state property being set. That state property is set when you access another property somewhere. Assuming that property is not accessed in the code path that I'm executing, the state property will not be set. HOWEVER, if I trace through the code and evaluate the property that sets the state property, it will end up setting my state, thus changing the way my code executes. Let's look at a concrete example:

class testthing
{
private string s = null;

public string PropString
{
get { if (s == null)s = "new"; return "new";}
set { s = value; }
}

public bool forceit = false;

public bool DoSomething()
{

if (forceit)
Console.WriteLine(PropString);

return s == null;
}

}

class Program
{
static void Main(string[] args)
{
Console.WriteLine("Starting run");
testthing t = new testthing();
bool x = t.DoSomething();

Console.WriteLine("result was: " + (string)(x ? "true" : "false"));
Console.ReadKey();

}
}

So the class testthing has a property PropString that doesn't set the internal value of the private field s until the Get is called. Therefore, if you never call PropString.Get, it never sets the value of s, and DoSomething() will return true because s defaults to null. Run the example code and observe this, it's pretty straightforward.

Now, run it a second time, except this time put a breakpoint on the first line in DoSomething(). When it breaks, hover over the Console.WriteLine(PropString) so that it forces the debugger to evaluate PropString. Now, execute the rest of the code (f5) and observe the output is false, because the debugger has executed the getter of PropString which had a side-effect.

So, the next time you debug an application in Visual Studio and it works in the debugger but not in the code, look at your variables within the method throwing the exception and see if any of them could possibly be changed through evaluation. If so, then you may have found the problem.

One final word: If you have a unit test for something like this, your unit test will fail since it won't evaluate the property when it's run. It would be far easier to write a failing unit test around the method that has the bug and then figure out why it's failing than to step through the method and hope that you can see where it's going wrong.

Visual Studio is not a build tool

Visual Studio (henceforth Visual Fucking Studio) is not a build tool. I have spent the last few days hating Visual Fucking Studio more than ever before. I have hated on it in the past for things like the "Add new item" dialog box that has about 100 different options in no particular order that are all just names for a text file. I have hated the "add reference" dialog box that takes forever to load because it always has to parse every .Net assembly in the GAC and every COM object ever created before it can load (I know they fixed this in .Net 2008 but it still takes a long time to load the first time and I wish it would default to the "project references" tab because that's what I always fucking use). Both of these things that I hate specifically, however, are nothing compared to the build issues that I've had in the last few days.

How we build normally

So on my TeamCity server, I build the project using a Nant script. This is really not hard and it works well. You can also run the build script locally, and thanks to a few ideas that I've borrowed from some people it all works quite nicely and has very few issues. In fact, the biggest problem that happens is when someone adds or removes a file in the Visual Studio project and forgets to modify Subversion to include/remove the file and breaks the build but this is easy enough to detect and fix.

However, people on the dev team absolutely have to have the capability to build the project using ctrl-shift-b and run it with f5 so that they can debug in the Visual Studio environment. This means that when I hit f5 it has to work. This is the way of pain.

Why oh why did I remove that project????

So our application needs to work with both Sql and Oracle with identical functionality. This is one of the most important requirements. As a result, we had to write a data access layer that was abstracted enough so that our factories didn't need to know what platform they were connecting to (there is a reason that we did this ourselves instead of using NHibernate or something similar but more about that in a future post). This project and it's associated dependencies is commonly referred to as the "Data Access Layer" for this project.

Now it turns out that we actually have three or four separate projects (and a few future projects) that will need the Data Access Layer in order for them to live. Initially, we wrote the Data Access Layer in our main solution file for the application, but as soon as we realized that we'd need it in other places, I decided to factor the Data Access Layer out of that solution and give it its own independent solution and build process (it took me about five minutes to modify team city and our NAnt script to set up this new project). So, I now have a separate Data Access Layer solution that I can reference in all projects that are dependent on it. Now I try to build the main application solution and that's when the fun started.

You can't reference that

So you can't reference a project in another solution using Visual Studio. Ok, not a problem, let's just use a file reference. Well, where do we point the reference to? Oh, well there's the obj/debug directory that has the DataAccess.dll and it's associated referenced dll's, let's point it to that. But wait, what about obj/Release? What if we change the build mode? Well, it's unlikely that the developer builds will not be in debug mode, unless there's a bug that is only reproducible in release mode or one of 1000000000 other scenarios that would necessitate it. That means that you'd have to change the reference. What if we move that project around more? You'd have to change the reference. What if we blah, blah, blah you get the idea.

So, I get a great idea: I'm going to create an Artifacts directory in a common location for the build. Then I can just reference whatever DLL is in there so it will work in whatever mode I built the Data Access Layer in. I'll create a postbuild step that deploys them into the artifacts directory and then keep them out of version control. That way, all you have to do is make sure you build Data Access Layer and everything else will build. Cool.

Batch files have been around for years and they're not hard

So I write a simple Copy command in the post build step in Visual Fucking Studio. I run it. "Error: The command exited with code 1." WHAT THE FUCK DOES THAT MEAN? I check the output that Visual Fucking Studio tells me its running and it turns out that some of its macros don't actually mean what you think they mean and append an extra slash on them, so ${projdir)\bin actually resolves to c:\some project\directory\\bin instead. I fix this, it runs. I run it again a few minutes later "Error: The command exited with code 9663." WTF, bitch? What do you want now? I finally start adding echo and dir commands and determine that the command is being run on the wrong directory so no files were originally copied. Now I run it a second time and I get "Error: The command exited with code 9347" or something. This is about when I wished I hadn't quit smoking. I forget what fixed this, but suddenly I'm getting "0 files copied" which is not what I want, particularly when there should be 3 files copied because the source directory has three files in it and copy *.* means copy all files, in this case THREE OF THEM.

Are you sure (y/n)?

It takes me a minute to figure out that the copy command wants to ask you if the files should be overwritten first and if you don't answer it assumes that you don't. Adding /Y to it only surpresses that message but it still assumes that you don't want to overwrite. Fuck you, DOS. I wish I had cygwin installed about now, but that isn't helping me fix the build. So I add a delete command to my artifacts directory prior to the copy and after another "exited with code 1" I determine that there's another one of those directory path issues and finally get it to work. Happiness.

We sure are lucky the build worked at all

So now I start looking at getting the remaining application solution to build. When I first start, it utterly fails for some reason. It turns out that no dependencies were set between projects and there was no build order, so the whole time we were working on this project it was coincidence that it was building correctly. I set some dependencies and changed the build order and put in a reference to my Artifacts directory and the build worked. So I hit f5 to run the application and what happens? App no work. Sadness. After a few more minutes, it turns out that since this is a web app and the UI is in its own project, all the DLL's need to be copied in the UI/bin directory and of course they're not. Our project is structured so that each project should only reference a project that is directly below it. This keeps UI from referencing Data Access Layer which is a good thing. However, without a reference, it won't copy the dll's over because Visual Fucking Studio is not smart enough to follow references in dependent projects. Not a problem, I'll add a postbuild event to copy all the dll's from artifacts and the other output directories into the UI/bin folder. Go back and re-read the previous paragraph because that's pretty much the same thing I went through to add this post build step. I even copied and pasted the commands from the Data Access Layer but it still didn't work. I don't even remember why. I think in total that I spent almost four hours trying to copy like six files into a few directories as part of a build that we're never even going to use except to set breakpoints in the app and run it. I really hate Visual Fucking Studio right now.

Things to fix:

Here's what I'd like to see changed in Visual Fucking Studio:

1. If I reference a project and that project has references to anything not in the GAC, chase down those references and copy every binary I need into the /bin folder where the app is going to run.
2. It appears that Visual Fucking Studio just puts all the commands that I add to prebuild and postbuild events into a .bat file and runs them. I would like to see that .bat file. Better yet, I would like that .bat file not to exist and for it to just run the commands.
3. I want to see WHICH command failed in my steps, not just some useless error code.
4. I would like to be able to add a reference to another project in a different solution.

I don't foresee myself not using Nant any time in the future.