Tuesday, April 29, 2008

I can haz spec# kthx

Greg Young has been talking about Spec# for a long time. I was skeptical. I was argumentative. I was interested in attending the Spec# talk at the Seattle Alt .Net open spaces. I was blind, but now I see.

DBC

DBC is Design By Contract. The Pragmatic Programmer dedicates a decent chunk of time to this concept (chapter 4, pp 109-119 in my copy) but I'm going to give a brief overview that will probably not be adequate to really explain it, but I hope it makes you want to go read about it.

The idea behind it is that it's hard to work with many different modules and their interactions with each other because invariably you'll make mistakes. In order to make this easier, you specify contracts that a module will abide by. These are defined by preconditions, postconditions, and invariants. A precondition is something that must be true in order for a module to do its job. It isn't user input validation, since the user might make mistakes. It's things that, if met, the object with them will guarantee a particular result. This result is defined by the postconditions. Invariants are things that may interact with that particular method but at the end of that method their conditions will be true, regardless of what may have happened during that method call. So here's an example:

Let's say that I have a method that searches a sorted array for a particular value. Preconditions would include:
  1. The list must be sorted
  2. The list object can never be null
Postconditions would be:

  1. The output object will be the object that was searched for if found.
  2. The ouptut object will be null if the input was not found.
Invariants might be:
  1. The order of items in the list will not change
  2. The list will never be null
So what does this mean in code? It means that in order for this method to be successful, I have to guarantee that the caller will sort the list first and that they'll NEVER pass a null object in the list. It also means that the search method will always return the object searched for or null if not found, and it will NEVER return anything else. Finally, it means that although the method may do things to the list while it executes, the list that exists after the method terminates will be identical to the list that was passed in (assuming you're passing the list by reference).

So how do you enforce this?

Short answer: you can't. The only thing you can do is to put Asserts on your contracts and let them fail if somehow your contract is not met. This means that somewhere, your code is not correct. These constraints define things that should NEVER happen. A breach of contract is NOT a bug, it's a part of your code that is incorrect and needs to be fixed. At runtime, it's possible that something totally messed up might occur, like something in your list getting overwritten due to a buffer overflow or getting an out-of-memory exception because Sql Server is consuming all of your available RAM (I've seen that happen). When that happens, your contracts will catch it and throw an exception with detailed error messages. That is, if you've left Asserts turned on in Release, which I know most of you don't. My lead would probably castrate me if I suggested this.

Enter The Dragon (where Dragon = Spec#)


Spec# allows you to explicitly define contracts in your code that are then checked at compile time (this is optional). You can also add invariant constraints to your properties and such. You can add preconditions and postconditions. This does a few things for you:
  1. Spec# sets up checks for violations of your contracts at runtime. It will automatically throw exceptions for you if they are violated, so no more checking if an object is null, just put a precondition on it.
  2. There is static code verification that analyzes your code to see if you are ever potentially violating your contracts. It will actually throw a compile error if you might be writing incorrect code (this is an option that you can disable though).
The halting problem

So, like me, you might have said "Well, what about the halting problem? It's provably true that you can never write a program to analyze another program to determine if it will ever halt. What about that, huh? How do you get around that? Look at me, I know Computer Science buzzwords even if I can never actually show you the proof of that problem or illustrate it on a Turing machine or anything. I don't know what a Turing machine is but I read about it in Cryptonomicon and it makes me sound smart."

Well, that isn't a problem because the static verification will only prove your program under certain conditions. There are methods that are too complex to prove, or conditions under which a proof is impossible. To help this, you get the "Assume" keyword which allows you to set assumptions about the circumstances of your methods so that static verification can try to prove it, but this may still break something (although your contracts should still catch this at runtime).

In conclusion

Go download spec# and see how awesome it is. Then email microsoft and tell them to release it as a product. I told Scott Hanselman that if he blogged about it, and I blogged about it, then the 30,000 people who read his blog will all email Microsoft about it and the eight or so people who read my blog will also email Microsoft about it and then they'll have 30,008 emails demanding Spec# and we'll get it released.

Monday, April 21, 2008

My brain is full and my liver is angry

So the Alt .Net open spaces event has concluded. This is the best conference I've ever been to (with DevTeach Vancouver being a close second) for a variety of reasons that I will get into later, but let me first start out with a brief description of what exactly this type of event is.

Format

First, there is no agenda, no speakers submit their sessions for approval, there is no keynote, and there isn't even a fixed list of topics. Everyone has equal input and there is collective decision making in what occurs. You start out by proposing topics and discussing them and then people decide what topics they most want to talk about. These become the sessions that you can go to.

Each session can be whatever the people want them to be. If the sessions are small, everyone can just discuss the topics. If they are larger, a fishbowl format is used, where some number of people n are sitting in n+1 chairs such that one chair is always empty. Anyone may go up and sit in the empty chair at any time and the person who has been there the longest has to get up. I like this format because everyone gets a change to speak and contribute. Sessions aren't death by powerpoint and they aren't some company just trying to sell you something.


People


A lot of people were there. Some of them were new to Alt.Net and were interested in learning more about the movement. There were quite a few people from Microsoft, including ScottGu, Scott Hanselman, and many others. Also, a lot of the usual Alt.Net people were there (most of whom are bloggers that I link to on my home page). I missed Justice Gray and I hope that he will be there for the next one (study your VB 6 hard, Justice, so you can be an MVP). I also got to meet a few local developers that I hope to stay in touch with (including one from my own company but a different division than I work for). To everyone whom I gave a business card: please keep in touch. To everyone who gave me a business card: I'll try to get back to all of you as soon as I can.

Topics

So many things were talked about that my head is spinning. There was discussion and the exchange of ideas pretty much nonstop throughout this conference, so I'll try to go over a few of the highlights that I remember.

Spec#


This was an interesting demonstration. Greg Young has long hyped the coming of Spec# because it enables true design-by-contract where contracts can be explicitly specified at design time and verified at compile time. This forces you to ensure that your code is 'correct' when it is built. I had my doubts (the halting problem comes to mind), but now that I've actually seen it work I think it's going to be a great tool. I hope that all eight of you who read my blog will join the 30,000 or so that read Hanselman's blog in sending emails to Microsoft demanding the release of Spec#.

Are we innovating or just porting?

This talk was about all the new tools in the Alt .Net community and asking if we're actually creating any truly "new" tools or if we're just porting them from the Java community. It was a great discussion with a lot of people saying a lot of things that I wish I could write more specifically about. Someone videotaped this, I hope that it gets posted somewhere.

How to talk to suits

Very interesting talk about how to sell Agile to management but also some good information about how to talk to business people in general. Something that was brought up was the idea of having respect and trust for each other which is something that I think is lacking in a lot of places. Some ideas for helping to "sell" Agile was to present different practices as a solution for existing pains that a company is feeling. Lots of good information here. I was also surprised by the number of consultants at this talk, it was well over 75% of the group.

Has software development failed?

A good talk about the current state of software development projects. There have been some very notable failures in the industry over the years. I brought up my question of "are we all wrong" and got some good discussion going on there.

Javascript- it's not just the bastard step-child of your web app

This was a cool talk. A lot of people tend to treat javascript like some sort of second class citizen. It's there, but it isn't too important. As a result, there aren't a lot of test frameworks and established patterns out there for javascript and its code quality often is sub-par. While there are some great frameworks out there (some of which I need to take a fresh look at), there is a real need for improving the tools, particularly now that javascript is becoming more of a tool for the presentation layer and contains business logic. It is a real programming language so all the rules of good design apply to it and I think people largely haven't realized that until recently. Justin Angel had a great tool also for running javascript unit tests at build time. It wasn't quite what I would want but it is still very slick and would definitely be useful.

Education in the industry

This was a great point that Scott Hanselman mentioned at least a dozen times and I still think he could have said it more without it being overkill because it's that important. There is a significant lack of education in basic software engineering principles out there in schools. Computer Science programs teach algorithms, O(n), data structures, and polymorphism. Those things are super-important to developing software. However, what's missing is things like concepts in good object-oriented design, how to test your software, unit testing, refactoring, patterns, and version control. He mentioned speaking to local schools about your career and software development in general. I think he's right, and I intend to actually go out and do something about it. I'm not sure what yet, but I did talk to a few friends of mine who happened to be students at digipen. I ran into them on the last day and they asked me about the conference and we ended up talking for an hour or so. I'll post more in the future as I continue to follow up on this.

Overall, this was an awesome conference. There was a lot of good dialog and exchange of knowledge as well as a lot of respect for everyone there. Someone there said that we're all leaders (might have been David Laribee who said that) and so we need to go out and lead. I'm going to go out and lead, and I'm going to challenge everyone who is reading this to do the same.

Also, if you're not at the next open spaces, you'd better be dead or in jail, and if you're in jail then break out.

Also, if you are interested in seeing some more of it, Jeff Palermo made some videos that are posted over on his blog.

Tuesday, April 1, 2008

Checkout rules speed up TeamCity builds a lot

I discovered something interesting today: TeamCity allows you customize what folders are checked out from your source control (we're using subversion). When we were building before, the actual build took about 5 seconds or so but the total length of time was on average a minute and a half. People complained about this and didn't lieke doing personal builds because it took too long (personal builds allow you to send a build to TeamCity and have it try to build with your updates before committing your code. This helps stop people from committing breaking changes). I found that the reason it was taking so long is that we have a lot of stuff under version control, such as our database, the QA test projects, some tools projects, as well as a bunch of other random crap that, while important to the whole project, are not directly used by the particular build configuration we care about for the website. So, after reading the manual, I discovered that I can restrict TeamCity to only check out certain directories. Here's what I did:

My svn root for my project (this is not my real svn tree, duh) is at https://127.0.0.1:8080/projects/trunk and the various parts of the application live in sub folders. I therefore have the following:

https://127.0.0.1:8080/projects/trunk/Application - our web application
https://127.0.0.1:8080/projects/trunk/Database - the database scripts
https://127.0.0.1:8080/projects/trunk/Externals - things like Rhino, MbUnit, etc.
https://127.0.0.1:8080/projects/trunk/Tools - useful tools that we write for application admin stuff
https://127.0.0.1:8080/projects/trunk/QA - our QA department's test stuff (BVT, regression, etc)

So in order to build our web application, everything we need is in Application and Externals. However, I can't just point TeamCity to https://127.0.0.1:8080/projects/trunk/Application because then I don't get Externals. I also can't just have an externals directory on my build agents because then I have to maintain them in multiple places (DRY principle- it's not just for code; it applies to project structure and build scripts too). So, I have to check out https://127.0.0.1:8080/projects/trunk which gives me about 100mb of extra crap that I don't need to build the web application. How do I deal with this?

Under TeamCity's project configuration on the Version Control Settings section, there is a thing that says "edit checkout rules" that, when clicked, will allow you to apply rules to what gets checked out. I clicked it and here is how it works:

You put each rule on its own line. A rule starts with either a + or a -, with + meaning to check out the thing and - meaning not to check it out. You would make rules relative to the root directory, so since my version control settings for the VCS root is set to https://127.0.0.1:8080/projects/trunk, I have to make all my rules relative to that path. Therefore, to exclude Database and save about 45 seconds of checkout time, I add a rule that restricts checkout of that directory. Since the directory is https://127.0.0.1:8080/projects/trunk/Database, I do this:

-:Database

and it now ignores the database (I believe that these are case sensitive). I applied a few more rules and now all that is checked out is https://127.0.0.1:8080/projects/trunk/Application and https://127.0.0.1:8080/projects/trunk/Externals for my main application build. This reduced my build time to about nine seconds on average. Much better. There are other things that you can do here with checkout rules also but I'm not actively using any of them yet.

I have been using TeamCity for a while now, I strongly suggest you check it out. There is a free version with limited numbers of Users (20), Build Agents (3), and Build Configurations (20). You can buy additional build agents individually or you can buy the full version which gives you more features. Check it out here: http://www.jetbrains.com/teamcity/