Showing posts with label Software Development. Show all posts
Showing posts with label Software Development. Show all posts

Friday, October 9, 2009

Don't comment it out - Delete that code!

You have a big, ugly, huge block of code that you want to improve. The first thing you do? Comment it all out. Then, try to re-implement in a better way. Finally, you just leave the commented out code in the file in case you need to reference it in the future. I've been guilty of this a lot in the past but I've recently come to the conclusion that instead of commenting it out, you should delete it immediately. Here's why:

It's bad code

You've already identified this code as bad. You don't want it to exist. If you keep it "commented out" so that you can reference it, you're referencing bad code. Why would you want to do that? If the code were good, you wouldn't have commented it out, right? Since it's bad, why would you possibly want to use it. If you aren't going to use it, why keep it around?

Copy and pasting of commented out code

I also notice that I sometimes will copy and paste parts of that commented out code in my new implementation. My realization is that if I'm actually using the code that I commented out, I should either:
  1. Stop using it. I commented it out for a reason.
  2. Don't comment it out. If it actually is good code, even if it's a small part, why did I comment it out? I should have left it in.

Leaving it in for posterity

A lot of code that gets left in "so you can see how it used to work." If you want to see how it used to work, you have version control, right? Just look at the history of the file and diff the current file. Problem solved.

Big block comments detailing how something works or what it does

These are also bad. If you find yourself writing one, it means that this part of your code is probably doing too much; split it up into understandable chunks. If you need an explanation of what the code is accomplishing, your unit tests are not descriptive enough. The only exception is if the code implements something intrinsicly complicated such as an an RFC, where even someone with domain knowledge may have difficulty understanding it (example: in RFC 2822, WTF is an addr-spec and what characters are allowed in a domain literal? I don't remember and I implemented it). In this case, two things are useful:

  1. Reference the RFC (or other standards documentation) by section and subsection. If I'm implementing an addr-spec reader, I can write "This implements the addr-spec in RFC 2822, section 3.4.1 (page 16)" and now someone reading the code can go read the RFC if they need to know more about what the code is doing.
  2. Write a functional spec. I know, in Agile we favor working software over comprehensive documentation. That's fine. Just because we favor working software doesn't mean we never document. A functional spec of complex logic is hardly "comprehensive documentation;" it's a useful vehicle for clarifying how something works or how different components interact. If that the most effective solution, there's nothing non-Agile about doing it.

In conclusion

If you find yourself commenting out code, ask yourself if you really need to keep any of it and if so, then don't comment that part out. Delete everything else immediately. I promise you that you won't miss them.

Monday, March 23, 2009

Don't speed up your code, make it do less instead

I just read an extremely insightful blog post here. It's about speeding up your code, and the author makes an interesting point that he learned long ago from a mentor of his:

So there is no way, really, to make code go faster, because there is no way to
make instructions execute faster. There is only such a thing as making the
machine do less.

This is fscking brilliant advice. I never thought about code this way, but upon reflection I think he's absolutely correct. I can think of 1000 ways to accomplish this in both managed and unmanaged code, but ultimately it comes from knowing how your code is being built and what instructions are being created from it. As I think about some of the general principles of OO design (i.e. SOLID and such) I really am starting to see how efficient, extensible code improves this. Maybe the right thing to do in managed or unmanaged code when you start analyzing your code for performance is to actually look at the assembly or MSIL that is being generated instead of staring at it and trying to think of ways to optimize. Read the article and learn.

Thursday, September 11, 2008

Let's just blame Microsoft!

This is a good one. Some guy named Steven J. Vaughn-Nichols is blaming the Sept. 10th London Stock Exchange crash on .Net. Wow, informative! It crashed. it runs on .Net, so that must be the reason. .Net isn't suited for real time systems! Right? Not so fast, dude.

Full disclosure

Before I start, let me just say that I do work for Microsoft and I work on the .Net framework. Does this make me biased? Probably, but I'm going to attempt to focus on other things besides "Microsoft good, .Net good" here and draw a logical conclusion.

What happens

So, what's the scenario? Well, apparently (according to Steven) the LSE runs some software called TradElec, which is a c# application. It also runs on Windows 2003 with Sql Server 2000. Clearly, the weak point is .Net here, nothing else it could possible be. Right?

You are full of fail


So Steven probably wrote all those "conclusions" down on a mat, which he then placed on the floor, so that he can "jump" to them. He clearly has. Something broke, so it's Microsoft's fault, because .Net just sucks for real-time applications. So does Sql Server 2000 and Windows Server 2003. There's nothing else that could have gone wrong, right?


There's no way it could be human error. No way at all

What he doesn't say is that this could possibly be programmer error. There are thousands of ways that a programmer could mess this up and just write crappy code. For network connections, the Asynchronous programming model is not trivial and requires some reasonably deep understanding before you can really make it work well for you. I see a lot of people mess this up, and unfortunately it's their fault and their problem most of the time because the performance you get through asynchronous programming comes at the price of being complex and involving multiple threads, which is something that a lot of people just don't understand.

Additionally, we don't know how they're doing their DB access here. Maybe they have some sort of transaction hell that's locking the shit out of their DB. Maybe they don't use stored procs (BIG performance issue in Sql2k, fixed in Sql2k5 so not a big deal there). Maybe they don't know how to create an index. My point is that we don't know, so we can't say for sure. Probably, however, this is an issue.

Finally, the .Net framework itself has some interesting quirks if you don't really understand the CLR well. I don't usually recommend books on specific software technologies, but go out and get a copy of CLR via C# by Jeffery Richter; I learned more about the CLR in that book in a month than I did in two years of using .Net every day. Granted, garbage collection takes away a lot of the complexities of memory management, which can be a big performance issue, however as a developer you STILL need to understand what the CLR is doing. Things like boxing and unboxing can take time, mis-using value types and reference types eat performance, even how you allocate objects can affect performance. For example, if you're using buffers for network traffic, if you allocate a new buffer each time, you may trigger garbage collection which will randomly hurt performance and be difficult to track down. if instead you allocate a massive pool of buffers and then just use those, they will live on the large object heap and they will NEVER trigger garbage collection so your app will be more consistent.

Blame Canada . . .um. . . er. . . .Net?

So do we blame .Net? With this much information, we really can't. It's far more likely that Sql 2000 is to blame (if anything), although I've seen shit databases created in open source just as often as MS Sql so it's entirely possible that it was just designed stupidly. It's also equally likely that the people who wrote this just screwed up, either in writing the code or improperly testing it. Again, these things would happen if the same programmers used open source software.

Wow, what a useful solution!!!

What does Steven suggest? Use linux. Wow, that will fix everything! I'll just go install it right now, with KDE and everything!!! Wait, no.

Next, he suggests Oracle. I've used Oracle and in some ways I love it way more than MS Sql Server but in other ways I hate it a lot. Oracle is better than Sql2k but I have yet to see proof that it's better than Sql2k5, however I won't pass judgement on that yet. Maybe Oracle would be a better db choice. Not that Oracle's open source or anything. It also works with .Net. I've used it.

Next, he recommends Java. Java, with the worst threading model in the history of the world (more on that later), is his recommendation for a fix! I have yet to see a case where a Java application works significantly better than a .Net application doing the same thing. A lot of the tools are similar. The languages are similar.

In conclusion, Steven is jumping to conclusions that Open Source software (+Oracle) is better for performance. He has no evidence other than "it was running .net and it crashed" to base this on. He is therefore wrong. I have an idea. So you take this mat, and you write various "conclusions" on it, and put it on the floor, so you can "jump" to them. I'll send him one!

And I KNOW it wasn't a .Net networking issue because

I am on the NCL team at Microsoft. We own the System.Net namespace, which is what handles networking in the .Net framework. It was my turn to handle issues that came that week. If it had been a .Net issue with networking, I would have heard about it. I heard nothing.

Friday, February 1, 2008

Why we can't build software like we build bridges

Many people say that software sucks. It has bugs, it doesn't work the way we want it to, it gets released late, it has useless features, it doesn't have a particular feature we need, the UI looks funny, I get epileptic seizures from its splash screen, etc. Granted, it's always going to be impossible to please everyone (duh), but with software people often find that they have a special kind of hatred (google "Windows Vista sucks" for a few examples). People say that if we build cars like we built software, they would crash every few miles, but you could completely fix them just by turning them off and then turning them back on. Obviously cars don't do that, so why must we suffer through software products that work exactly like that? Why is software so hard? Why can't we build software like we build bridges?

First of all, a bridge is a concrete thing (literally and figuratively). I can touch a bridge, I can walk across it, it can kill me if it falls on me, and it has a specific function that is well defined. On the other hand, software is an abstract concept. You can't touch it, you can't hit it with something, you can't drive across it, and it can't touch you (if it can, well, let's not go there). Software is an idea. It's an abstract concept of something concrete. Someone, somewhere said "I want to do some task and I want some software to help me with that task somehow" and then asked the question "how do I do that?" and unfortunately the answer is usually something along the lines of "it depends."

Software itself is really just a set of rules (or states, but let's not go there) that a computer has to follow. A programmer's job is to attempt to turn that abstract concept that the product owner is thinking of into a concrete set of rules that covers every possible scenario that may arise as a result of the user's interactions with the computer. There is no room for interpretation, there is only what the instructions say to do. If you try to do something that there isn't an instruction for, then the computer will most likely not behave in the way you want it to because the programmer didn't think of what to tell the computer to do when you did that. In order for software to be perfect, the programmer has to think of every possible state that the software could possibly ever be in and then program rules for all of them. This may sound impossible, and that's probably because it is.

A bridge is an easy thing to build. Someone has a problem that involves getting from here to there, and usually in between here and there is an obstacle of some sort that is difficult or impossible to traverse. The person usually has a means that they wish to use to traverse that distance which is pretty much limited to walking or driving. That makes it easy. It's possible to absolutely calculate the distance so we know how big it needs to be. It's possible to calculate the maximum possible number of people or cars or whatever that could ever occupy that bridge. You can calculate the strength of various materials to know exactly what to build your bridge out of. You can measure the stability and composition of the ground to know how you have to anchor the bridge. You can even estimate the external stresses that it's likely to encounter by observing the environment around the bridge.

This is all a complex process, but it still has an answer that exactly meets the criteria of "here to there without collapsing." Also (and possibly most important) is that once you build the bridge, nothing will change. The strength of the materials is constant (let's ignore things like rust here, but those can also be calculated at design time). The distance is constant. The environment is even constant within a certain range of parameters. The maximum load of the bridge is known (you can measure it) and it will never change. In fact, if any of those things ever did change, the bridge would most likely fail catastrophically.

Let me know tell you how a piece of software would look if it was a bridge. First, it needs to be able to connect any distance provided that there is a start point and an end point. If the distance changes, it should be able to lengthen or shorten itself as needed. Also, it needs to be able to hold up as many things as possible and should be able to change itself easily and quickly if it needs to support more things than it is currently able to. I should be able to build a bridge in one place and then immediately just copy that bridge to any other location that looks similar and has a start point and an end point. I also need to make my bridge support anything that looks like it might be able to cross a bridge, such as cars, people, trucks, airplanes, boats with wheels, motorcycles, tanks, trains, platypuses (platypi? whatever), and so on. Also, if it collapses for any reason, I should be able to just turn it off and turn it back on and it will be fine again. It should also be immune to all forms of external attack and should only allow people across who are authorized to cross the bridge. Finally, if any part of it is revealed to be prone to failure or vulnerable to attack or otherwise fails to allow objects across, I should be able to rebuild just that small part of the bridge and send it to you, where you can open the package and it will automatically install itself onto the bridge and fix your problem. That's the bridge you want me to build.

Do you know a piece of software that this could describe? A website. That's right, a fucking website looks like this. It needs to scale up and out, I should be able to patch it remotely if it's broken, it must be secure, it must be able to support anything that looks like a web browser and can speak HTTP, it needs to be installable on any web server that supports the language it's written in and HTTP, it should restart itself if I restart IIS, and it needs to work. And it's only a website. It's not AI, DSP, computer vision, distributed processing, self-modifying code, or anything else complicated like that, it's just a bunch of string concatenations. It's the "A" and the "P" of the LAMP stack. It's a couple of services out of hundreds on windows. And it is infinitely more complicated than building a bridge.

Wednesday, January 30, 2008

So, I have a blog now. It's about Agile and stuff. . .

Due to overwhelming demand, I am finally starting my own blog. Or, more accurately, a number of people lately have been complaining about me using the comments section of their blogs to write my own blog. Jeremy Miller also makes a good point about blogging for Software Developers, so here I am.

Whenever I'm giving a technical interview, I almost always ask candidate the question "What does Agile mean to you?" I also point out that if you ask this question to ten different people, you'll probably get ten different answers. I find that this question often tells me a great deal about a person's experience with and understanding of Agile, which is why I ask it. Anyway, to start off my blog, I'm going to give you my answer (Note to interview candidates: If I ask you this, you will not get bonus points for reciting my answer back to me).

To me, Agile is not a process and it's not even really an SDLC; it's an idea. It's a way of thinking about software development, and from this, several processes have come into existence that are built around the idea of Agile such as Scrum, XP, etc. The idea behind Agile starts from the fact that Waterfall (the traditional approach) tends to fail because it is very poor at handling change throughout the project's lifetime and that implementing changes tends to increase the amount of time and work exponentially as the project ages. Agile was designed to take into account that change is inevitable in software, so doing extra work that would need to be re-done when something changes is pointless.

Another idea behind Agile is that non-working software has no value to the company making it. If you can't sell it, people won't use it and you can't make any money from it. Agile addresses this by focusing on delivering working software as rapidly as possible. Ok, so maybe we didn't implement every last little feature, no, but basically we wrote the software. What we write over each iteration, indeed the reason for short iterations, is that we can deliver something that works and has at least some value right away. If the project ends abruptly, hits a deadline, whatever, we may have something that's functional enough to start delivering value to the company and that people may want to use. We can also get rapid feedback on how well the software works.

In contrast, Waterfall is all-or-nothing. You start designing, then you code, then you test, then you release. You get no value until you release, which isn't until the very end. If you fail to get to the very end for any reason, you have nothing that works and therefore a useless block of writing that is your code base and a big pile of money that is no longer yours.

As an aside, I believe that part of the reason that Waterfall fails and Agile succeeds also revolves around why software is hard to write (i.e "Why can't we build software like we build bridges?"). I have an answer to this, but I'll save it for another blog.

Anyway, in order to realize the benefits of Agile, a few basic principles have been thought up that allows these things to occur. They are mostly outlined in the Agile Manifesto (which I highly suggest you read). I am going to attempt to give you an overview of my interpretation of these principles.

We always want working software, so the best metric to measure project by is working software. Lines of code or partial requirement satisfaction is meaningless if those things don't provide something that someone can use.

We favor interacting with people because people are the ones who use the software and whose ideas the software is supposed to implement. You can't ask a document questions. A spec can only say so much. A feature like "User can save their data" means something entirely different to a customer and a developer. Dialog helps to clarify what is needed because by asking questions of each other, people can come to a better understanding of what the customer wants, which is all that really matters anyway (more on that in a later post).

We also favor responsiveness to change because software is all about change. We start to understand the business domain more and more as we work on a product for it. As we understand the business domain better, we start to realize that we made mistakes in implementation (communication helps with this also) so we have to make changes. We also understand that when the customer actually sees the software, sometimes they can say "no, I didn't actually want it to work like that" and we have to rebuild it. This is why we only work on a small chunk of the project at a time and show it to the customer frequently. The more time that elapses between the introduction of a bug and the discovery of it, the harder it is to fix and the more code that might be dependent on the buggy code.

We like to have rapid feedback. We do continuous integration in order to get feedback about if the software works as rapidly as possible. If something breaks, we can fix it right away before it has a chance to corrupt anything else. If someone's code fails to integrate, we can fix it right away because we know about it and it's still easy to fix. If we wait, it becomes harder to fix and harder to integrate.

We are mindful of the whole of the application as we build it, so we know when we must make a decision on when to implement something. We don't build things that we don't know about now. If we build something that we're going to use in the future, it's possible that the requirements will change before the future arrives. We do build things so that we can incorporate new functionality into our code later, but we are using those things now; we'll expand on them later. We build things so that we can change them as the need for change arises, because we know that change is inevitable.

We don't focus on process, but we evaluate our own process continuously. We recognize that our process can always improve and we always find better ways to do what we do. Improving our own process is simply another way of responding to change, and we know that change will always happen.

We also don't ignore things like documentation, process, and tools. They have their place. We know their place. However, when given a choice, we tend to prefer talking to each other and writing working code. We write as much documentation and use whatever tools we need but we realize that these tools exist only to make us more productive in what we already do, not to ensure the success of our project. We do not use OR/M as a crutch because we don't know how to persist our data. We do not use IoC containers because we don't know how to resolve our own dependencies. We do not use a Continuous Integration because we don't know how to keep track of our builds and integrate our software. We use these tools because we know that they make it easier for us to focus on building software. We also use them because we have had to do these things manually before and while we understand how to do them, we recognize that tools can save us time by doing for us what we would otherwise do exactly the same way.

Now, a number of people have created a number of tools, ideas, and methods that help facilitate these things. Processes like Scrum and eXtreme Programming are ways to realize the benefits of Agile, but just because you do them, you are not Agile. There are coding principles such as inversion of control, the single responsibility principle, don't repeat yourself, test driven development, etc, but you aren't being Agile by just blindly following them. You can write enough stories to consume three acres of rainforest's worth of note cards every hour, but that won't make you Agile either. You can have iterations and not be Agile. You can write unit tests and not be Agile. You can do anything you can think of and still not be Agile, because Agile isn't about what you do.

In order to truly BE Agile, you must embrace that Agile is just an idea. It is a way of thinking about development that will help you to deliver a successful software project. There are many interpretations of what exactly that means and many processes that can help you in implementing the idea of being Agile. Ultimately, as long as you are realizing the benefits of being responsive to change and delivering working software that has business value, you are Agile, no matter what you're doing.