Tag Archives: Version Control

Continuous Integration: Just A Pipe Dream?

Posted by on October 10, 2009 No comments

Agile Programming ComicOne of the fundamental principles of agile development is continuous integration. I’ve always been a huge fan of agile development, but in my experience it doesn’t seem to be something put into practice in the game industry. A lot of developers speak highly of it, however sometimes for business or political reasons we programmers don’t always get to do things that way.

Martin Fowler, who in my opinion is one of the greatest experts regarding continuous integration, has many good articles on the subject. I love the concept and I like everything he says it should be, but will I ever get a chance to see it in action in my career? I’ve talked with co-workers in the past about it and tried to get it adopted but it’s a very difficult process.

Writing unit tests, documenting code, and setting up build systems tailored for continuous integration all require time. The time spent doing this might be considered wasted by managers, executives, or producers, because “the customer doesn’t see the code”, and thus is of no value to them. The thing they fail to realize, however, is that by implementing these practices, everyone benefits. Software is released quicker because it goes through shorter and fewer testing cycles- and we all know that time is money. Unit tests, while imperfect, provide a scalable level of coverage and can catch most of, if not all, the issues you will typically encounter in a product. Continuous commits to a version control system ensures fewer conflicts and quicker turn-around times for resolving issues.

There are a lot of reasons why a company should use continuous integration, and no reasons not to. There are a lot of things I don’t fully understand in this world and the reason why companies choose waterfall management over agile is one of them. As much as I’d love to cover every little detail here, I would only be redundant. Everyone should check out Martin Fowler’s website for some good reading on everything I’ve discussed here and much, much more. He does a much better job explaining it than I could.

Dependency Management in Version Control

Posted by on October 6, 2009 No comments

Hello everyone,

A few months ago I came up with a pretty great way to manage third party dependencies in projects and I’d like to share it with the world. In my experience, dependency management has been pretty straight forward. If I am working on a project that uses three different open source libraries, normally I would compile them into binaries and commit the headers and the libraries into version control. There are a couple of drawbacks to this approach:

  1. It consumes an enormous amount of storage space on the server. On a standard HDD, this opens the door for more fragmentation and could actually affect the performance of your version control system.
  2. It consumes an outrageous amount of bandwidth. Whether you all have your own tube or you share one, this is bad and checking in new libraries or updates to existing ones could mean a 30 minute update that blocks your entire team.
  3. Depending on the version control system you use, it could slow down the performance of your working copy. For example, updates could become slower because there are more directories to recurse and larger files to process. With Subversion, the extra recursion depth can really kill your update speed locally, not just in bandwidth.
  4. If you are targeting multiple compilers or platforms, you will have to build and maintain multiple copies of the binaries for each third party library you have. This is a maintenance nightmare for a team.

Some teams have solved a few of these issues by creating a compressed archive of all of the pre-built third party libraries the project depends on and placing that on a server somewhere separate from the version control system. This creates a few issues of its own, however, such as complicating the checkout process. Ideally checkouts should be as close to one step as possible. It also doesn’t solve some of the more annoying issues, such as point #4 in the list above.

I believe I’ve come up with a solution that addresses all of the issues above. In my opinion, this is the best solution to maintaining third party dependencies for a project. In a simple case, simply write a script that will download, build, and install (i.e. restructure libraries’ headers and binaries locally to the project in a way that it expects them to be hierarchically) each individual third party library your project depends on. For example, if you wrote this script in Python, you would be able to make it portable so that it would build those libraries appropriately for every platform you are going to support.

I have implemented such a script in my HareSVN project and it works pretty good. For now, however, it only works on Windows as I do not have the resources available to test on other platforms. But if anyone is curious to see it in action feel free to do a checkout and test it for yourself!