Saturday, 3 July 2010

Hyper Geo 64

As a lot of people who know me will admit to, I am a fan of video games. Pretty much where ever there is a TV in my house, there will be a video game console attached to it.

If you are a guy round about the same age as me, it is possible that you will remember some of the great video game consoles of the 1990's. One that may stick out a bit was the Neo Geo made by SNK. I remember this console having a near mythological impact on video gaming, the games themselves cost almost as much as the console itself, and chances are you never met someone who owned a Neo Geo.

The console itself was pretty much a stripped down version of an arcade machine, this is why the games cost so much money. The AES, which was the home version of the console, provided arcade perfect graphics during a time where people were arguing about whether or not the SNES was superior to the Mega Drive. I think there were a number of attempts by SNK to create a newer, cheaper console for home use - they came up with the Neo Geo CD as well as two handheld consoles.

I think one of the most interesting consoles they came up with was the Hyper Geo 64. From what I can make out, this was SNK's attempt to flirt with 3D graphics. It never made it to the home market though and failed as an arcade machine with only a few games made for it.

Given my interest in consoles, I have embarked on a search to pick up a Hyper Geo 64 and see if I can get it firing. I love old tech like this, so I am hoping that it wont be too expensive to get one.

Continuous Integration and SCM

Last month, I went on a bit talking about Continuous Integration and how I have developed myself and my skills in relation to this.

I left off by saying that the needs of the company had expanded some what as had the needs of individual development teams. My team was slightly different to the main production and new build teams. The work and code we produced was made to facilitate the development of the applications we offered to our customers. So for us, there was very little in the way of UI development etc. The code that my team developed was primarily to support the rest of the development teams, enabling them to get a birds eye view of development at any one time and across all projects. We didn't just restrict this to management, it would be a bit daft to do that. Pre CI, we often heard the following complaint "we just do not know what is happening". In the build team we were responsible for maintaining all of the build and test environments, infrastructure took care of the guts of them and we maintained the environment based on a development teams specification.

So, to start developing version 2.0 of the CI system, we need to appraise a few new things. First up was our SCM tool. Our current tool was just not geared up to what we wanted it to do. We had been using the same tool for quite some time prior to me joining the company, it was fine in the pre-.Net days where there was only one customer, but as the company grew we needed to get something a little more robust, extensible and modern in. So, one of the first things we did was move away from Surround as our SCM. There were a number of options available to us at this point in time. Team Foundation Server could cater for us at version 1.0, it had a SCM system, it could carry out builds on more than one server etc. The only trouble with it was that it was v1.0 software, even though it was a production release, it was a pain to get it to work. Thankfully, the head of development had looked into this issue quite some time ago with a view to migrating from Surround to a newer tool. His initial efforts at the time met with apathy - there was no need for a new tool as the one they had was fine. So, when the time came the head of development pointed me in the direction of a package called Accurev. By this point in time, I had been made up to the head of deployment and testing, my main aim here was to get a complete CI system, from build to testing logic, to deployment to an environment and then further testing (UI stuff, this was a web app). I took a look at Accurev and liked it pretty much straight away. It was a very modern tool, it allowed you to work under the Agile umbrella of software development and allowed my team to quickly configure code and builds whilst at the same time gave the releasers time to make sure the code we were about to release was the correct code (there was a time prior to Accurev that code hadn't been checked in or re-based properly).

To fit into our CI system, all we needed to do was to create a number of NAnt tasks to work with Accurev, these were nice little tasks to make sure that the code was being updated on the build server. Integrating Accurev, Cruisecontrol.Net, NAnt was pretty simple in the long run, and it let management chill out when we were able to demonstrate how a fairly significant migration could take place with out a lot of drama.

With our new SCM in place and all the developers and other people who needed access to code trained up on how to use it, we were ready for take-off. Took a weekend to export the code we needed from our old SCM into our new one. It didn't take long to script the whole event, but did take a while to get the code from a to b. Once it was done, we were ready. We had a nice new shiny SCM that everyone knew how to work and we were ready to re task the deployment/release team into a new method of CI.

Of course, there were some hiccups along the way. I think the scariest one we had was just after migration, one of my colleagues accidentally deleted some very important code a few days in-front of a major release. bit of panic at the beginning, but we sorted out a major catastrophe fairly quickly. One of the good things that Accurev offers is the fact that it never, ever forgets code. It actually stores all your code ad infinitum, so a quick look through the manual and we were able to bring back all the code we needed after a few bits of code on the command line.

With this migration complete, we were only a few steps away from providing a major update to our CI system.

Thursday, 10 June 2010

Unit Testing .Net Automatically

When I am working on a project, I try to keep the tools we use for building and testing in the same language as any production code uses. So, if I am working on a .Net project, I will endeavour to use .Net wherever possible. The same can be said for things like Java, although I rarely develop in this language, I will try and make sure that the tools used to build, test and deploy the code are based in Java.

Recently though, I have been working on pure c# projects. Since becoming a contract developer, I have encountered a number of lead or senior developers that simply turn around and say "no unit testing, just make it work". It makes me a little sad when I hear comments like this. As far as I am concerned, unit testing is one of the most useful processes a developer can exploit. It cuts down the amount of logic fails in testing and in production. For a relatively short amount of work prior to development, both analysts and developers can agree the unit tests that support the specification - once the test is defined for each unit of code it never has to be changed again (or rarely changed). This cuts down the amount of fails picked up in testing. Some lead developers I have worked with recently have turned down unit testing flat simply because they do not understand what a unit test actually is. You find this situation in teams that are under resourced and over time/budget. The embattled lead developer, possibly dealing with a bit of cynicism, just wants to make the deadline. But, one thing I have seen is that if you take the time to explain the benefits of unit testing, they normally come round and embrace the concept.

These days though, Visual Studio comes along with built in unit testing support. I think it is about time this has happened, even though I am not a fan of what is bundled. My unit testing tool of choice is MbUnit. These days it is part of a much larger test automation platform called Gallio. I like MbUnit for one reason you can test against a DB and roll back any changes once the test is complete, fail or not. I know that typically this is the way you should use unit testing with a DB, but for small things its fine.

Most unit testing tools will come with a test runner that you can use to run your tests, there are also VS add-ins that will do this for you - but I am not going to teach anyone to suck eggs here. Instead, I am going to examine just how unit testing can be automated as part of a larger build process. Some unit testing frameworks will come with an example NAnt or MSBUILD task that can be slotted straight into your build process. There are some out there that will not do this and/or you have the need to execute tests with a test runner outside of NAnt or MSBUILD. Another bit of win for Gallio is that it has very good support for NAnt. There is a task provided by Gallio that allows you to use the test framework automatically, there is a really good example of how this can be done here. The fragment of xml on that site is enough to demonstrate how Gallio can be used with NAnt, however in practise you may find that it is too simplistic, especially for larger builds.

The trick comes down to maintaining your unit tests. IMHO, creating a requirement for naming unit test assemblies is a must, for instance calling your unit tests someting like foo.tests.mbunit is very important for managing the fixtures that will carry out your tests. Personally, I like to provide unit tests within the project it tests. However, I suppose that you could create a separate solution containing all of the test fixtures, I am just of the mind that if you have a project then the fixture should go in that project to maintain encapsulation.

Anyway, regardless of where you keep your fixtures, the naming convention is pretty important as you can make NAnt loop through all assemblies that use a certain naming convention. You could, of course, use NAnt to run an external tool to do this for you etc.

One complaint I used to here from developers when they were just getting used to unit testing was the fact that they needed to run a command on the CLI or use a test runner to prove their code. Again, there is a simple solution to this problem called Test Driven. They provide a fantastic little tool that allows you to test fixtures from within VS itself. I know this may seem like teaching some people to suck eggs but describing this, but there are many new developers who are new to .Net or development in itself who may not know about some of the tools. I am not going to describe any NAnt code here to explain how it can be done. There are lots of NAnt scripts I have written for other bits and pieces I have done in the past, these coupled with the support docs provided by Gallio, NAnt and et al will show you how this can be done.

Wednesday, 14 April 2010

Back once again with the renegade master

Been a while since I last posted on here. My aim to put up at least one worthwhile post every month seemed to fall by the wayside.

I still havent setup my old Acorn machine correctly yet, and cunningly my main desktop has decided it doesnt want to speak to my HDD any more.

Good news though, I have a new job that is very interesting to say the least and I have move house.

Bad news, totalled my bike and I have suffered from bad TV and travel karma over the last few weeks. But onwards and upwards! over the next few weeks I intend to put up at least one post concerning the Levenshtein Distance. There is lots of interesting proofs of concept out in the wild at the moment, but there are none concerning an actual useful example. I know that not everyone will be able to implement a really good peice of code to do this (and I doubt anything I produce will be class leading). But this is something I have to work on in my new job, so this will help me figure it out in my head :)

Saturday, 3 April 2010

Continuous Intergration

It's about time I started blogging about CI again, given that I have developed and worked with some of the best CI systems I have had the pleasure to encounter.

When I first got involved with CI it was with Java development. At the start we simply used Ant to carry out our builds and tests on remote servers. As time progressed, we started to look at some of the CI build tools that were available to us. We came across Cruisecontrol, which I think was the tool de jour when it came to build servers. I was naturally impressed with the features offered by Cruisecontrol at the time. We were able to put together a simple CI environment to carry out our builds and/or churns. As time progressed, we looked for ways to integrate Cruisecontrol into our main repository - the company I was working with at the time used Perforce as it's SCM, we also had a tool called the repository that held all of our application meta data (we made mobile phone apps, so the meta data was a collection of definitions per handset and carrier). By the time I left that company, we had almost a complete integration of our build environment with our repository and SCM.

From here, I moved to a .Net software house. I brought with me the concept of Cruisecontrol and CI. My new employers could be best described as a group of energetic younger people who worked on the bleeding edge. This time round, it wasn't mobile phone apps, instead it was the ultra serious realm of financial software. When I joined this company, the guys had just put the finishing touches to a migration to .Net, easy to implement but unforseen issues cropped up - that is a story for another day ;). These guys were up for anything when it came to CI and automated testing. I introduced them to Cruisecontrol and Ant for creating a CI build environment, the head of development (my boss) at the time was very interested in implementing a unit testing regime for the newly migrated .Net code. I remember working with an intern who had been tasked with creating a set of guidelines for unit testing within the integration code, this was about the time I started looking at Cruisecontrol.Net and Nant.

Given that my new place was a .Net house, it made little sense to me to implement a suite of Java based tools when there were viable .Net versions available. With this said and done, the company branched out into the new realms of .Net CI! It was a success from the get go, the upper management liked the idea of seeing nice graphs and reports on build status, the developers like the idea of being told when a specific build passed testing - it allowed pretty much anyone in the developer and analyst pool to see what was happening with their build and others.

I feel that good CI is often based on some simple foundations. First off would have to be good code. It doesn't matter if it doesn't pass the build at a lower level, those guys should be used to new build to go through some teething problems with production code, this is something that CI would come to reduce further down the line. Next, we needed a decent environment in which to build our code. At the start, we used Crusiecontrol.Net on one builds server, but we had another three servers available to us should we need them. As unit testing was rolled out for all of development, this allowed everyone to see problems before a build made it out onto a test environment. Naturally, this annoyed some developers who didn't see the point in unit testing their code, being in the build development team I could see the importance of unit testing and so could my colleagues. The next important thing that makes up a good CI environment is Software Configuration Management. I have worked with some excellent SCM's and some truly awful (read Visual Source Safe). At this point the company was using a tool called Surround (made by Seapine). It was a fairly good SCM, a little clunky and had some performance issues, we needed to move away from it however as we needed to get to a point where we could run multiple projects from a common library of assemblies. Surround wasn't really up to this. Being a .Net house, we had access to all of the latest tools from MS, naturally Visual Studio was in there. At around about this point in time, MS was starting to promote a tool called Team Foundation Server, this was essentially MS's take on CI. It came with all all the bells and whistles that a good CI system should offer, there was built in SCM, built in reporting, built in data collection and the ability to carry out Team Builds (MS's take on a Cruisecontrol style too). It also came with a tool called MSBUILD, which I like very much. MSBUILD was the replacement for NMake, I am going to assume that most people reading this article will be familiar with Apache Ant and/or Nant. MSBUILD borrows a lot from these tools. Essentially, it is a XML based script that allowed you to automate a great deal of pre-build, build and post-build activities. At it's simplest, it allows you to carry out a build on the command line.

Now, when it came to the CI system that I built, I didn't want to use MSBUILD, at the time it seemed to constricted for me, instead I turned to what is probably MSBUILD's biggest rival, Nant. At the present time, both these tools are pretty much identical in what they can do and how they are built and maintained, back in the day though Nant offered slightly more flexibility over MSBUILD.

Now it came down to the system that I built. One of the strong points of Nant (and MSBUILD) is that you can write your own tasks for your build/test configuration. This is great when you need to add some small things into the config - this could be something as simple as plugging in a third party tool etc. Both of these tools offer a lot of functionality from the get go, all I had to write for the first CI system there was integration with our SCM, I needed to create some Nant tasks that would work with our SCM.

When we first went live with this system, there were some complaints. These mainly came from the developers working on new build, they would now be informed of breaks very early on in the build process automatically, Cruisecontrol.Net allowed you to do this. There was some resistance to unit testing also at this time, some developers simply did not want to see their code fail at such an early stage. Both the head of development and myself held the same view - unit tests are a must have as they find issues way in advance of a build making it out into test, where an analyst may or may not find the issue. Also, the build development team was also subject to unit tests, so across the entire company unit testing became a must be done (especially when managerial start looking at lovely reports that showed them test coverage and pass rates. A lot of the developers that complained were of the "just get it to run" frame of mind. Thankfully, these developers were vastly in the majority.

As time progressed, we found that Cruisecontrol.Net was beginning to be a little constrictive for us. We were only using it for build and test for the entire company (read multiple projects) on one build server. By this time, upper management was extremely impressed with what had been achieved with the setup, it has sped up the build and test time across the board. When companies are successful, they often start branching out to get more clients to use their systems, this was no different to the one I was currently working for. It was financial software, so the money to be made from efficient development was astronomic. As the shape of the overall development changed, so did our attitude to building it changed also. With the current setup, we couldn't handle these sorts of builds on a daily basis, at this point we were running CI for one workstream only and the system would not be able to cater for different builds for different projects. We were under using the build environment we had, we only used one machine out of the four available to us. We knew we needed to change our build tools and management was concerned that we would loose development time having to migrate from one system to another. We quickly put that fear down by explaining that the build scripts we were using were not dependant on the CI system, in other words, the Nant scripts we used for all of the builds would work with any CI tool. This brought a sigh of relief to many a manager a team lead. Right up until this point, we were running a complete build and test for the new build for just one customer automatically. This would take place all the way through the day at scheduled times, this meant that developers could work away to their hearts content pretty much 24/7 (and some did come close to this).

This made moving to a new CI system fairly painless, it also allowed us to see what we could do for the build, not just a test and build but hopefully a test, build, deployment and further testing. A complete lights off system that would roll back to a working build should the current code fail.

Tuesday, 3 November 2009

Building a data access layer

I recently agreed to help a student out with a software development assignment out for university. At first glance, it looks to be a pretty heavyweight peice of work for an undergrad to be given, even more so when the undergrad doesnt have a lot of experience in software development.

Essentially, the assignment is to create an Ebay-esque Windows Form app. The wording of the assignment makes it look harder than it actually is, when you get around the features that are being requested the actual technical requirement is really very simple. The assignment needs to be split into several peices, something that a lot of students probably wont be familiar with as it simply doesnt get taught at uni. The first section is the persistent data storage, I don't yet know what options are available at the moment, but the assignment spec asks for data to be retrieved from either a database or some XML. Now, I can understand why these features would be requested, it allows the application to be able to pick up its data from the local infrastructure or from a web service.

So, they key technical part here? Well, it's a data access layer. I have put together a few of these now, normally as part of an architectural migration to ease the switch over, all this assignment really needs is an appropriate DAL in order to get its data into the Windows app. It struck me that there really isnt a lot of info on how to do this rather simple thing, I think with the maturation of ORM like nHibernate and Linq-to-SQL a lot of people are over looking this for smaller projects. Anyway, I plan to put together a short howto on building your own DAL, because once you understand how the things work in the firs place, it makes implementing your own or using one of the ORM's already out there a lot more intuitive.

Sunday, 11 October 2009

Old School Computing, in its most truest sense.

Recently, I posted about the excellent machines that a rejuvinated Acorn Computers are now selling on the Internet. It got me thinking, anyone who went through an English school suring the late 80's and early 90's should have at least some distant memory of these rather fine machines.

For those that don't know, Acorn was one of the UK's only computer builders, a lot of people wont even remember them these days, but the ones who do will no doubt have fond memories of the BBC Master and probably playing Chuckie Egg on it.

Acorn made computers from 1978 to 1998, when they finally succumbed to the dominance of the PC architeture. There is loads about them on Wikipedia: http://en.wikipedia.org/wiki/Acorn_Computers, there are loads of these machines about now, mainly because of the sheer amount that were installed in schools across the country for so long. The first ever computer I owned was an Acorn Electron back in the early 80's. My parents thought that it might improve my hand/eye coordination to play games on it!! I think what many people wont realise is the lasting and prominent impact Acorn Computers have had on our everyday lives. Not only did they build their own computers, but they made their own OS, architecture and hardware. Acorn are resposible for developing the ARM processor, a peice of hardware that pretty much everyone carries around in their pocket, powering their mobile phone. It would be a fair bet to make that there is also one in your set top box at home, powering Sky, Virgin Media or your freeview/sat tuner. It doesnt start and end there either, ARM processors powered another venerable computing device towards the end of the 90's, the PSION Series 5. This is a very notable device as it was the precursor to the S60 mobile phone platform, the OS that powered it was called EPOC and it pretty much turned into Symbian - now things should start to be recogniseable!

With that said, I have now in front of me an original Acorn A7000+ desktop computer from a school down in England. It should run, I have powered it on and got the desktop up and displayed on my monitor. I plan to rennovate this UK computing relic to get it back up and running properly. I remember schools were using these machines right up until 2001 in some parts of the UK, this one I have here is a clear indication as it comes with an all important ethernet adapter.

One of the notable things about these machines is that they don't really need hard drives, the entire OS and all its supporting applications are stored on ROM. When you want to upgrade the OS, you simply replace the ROMs with your new set. This isn't something we see at all these days in a desktop computer, unless you are running a setup with embedded Linux.

I recall from the last time I saw an Acorn machine in operation, that it had a fully functional web browser. So, my plan right now is to get this bad boy up and running and connected to the Internet so that I can try and do some simple things like email and facebook. The example I have here, an A7000+, has a hard drive installed, as well as a CD writer, so this means I will be able to install software to it if I need to. From what I can see from the boot up screen, this machine is running RISC OS 3.7. From what I can remember, this was the last main release of the OS prior to Acorns demise, however since then there have been three seperate releases of newer OS's. In a rather different move to the normal distributions of Windows and Linux etc that we see now, a new company was formed in 1999 to release RISC OS 4. Not long after this, a seperate company release a RISC OS 5 for a seperate line of devices, but as far as I can tell this version of the OS is only for machines made by a company called Castle. This confuses me a bit, as far as I can tell this version of the OS has never really been available seperately.

But, a further twist is this, a new OS has been recently released, RISC OS 6... I have no idea how much of a departure this will be from the RISC OS I know and love. One thing that is very notable about RISC OS is that it pretty much introduced the concept of the task bar (as far as I know). This is very familiar to Apple users and now, to Windows 7 users, its a feature that Acorn introduced in 1988 though... As soon as I get this one up and running, I will start to get some screen shots up and running.