Simon Green's Developer Blog
Developing .NET in the cold white north ...

Bootstrap your build - good practice for CruiseControl.NET

Sunday, 16 December 2007 03:43 by Simon

If you are reading this (a developers blog) then chances are you are already in the 20% of developers who know there is more to development than 'developing' (code). So, you are probably using version control such as Subversion, SourceGear Vault or heck, even SourceSafe (for all people knock it, it is infinitely better than nothing at all!) and maybe have a continuous integration system such as CruiseControl.NET or Draco.NET setup to automatically do a build when the source-code changes.

What many people leave out though is making sure that the build scripts themselves and the whole source control configuration is itself protected by version control so that in an emergency a new build machine can be setup quickly and easily without having to develop everything from scratch or remember all the triggers, rules and targets that were in the original system.

Here is a technique that I've found useful and also helps to keep the build configuration for each project with the project that it belongs to (i.e. in it's repository) rather than off somewhere else. It also enables you to make all changes to your build system via the normal Subversion working-copy / commit process.

For this example, lets assume that we have 3 subversion repositories:

The build repository would contain the default CruiseControl.NET files including the ccnet.config file in the server folder which is where the configuration for each project normally resides. We're going to move pieces of that file into each separate project repository.

The CruiseControl.NET wiki describes how to Configure CruiseControl.Net to Automatically Update its Config File which is the bootstrap part of the process - this allows us to make changes to the build config file from another machine without having to get onto the build server and also ensures that all the configuration is stored in a repository.

My ccnt.config file is slightly different to the one shown in the wiki to enable the individual project configurations to be stored in their own repositories so each project has it's own folder:

<!DOCTYPE cruisecontrol [
  &lt;!ENTITY accelerator SYSTEM "file:accelerator\ccnet.config">
  <!ENTITY redirector SYSTEM "file:redirector\ccnet.config">
]>
<cruisecontrol>
  <project name="ccnet">
    <sourcecontrol type="svn">
      <trunkUrl>http://buildserver/svn/build/trunk/CruiseControl.NET/server/config</trunkUrl>
      <workingDirectory>C:\Program Files (x86)\CruiseControl.NET\server\config</workingDirectory>
      <executable>C:\Program Files (x86)\VisualSVN Server\bin\svn.exe</executable>
    </sourcecontrol>
    <triggers>
      <intervalTrigger name="ci" seconds="60" buildCondition="IfModificationExists" />
    </triggers>
  </project>
  &accelerator;
  &redirector;
</cruisecontrol>

 

Remember that the ccservice.exe.config and / or ccnet.exe.config file will need to be changed to tell CruiseControl.NET where to look for the ccnet.config file now that it isn't in the original place:

 <appSettings>
  <!-- Without this appSetting ccservice will look for ccnet.config in its own directory. -->
  <add key="ccnet.config" value=".\config\ccnet.config"/>
  <add key="service.name" value="CCService"/>
  <add key="remoting" value="on"/>
  <add key="ServerLogFilePath" value="ccnet.log"/>
  <!-- Used by the WebDashboard ServerLog plugin to locate the log file produced by the LogFileTraceListener (above) -->
  <add key="ServerLogFileLines" value="100"/>
  <!-- Used by the WebDashboard ServerLog plugin to determine how many lines from the log file should be read -->
  <add key="WatchConfigFile" value="true"/>
  <!-- Turns on or off the file watcher used to monitor the ccnet.config file -->
 </appSettings>

 

The next step is to supply the ccnet.config files for each project which will come from the repositories for those projects.

I've created a 'build' folder in each ...

... and inside that is the ccnet.config project snippet for that project:

  <project name="redirector">
    <workingDirectory>E:\ccnet\redirector</workingDirectory>
    <sourcecontrol type="svn">
      <trunkUrl>http://buildserver/svn/redirector/trunk</trunkUrl>
      <executable>C:\Program Files (x86)\VisualSVN Server\bin\svn.exe</executable>
    </sourcecontrol>
  </project>

 

(the real-life project contains a lot more - labellers, targets etc...)

The final piece to tie everything together is to make these project-specific files part of the config folder that the build ccnet.config file is in above and for this we use the svn:externals facility.

I used TortoiseSVN to add the property to a checked-out copy of the config folder:

NOTE: The screeshot above only shows the first entry. An additional entry would be made for the accelerator project.

After committing all these files the build server will automatically update it's own ccnet.config and get the separate snippet of the file for each project from the separate project repositories because of the svn:externals. The folder structure will then appear as below:

Everything is stored in source control which makes moving to a new build machine easier and the configuration for each project is stored in the same repository as the project itself.

Adding or removing projects from the continuous integration / build system is simply a case of editing the root ccnet.config file and setting the appropriate svn:externals property on the config folder which can be done on a working copy of the build repository. An additional advantage is that all changes can now be done via subversion itself without needing direct access to the build machine so normal repository security can be used.

Incidentally, in case you are wondering what happens if a non-working configuration is committed don't worry - CruiseControl.NET is smart enough to avoid using something that won't load and keeps the previous version in memory so all you need to do is correct it and commit a new working version and it will then pick that up and carry on.

Currently rated 4.7 by 3 people

  • Currently 4.666667/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Using VistaDB for unit testing (even if you target SQL Server)

Wednesday, 21 November 2007 16:06 by Simon

In case you aren't familiar with it, VistaDB is a lightweight, embedded database engine written entirely in managed code. It is a fantastic product that is coming on in leaps and bounds. Their aim is to be fully SQL compatible with SQL Server and are even adding support for TSQL.

I've used in on a few of my own projects and have been very impressed (I have a forum application with a 4Gb SQL Server database and approximately 2.5 million posts that I have just started converting to use it).

Even if you decide that you don't need it though it may still be worth looking at for testing purposes.

The client project I'm currently working on uses NHibernate for the persistence layer. Having developed an NHibernate Dialect and Driver for VistaDB (which I need to publish!) we've been able to use this when running unit tests. Instead of having to setup databases we can just create an in-memory database for use by the unit tests which is destroyed when they are finished. It is quick, convenient and avoids a lot of dependency issues PLUS it runs Linux / MONO (our app is cross platform).

 

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

Tags:   , ,
Categories:   .NET | Build
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

Generate build, solution and project files from Xml

Wednesday, 21 November 2007 15:35 by Simon

The current project that I am working on has experienced a few problems recently where developers have made changes to the build files (such as adding assembly references) but not the Visual Studio project files or vice versa which results in the rest of the team coping with a broken build or a solution that appears broken when first opened.

While the obvious and probably most common solution would be to use the Visual Studio definition of the projects as the master and create the build file from that (or use MSBuild directly) we do have some added complications that prevent this approach:

  • The application is cross-platform and has to run on Linux / MONO as well as Windows / .NET
  • There are C#, C++ and FORTRAN modules in the application (scientific computing)
  • Some developers work on Linux and do not want to use Visual Studio

I have evaluated and had some success with the Prebuild project which lets you define all the project settings and dependencies in an Xml file (which isn't too dissimilar to the Visual Studio format but is probably 'politically acceptable'). From this Xml file the tool generates the Visual Studio solution & project files plus the (NAnt) build files. It can also generate Mono Develop or Sharp Develop projects for use on Linux to keep everyone happy.

While it's a great tool there are a few things I don't like - I wish you could define your own templates for build files instead of being forced to use the one provided (although you can change the source to alter what it generates). Also, it doesn't directly support the build tool we are using (SCons) but I'm hoping we can switch to using NAnt at some point to fix some other issues.

It does address the main problem though - ensuring that there is only one 'truth' and a single way to add project references (in the Xml file). Adding it to the build files or the project files directly is pointless as they are both generated as part of the build process (which will then fail if the Xml definition is wrong).

Also, it's quite nice to just have source code checked into the repository ... no build files, no solution, no projects. Everything is created as needed which may also help as the team migrates to VS.NET 2008.

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5

NAnt task for Red-Gate SQL Compare

Sunday, 7 May 2006 16:47 by Simon

One of the aims of Continuous Integration is to always have a releasable version of a project ready. For the code itself this is no problem but if the project includes a database with schema changes then it can be a little more complicated, especially if you have several versions of an application that are supported. You need a way of creating database change scripts for each version of the database as part of the build process.

Red-Gate make an excellent product called SQL Compare which can compare two database versions and produce a script to convert one version to the other and a SQL Toolkit that allows you to automate the process. They had a link to a NAnt task on their site but I found this a little inflexible so I created the one below which gives more control over what is compared and also allows you to set various SQL Compare options.

The main feature are that you can compare the current development SQL Server database against other databases and / or database snapshots with script generated to convert to and from each version. You can also save a database snapshot which can be put back into source control (as can the change scripts).

Here is an example of the configuration:

<?xml version="1.0"?>
<project name="comparesql" default="comparesql" basedir=".">
 <loadtasks assembly="C:\Program Files\nantsql-0.85-rc3\bin\NAnt.SqlTasks.dll" />


 <!-- target .NET framework and corresponding compiler directives -->
 <property name="nant.settings.currentframework" value="net-1.1"/>


 <target name="comparesql">
  <sql-compare
   folder="c:\output"
   title="TRUNK"
   server="DEVSERVER"
   database="AppDb"
   uid="dbuser"
   pwd="dbpassword"
   snapshotFilename="trunk.snp"
   scriptFilename="trunk.sql"
   saveScript="true"
   saveSnapshot="true">
   <objects>
    <object type="Field"/>
    <object type="Function"/>
    <object type="Index"/>
    <object type="Rule"/>
    <object type="StoredProcedure"/>
    <object type="Table"/>
    <object type="Trigger"/>
    <object type="UserDefinedType"/>
    <object type="View"/>
   </objects>
   <options>
    <option type="IncludeDependencies"/>
    <option type="IgnoreFullTextIndexing"/>
    <option type="IgnoreUserProperties"/>
    <option type="IgnoreUsers"/>
    <option type="IgnoreOwners"/>
    <option type="IgnorePermissions"/>
    <option type="TargetIsPreYukon"/>
   </options>
   <databases>
    <database
     title="BRANCH100"
     server="DEVSERVER"
     database="AppDb"
     uid="dbuser"
     pwd="dbpassword"
     snapshotFilename="BRANCH100.snp"
     scriptFilename="BRANCH100.sql"
     saveSnapshot="true"
     saveScript="true"/>
    <database
     title="BRANCH115"
     server="DEVSERVER"
     database="AppDb"
     uid="dbuser"
     pwd="dbpassword"
     snapshotFilename="BRANCH115.snp"
     scriptFilename="BRANCH115.sql"
     saveSnapshot="true"
     saveScript="true"/>
   </databases>
  </sql-compare>  
 </target>
</project>

I will post the source code on the main InteSoft.NET site for download.

 

Currently rated 2.5 by 2 people

  • Currently 2.5/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Tags:   ,
Categories:   .NET | Build
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed

Build versioning with NAnt, Draco.NET and SourceGear Vault

Friday, 21 October 2005 16:33 by Simon

One aim of continuous integration should be to automate as much of the build process as possible. If something is left to human intervention then inevitably it will at some point be completed incorrectly or inconsistently or forgotten completely.

Part of a good build process is to update the version number of the application so that each build (triggered by changes in the source) will have a unique number and the source control system should label the source so that the code that created that build can be identified. This is my solution to updating the version number when using NAnt, Draco.Net and SourceGear Vault.

The build process consists of the following:

Draco.Net monitors the project folder in the SourceGear Vault repository.
A developer checks in some changes which causes Vault to update the folder version number (based on the changeset).
Draco.Net detects the changes and gets the latest version of the project to a temporary build folder
The NAnt .build file (part of the project) is started by Draco.Net
Within NAnt a custom function gets the folder version from Vault and sets the revision part of the version number based on it
The project folder in Vault is labeled with the build revision number
The assembly is compiled using the version number by overriding the AssemblyInfo.cs file (using the standard <adminfo> task)
The NUnit tests are run and reports produced, MSI files created etc.
Everything is pretty straightforward using the standard functions of NAnt, Draco.Net and the SourceGear Vault NAnt Tasks other than step #5 which required a custom NAnt function to return the folder tree version number. The code for this is below which I added to the Vault NAnt Tasks project (I'm not sure yet of the process for feeding this into the project as an update):

<?xml version="1.0"?>
<project name="NUnitAspInjector" default="build" basedir=".">
 <!-- set by Draco.Net for automated builds -->
 <property name="draco" value="false"/>
 <property name="build.id" value="0"/>
 <!-- project properties -->
 <property name="project.name" value="NUnitAspInjector"/>
 <!-- base project version for this trunk / branch -->
 <property name="build.version.major" value="1"/>
 <property name="build.version.minor" value="0"/>
 <property name="build.version.build" value="5000"/>
 <property name="build.version.revision" value="0"/>
 <!-- if automated build then set revision number based on version in vault -->
 <if test="${draco}">
  <!-- vault parameters (set in Draco.Net project properties) -->
  <!--  <property name="vault.url"  value="http://localhost"/>  <property name="vault.repository" value="default"/>  <property name="vault.path"  value="$/NUnitAspInjector/trunk"/>  <property name="vault.username"  value="build"/>  <property name="vault.password"  value="password"/>   -->
  <!-- set revision number based on tree version from vault -->
  <property name="build.version.revision" value="${vault::getversion(vault.url, vault.repository, vault.path, vault.username, vault.password)}"/>
  <!-- label the build -->
  <!-- in case a build is forced, this avoids a duplicate label error -->
  <vaultdeletelabel    url="${vault.url}"    username="${vault.username}"    password="${vault.password}"    repository="${vault.repository}"    path="${vault.path}"    labelstring="build ${build.version.revision}"/>
  <!-- apply the label to the source folder -->
  <vaultlabel    url="${vault.url}"    username="${vault.username}"    password="${vault.password}"    repository="${vault.repository}"    path="${vault.path}"    labelstring="build ${build.version.revision}"/>
 </if>
 <!-- set the full build version based on major.minor.build.revision -->
 <property name="build.version" value="${build.version.major}.${build.version.minor}.${build.version.build}.${build.version.revision}"/>
 <!-- output directory -->
 <property name="build.outdir" value="C:\working\buildoutput\${project.name}\${build.version}"/>
 <!-- target .NET framework and corresponding compiler directives -->
 <property name="nant.settings.currentframework" value="net-1.1"/>
 <property name="csc.define" value="NET11"/>
 <target name="build">
  <echo message="building: ${project.name} ${build.version}"/>
  <echo message="build id: ${build.id}" if="${draco}"/>
  <!-- build the solution in debugh mode so that the test are created -->
  <!-- this is a shortcut until a proper test project is created -->
  <solution solutionfile="src\NUnitAspInjector.sln" configuration="debug">
   <webmap>
    <map url="http://localhost/NUnitAspInjector.Sample/" path="src\NUnitAspInjector.Sample\"/>
   </webmap>
  </solution>
  <!-- make a folder for the assembly -->
  <mkdir dir="bin"/>
  <!-- if automated build then create assembly version info -->
  <if test="${draco}">
   <attrib file="src\NUnitAspInjector\AssemblyInfo.cs" readonly="false"/>
   <asminfo output="src\NUnitAspInjector\AssemblyInfo.cs" language="CSharp">
    <imports>
     <import name="System" />
     <import name="System.Reflection" />
     <import name="System.EnterpriseServices" />
     <import name="System.Runtime.InteropServices" />
    </imports>
    <attributes>
     <attribute type="ApplicationNameAttribute" value="${project.name}" />
     <attribute type="AssemblyProductAttribute" value="${project.name}" />
     <attribute type="AssemblyTitleAttribute" value="InteSoft ${project.name} ${build.version.major}.${build.version.minor} for .NET 1.1" />
     <attribute type="AssemblyDescriptionAttribute" value="${project.name} for ASP.NET" />
     <attribute type="AssemblyCompanyAttribute" value="InteSoft IT Ltd." />
     <attribute type="AssemblyCopyrightAttribute" value="Copyright &#x00A9; 2005 InteSoft IT Ltd." />
     <!-- the version stays at x.x.x.0 for binding -->
     <attribute type="AssemblyVersionAttribute" value="${build.version.major}.${build.version.minor}.${build.version.build}.0" />
     <!-- the information versions show the revision number -->
     <attribute type="AssemblyFileVersionAttribute" value="${build.version}" />
     <attribute type="AssemblyInformationalVersionAttribute" value="${build.version}" />
     <attribute type="AssemblyDelaySignAttribute" value="false" />
     <attribute type="AssemblyKeyFileAttribute" value="" />
     <attribute type="AssemblyKeyNameAttribute" value="" />
     <attribute type="CLSCompliantAttribute" value="true" />
     <attribute type="ComVisibleAttribute" value="false" />
    </attributes>
    <references>
     <include name="System.EnterpriseServices.dll" />
    </references>
   </asminfo>
  </if>
  <!-- now build the proper release version of our component -->
  <csc target="library" output="bin\NUnitAspInjector.dll" debug="false" define="${csc.define};" optimize="true" rebuild="true">
   <sources basedir="src\NUnitAspInjector">
    <include name="AssemblyInfo.cs"/>
    <include name="Gdi32.cs"/>
    <include name="Handler.cs"/>
    <include name="Html2Image.cs"/>
    <include name="InjectorModule.cs"/>
    <include name="InjectorWebFormTestCase.cs"/>
    <include name="RenderAssertion.cs"/>
    <include name="TestHostAttribute.cs"/>
    <include name="TestInjectorAttribute.cs"/>
    <include name="TestRenderAttribute.cs"/>
    <include name="TestServer.cs"/>
    <include name="cassini\ByteParser.cs"/>
    <include name="cassini\ByteString.cs"/>
    <include name="cassini\Connection.cs"/>
    <include name="cassini\Host.cs"/>
    <include name="cassini\Messages.cs"/>
    <include name="cassini\Request.cs"/>
    <include name="cassini\Server.cs"/>
   </sources>
   <resources prefix="NUnit.Extensions.Asp" basedir="src\NUnitAspInjector">
    <include name="Html2Image.resx"/>
   </resources>
   <references>
    <include asis="true" name="System.Web.dll"/>
    <include asis="true" name="System.Web.Services.dll"/>
    <include asis="true" name="System.Runtime.Remoting.dll"/>
    <include asis="true" name="System.XML.dll"/>
    <include asis="true" name="System.Windows.Forms.dll"/>
    <include asis="true" name="System.Drawing.dll"/>
    <include asis="true" name="lib\AForge.Imaging.dll"/>
    <include asis="true" name="lib\AForge.Math.dll"/>
    <include asis="true" name="lib\nmock.dll"/>
    <include asis="true" name="lib\nunit.framework.dll"/>
    <include asis="true" name="lib\NUnitAsp.dll"/>
    <include asis="true" name="lib\Interop.SHDocVw.dll"/>
    <include asis="true" name="lib\AxInterop.SHDocVw.dll"/>
    <include asis="true" name="lib\Microsoft.mshtml.dll"/>
   </references>
  </csc>
  <!-- copy this release version to the test app -->
  <copy file="bin\NUnitAspInjector.dll" todir="src\NUnitAspInjector.Sample\bin" overwrite="true"/>
  <!-- run the NUnit tests -->
  <nunit2 failonerror="true">
   <formatter type="Xml" usefile="true" extension=".xml" outputdir="${build.outdir}"/>
   <test assemblyname="src\NUnitAspInjector.Sample\bin\NUnitAspInjector.Sample.dll" appconfig="src\NUnitAspInjector.Sample\web.config">
    <categories>
     <exclude name="IIS"/>
    </categories>
   </test>
  </nunit2>
  <!-- create NUnit report -->
  <nunit2report out="NUnitReport.htm" todir="${build.outdir}" opendesc="true">
   <fileset>
    <include name="${build.outdir}\NUnitAspInjector.Sample.dll-results.xml"/>
   </fileset>
  </nunit2report>
  <!-- copy this release version to the output folder -->
  <copy file="bin\NUnitAspInjector.dll" todir="${build.outdir}" overwrite="true"/>
 </target>
</project>

The key line is this:

<!-- set the revision number based on a build startdate -->
<property name="build.startdate" value="01 Oct 2005"/>
<script language="C#">
 <imports>
  <import namespace="System.Globalization"/>
  <import namespace="System.Threading"/>
 </imports>
 <code>
 <![CDATA[
  public static void ScriptMain(Project project)
  {
   DateTime start = Convert.ToDateTime(project.Properties["build.startdate"]);   
   Calendar calendar = Thread.CurrentThread.CurrentCulture.Calendar;   
   int months = ((calendar.GetYear(DateTime.Today) - calendar.GetYear(start)) * 12) + calendar.GetMonth(DateTime.Today) - calendar.GetMonth(start);   
   int day = DateTime.Now.Day;   
   int revision = (months * 100) + day;   
   project.Properties["build.version.revision"] = revision.ToString();  
  }  
 ]]>
 </code>
</script>

However, I like the fact that the build number only changes when the source changes (due to checkins) rather than incrementing simply due to time. Also, with Vault, the version number of the folder ends up matching the build label applied to it which just seems neat. I like neat :)

That's it. Hope someone finds this of some use!

 

Be the first to rate this post

  • Currently 0/5 Stars.
  • 1
  • 2
  • 3
  • 4
  • 5
Tags:   , , ,
Categories:   .NET | Build
Actions:   E-mail | del.icio.us | Permalink | Comments (0) | Comment RSSRSS comment feed