<<O>>  Difference Topic NIghtlyBuidls (r1.3 - 06 Nov 2007 - JamesGallagher)

META TOPICPARENT WebHome

Nightly Builds

Added:
>
>
Update: The nightly builds are now run using a simple web-service system. The results are available at: http://test.opendap.org/cgi-bin/build_reader.pl?show=current&sort=yes. At the main Trac page there are notes about the nightly build access service and how it can be used to access older/different results.

N.B.: The nightly build system has been running for about two years, so there's quite a bit of data there.

-- JamesGallagher - 06 Nov 2007


Note: The nightly builds are no longer done on machines that also run web servers, so the results are only available using email.

The SVN project 'nbuilds' contains the scripts for the builds we now run.


 <<O>>  Difference Topic NIghtlyBuidls (r1.2 - 06 Nov 2005 - JamesGallagher)

META TOPICPARENT WebHome

Nightly Builds

Added:
>
>
Note: The nightly builds are no longer done on machines that also run web servers, so the results are only available using email.

The SVN project 'nbuilds' contains the scripts for the builds we now run.

-- JamesGallagher - 13 Jul 2005

Old nightly build information


Nightly builds are run for the current release and main development versions on most of the platforms we support. Here, most of those builds are accessible. The Name/Log link will take you to the log for the current night's build. The ten most recent logs are also kept in the old_builds sub directory. The Binaries link will take you to snapshot binaries from the most recent build. We may not have snapshot binaries for all of the platforms or versions listed here.

Current release (3.4)

Line: 33 to 41

Solaris/SPARC

IRIX/MIPS

HP/UX

Changed:
<
<
To get the 3.4 source to build on an HP/UX machine (B11.11 rev of the OS; uname -a --> HP-UX eclipse B.11.11 U 9000/785 2009737488 unlimited-user license), I had to removed all the pragma interface/implementation lines. I also had to include libnsl.a (for the XDR functions). I was then able to build hte code using gcc 3.2. I understand that gcc 3.4 will process the pragmas (although they seem to be of limited value with the newer compilers...)
>
>
To get the 3.4 source to build on an HP/UX machine (B11.11 rev of the OS; uname -a --> HP-UX eclipse B.11.11 U 9000/785 2009737488 unlimited-user license), I had to removed all the pragma interface/implementation lines. I also had to include libnsl.a (for the XDR functions). I was then able to build the code using gcc 3.2. I understand that gcc 3.4 will process the pragmas (although they seem to be of limited value with the newer compilers...)

Changed:
<
<
To get the ml-cmdln client to link, I: 1) added -fpic to CFLAGS and made sure all of the loaddods *.c files were compiled using that option, as well as those already set in the Makefile. 2) I grabed the mexopts.sh file and removed the option '-mt' from COMPFLAGS for the hpux case. By default mex uses a mexopts.sh file in the CWD over the default copy in MATLAB/bin. It may be I had to do this because the compiler was a little out of date (version B.11.11.02 while the ML docs talk about B11.11.06 as the requried version), so that step might not be neccessary with a newer compiler. Compiling with gcc and linking with the HP/UX cc (not the bundled compiler) produced a working loaddods. -- JamesGallagher - 13 Jul 2004
>
>
To get the ml-cmdln client to link, I: 1) added -fpic to CFLAGS and made sure all of the loaddods *.c files were compiled using that option, as well as those already set in the Makefile. 2) I grabed the mexopts.sh file and removed the option '-mt' from COMPFLAGS for the hpux case. By default mex uses a mexopts.sh file in the CWD over the default copy in MATLAB/bin. It may be I had to do this because the compiler was a little out of date (version B.11.11.02 while the ML docs talk about B11.11.06 as the required version), so that step might not be necessary with a newer compiler. Compiling with gcc and linking with the HP/UX cc (not the bundled compiler) produced a working loaddods. -- JamesGallagher - 13 Jul 2004


Line: 49 to 57

    1. gcc
    2. solaris
    3. MS
Changed:
<
<
  1. Nightly summaies of the suite of builds
>
>
  1. Nightly summaries of the suite of builds

  1. Web-accessible (build logs and summary)
Changed:
<
<

Other things the nighly builds do:

>
>

Other things the nightly builds do:


Changed:
<
<
  1. They can make snapshot binaries. When the platform's nataive compiler is not available, the binaries should be statically linked. To do this, set LDFLAGS='-static' in nightly_dods_build.conf. Remember to export LDFLAGS. This will make all the binaries static which means they will be quite large. -- JamesGallagher - 28 Apr 2004
>
>
  1. They can make snapshot binaries. When the platform's native compiler is not available, the binaries should be statically linked. To do this, set LDFLAGS='-static' in nightly_dods_build.conf. Remember to export LDFLAGS. This will make all the binaries static which means they will be quite large. -- JamesGallagher - 28 Apr 2004

What other 'nightly' tests should we do?


 <<O>>  Difference Topic NIghtlyBuidls (r1.1 - 14 Apr 2005 - Main.apache)
Line: 1 to 1
Added:
>
>
META TOPICPARENT WebHome

Nightly Builds

Nightly builds are run for the current release and main development versions on most of the platforms we support. Here, most of those builds are accessible. The Name/Log link will take you to the log for the current night's build. The ten most recent logs are also kept in the old_builds sub directory. The Binaries link will take you to snapshot binaries from the most recent build. We may not have snapshot binaries for all of the platforms or versions listed here.

Current release (3.4)

Name/log Binaries CPU Type OS
comet gcc 3.2.2; dynamic link ix86 Fedora 1 (linux 2.4.22-1.2115.nptl)
sunshine gcc 3.2; static link ix86 RH 8.0 ( 2.4.18-14smp #1 SMP )
bette gcc 3.0; static link ix86 RH 7.3 (2.4.25-smp #2 SMP)
dcz gcc 2.96 (native compiler); static link ix86 RH 7.3
laraine gcc 3.2; dynamic link SPARC solaris
flip gcc 3.3; dynamic link MIPS IRIX64 6.5
dolphin gcc 3.3; dynamic link alphaev56 OSF 5.1

Trunk

Name/Log Binaries CPU Type OS
comet gcc 3.3.2; dynamic link ix86 Fedora 1 (linux 2.4.22-1.2115.nptl)

Platform specific notes

Linux/ix86

Static linking seems to be the rule for Linux, except that Fedora Core 1 with the stock compiler (gcc 3.2.3) creates static binaries that seg fault whenever an exception is thrown (not just when an exception is not caught). Other builds don't have this problem. Thus the FC1 build is dynamically linked. -- JamesGallagher - 15 Jul 2004

OSF/Alpha

To get both the hdf and dsp servers to link I had to force the linker (which is the native ld; GNU ld is not supported) to use .a libraries instead of .so ones. There are conflicts between libraries that come with the OS and those that the HDF and DSP packages use (jpeg and rt, resp.). In addition, for the dsp server which has a mix of C, Fortran and C++, I had to add -lfor and -lFutil to get the server to link. I made all of these changes in the configure.in scripts but the tests are specific to the alpha-osf platform string that config.guess/sub generates.

I did not bother to build ncview since warnings about pointer/integer size conflicts are generated by code that we did not write. I was leery of spending too much time working on it right now. -- JamesGallagher - 30 Jun 2004

Solaris/SPARC

IRIX/MIPS

HP/UX

To get the 3.4 source to build on an HP/UX machine (B11.11 rev of the OS; uname -a --> HP-UX eclipse B.11.11 U 9000/785 2009737488 unlimited-user license), I had to removed all the pragma interface/implementation lines. I also had to include libnsl.a (for the XDR functions). I was then able to build hte code using gcc 3.2. I understand that gcc 3.4 will process the pragmas (although they seem to be of limited value with the newer compilers...)

To get the ml-cmdln client to link, I: 1) added -fpic to CFLAGS and made sure all of the loaddods *.c files were compiled using that option, as well as those already set in the Makefile. 2) I grabed the mexopts.sh file and removed the option '-mt' from COMPFLAGS for the hpux case. By default mex uses a mexopts.sh file in the CWD over the default copy in MATLAB/bin. It may be I had to do this because the compiler was a little out of date (version B.11.11.02 while the ML docs talk about B11.11.06 as the requried version), so that step might not be neccessary with a newer compiler. Compiling with gcc and linking with the HP/UX cc (not the bundled compiler) produced a working loaddods. -- JamesGallagher - 13 Jul 2004


Goals of the nightly build process:

  • Detect errors in code checked into cvs ASAP
  • Run automated tests

  1. Build on all supported platforms
  2. Build using a range of compilers
    1. gcc
    2. solaris
    3. MS
  3. Nightly summaies of the suite of builds
  4. Web-accessible (build logs and summary)

Other things the nighly builds do:

  1. They can make snapshot binaries. When the platform's nataive compiler is not available, the binaries should be statically linked. To do this, set LDFLAGS='-static' in nightly_dods_build.conf. Remember to export LDFLAGS. This will make all the binaries static which means they will be quite large. -- JamesGallagher - 28 Apr 2004

What other 'nightly' tests should we do?

  1. Dan's dataset list test
  2. Build the Java code

How to set up a nightly build of your very own:

  • Host must be 'network accessible.' This means either running httpd or cross-mounting a directory from a machine that runs httpd.
  • Host must be able to access a remote CVS repository.
  • Host must be able to run jobs unattended (unless you're very dedicated).

  1. Create a directory for the build (e.g., /usr/local/opendap_builds).
  2. Make a sub dir for the specific build (e.g., DODS-3.4); call this $bdir.
  3. In $bdir use cvs to checkout the code you want to build (e.g., cvs co -r release-3-4).
  4. In $bdir, create a symbolic link to DODS/etc/nightly_dods_build.sh.
  5. Copy DODS/etc/nightly_dods_build.conf.example to $bdir.
  6. Edit nightly_dods_build.conf.example and rename it to nightly_dods_build.conf.
    1. Set the various paths, including the one to the net-accessible directory
    2. Define SEPARATE_TARS = "Yes" if you want to build nightly binary snapshots
    3. Define UPDATE_CVS = "Yes" to enable automatic updates from CVS.
    4. If you want to do something special with the client-side caching, which can be important if you're running the tests, make a subdir in $bdir called dods_cache and create your own local dodsrc file. Reference this dodsrc file in the nightly_dods_build.conf file.
  7. Set up cron (or the equivalent) to run the script.
  8. Let someone know about the build so we can add it to the nightly build summary.

What the nightly build script does

  • When it runs, the nightly build script:
    1. checks out code from CVS
    2. runs configure
    3. compiles
    4. optionally runs automated tests
    5. optionally builds snapshot binaries
    6. moves a log of all this activity to a web-accessible directory
      1. maintains the 10 oldest logs in an 'old_builds' sub-directory

  • A separate script scans the web-accessible logs and builds a summary report.


-- JamesGallagher - 07 Apr 2004, Updated -- JamesGallagher - 06 Aug 2004