1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"> 2<book id="BuildSystemManual" lang="en"> 3 4<bookinfo> 5<title>ASE BuildSystem Manual</title> 6<authorgroup> 7<author> 8<firstname>Matthew</firstname> 9<othername>G.</othername> 10<surname>Knepley</surname> 11</author> 12</authorgroup> 13<date>July, 2005</date> 14<releaseinfo>Release tag ???</releaseinfo> 15</bookinfo> 16 17<chapter id="Introduction"> 18<title>Introduction</title> 19 20<para>The BuildSystem from ASE is intended to be a Python replacement for the GNU autotools. It actually encompasses 21somewhat more, as it supports integrated version control and automatic code generation. However, the most useful 22comparisons will come from <command>autoconf</command>, <command>make</command>, and <command>libtool</command>. The 23system is not designed to be monolithic. Thus each component may be used independently, meaning logging, configuration, 24and build are all separate modules which do not require each other. This allows a user to incremenetally adopt the most 25useful portions of the package.</para> 26 27</chapter> 28 29<chapter id="Configure"> 30<title>Configure</title> 31 32<sect1 id="Configure-Design-Sketch"> 33<title>Configure Design Sketch</title> 34 35<para>The system is based upon an autonomous unit, objects of class <classname>config.base.Configure</classname>, which 36are responsible for discovering configuration information for a particular package or purpose. The only interface which 37must be supported is the <methodname>configure</methodname> method, as shown below. Support for lower-level operations 38such as compiling and linking will be discussed in section ???.</para> 39 40<classsynopsis language="python"> 41<ooclass> 42<classname>Configure</classname> 43</ooclass> 44<methodsynopsis> 45<void/><methodname>configure</methodname><methodparam><parameter>self</parameter></methodparam> 46</methodsynopsis> 47</classsynopsis> 48 49<para>This collection of configure objects is managed by a <classname>config.base.Framework</classname> object. As we 50will see in section ???, the framework manages all dependecies between modules and output of configure information. The 51framework is itself a subclass of <classname>config.base.Configure</classname> for which the 52<methodname>configure</methodname> method manages the entire configuration process. In order to associate a module with 53the given framework, it also provides the <methodname>require</methodname> method, discussed in section ???. Thus, the 54minimal framework interface is given by:</para> 55 56<classsynopsis language="python"> 57<ooclass> 58<classname>Framework</classname> 59</ooclass> 60<ooclass> 61<classname>config.base.Configure</classname> 62</ooclass> 63<methodsynopsis> 64<void/><methodname>require</methodname> 65 <methodparam><parameter>self</parameter></methodparam> 66 <methodparam><parameter>moduleName</parameter></methodparam> 67 <methodparam><parameter>depChild</parameter></methodparam> 68 <methodparam><parameter>keywordArgs</parameter><initializer>{}</initializer></methodparam> 69</methodsynopsis> 70<methodsynopsis> 71<void/><methodname>configure</methodname><methodparam><parameter>self</parameter></methodparam> 72</methodsynopsis> 73</classsynopsis> 74 75<para>This design allows user modules to be seamlessly integrated into the framework without changing the paradigm, or 76even any of the original code. Modules can be specified on the command line, or left in special directories. Although it 77is common to derive from <classname>config.base.Configure</classname>, the only necessity is that the user provide a 78<methodname>configure</methodname> method for the framework to execute.</para> 79 80<para>The framework does provide the traditional output mechanisms from <command>autoconf</command>, namely 81<methodname>#define</methodname> statements and file substitutions, to which we add make variables and 82rules. However, the preferred interaction mechanism is to use member variables directly from the configure objects. This 83is illustrated in section ???</para> 84 85</sect1> 86 87<sect1 id="Running-configure"> 88<title>Running configure</title> 89 90<para>The first step in running configure is to show the help: 91<screen> 92<prompt>bash$</prompt> <command>framework.py -help</command> 93<computeroutput> 94Python Configure Help 95 Comma separated lists should be given between [] (use \[ \] in tcsh/csh) 96 For example: --with-mpi-lib=\[/usr/local/lib/libmpich.a,/usr/local/lib/libpmpich.a\] 97---------------------------------------------------------------------------------------- 98Script: 99 --help : Print this help message current: 1 100 --h : Print this help message current: 0 101Framework: 102 --configModules : A list of Python modules with a Configure class current: [] 103 --ignoreCompileOutput : Ignore compiler output current: 1 104 --ignoreLinkOutput : Ignore linker output current: 1 105 --ignoreWarnings : Ignore compiler and linker warnings current: 0 106 --doCleanup : Delete any configure generated files (turn off for debugging) current: 1 107 --with-alternatives : Provide a choice among alternative package installations current: 0 108 --with-executables-search-path : A list of directories used to search for executables current: [] 109 --with-packages-search-path : A list of directories used to search for packages current: [] 110 --with-batch : Machine uses a batch system to submit jobs current: 0 111</computeroutput> 112</screen> 113The options shown will depend upon the modules loaded with <option>-configModules</option>. For instance, we will 114normally load the compiler module, which reveals the host of optios controlling preprocessors, compilers, and linkers. 115<screen> 116<prompt>bash$</prompt> <command>framework.py -configModules=[config.compilers] -help</command> 117<computeroutput> 118Python Configure Help 119 Comma separated lists should be given between [] (use \[ \] in tcsh/csh) 120 For example: --with-mpi-lib=\[/usr/local/lib/libmpich.a,/usr/local/lib/libpmpich.a\] 121---------------------------------------------------------------------------------------- 122Script: 123 --help : Print this help message current: 1 124 --h : Print this help message current: 0 125Framework: 126 --configModules : A list of Python modules with a Configure class current: [] 127 --ignoreCompileOutput : Ignore compiler output current: 1 128 --ignoreLinkOutput : Ignore linker output current: 1 129 --ignoreWarnings : Ignore compiler and linker warnings current: 0 130 --doCleanup : Delete any configure generated files (turn off for debugging) current: 1 131 --with-alternatives : Provide a choice among alternative package installations current: 0 132 --with-executables-search-path : A list of directories used to search for executables current: [] 133 --with-packages-search-path : A list of directories used to search for packages current: [] 134 --with-batch : Machine uses a batch system to submit jobs current: 0 135Compilers: 136 --with-cpp=<prog> : Specify the C preprocessor 137 --with-cc=<prog> : Specify the C compiler 138 --with-cxx=<prog> : Specify the C++ compiler 139 --with-fc=<prog> : Specify the Fortran compiler 140 --with-64-bit-pointers=<bool> : Use 64 bit compilers and libraries current: 0 141 --CPP=<prog> : Specify the C preprocessor 142 --CPPFLAGS=<string> : Specify the C preprocessor options current: 143 --CXXPP=<prog> : Specify the C++ preprocessor 144 --CC=<prog> : Specify the C compiler 145 --CFLAGS=<string> : Specify the C compiler options current: 146 --CXX=<prog> : Specify the C++ compiler 147 --CXXFLAGS=<string> : Specify the C++ compiler options current: 148 --CXX_CXXFLAGS=<string> : Specify the C++ compiler-only options current: 149 --FC=<prog> : Specify the Fortran compiler 150 --FFLAGS=<string> : Specify the Fortran compiler options current: 151 --LD=<prog> : Specify the default linker 152 --CC_LD=<prog> : Specify the linker for C only 153 --CXX_LD=<prog> : Specify the linker for C++ only 154 --FC_LD=<prog> : Specify the linker for Fortran only 155 --LDFLAGS=<string> : Specify the linker options current: 156 --with-ar : Specify the archiver 157 -AR : Specify the archiver flags 158 -AR_FLAGS : Specify the archiver flags 159 --with-ranlib : Specify ranlib 160 --with-shared-libraries : Enable shared libraries current: 1 161 --with-shared-ld=<prog> : Specify the shared linker 162 --with-f90-header=<file> : Specify the C header for the F90 interface, e.g. f90_intel.h 163 --with-f90-source=<file> : Specify the C source for the F90 interface, e.g. f90_intel.c 164</computeroutput> 165</screen> 166The syntax for list and dictionary option values is identical to Python syntax. However, in some shells (like 167<command>csh</command>), brackets must be escaped, and braces will usually have to be enclosed in quotes.</para> 168 169<para>The modules indicated with <option>-configModules</option> are located using <envar>PYTHONPATH</envar>. Since 170specifying environment variables can be inconvenient and error prone, it is common to provide a driver which alters 171<varname>sys.path</varname>, as is done for PETSc. In fact, the PETSc driver 172<itemizedlist> 173 <listitem><para>Verifies <envar>PETSC_ARCH</envar></para></listitem> 174 <listitem><para>Checks for invalid Cygwin versions</para></listitem> 175 <listitem><para>Checks for RedHat 9, which has a threads bug</para></listitem> 176 <listitem><para>Augments <envar>PYTHONPATH</envar></para></listitem> 177 <listitem><para>Adds the default PETSc configure module</para></listitem> 178 <listitem><para>Persists the configuration in <filename>RDict.db</filename></para></listitem> 179 <listitem><para>Handles exceptions</para></listitem> 180</itemizedlist> 181</para> 182 183</sect1> 184 185<sect1 id="Adding-a-module"> 186<title>Adding a module</title> 187 188<para>As we discussed in the introduction, all that is strictly necessary for a configure module, is to provide a class 189named <classname>Configure</classname> with a method <methodname>configure</methodname> taking no arguments. However, 190there are a variety of common operations, which will be illustrated in the sections below.</para> 191 192 <sect2 id="Using-other-modules"> 193 <title>Using other modules</title> 194 195 <para>We will often want to use the methods or results of other configure modules in order to perform checks in our 196own. The framework provides a mechanism for retrieving the object for any given configure module. As an example, 197consider checking for the <methodname>ddot</methodname> function in the BLAS library. The relevant Python code would 198be 199<programlisting> 200import config.base 201 202class Configure(config.base.Configure): 203 def __init__(self, framework): 204 config.base.Configure.__init__(self, framework) 205 self.compilers = self.framework.require('config.compilers', self) 206 self.libraries = self.framework.require('config.libraries', self) 207 return 208 209 def configure(self): 210 return self.libraries.check('libblas.a', 'ddot', otherLibs = self.compilers.flibs, 211 fortranMangle = 1) 212</programlisting> 213The <methodname>require</methodname> call will return the configure object from the given module, creating it if 214necessary. If the second argument is given, the framework will ensure that the returned configure object runs 215<emphasis>before</emphasis> the passed configure object. Notice that we can use the returned object either to call 216methods, like <methodname>check</methodname> from <classname>config.libraries</classname>, or use member variables, such 217as the list of Fortran compatibility libraries <methodname>flibs</methodname> from 218<classname>config.compilers</classname>. 219</para> 220 221<para>The underlying implementation in the framework uses a directed acyclic graph to indicate dependencies among 222modules. The vertices of this graph, configure objects, are topologically sorted and then executed. Moreover, child 223objects can be added to the framework without respecting the dependency structure, but this is discouraged.</para> 224 225 </sect2> 226 227 <sect2 id="Adding-a-test"> 228 <title>Adding a test</title> 229 230 <para>A user could of course perform all tests in the object's <methodname>configure</methodname> method, but the base 231class provides useful logging support for this purpose. Consider again the BLAS example, which will now become, 232<programlisting> 233 def checkDot(self): 234 '''Verify that the ddot() function is contained in the BLAS library''' 235 return self.libraries.check('libblas.a', 'ddot', otherLibs = self.compilers.flibs, 236 fortranMangle = 1) 237 238 def configure(self): 239 self.executeTest(self.checkDot) 240 return 241</programlisting> 242Passing our test module to the framework, 243<screen> 244<prompt>docs$</prompt> <command>PYTHONPATH=`pwd` ../config/framework.py --configModules=[examples.blasTest]</command> 245</screen> 246we produce the following log output in <filename>configure.log</filename>. Notice that it not only records the method and module, but the method doc string, 247all shell calls, and any output actions as well.</para> 248<screen> 249<computeroutput> 250================================================================================ 251TEST checkDot from examples.blasTest(/PETSc3/sidl/BuildSystem/docs/examples/blasTest.py:10) 252TESTING: checkDot from examples.blasTest(/PETSc3/sidl/BuildSystem/docs/examples/blasTest.py:10) 253 Verify that the ddot() function is contained in the BLAS library 254 Checking for functions ['ddot'] in library ['libblas.a'] ['-lfrtbegin', '-lg2c', '-lm', 255 '-L/usr/lib/gcc-lib/i486-linux/3.3.5', '-L/usr/lib/gcc-lib/i486-linux/3.3.5/../../..', 256 '-lm', '-lgcc_s'] 257sh: gcc -c -o conftest.o -fPIC conftest.c 258Executing: gcc -c -o conftest.o -fPIC conftest.c 259sh: 260sh: gcc -o conftest -fPIC conftest.o -lblas -lfrtbegin -lg2c -lm 261 -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s 262Executing: gcc -o conftest -fPIC conftest.o -lblas -lfrtbegin -lg2c -lm 263 -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s 264sh: 265Defined HAVE_LIBBLAS to 1 in config.libraries 266</computeroutput> 267</screen> 268 269 </sect2> 270 271 <sect2 id="Checking-for-headers"> 272 <title>Checking for headers</title> 273 274 <para>Often, we would like to test for the presence of certain headers. This is done is a completely analogous way to 275the library case, using instead the <classname>config.headers</classname> module. Below, we test for the presence of the 276<command>curses</command> header. 277<programlisting> 278import config.base 279 280class Configure(config.base.Configure): 281 def __init__(self, framework): 282 config.base.Configure.__init__(self, framework) 283 self.headers = self.framework.require('config.headers, self) 284 return 285 286 def checkCurses(self): 287 'Verify that we have the curses header' 288 return self.headers.check('curses.h') 289 290 def configure(self): 291 self.executeTest(self.checkCurses) 292 return 293</programlisting> 294Running this test 295<screen> 296<prompt>docs$</prompt> <command>PYTHONPATH=`pwd` ../config/framework.py --configModules=[examples.cursesTest]</command> 297</screen> 298produces the following log output.</para> 299<screen> 300<computeroutput> 301================================================================================ 302TEST checkCurses from examples.cursesTest(/PETSc3/sidl/BuildSystem/docs/examples/cursesTest.py:9) 303TESTING: checkCurses from examples.cursesTest(/PETSc3/sidl/BuildSystem/docs/examples/cursesTest.py:9) 304 Verify that we have the curses header 305Checking for header: curses.h 306sh: gcc -E conftest.c 307Executing: gcc -E conftest.c 308sh: # 1 "conftest.c" 309# 1 "<built-in>" 310# 1 "<command line>" 311# 1 "conftest.c" 312# 1 "confdefs.h" 1 313# 2 "conftest.c" 2 314# 1 "conffix.h" 1 315# 3 "conftest.c" 2 316# 1 "/usr/include/curses.h" 1 3 4 317# 58 "/usr/include/curses.h" 3 4 318# 1 "/usr/include/ncurses_dll.h" 1 3 4 319# 59 "/usr/include/curses.h" 2 3 4 320# 99 "/usr/include/curses.h" 3 4 321typedef unsigned long chtype; 322# 1 "/usr/include/stdio.h" 1 3 4 323# 28 "/usr/include/stdio.h" 3 4 324# 1 "/usr/include/features.h" 1 3 4 325# 295 "/usr/include/features.h" 3 4 326# 1 "/usr/include/sys/cdefs.h" 1 3 4 327# 296 "/usr/include/features.h" 2 3 4 328# 318 "/usr/include/features.h" 3 4 329#... 330... W* win,int* y, int* x, _Bool to_screen); 331extern _Bool mouse_trafo (int*, int*, _Bool); 332extern int mcprint (char *, int); 333extern int has_key (int); 334extern void _tracef (const char *, ...) ; 335extern void _tracedump (const char *, WINDOW *); 336extern char * _traceattr (attr_t); 337extern char * _traceattr2 (int, chtype); 338extern char * _nc_tracebits (void); 339extern char * _tracechar (int); 340extern char * _tracechtype (chtype); 341extern char * _tracechtype2 (int, chtype); 342# 1203 "/usr/include/curses.h" 3 4 343extern char * _tracemouse (const MEVENT *); 344extern void trace (const unsigned int); 345# 4 "conftest.c" 2 346 347Defined HAVE_CURSES_H to 1 in config.headers 348</computeroutput> 349</screen> 350 351<para>Alternatively, we could have specified that this header be included in the list of header files checked by default.</para> 352<programlisting> 353import config.base 354 355class Configure(config.base.Configure): 356 def __init__(self, framework): 357 config.base.Configure.__init__(self, framework) 358 self.headers = self.framework.require('config.headers, self) 359 self.headers.headers.append('curses.h') 360 return 361 362 def checkCurses(self): 363 'Verify that we have the curses header' 364 return self.headers.haveHeader('curses.h') 365 366 def configure(self): 367 self.executeTest(self.checkCurses) 368 return 369</programlisting> 370 371<para>In addition, the base class does include lower level support for preprocessing files. The 372<methodname>preprocess</methodname> method takes a code string as input and return a tuple of the 373<command>(stdout,stderr,error code)</command> for the run. The <methodname>outputPreprocess</methodname> method returns 374only the standard output, and <methodname>checkPreprocess</methodname> returns true if no error occurs.</para> 375 376 </sect2> 377 378 <sect2 id="Checking-for-libraries"> 379 <title>Checking for libraries</title> 380 381 <para>We have already demonstrated a test for the existence of a function in a library. However the 382<methodname>check</methodname> method is much more general. It allows the specification of multiple libraries and 383multiple functions, as well as auxiliary libraries. For instance, to check for the <methodname>MPI_Init</methodname> and 384<methodname>MPI_Comm_create</methodname> functions in MPICH when the Fortran bindings are active, we would use: 385<programlisting> 386 self.libraries.check(['libmpich.so', 'libpmpich.so'], ['MPI_Init', 'MPI_Comm_create'], 387 otherLibs = self.compilers.flibs) 388</programlisting> 389As in the BLAS example, we can also turn on Fortran name mangling. The caller may also supply a function prototype and 390calling sequence, which are necessary if the current language is C++. 391</para> 392 393<para>It is also necessary at some times to determine whether a given library is a shared object. This can be 394accomplished using the <methodname>checkShared</methodname> method, as we demonstrate with the MPICH library in a call 395taken from the MPI configure module in PETSc. 396<programlisting> 397 self.libraries.checkShared('#include <mpi.h>\n', 'MPI_Init', 'MPI_Initialized', 398 'MPI_Finalize', checkLink = self.checkMPILink, 399 libraries = self.lib) 400</programlisting> 401The theory for the check is that a shared object will have only one copy of any global variable. Thus functions such as 402<methodname>MPI_Initialized</methodname> will render consistent results across other libraries. The test begins by 403creating two dynamic libraries, both of which link the given library. Then an executable is constructed which loads the 404libraries in turn. The first library calls the initizlization functions, here <methodname>MPI_Init</methodname>, and the 405second library calls the initialization check function, here <methodname>MPI_Initialized</methodname>. The check 406function will return true if the given library is a shared object. This organization is shown in figure ???</para> 407<para> 408<inlinemediaobject> 409<imageobject><imagedata fileref="sharedLibraryCheck" format="EPS"/></imageobject> 410<imageobject><imagedata fileref="sharedLibraryCheck" format="JPG"/></imageobject> 411<!-- <textobject><phrase>A diagram of the link structure for the shared library test</phrase></textobject> --> 412</inlinemediaobject> 413</para> 414 415 <para>The lower level interface to compiling and linking in the base class mirrors that for preprocessing. The 416<methodname>outputCompile</methodname> and <methodname>checkCompile</methodname> methods function in the same way. The 417code is now broken up into four distinct sections. There are includes, the body of <methodname>main</methodname>, and a 418possible replacement for the beginning and end of the <methodname>main</methodname> declaration. The linking methods, 419<methodname>outputLink</methodname> and <methodname>checkLink</methodname>, are exactly analogous.</para> 420 421 <para>There are also some convenience methods provided to handle compiler and linker flags. The 422<methodname>checkCompilerFlag</methodname> and <methodname>checkLinkerFlag</methodname> try to determine whether a given 423flag is accepted by the processor, while <methodname>addCompilerFlag</methodname> and 424<methodname>addLinkerFlag</methodname> will do that check and add any valid flag to the list of default flags.</para> 425 426 </sect2> 427 428 <sect2 id="Checking-for-executables"> 429 <title>Checking for executables</title> 430 431 <para>The <methodname>getExecutable</methodname> method is used to locate executable files. For instance, this code 432would allow us to locate the <command>valgrind</command> binary. 433<programlisting> 434 self.getExecutable('valgrind') 435</programlisting> 436If the program is found, a member variable of the same name will be set in the object to the program name, and a make 437macro defined to it as well. We can opt for these to contain the full path by using the <option>getFullPath</option> 438argument. In addition, we can change the name of the member variable and macro using the <option>resultName</option> 439argument. 440</para> 441 442<para>We also have control over the search path used. If we give no arguments, the default path from the environment is 443used. This can be overridden with a new path using the <option>path</option> argument, either as a Python list or a 444colon separated string. Furthermore, the default path can be added to this custom path using the 445<option>useDefaultPath</option> argument. For instance, this call 446<programlisting> 447 self.getExecutable('valgrind', path=['/opt/valgrind-1.0'], getFullPath=1, 448 useDefaultPath=1, resultName='grinder') 449</programlisting> 450will check for <command>valgrind</command> first in <filename>/opt/valgrind-1.0</filename> and then along the default 451path. If found in the first location, it will set <varname>self.grinder</varname> to 452<filename>/opt/valgrind-1.0/valgrind</filename> as well as define <envar>GRINDER</envar> to the same value in makefiles. 453</para> 454 455 <para>As in the cases of preprocessing, compiling, and linking, the lower level operations are also exposed. The 456<methodname>checkRun</methodname> method takes in a code string and returns true if the executable runs without 457error. The <methodname>outputRun</methodname> method returns the output and status code. Both methods us the safe 458execution routine <methodname>config.base.Configure.executeShellCommand</methodname> which accepts a timeout. Moreover, 459there commands can run in the special batch mode described in section ???.</para> 460 461 </sect2> 462 463 <sect2 id="Output-results"> 464 <title>Output results</title> 465 466 <para>The BuildSystem configure includes the traditional output methods employed by <command>autoconf</command> to 467enable communication with <command>make</command>. Individual configure modules use the 468<methodname>addDefine</methodname> method to add C <methodname>#define</methodname> statements to a configuration header 469and the <methodname>addSubstitution</methodname> to setup substitution rules for specified files. For instance, to 470activate the parmetis package, we might provide 471<programlisting> 472 self.addDefine('HAVE_PARMETIS', 1) 473</programlisting> 474and then for the make process 475<programlisting> 476 self.addSubstitution('PARMETIS_INCLUDE', ' '.join([self.libraries.getIncludeArgument(i) 477 for i in self.include])) 478 self.addSubstitution('PARMETIS_LIB, ' '.join([self.libraries.getLibArgument(l) 479 for l in self.lib])) 480</programlisting> 481</para> 482 483<para>The actual output of this data is controlled by the framework. The user specifies the header file using the 484<varname>header</varname> field of the framework, and then the file is created automatically during the configure 485process, but can be output at any time using the <methodname>outputHeader</methodname> method. Furthermore, the 486<methodname>addSubstitutionFile</methodname> method can be used to tag a file for substitution, and also specify a 487different file for the result of the substitution.</para> 488 489<para>In the <command>autoconf</command> approach, separating the defines and substitutions for different packages 490becomes troublesome, and in some cases impossible to maintain. To help with this, we have introduced 491<emphasis>prefixes</emphasis> for the defines and substitutions. The are strings, unique to each module, which are 492prepended with an underscore to each identifier defined or substituted. These are set on a per object basis using the 493<varname>headerPrefix</varname> and <varname>substPrefix</varname> members. For instance, in our 494parmetis example, if we instead used the code 495<programlisting> 496 self.headerPrefix = 'MATT' 497 self.addDefine('HAVE_PARMETIS', 1) 498</programlisting> 499in our configuration header we would see 500<programlisting> 501 #ifndef MATT_HAVE_PARMETIS 502 #define MATT_HAVE_PARMETIS 1 503 #endif 504</programlisting> 505Note that the value of the prefix is used at output time, not at the time that the define or substitution is set. 506</para> 507 508<para>Another extension of the old-style output mechanisms adds more C structure to the interface. The 509<methodname>addTypedef</methodname> method allows a typedef from one typename to another, which in 510<command>autoconf</command> is handled by a define. Likewise <methodname>addPrototype</methodname> can add a missing 511function prototype to a header. Since these are C specific structures, they are output into a separate configuration 512header file, which is controlled by the <varname>cHeader</varname> member variable.</para> 513 514<para>Extending in a different direction, we allow makefile structures to be specified directly rather than through 515substitutions. Using <methodname>addMakeMacro</methodname>, we can add variable definitions to the configuration 516makefile, whereas <methodname>addMakeRule</methodname> allows the user to specify a make target, complete with 517dependencies and action. As an example, we will replace our parmetis example from above with the following code 518<programlisting> 519 self.addMakeMacro('PARMETIS_INCLUDE', ' '.join([self.libraries.getIncludeArgument(i) 520 for i in self.include])) 521 self.addMakeMacro('PARMETIS_LIB, ' '.join([self.libraries.getLibArgument(l) 522 for l in self.lib])) 523 self.addMakeRule('.c.o', '', ['${CC} -c -o $@ -I${PARMETIS_INCLUDE} $<']) 524 self.addMakeRule('myApp', '${.c=.o:SOURCE}', ['${CC} -o $@ $< ${PARMETIS_LIB}']) 525</programlisting> 526which will produce 527<programlisting> 528 PARMETIS_INCLUDE = -I/home/knepley/petsc-dev/externalpackages/parmetis/build/Darwin-x86_64/include 529 PARMETIS_LIB = -L/home/knepley/petsc-dev/externalpackages/parmetis/build/Darwin-x86_64/lib -lparmetis -lmetis 530</programlisting> 531in the file specified by the <varname>makeMacroHeader</varname> member variable, and 532<programlisting> 533 myApp: ${.c=.o:SOURCE} 534 ${CC} -i $@ $< ${PARMETIS_LIB} 535</programlisting> 536in the file specified by the <varname>makeRuleHeader</varname> member variable.</para> 537 538<para>The above output methods are all specified on a per configure object basis, however this may become confusing in a 539large project. All the prefixes and output filenames would have to be coordinated. A common strategy is to use the 540framework for coordination, putting all the output into the framework object itself. For instance, we might have 541<programlisting> 542 self.framework.addDefine('HAVE_PARMETIS', 1) 543</programlisting> 544which would allow the define to appear in the headre specified by the framework with the framework prefix. 545</para> 546 547 </sect2> 548 549</sect1> 550 551<sect1 id="Configuring-batch-systems"> 552<title>Configuring batch systems</title> 553 554<para>It is not uncommon for large clusters or supercomputing centers to have a batch execution policy, making it 555difficult for configure to execute the few tests that depend on executing code, rather than compiling and linking it. To 556handle this case, we provide the <option>--with-batch</option> argument. The code to be run is collected in a single 557executable which the user must submit to the system. This executable produces a <emphasis>reconfigure</emphasis> script 558which may then be run to fully configure the system.</para> 559 560<para>When configure is run with the <option>--with-batch</option> option, the following message will appear. 561<screen> 562<prompt>petsc-dev$</prompt> <command>./config/configure.py --with-batch</command> 563</screen> 564produces the following log output. 565<screen> 566<computeroutput> 567================================================================================= 568 Since your compute nodes require use of a batch system or mpirun you must: 569 1) Submit ./conftest to your batch system (this will generate the file reconfigure) 570 2) Run "python reconfigure" (to complete the configure process). 571================================================================================= 572</computeroutput> 573</screen> 574The user must then execute the <filename>conftest</filename> binary, and then run the <command>python 575reconfigure</command> command. 576</para> 577 578<para>If a user defined test relies upon running code, he may make it suitable for a batch system. The 579<methodname>checkRun</methodname> method takes the <option>defaultArg</option> argument which names a configure option 580whose value may substitute for the outcome of the test, allowing a user to preempt the run. For instance, the 581<methodname>config.types.checkEndian</methodname> method contains the code 582<programlisting> 583 if self.checkRun('', body, defaultArg = 'isLittleEndian'): 584</programlisting> 585which means the <option>isLittleEndian</option> option can be given to replace the output of the run. However, this does 586the require the user to supply the missing option.</para> 587 588<para>To automate this process, the test should first check for batch mode. Using the 589<methodname>addBatchInclude</methodname> and <methodname>addBatchBody</methodname> methods, code can be included in the 590batch executable. We return to the endian test to illustrate this usage. 591<programlisting> 592 if not self.framework.argDB['with-batch']: 593 body = ''' 594 /* Are we little or big endian? From Harbison & Steele. */ 595 union 596 { 597 long l; 598 char c[sizeof(long)]; 599 } u; 600 u.l = 1; 601 exit(u.c[sizeof(long) - 1] == 1); 602 ''' 603 if self.checkRun('', body, defaultArg = 'isLittleEndian'): 604 endian = 'little' 605 else: 606 endian = 'big' 607 else: 608 self.framework.addBatchBody( 609 ['{', 610 ' union {long l; char c[sizeof(long)];} u;', 611 ' u.l = 1;', 612 ' fprintf(output, " \'--with-endian=%s\',\\n",\ 613 (u.c[sizeof(long) - 1] == 1) ? "little" : "big");', 614 '}']) 615 # Dummy value 616 endian = 'little' 617</programlisting> 618The batch body code should output configure options to the <varname>output</varname> file descriptor. These are 619collected for the new configure run in the <filename>reconfigure</filename> script. 620</para> 621 622</sect1> 623 624</chapter> 625 626<chapter id="Build"> 627<title>Build</title> 628 629<para>The build operation now encompasses the configure, compile, link, install, and update operations.</para> 630 631<sect1 id="Running-make"> 632<title>Running make</title> 633 634<para>All options for both configuration and build are given to <filename>make.py</filename>. Thus, the simplest build 635is merely 636<screen> 637<prompt>petsc-dev$</prompt> <command>./make.py</command> 638</screen> 639The help is also given by <option>-help</option>, but this time it will also include build switches. 640<screen> 641<prompt>petsc-dev$</prompt> <command>./make.py -help</command> 642<computeroutput> 643Script Help 644----------- 645Script: 646 --help : Print this help message current: 1 647 --h : Print this help message current: 0 648Make: 649 -forceConfigure : Force a reconfiguration current: 0 650 -ignoreCompileOutput : Ignore compiler output current: 1 651 -defaultRoot : Directory root for all packages current: ../.. 652 -prefix : Root for installation of libraries and binaries 653SIDLMake: 654 -bootstrap : Generate the bootstrap client current: 0 655 -outputSIDLFiles : Write generated files to disk current: 1 656 -excludeLanguages=<languages> : Do not load configurations from RDict for the given languages current: [] 657 -excludeBasenames=<names> : Do not load configurations from RDict for these SIDL base names current: [] 658</computeroutput> 659</screen> 660</para> 661 662</sect1> 663 664<sect1 id="Makers"> 665<title>Makers</title> 666 667<para>The build operation now encompasses configure, compile, and link operations, which are coordinated by objects of 668class <classname>maker.Maker</classname>. This object manages: 669<itemizedlist> 670 <listitem><para>configuration,</para></listitem> 671 <listitem><para>build,</para></listitem> 672 <listitem><para>install, and</para></listitem> 673 <listitem><para>project dependencies</para></listitem> 674</itemizedlist> 675All options, no matter which component they are intended for, are given uniformly to <filename>make.py</filename>. 676</para> 677 678 <sect2 id="SIDLMaker"> 679 <title>SIDLMaker</title> 680 681<para>This is a subclass which handles source generation from SIDL.</para> 682 683 </sect2> 684 685</sect1> 686 687<sect1 id="Builders"> 688<title>Builders</title> 689 690<para>The build operation now encompasses the configure, compile, and link operations.</para> 691 692</sect1> 693 694<sect1 id="LanguageProcessors"> 695<title>LanguageProcessors</title> 696 697<para>The build operation now encompasses the configure, compile, and link operations.</para> 698 699</sect1> 700 701<sect1 id="Interaction-with-Configure"> 702<title>Interaction with Configure</title> 703 704<para>The pickled configure is loaded by Maker, and then the config.compile objects are jacked into the Builder.</para> 705 706</sect1> 707 708</chapter> 709 710</book> 711