1<!DOCTYPE book PUBLIC "-//OASIS//DTD DocBook XML V4.2//EN"> 2<book id="BuildSystemManual" lang="en"> 3 4<bookinfo> 5<title>ASE BuildSystem Manual</title> 6<authorgroup> 7<author> 8<firstname>Matthew</firstname> 9<othername>G.</othername> 10<surname>Knepley</surname> 11</author> 12</authorgroup> 13<date>July, 2005</date> 14<releaseinfo>Release tag ???</releaseinfo> 15</bookinfo> 16 17<chapter id="Introduction"> 18<title>Introduction</title> 19 20<para>The BuildSystem from ASE is intended to be a Python replacement for the GNU autotools. It actually encompasses 21somewhat more, as it supports integrated version control and automatic code generation. However, the most useful 22comparisons will come from <command>autoconf</command>, <command>make</command>, and <command>libtool</command>. The 23system is not designed to be monolithic. Thus each component may be used independently, meaning logging, configuration, 24and build are all separate modules which do not require each other. This allows a user to incremenetally adopt the most 25useful portions of the package.</para> 26 27</chapter> 28 29<chapter id="Configure"> 30<title>Configure</title> 31 32<sect1 id="Configure-Design-Sketch"> 33<title>Configure Design Sketch</title> 34 35<para>The system is based upon an autonomous unit, objects of class <classname>config.base.Configure</classname>, which 36are responsible for discovering configuration information for a particular package or purpose. The only interface which 37must be supported is the <methodname>configure</methodname> method, as shown below. Support for lower-level operations 38such as compiling and linking will be discussed in section ???.</para> 39 40<classsynopsis language="python"> 41<ooclass> 42<classname>Configure</classname> 43</ooclass> 44<methodsynopsis> 45<void/><methodname>configure</methodname><methodparam><parameter>self</parameter></methodparam> 46</methodsynopsis> 47</classsynopsis> 48 49<para>This collection of configure objects is managed by a <classname>config.base.Framework</classname> object. As we 50will see in section ???, the framework manages all dependecies between modules and output of configure information. The 51framework is itself a subclass of <classname>config.base.Configure</classname> for which the 52<methodname>configure</methodname> method manages the entire configuration process. In order to associate a module with 53the given framework, it also provides the <methodname>require</methodname> method, discussed in section ???. Thus, the 54minimal framework interface is given by:</para> 55 56<classsynopsis language="python"> 57<ooclass> 58<classname>Framework</classname> 59</ooclass> 60<ooclass> 61<classname>config.base.Configure</classname> 62</ooclass> 63<methodsynopsis> 64<void/><methodname>require</methodname> 65 <methodparam><parameter>self</parameter></methodparam> 66 <methodparam><parameter>moduleName</parameter></methodparam> 67 <methodparam><parameter>depChild</parameter></methodparam> 68 <methodparam><parameter>keywordArgs</parameter><initializer>{}</initializer></methodparam> 69</methodsynopsis> 70<methodsynopsis> 71<void/><methodname>configure</methodname><methodparam><parameter>self</parameter></methodparam> 72</methodsynopsis> 73</classsynopsis> 74 75<para>This design allows user modules to be seamlessly integrated into the framework without changing the paradigm, or 76even any of the original code. Modules can be specified on the command line, or left in special directories. Although it 77is common to derive from <classname>config.base.Configure</classname>, the only necessity is that the user provide a 78<methodname>configure</methodname> method for the framework to execute.</para> 79 80<para>The framework does provide the traditional output mechanisms from <command>autoconf</command>, namely 81<methodname>#define</methodname> statements and file substitutions, to which we add make variables and 82rules. However, the preferred interaction mechanism is to use member variables directly from the configure objects. This 83is illustrated in section ???</para> 84 85</sect1> 86 87<sect1 id="Running-configure"> 88<title>Running configure</title> 89 90<para>The first step in running configure is to show the help: 91<screen> 92<prompt>bash$</prompt> <command>framework.py -help</command> 93<computeroutput> 94Python Configure Help 95 Comma separated lists should be given between [] (use \[ \] in tcsh/csh) 96 For example: --with-mpi-lib=\[/usr/local/lib/libmpich.a,/usr/local/lib/libpmpich.a\] 97---------------------------------------------------------------------------------------- 98Script: 99 --help : Print this help message current: 1 100 --h : Print this help message current: 0 101Framework: 102 --configModules : A list of Python modules with a Configure class current: [] 103 --ignoreCompileOutput : Ignore compiler output current: 1 104 --ignoreLinkOutput : Ignore linker output current: 1 105 --ignoreWarnings : Ignore compiler and linker warnings current: 0 106 --doCleanup : Delete any configure generated files (turn off for debugging) current: 1 107 --with-alternatives : Provide a choice among alternative package installations current: 0 108 --search-dirs : A list of directories used to search for executables current: [] 109 --package-dirs : A list of directories used to search for packages current: [] 110 --with-batch : Machine uses a batch system to submit jobs current: 0 111</computeroutput> 112</screen> 113The options shown will depend upon the modules loaded with <option>-configModules</option>. For instance, we will 114normally load the compiler module, which reveals the host of optios controlling preprocessors, compilers, and linkers. 115<screen> 116<prompt>bash$</prompt> <command>framework.py -configModules=[config.compilers] -help</command> 117<computeroutput> 118Python Configure Help 119 Comma separated lists should be given between [] (use \[ \] in tcsh/csh) 120 For example: --with-mpi-lib=\[/usr/local/lib/libmpich.a,/usr/local/lib/libpmpich.a\] 121---------------------------------------------------------------------------------------- 122Script: 123 --help : Print this help message current: 1 124 --h : Print this help message current: 0 125Framework: 126 --configModules : A list of Python modules with a Configure class current: [] 127 --ignoreCompileOutput : Ignore compiler output current: 1 128 --ignoreLinkOutput : Ignore linker output current: 1 129 --ignoreWarnings : Ignore compiler and linker warnings current: 0 130 --doCleanup : Delete any configure generated files (turn off for debugging) current: 1 131 --with-alternatives : Provide a choice among alternative package installations current: 0 132 --search-dirs : A list of directories used to search for executables current: [] 133 --package-dirs : A list of directories used to search for packages current: [] 134 --with-batch : Machine uses a batch system to submit jobs current: 0 135Compilers: 136 --with-cpp=<prog> : Specify the C preprocessor 137 --with-cc=<prog> : Specify the C compiler 138 --with-cxx=<prog> : Specify the C++ compiler 139 --with-fc=<prog> : Specify the Fortran compiler 140 --with-gnu-compilers=<bool> : Try to use GNU compilers current: 1 141 --with-vendor-compilers=<vendor> : Try to use vendor compilers (no argument all vendors, 0 no vendors) current: 142 --with-64-bit-pointers=<bool> : Use 64 bit compilers and libraries current: 0 143 --CPP=<prog> : Specify the C preprocessor 144 --CPPFLAGS=<string> : Specify the C preprocessor options current: 145 --CXXPP=<prog> : Specify the C++ preprocessor 146 --CC=<prog> : Specify the C compiler 147 --CFLAGS=<string> : Specify the C compiler options current: 148 --CXX=<prog> : Specify the C++ compiler 149 --CXXFLAGS=<string> : Specify the C++ compiler options current: 150 --CXX_CXXFLAGS=<string> : Specify the C++ compiler-only options current: 151 --FC=<prog> : Specify the Fortran compiler 152 --FFLAGS=<string> : Specify the Fortran compiler options current: 153 --LD=<prog> : Specify the default linker 154 --CC_LD=<prog> : Specify the linker for C only 155 --CXX_LD=<prog> : Specify the linker for C++ only 156 --FC_LD=<prog> : Specify the linker for Fortran only 157 --LDFLAGS=<string> : Specify the linker options current: 158 --with-ar : Specify the archiver 159 -AR : Specify the archiver flags 160 -AR_FLAGS : Specify the archiver flags 161 --with-ranlib : Specify ranlib 162 --with-shared-libraries : Enable shared libraries current: 1 163 --with-shared-ld=<prog> : Specify the shared linker 164 --with-f90-header=<file> : Specify the C header for the F90 interface, e.g. f90_intel.h 165 --with-f90-source=<file> : Specify the C source for the F90 interface, e.g. f90_intel.c 166</computeroutput> 167</screen> 168The syntax for list and dictionary option values is identical to Python syntax. However, in some shells (like 169<command>csh</command>), brackets must be escaped, and braces will usually have to be enclosed in quotes.</para> 170 171<para>The modules indicated with <option>-configModules</option> are located using <envar>PYTHONPATH</envar>. Since 172specifying environment variables can be inconvenient and error prone, it is common to provide a driver which alters 173<varname>sys.path</varname>, as is done for PETSc. In fact, the PETSc driver 174<itemizedlist> 175 <listitem><para>Verifies <envar>PETSC_ARCH</envar></para></listitem> 176 <listitem><para>Checks for invalid Cygwin versions</para></listitem> 177 <listitem><para>Checks for RedHat 9, which has a threads bug</para></listitem> 178 <listitem><para>Augments <envar>PYTHONPATH</envar></para></listitem> 179 <listitem><para>Adds the default PETSc configure module</para></listitem> 180 <listitem><para>Persists the configuration in <filename>RDict.db</filename></para></listitem> 181 <listitem><para>Handles exceptions</para></listitem> 182</itemizedlist> 183</para> 184 185</sect1> 186 187<sect1 id="Adding-a-module"> 188<title>Adding a module</title> 189 190<para>As we discussed in the introduction, all that is strictly necessary for a configure module, is to provide a class 191named <classname>Configure</classname> with a method <methodname>configure</methodname> taking no arguments. However, 192there are a variety of common operations, which will be illustrated in the sections below.</para> 193 194 <sect2 id="Using-other-modules"> 195 <title>Using other modules</title> 196 197 <para>We will often want to use the methods or results of other configure modules in order to perform checks in our 198own. The framework provides a mechanism for retrieving the object for any given configure module. As an example, 199consider checking for the <methodname>ddot</methodname> function in the BLAS library. The relevant Python code would 200be 201<programlisting> 202import config.base 203 204class Configure(config.base.Configure): 205 def __init__(self, framework): 206 config.base.Configure.__init__(self, framework) 207 self.compilers = self.framework.require('config.compilers', self) 208 self.libraries = self.framework.require('config.libraries', self) 209 return 210 211 def configure(self): 212 return self.libraries.check('libblas.a', 'ddot', otherLibs = self.compilers.flibs, 213 fortranMangle = 1) 214</programlisting> 215The <methodname>require</methodname> call will return the configure object from the given module, creating it if 216necessary. If the second argument is given, the framework will ensure that the returned configure object runs 217<emphasis>before</emphasis> the passed configure object. Notice that we can use the returned object either to call 218methods, like <methodname>check</methodname> from <classname>config.libraries</classname>, or use member variables, such 219as the list of Fortran compatibility libraries <methodname>flibs</methodname> from 220<classname>config.compilers</classname>. 221</para> 222 223<para>The underlying implementation in the framework uses a directed acyclic graph to indicate dependencies among 224modules. The vertices of this graph, configure objects, are topologically sorted and then executed. Moreover, child 225objects can be added to the framework without respecting the dependency structure, but this is discouraged.</para> 226 227 </sect2> 228 229 <sect2 id="Adding-a-test"> 230 <title>Adding a test</title> 231 232 <para>A user could of course perform all tests in the object's <methodname>configure</methodname> method, but the base 233class provides useful logging support for this purpose. Consider again the BLAS example, which will now become, 234<programlisting> 235 def checkDot(self): 236 '''Verify that the ddot() function is contained in the BLAS library''' 237 return self.libraries.check('libblas.a', 'ddot', otherLibs = self.compilers.flibs, 238 fortranMangle = 1) 239 240 def configure(self): 241 self.executeTest(self.checkDot) 242 return 243</programlisting> 244Passing our test module to the framework, 245<screen> 246<prompt>docs$</prompt> <command>PYTHONPATH=`pwd` ../config/framework.py --configModules=[examples.blasTest]</command> 247</screen> 248we produce the following log output in <filename>configure.log</filename>. Notice that it not only records the method and module, but the method doc string, 249all shell calls, and any output actions as well.</para> 250<screen> 251<computeroutput> 252================================================================================ 253TEST checkDot from examples.blasTest(/PETSc3/sidl/BuildSystem/docs/examples/blasTest.py:10) 254TESTING: checkDot from examples.blasTest(/PETSc3/sidl/BuildSystem/docs/examples/blasTest.py:10) 255 Verify that the ddot() function is contained in the BLAS library 256 Checking for functions ['ddot'] in library ['libblas.a'] ['-lfrtbegin', '-lg2c', '-lm', 257 '-L/usr/lib/gcc-lib/i486-linux/3.3.5', '-L/usr/lib/gcc-lib/i486-linux/3.3.5/../../..', 258 '-lm', '-lgcc_s'] 259sh: gcc -c -o conftest.o -fPIC conftest.c 260Executing: gcc -c -o conftest.o -fPIC conftest.c 261sh: 262sh: gcc -o conftest -fPIC conftest.o -lblas -lfrtbegin -lg2c -lm 263 -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s 264Executing: gcc -o conftest -fPIC conftest.o -lblas -lfrtbegin -lg2c -lm 265 -L/usr/lib/gcc-lib/i486-linux/3.3.5 -L/usr/lib/gcc-lib/i486-linux/3.3.5/../../.. -lm -lgcc_s 266sh: 267Defined HAVE_LIBBLAS to 1 in config.libraries 268</computeroutput> 269</screen> 270 271 </sect2> 272 273 <sect2 id="Checking-for-headers"> 274 <title>Checking for headers</title> 275 276 <para>Often, we would like to test for the presence of certain headers. This is done is a completely analogous way to 277the library case, using instead the <classname>config.headers</classname> module. Below, we test for the presence of the 278<command>curses</command> header. 279<programlisting> 280import config.base 281 282class Configure(config.base.Configure): 283 def __init__(self, framework): 284 config.base.Configure.__init__(self, framework) 285 self.headers = self.framework.require('config.headers, self) 286 return 287 288 def checkCurses(self): 289 'Verify that we have the curses header' 290 return self.headers.check('curses.h') 291 292 def configure(self): 293 self.executeTest(self.checkCurses) 294 return 295</programlisting> 296Running this test 297<screen> 298<prompt>docs$</prompt> <command>PYTHONPATH=`pwd` ../config/framework.py --configModules=[examples.cursesTest]</command> 299</screen> 300produces the following log output.</para> 301<screen> 302<computeroutput> 303================================================================================ 304TEST checkCurses from examples.cursesTest(/PETSc3/sidl/BuildSystem/docs/examples/cursesTest.py:9) 305TESTING: checkCurses from examples.cursesTest(/PETSc3/sidl/BuildSystem/docs/examples/cursesTest.py:9) 306 Verify that we have the curses header 307Checking for header: curses.h 308sh: gcc -E conftest.c 309Executing: gcc -E conftest.c 310sh: # 1 "conftest.c" 311# 1 "<built-in>" 312# 1 "<command line>" 313# 1 "conftest.c" 314# 1 "confdefs.h" 1 315# 2 "conftest.c" 2 316# 1 "conffix.h" 1 317# 3 "conftest.c" 2 318# 1 "/usr/include/curses.h" 1 3 4 319# 58 "/usr/include/curses.h" 3 4 320# 1 "/usr/include/ncurses_dll.h" 1 3 4 321# 59 "/usr/include/curses.h" 2 3 4 322# 99 "/usr/include/curses.h" 3 4 323typedef unsigned long chtype; 324# 1 "/usr/include/stdio.h" 1 3 4 325# 28 "/usr/include/stdio.h" 3 4 326# 1 "/usr/include/features.h" 1 3 4 327# 295 "/usr/include/features.h" 3 4 328# 1 "/usr/include/sys/cdefs.h" 1 3 4 329# 296 "/usr/include/features.h" 2 3 4 330# 318 "/usr/include/features.h" 3 4 331#... 332... W* win,int* y, int* x, _Bool to_screen); 333extern _Bool mouse_trafo (int*, int*, _Bool); 334extern int mcprint (char *, int); 335extern int has_key (int); 336extern void _tracef (const char *, ...) ; 337extern void _tracedump (const char *, WINDOW *); 338extern char * _traceattr (attr_t); 339extern char * _traceattr2 (int, chtype); 340extern char * _nc_tracebits (void); 341extern char * _tracechar (int); 342extern char * _tracechtype (chtype); 343extern char * _tracechtype2 (int, chtype); 344# 1203 "/usr/include/curses.h" 3 4 345extern char * _tracemouse (const MEVENT *); 346extern void trace (const unsigned int); 347# 4 "conftest.c" 2 348 349Defined HAVE_CURSES_H to 1 in config.headers 350</computeroutput> 351</screen> 352 353<para>Alternatively, we could have specified that this header be included in the list of header files checked by default.</para> 354<programlisting> 355import config.base 356 357class Configure(config.base.Configure): 358 def __init__(self, framework): 359 config.base.Configure.__init__(self, framework) 360 self.headers = self.framework.require('config.headers, self) 361 self.headers.headers.append('curses.h') 362 return 363 364 def checkCurses(self): 365 'Verify that we have the curses header' 366 return self.headers.haveHeader('curses.h') 367 368 def configure(self): 369 self.executeTest(self.checkCurses) 370 return 371</programlisting> 372 373<para>In addition, the base class does include lower level support for preprocessing files. The 374<methodname>preprocess</methodname> method takes a code string as input and return a tuple of the 375<command>(stdout,stderr,error code)</command> for the run. The <methodname>outputPreprocess</methodname> method returns 376only the standard output, and <methodname>checkPreprocess</methodname> returns true if no error occurs.</para> 377 378 </sect2> 379 380 <sect2 id="Checking-for-libraries"> 381 <title>Checking for libraries</title> 382 383 <para>We have already demonstrated a test for the existence of a function in a library. However the 384<methodname>check</methodname> method is much more general. It allows the specification of multiple libraries and 385multiple functions, as well as auxiliary libraries. For instance, to check for the <methodname>MPI_Init</methodname> and 386<methodname>MPI_Comm_create</methodname> functions in MPICH when the Fortran bindings are active, we would use: 387<programlisting> 388 self.libraries.check(['libmpich.so', 'libpmpich.so'], ['MPI_Init', 'MPI_Comm_create'], 389 otherLibs = self.compilers.flibs) 390</programlisting> 391As in the BLAS example, we can also turn on Fortran name mangling. The caller may also supply a function prototype and 392calling sequence, which are necessary if the current language is C++. 393</para> 394 395<para>It is also necessary at some times to determine whether a given library is a shared object. This can be 396accomplished using the <methodname>checkShared</methodname> method, as we demonstrate with the MPICH library in a call 397taken from the MPI configure module in PETSc. 398<programlisting> 399 self.libraries.checkShared('#include <mpi.h>\n', 'MPI_Init', 'MPI_Initialized', 400 'MPI_Finalize', checkLink = self.checkMPILink, 401 libraries = self.lib) 402</programlisting> 403The theory for the check is that a shared object will have only one copy of any global variable. Thus functions such as 404<methodname>MPI_Initialized</methodname> will render consistent results across other libraries. The test begins by 405creating two dynamic libraries, both of which link the given library. Then an executable is constructed which loads the 406libraries in turn. The first library calls the initizlization functions, here <methodname>MPI_Init</methodname>, and the 407second library calls the initialization check function, here <methodname>MPI_Initialized</methodname>. The check 408function will return true if the given library is a shared object. This organization is shown in figure ???</para> 409<para> 410<inlinemediaobject> 411<imageobject><imagedata fileref="sharedLibraryCheck" format="EPS"/></imageobject> 412<imageobject><imagedata fileref="sharedLibraryCheck" format="JPG"/></imageobject> 413<!-- <textobject><phrase>A diagram of the link structure for the shared library test</phrase></textobject> --> 414</inlinemediaobject> 415</para> 416 417 <para>The lower level interface to compiling and linking in the base class mirrors that for preprocessing. The 418<methodname>outputCompile</methodname> and <methodname>checkCompile</methodname> methods function in the same way. The 419code is now broken up into four distinct sections. There are includes, the body of <methodname>main</methodname>, and a 420possible replacement for the beginning and end of the <methodname>main</methodname> declaration. The linking methods, 421<methodname>outputLink</methodname> and <methodname>checkLink</methodname>, are exactly analogous.</para> 422 423 <para>There are also some convenience methods provided to handle compiler and linker flags. The 424<methodname>checkCompilerFlag</methodname> and <methodname>checkLinkerFlag</methodname> try to determine whether a given 425flag is accepted by the processor, while <methodname>addCompilerFlag</methodname> and 426<methodname>addLinkerFlag</methodname> will do that check and add any valid flag to the list of default flags.</para> 427 428 </sect2> 429 430 <sect2 id="Checking-for-executables"> 431 <title>Checking for executables</title> 432 433 <para>The <methodname>getExecutable</methodname> method is used to locate executable files. For instance, this code 434would allow us to locate the <command>valgrind</command> binary. 435<programlisting> 436 self.getExecutable('valgrind') 437</programlisting> 438If the program is found, a member variable of the same name will be set in the object to the program name, and a make 439macro defined to it as well. We can opt for these to contain the full path by using the <option>getFullPath</option> 440argument. In addition, we can change the name of the member variable and macro using the <option>resultName</option> 441argument. 442</para> 443 444<para>We also have control over the search path used. If we give no arguments, the default path from the environment is 445used. This can be overridden with a new path using the <option>path</option> argument, either as a Python list or a 446colon separated string. Furthermore, the default path can be added to this custom path using the 447<option>useDefaultPath</option> argument. For instance, this call 448<programlisting> 449 self.getExecutable('valgrind', path=['/opt/valgrind-1.0'], getFullPath=1, 450 useDefaultPath=1, resultName='grinder') 451</programlisting> 452will check for <command>valgrind</command> first in <filename>/opt/valgrind-1.0</filename> and then along the default 453path. If found in the first location, it will set <varname>self.grinder</varname> to 454<filename>/opt/valgrind-1.0/valgrind</filename> as well as define <envar>GRINDER</envar> to the same value in makefiles. 455</para> 456 457 <para>As in the cases of preprocessing, compiling, and linking, the lower level operations are also exposed. The 458<methodname>checkRun</methodname> method takes in a code string and returns true if the executable runs without 459error. The <methodname>outputRun</methodname> method returns the output and status code. Both methods us the safe 460execution routine <methodname>config.base.Configure.executeShellCommand</methodname> which accepts a timeout. Moreover, 461there commands can run in the special batch mode described in section ???.</para> 462 463 </sect2> 464 465 <sect2 id="Output-results"> 466 <title>Output results</title> 467 468 <para>The BuildSystem configure includes the traditional output methods employed by <command>autoconf</command> to 469enable communication with <command>make</command>. Individual configure modules use the 470<methodname>addDefine</methodname> method to add C <methodname>#define</methodname> statements to a configuration header 471and the <methodname>addSubstitution</methodname> to setup substitution rules for specified files. For instance, to 472activate the parmetis package, we might provide 473<programlisting> 474 self.addDefine('HAVE_PARMETIS', 1) 475</programlisting> 476and then for the make process 477<programlisting> 478 self.addSubstitution('PARMETIS_INCLUDE', ' '.join([self.libraries.getIncludeArgument(i) 479 for i in self.include])) 480 self.addSubstitution('PARMETIS_LIB, ' '.join([self.libraries.getLibArgument(l) 481 for l in self.lib])) 482</programlisting> 483</para> 484 485<para>The actual output of this data is controlled by the framework. The user specifies the header file using the 486<varname>header</varname> field of the framework, and then the file is created automatically during the configure 487process, but can be output at any time using the <methodname>outputHeader</methodname> method. Furthermore, the 488<methodname>addSubstitutionFile</methodname> method can be used to tag a file for substitution, and also specify a 489different file for the result of the substitution.</para> 490 491<para>In the <command>autoconf</command> approach, separating the defines and substitutions for different packages 492becomes troublesome, and in some cases impossible to maintain. To help with this, we have introduced 493<emphasis>prefixes</emphasis> for the defines and substitutions. The are strings, unique to each module, which are 494prepended with an underscore to each identifier defined or substituted. These are set on a per object basis using the 495<varname>headerPrefix</varname> and <varname>substPrefix</varname> members. For instance, in our 496parmetis example, if we instead used the code 497<programlisting> 498 self.headerPrefix = 'MATT' 499 self.addDefine('HAVE_PARMETIS', 1) 500</programlisting> 501in our configuration header we would see 502<programlisting> 503 #ifndef MATT_HAVE_PARMETIS 504 #define MATT_HAVE_PARMETIS 1 505 #endif 506</programlisting> 507Note that the value of the prefix is used at output time, not at the time that the define or substitution is set. 508</para> 509 510<para>Another extension of the old-style output mechanisms adds more C structure to the interface. The 511<methodname>addTypedef</methodname> method allows a typedef from one typename to another, which in 512<command>autoconf</command> is handled by a define. Likewise <methodname>addPrototype</methodname> can add a missing 513function prototype to a header. Since these are C specific structures, they are output into a separate configuration 514header file, which is controlled by the <varname>cHeader</varname> member variable.</para> 515 516<para>Extending in a different direction, we allow makefile structures to be specified directly rather than through 517substitutions. Using <methodname>addMakeMacro</methodname>, we can add variable definitions to the configuration 518makefile, whereas <methodname>addMakeRule</methodname> allows the user to specify a make target, complete with 519dependencies and action. As an example, we will replace our parmetis example from above with the following code 520<programlisting> 521 self.addMakeMacro('PARMETIS_INCLUDE', ' '.join([self.libraries.getIncludeArgument(i) 522 for i in self.include])) 523 self.addMakeMacro('PARMETIS_LIB, ' '.join([self.libraries.getLibArgument(l) 524 for l in self.lib])) 525 self.addMakeRule('.c.o', '', ['${CC} -c -o $@ -I${PARMETIS_INCLUDE} $<']) 526 self.addMakeRule('myApp', '${.c=.o:SOURCE}', ['${CC} -o $@ $< ${PARMETIS_LIB}']) 527</programlisting> 528which will produce 529<programlisting> 530 PARMETIS_INCLUDE = -I/home/knepley/petsc-dev/externalpackages/parmetis/build/Darwin-x86_64/include 531 PARMETIS_LIB = -L/home/knepley/petsc-dev/externalpackages/parmetis/build/Darwin-x86_64/lib -lparmetis -lmetis 532</programlisting> 533in the file specified by the <varname>makeMacroHeader</varname> member variable, and 534<programlisting> 535 myApp: ${.c=.o:SOURCE} 536 ${CC} -i $@ $< ${PARMETIS_LIB} 537</programlisting> 538in the file specified by the <varname>makeRuleHeader</varname> member variable.</para> 539 540<para>The above output methods are all specified on a per configure object basis, however this may become confusing in a 541large project. All the prefixes and output filenames would have to be coordinated. A common strategy is to use the 542framework for coordination, putting all the output into the framework object itself. For instance, we might have 543<programlisting> 544 self.framework.addDefine('HAVE_PARMETIS', 1) 545</programlisting> 546which would allow the define to appear in the headre specified by the framework with the framework prefix. 547</para> 548 549 </sect2> 550 551</sect1> 552 553<sect1 id="Configuring-batch-systems"> 554<title>Configuring batch systems</title> 555 556<para>It is not uncommon for large clusters or supercomputing centers to have a batch execution policy, making it 557difficult for configure to execute the few tests that depend on executing code, rather than compiling and linking it. To 558handle this case, we provide the <option>--with-batch</option> argument. The code to be run is collected in a single 559executable which the user must submit to the system. This executable produces a <emphasis>reconfigure</emphasis> script 560which may then be run to fully configure the system.</para> 561 562<para>When configure is run with the <option>--with-batch</option> option, the following message will appear. 563<screen> 564<prompt>petsc-dev$</prompt> <command>./config/configure.py --with-batch</command> 565</screen> 566produces the following log output. 567<screen> 568<computeroutput> 569================================================================================= 570 Since your compute nodes require use of a batch system or mpirun you must: 571 1) Submit ./conftest to your batch system (this will generate the file reconfigure) 572 2) Run "python reconfigure" (to complete the configure process). 573================================================================================= 574</computeroutput> 575</screen> 576The user must then execute the <filename>conftest</filename> binary, and then run the <command>python 577reconfigure</command> command. 578</para> 579 580<para>If a user defined test relies upon running code, he may make it suitable for a batch system. The 581<methodname>checkRun</methodname> method takes the <option>defaultArg</option> argument which names a configure option 582whose value may substitute for the outcome of the test, allowing a user to preempt the run. For instance, the 583<methodname>config.types.checkEndian</methodname> method contains the code 584<programlisting> 585 if self.checkRun('', body, defaultArg = 'isLittleEndian'): 586</programlisting> 587which means the <option>isLittleEndian</option> option can be given to replace the output of the run. However, this does 588the require the user to supply the missing option.</para> 589 590<para>To automate this process, the test should first check for batch mode. Using the 591<methodname>addBatchInclude</methodname> and <methodname>addBatchBody</methodname> methods, code can be included in the 592batch executable. We return to the endian test to illustrate this usage. 593<programlisting> 594 if not self.framework.argDB['with-batch']: 595 body = ''' 596 /* Are we little or big endian? From Harbison & Steele. */ 597 union 598 { 599 long l; 600 char c[sizeof(long)]; 601 } u; 602 u.l = 1; 603 exit(u.c[sizeof(long) - 1] == 1); 604 ''' 605 if self.checkRun('', body, defaultArg = 'isLittleEndian'): 606 endian = 'little' 607 else: 608 endian = 'big' 609 else: 610 self.framework.addBatchBody( 611 ['{', 612 ' union {long l; char c[sizeof(long)];} u;', 613 ' u.l = 1;', 614 ' fprintf(output, " \'--with-endian=%s\',\\n",\ 615 (u.c[sizeof(long) - 1] == 1) ? "little" : "big");', 616 '}']) 617 # Dummy value 618 endian = 'little' 619</programlisting> 620The batch body code should output configure options to the <varname>output</varname> file descriptor. These are 621collected for the new configure run in the <filename>reconfigure</filename> script. 622</para> 623 624</sect1> 625 626</chapter> 627 628<chapter id="Build"> 629<title>Build</title> 630 631<para>The build operation now encompasses the configure, compile, link, install, and update operations.</para> 632 633<sect1 id="Running-make"> 634<title>Running make</title> 635 636<para>All options for both configuration and build are given to <filename>make.py</filename>. Thus, the simplest build 637is merely 638<screen> 639<prompt>petsc-dev$</prompt> <command>./make.py</command> 640</screen> 641The help is also given by <option>-help</option>, but this time it will also include build switches. 642<screen> 643<prompt>petsc-dev$</prompt> <command>./make.py -help</command> 644<computeroutput> 645Script Help 646----------- 647Script: 648 --help : Print this help message current: 1 649 --h : Print this help message current: 0 650Make: 651 -forceConfigure : Force a reconfiguration current: 0 652 -ignoreCompileOutput : Ignore compiler output current: 1 653 -defaultRoot : Directory root for all packages current: ../.. 654 -prefix : Root for installation of libraries and binaries 655SIDLMake: 656 -bootstrap : Generate the boostrap client current: 0 657 -outputSIDLFiles : Write generated files to disk current: 1 658 -excludeLanguages=<languages> : Do not load configurations from RDict for the given languages current: [] 659 -excludeBasenames=<names> : Do not load configurations from RDict for these SIDL base names current: [] 660</computeroutput> 661</screen> 662</para> 663 664</sect1> 665 666<sect1 id="Makers"> 667<title>Makers</title> 668 669<para>The build operation now encompasses configure, compile, and link operations, which are coordinated by objects of 670class <classname>maker.Maker</classname>. This object manages: 671<itemizedlist> 672 <listitem><para>configuration,</para></listitem> 673 <listitem><para>build,</para></listitem> 674 <listitem><para>install, and</para></listitem> 675 <listitem><para>project dependencies</para></listitem> 676</itemizedlist> 677All options, no matter which component they are intended for, are given uniformly to <filename>make.py</filename>. 678</para> 679 680 <sect2 id="SIDLMaker"> 681 <title>SIDLMaker</title> 682 683<para>This is a subclass which handles source generation from SIDL.</para> 684 685 </sect2> 686 687</sect1> 688 689<sect1 id="Builders"> 690<title>Builders</title> 691 692<para>The build operation now encompasses the configure, compile, and link operations.</para> 693 694</sect1> 695 696<sect1 id="LanguageProcessors"> 697<title>LanguageProcessors</title> 698 699<para>The build operation now encompasses the configure, compile, and link operations.</para> 700 701</sect1> 702 703<sect1 id="Interaction-with-Configure"> 704<title>Interaction with Configure</title> 705 706<para>The pickled configure is loaded by Maker, and then the config.compile objects are jacked into the Builder.</para> 707 708</sect1> 709 710</chapter> 711 712</book> 713