Work Plan In Memory Adapt - Revision history

Cwsmith: /* Detailed Description of Phases */

2015-11-20T23:01:14Z

‎Detailed Description of Phases

Cwsmith: /* Summary of Phases */

2015-11-20T22:59:55Z

‎Summary of Phases

Cwsmith: /* Detailed Description of Phases */

2015-07-23T15:20:45Z

‎Detailed Description of Phases

Cwsmith: /* phastaIO */

2015-07-23T15:15:52Z

‎phastaIO

Cwsmith: /* Detailed Description of Phases */

2015-07-23T15:13:56Z

‎Detailed Description of Phases

Cwsmith: /* Summary of Phases */

2015-07-23T15:13:20Z

‎Summary of Phases

Cwsmith: /* preliminary design plan */

2015-07-23T15:12:11Z

‎preliminary design plan

Cwsmith: /* preliminary design plan */

2015-07-23T15:11:52Z

‎preliminary design plan

Cwsmith: /* phastaIO */

2015-07-23T15:09:25Z

‎phastaIO

Cwsmith: /* Thorny Issues */

2015-07-23T15:08:51Z

‎Thorny Issues

@@ Line 48: / Line 48: @@
 ## Debug to make sure that files written are correct (two successive runs) - done
 #phase2
-## Create ability for Chef to write to memory location using posixIO pointed at a memory location instead of a file
+## Create ability for Chef to write to memory location using posixIO pointed at a memory location instead of a file - done
-## Create ability to pass the descriptor of that memory location to PHASTA  (I think it is safe to assume that since we are one “file” per part in both Chef and PHASTA that this is just the starting location of where Chef started writing)
+## Create ability to pass the descriptor of that memory location to PHASTA  (I think it is safe to assume that since we are one “file” per part in both Chef and PHASTA that this is just the starting location of where Chef started writing) - done
-## Create ability to have phasta initialize itself with posix-based PHASTAIO reads from this in-memory file/state that Chef wrote
+## Create ability to have phasta initialize itself with posix-based PHASTAIO reads from this in-memory file/state that Chef wrote - done
 #phase3
-## Create ability for PHASTA to write  adapt-fields (solution, d2wall, error, ybar) using posix-based PHASTAIO writes to a memory location
+## Create ability for PHASTA to write  adapt-fields (solution, d2wall, error, ybar) using posix-based PHASTAIO writes to a memory location - done
-## Create ability to pass the descriptor of that memory location back to Chef
+## Create ability to pass the descriptor of that memory location back to Chef - done
-## Create ability for Chef to read adapt-fields when passed descriptor
+## Create ability for Chef to read adapt-fields when passed descriptor - done
-## Invoke a UR with solution transfer and write files to confirm phase3 success
+## Invoke a UR with solution transfer and write files to confirm phase3 success - done
 #phase4
-## Should be trivial but have Chef write result of UR to a new memory location and pass descriptor to PHASTA to read and run again (comparing to file based version of the same which we are already doing at scale)
+## Should be trivial but have Chef write result of UR to a new memory location and pass descriptor to PHASTA to read and run again (comparing to file based version of the same which we are already doing at scale) - done
 #phase5
-## Demonstrate that we can do file based local adaptive refinement with Chef  (I will let Dan and Cameron described the subtask developments here but am more than happy to help plan and assist in any way I can because we really need this capability…I am fine with ignoring/freezing boundary layers to get a working case going before taking on that much tougher nut).
+## Demonstrate that we can do file based local adaptive refinement with Chef  (I will let Dan and Cameron described the subtask developments here but am more than happy to help plan and assist in any way I can because we really need this capability…I am fine with ignoring/freezing boundary layers to get a working case going before taking on that much tougher nut). - done
 ==Thorny Issues==

@@ Line 3: / Line 3: @@
 *phase0 wrapper layer that supports both syncio and posix - done 7/20/2015
 *phase1 PHASTA can read AND write both - done 7/22/2015
-*phase2 PHASTA runs after chef though an in-memory read of Chef's written in-memory data
+*phase2 PHASTA runs after chef though an in-memory read of Chef's written in-memory data - done
-*phase3 PHASTA writes its field back to chef and lets chef do a UR on its mesh and transfers fields
+*phase3 PHASTA writes its field back to chef and lets chef do a UR on its mesh and transfers fields - done
-*phase4 bring that result back to phasta (so chef->PHASTA->chef-UR->PHASTA-UR)
+*phase4 bring that result back to phasta (so chef->PHASTA->chef-UR->PHASTA-UR) - done
-*phase5 replace UR with local adaptation
+*phase5 replace UR with local adaptation - done
-*phase6 loop and we are done
+*phase6 loop and we are done - done
 == phastaIO ==

@@ Line 38: / Line 38: @@
 # phase0
-## Move field string construction into C)-  status:  CWS has done this for  syncio (top of routines so not yet in a syncio conditional but this is not necessary until phase1)
+## Move field string construction into phio layer - hardcoded in phasta - see https://github.com/cwsmith/phastaChef/issues/17
-## Move file name  construction into B)-  status: not started but I think CWS indicated it was basically same work as 0.1 so leaving that with him. Note we don’t want this in init (at least for restarts) because we init only once but possibly write multiple times and the filename changes.
+## Move file name  construction into - done
-### Move query and init into B)
+## Move query and init into phio_opefile_* - done
-## Migrating all of PHASTA to use 0.1 dev - status:  CWS did it for readnblk.f and this built and ran (YAY), KEJ is migrating it to the balance of PHASTA (DONE I THINK)
+## Migrating all of PHASTA to use wrapper apis - done
 #phase1
-## Migrate string constructors into  conditional for syncio
+## Migrate string constructors into conditional for syncio - done
-## Create posix counterparts and put them in the posix branch of conditional
+## Create posix counterparts and put them in the posix branch of conditional - done
-## make init and finalize functions that have a universal call but, via conditionals (likely posix just returns but syncio has stuff to do but we want the PHASTA side to look the same regardless of IO chosen)
+## Debug to be sure that at least all tet meshes can be read with both posix/syncio - done
-## Debug to be sure that at least all tet meshes can be read with both posix/syncio
+## Debug to make sure that files written are correct (two successive runs) - done
 #phase2
 ## Create ability for Chef to write to memory location using posixIO pointed at a memory location instead of a file

@@ Line 20: / Line 20: @@
 initalize
 </pre>
 To make the interface more simple, I propose, we push init into open and finalize into close.  Furthermore I propose that we consider a wrapper layer approach
 where the functions that the PHASTA workflow  directly uses to perform IO are:
-*phio_openfile_read (in addition to specializing filename string for particular library, this function can call queryphmpiio and initphmpiio in the case of syncio)
+*phio_openfile_[read|write] (in addition to specializing filename string for particular library, this function can call queryphmpiio and initphmpiio in the case of syncio)
 *phio_readheader
 *phio_readdatablock
 *phio_writeheader
 *phio_writedatablock
-*phio_closefile
+*phio_closefile_[read|write]
-These functions will be called by readnblk (and a few others) for reading and Write_restart (and a few clones) for writing.  Principally, these 6 functions will be wrappers that take in partial strings and the usual arguments for their PhastaIO functions  and then perform the necessary string specializations for the particular flavor of PhastaIO being used (e.g,, syncio or posix) before then ultimately calling the PhastaIO function (that already has a branching based on the two flavors).  Cameron has proposed housing those seven wrappers in phIO.c which seems fine to me.  Helper functions can go in the same file.
+These functions will be called by readnblk (and a few others) for reading and Write_restart (and a few clones) for writing.  Principally, these eight functions will be wrappers that take in partial strings and the usual arguments for their PhastaIO functions  and then perform the necessary string specializations for the particular flavor of PhastaIO being used (e.g,, syncio or posix) before then ultimately calling the PhastaIO function (that already has a branching based on the two flavors).  Cameron has proposed housing those seven wrappers in phIO.cc which seems fine to me.  Helper functions can go in the same file.
 Initially, <code>setIOparam()</code> counting of syncio files will be used to determine if sync or posix files will be used.  If the count is zero then assume posix.  Use the count as a switch for file format everywhere (can be replaced later by agreed upon flag for file format...candidates: iotype= binary-posix, binary-mpiio instead of binary (not sure we want to preserve ascii unless it easy to do)

@@ Line 52: / Line 52: @@
 ## Debug to make sure that files written are correct (two successive runs)
 #phase2
-## 2.1 Create ability for Chef to write to memory location using posixIO pointed at a memory location instead of a file
+## Create ability for Chef to write to memory location using posixIO pointed at a memory location instead of a file
-## 2.2 Create ability to pass the descriptor of that memory location to PHASTA  (I think it is safe to assume that since we are one “file” per part in both Chef and PHASTA that this is just the starting location of where Chef started writing)
+## Create ability to pass the descriptor of that memory location to PHASTA  (I think it is safe to assume that since we are one “file” per part in both Chef and PHASTA that this is just the starting location of where Chef started writing)
-## 2.3 Create ability to have phasta initialize itself with posix-based PHASTAIO reads from this in-memory file/state that Chef wrote
+## Create ability to have phasta initialize itself with posix-based PHASTAIO reads from this in-memory file/state that Chef wrote
 #phase3
 ## Create ability for PHASTA to write  adapt-fields (solution, d2wall, error, ybar) using posix-based PHASTAIO writes to a memory location

← Older revision		Revision as of 15:12, 23 July 2015
Line 76:		Line 76:
	### query is called before file is read the first time (same file read again can get by without it e.g., readlesprojection reads restart a second time)		### query is called before file is read the first time (same file read again can get by without it e.g., readlesprojection reads restart a second time)
	### init is called for both reads and writes of syncio files		### init is called for both reads and writes of syncio files
−
−	~~== preliminary design plan ==~~
−	* use setIOparam() to count syncio files. If count=0 then assume posix. use the count as a switch for file format everywhere (can be replaced later by agreed upon flag for file format...candidates: iotype= binary-posix, binary-mpiio instead of binary (not sure we want to preserve ascii unless it easy to do)
−	* push all init stuff into openfile so that there are only 6 functions that any users sees in phastaIO for either syncio or posix openfile, closefile, {read,write}X{header,datablock}

@@ Line 35: / Line 35: @@
 These functions will be called by readnblk (and a few others) for reading and Write_restart (and a few clones) for writing.  Principally, these 6 functions will be wrappers that take in partial strings and the usual arguments for their PhastaIO functions  and then perform the necessary string specializations for the particular flavor of PhastaIO being used (e.g,, syncio or posix) before then ultimately calling the PhastaIO function (that already has a branching based on the two flavors).  Cameron has proposed housing those seven wrappers in phIO.c which seems fine to me.  Helper functions can go in the same file.
-=== preliminary design plan ===
+Initially, <code>setIOparam()</code> counting of syncio files will be used to determine if sync or posix files will be used.  If the count is zero then assume posix.  Use the count as a switch for file format everywhere (can be replaced later by agreed upon flag for file format...candidates: iotype= binary-posix, binary-mpiio instead of binary (not sure we want to preserve ascii unless it easy to do)
 == Detailed Description of Phases ==

← Older revision		Revision as of 15:09, 23 July 2015
Line 34:		Line 34:

	These functions will be called by readnblk (and a few others) for reading and Write_restart (and a few clones) for writing. Principally, these 6 functions will be wrappers that take in partial strings and the usual arguments for their PhastaIO functions and then perform the necessary string specializations for the particular flavor of PhastaIO being used (e.g,, syncio or posix) before then ultimately calling the PhastaIO function (that already has a branching based on the two flavors). Cameron has proposed housing those seven wrappers in phIO.c which seems fine to me. Helper functions can go in the same file.		These functions will be called by readnblk (and a few others) for reading and Write_restart (and a few clones) for writing. Principally, these 6 functions will be wrappers that take in partial strings and the usual arguments for their PhastaIO functions and then perform the necessary string specializations for the particular flavor of PhastaIO being used (e.g,, syncio or posix) before then ultimately calling the PhastaIO function (that already has a branching based on the two flavors). Cameron has proposed housing those seven wrappers in phIO.c which seems fine to me. Helper functions can go in the same file.
		+
		+	=== preliminary design plan ===
		+	* use setIOparam() to count syncio files. If count=0 then assume posix. use the count as a switch for file format everywhere (can be replaced later by agreed upon flag for file format...candidates: iotype= binary-posix, binary-mpiio instead of binary (not sure we want to preserve ascii unless it easy to do)
		+	* push all init stuff into openfile so that there are only 6 functions that any users sees in phastaIO for either syncio or posix openfile, closefile, {read,write}X{header,datablock}

	== Detailed Description of Phases ==		== Detailed Description of Phases ==

@@ Line 66: / Line 66: @@
 #phase0
-## 1) extra write files.  Seems to me that there should be either
+## extra write files.  Seems to me that there should be either
-### a) only 2  write_restart and write_field, or
+### only 2  write_restart and write_field, or
-### b) an init function and then only write_field
+### an init function and then only write_field
-## 2) dwall currently has two modes (in restart or in a separate file….do we really need to preserve this)
+## dwall currently has two modes (in restart or in a separate file….do we really need to preserve this)
-## 3) usr.c opens restart to read saved projection vectors.  Not a problem per se but might be better to read this while we have the restart already open so that the IO code is more compact.  This may involve moving more leslib stuff forward in the code
+## usr.c opens restart to read saved projection vectors.  Not a problem per se but might be better to read this while we have the restart already open so that the IO code is more compact.  This may involve moving more leslib stuff forward in the code
-## 4) KEJ is still trying to sort out the roles of {query,init}phmpiio.
+## KEJ is still trying to sort out the roles of {query,init}phmpiio.
-### a) query is called before file is read the first time (same file read again can get by without it  e.g., readlesprojection reads restart a second time)
+### query is called before file is read the first time (same file read again can get by without it  e.g., readlesprojection reads restart a second time)
-### b) init is called for both reads and writes of syncio files
+### init is called for both reads and writes of syncio files
 == preliminary design plan ==
 * use setIOparam() to count syncio files.  If count=0 then assume posix.  use the count as a switch for file format everywhere (can be replaced later by agreed upon flag for file format...candidates: iotype= binary-posix, binary-mpiio instead of binary (not sure we want to preserve ascii unless it easy to do)
 * push all init stuff  into openfile so that there are only 6 functions that any users sees in phastaIO for either syncio or posix openfile, closefile, {read,write}X{header,datablock}