| dff009be | 24-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add composed functions for default log handler functions
This makes them extensible to other implementations.
Where appropriate, the global log handler will now dispatch
these functions t
Profiling: Add composed functions for default log handler functions
This makes them extensible to other implementations.
Where appropriate, the global log handler will now dispatch
these functions to all running log handlers.
It is not always appropriate to dispatch to all running log handlers:
when a single return value is expected or when a filename
is given, it will continue to dispatch only to the default
log handler.
show more ...
|
| b665b14e | 18-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Switch logging over to the new PetscLogHandler infrastructure
- Move data structures only needed by the default logger now (Action,
Object) to logdefault.c
- Create petsclogdeprecated.
Profiling: Switch logging over to the new PetscLogHandler infrastructure
- Move data structures only needed by the default logger now (Action,
Object) to logdefault.c
- Create petsclogdeprecated.h and petsc/private/logimpldeprecated.h
for publicly visible symbols that are now deprecated. Yes,
lots of symbols in petsc/private/logimpl.h were not only visible
but had docstrings.
- The macros PetscLogEventBegin(), PetscLogEventEnd(),
PetscLogEventSync(), PetscLogObjectCreate(), and
PetscLogObjectDestroy() now looks through the log handlers
in PetscLogHandlers, sees if they have they appropriate
callbacks (Exposed in PetscLogHandlerHot), and calls them.
Up to 4 log handlers can run simultaneously.
- All queries about the registered things now go through
petsc_log_state.
- Output functions dispatch to the appropriate PetscLogHandler
- There is no need for xmllogevent.c, xmlviewer.c, eventlog.c,
classlog.c, and stagelog.c anymore. Some orphaned functions
(PetscLogSetThreshold(), PetscLogEventSetDof(),
PetscLogEventSetError(), PetscASend(), PetscARecv(), PetscAReduce())
- PetscFEOpenCLLogResidual() did some intrusive things related
to the old logging infrastruture: the private header
for the default log handler is now imported into feopencl.c
so that it can do equivalent things with the new infrastructure.
- PetscLogPauseCurrentEvent_Internal() is replaced with
PetscLogEventsPause()
Deprecated symbols include:
- All functions related to PetscStageLog and the resizable array structs
that it contained.
- The legacy EventBegin()/EventEnd() callbacks.
- PetscLogAllBegin(), which actually reported less information than
PetscLogDefaultBegin() because it was not kept up-to-date with
regards to GPU profiling.
show more ...
|
| 53e0a2f3 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add PetscLogState and PetscLogHandler to global logging
Without removing the existing global objects (such as petsc_stageLog),
This commit creates a global PetscLogState (petsc_log_state)
Profiling: Add PetscLogState and PetscLogHandler to global logging
Without removing the existing global objects (such as petsc_stageLog),
This commit creates a global PetscLogState (petsc_log_state) and
connects it to PetscLogInitialize() and PetscLogFinalize().
The functions PetscLogHandlerStart() and PetscLogHandlerStop() are added
to connect a general PetscLogHandler() to the global petsc_log_state,
A small array of slots for log handlers, PetscLogHandlers, is
added. It will be used in the macros like `PetscLogEventBegin()`
and `PetscLogEventEnd()`, so we make a wrapper type
`PetscLogHandlerHot` that exposes just as much of the internals
as are necessary to implement the macros.
Because they do not conflict with existing functions,
the following functions that depend on the new infrastructure
are added:
- PetscLogStageGetName(), PetscLogEventGetName(), PetscLogClassGetId(),
and PetscLogClassIdGetName() for completeness and to bring
of the functionality from petsc4pys' custom header into the
library proper.
- PetscLogEventsPause() and PetscLogEventsResume(), which
will replace PetscLogPauseCurrentEvent_Internal() and
PetscLogEventResume_Internal()
- PetscLogPerfstubsBegin() and PetscLogLegacyCallbacksBegin()
show more ...
|
| ccf0b5c1 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add tau perfstubs implementation of PetscLogHandler
In the existing logging, perfstubs runs automatically with -log_view
if PETSc is configured --with-tau-perfstubs. This new PetscLogHan
Profiling: Add tau perfstubs implementation of PetscLogHandler
In the existing logging, perfstubs runs automatically with -log_view
if PETSc is configured --with-tau-perfstubs. This new PetscLogHandler
implementation will be used with a new -log_perfstubs command line
option that will only run this handler and not the default handler.
show more ...
|
| b9321188 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add nested PetscLogHandler implementation
This log handler has behavior like the behavior of
PetscLogNestedBegin().
Unlike the implementation of PetscLogHandler_Default, I did not try to
Profiling: Add nested PetscLogHandler implementation
This log handler has behavior like the behavior of
PetscLogNestedBegin().
Unlike the implementation of PetscLogHandler_Default, I did not try to
do a faithful translation of the existing nested logging behavior, for
two reasons.
- The existing implementation keeps sorted lists of nested paths ("Event
A;Event B;"), and for each of those paths keeps a sorted list of the
top-level events ("Event C", "Event D") that map that path to other
paths (path + event => other path). It's a complicated data structure,
and the benefit of the sorting (O(log n) lookup) is probably not worth
the more expensive insertion cost each time a new path + event combo
appears. Instead I implemented the path + event => other path
lookup as a hash map, which simplified the implementation
significantly.
- The existing implementation assigns an event to each path, modifying
the event registry in place. This messes things up if you want
to both -log_view and -log_view ::ascii_flamegraph output, the
original motivation for refactoring the profiling infrastructure.
In the new implementation, the nested handler has its own
PetscLogState that it modifies, and an inner copy of
PetscLogHandler_Default the acts on that nested state.
- The PetscLogGlobalNames data structure also simplifies the
construction of a linearized tree of events in the final
viewing stage, although the low-level xml construction
routines are copied almost verbatim from the previous
implementation.
show more ...
|
| 78f1b9b4 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Implement the default PetscLogHandler
The existing default log handling that uses the PetscStageLog
is translated in the PetscLogHandler_Default implementation
of PetscLogHandler.
- Most
Profiling: Implement the default PetscLogHandler
The existing default log handling that uses the PetscStageLog
is translated in the PetscLogHandler_Default implementation
of PetscLogHandler.
- Most data that is global in the existing log handling
is moved into the struct, with the exception of the
raw counters.
- The registry data structures of PetscStageLog (eventLog,
classLog) are not reproduced because they are now in
the PetscLogState interface.
- PETSC_LOG_RESIZABLE_ARRAY is used to implement
resizable arrays instead of PetscStageInfo,
PetscEventPerfLog, and PetscClassPerfLog
- The tau perfstubs and MPE functionality is not
reproduced because they will become new
handlers on their own.
- The PetscLogGlobalNames provided by the
PetscLogRegistry interface make the logging
robust to differences in which events / stages are
registered on which process and in which order.
Otherwise the functions are relatively faithful translations
of the existing PetscStageLog functions into the new interface.
When reviewing this commit it might help to have a diff
between the PetscStageLog and PetscLogHandler_Default functions
to see what changes between them. I am working on a way to attach
that to the MR associated with this change.
show more ...
|
| 19ef957c | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add PetscLogHandler interface
PetscStageLog tries to handle multiple ways of interpreting profiling
data:
* -log_view (default),
* -log_view ::ascii_xml and -log_view ::ascii_flamegraph
Profiling: Add PetscLogHandler interface
PetscStageLog tries to handle multiple ways of interpreting profiling
data:
* -log_view (default),
* -log_view ::ascii_xml and -log_view ::ascii_flamegraph (nested),
* -log_trace,
* -log_mpe,
* tau perfstubs (implicitly),
* nvidia
They can interact with petsc_logState in incompatible ways.
PetscLogHandler gives each handler its own interface so that
it can have its own data structures. They key operations
that a log handler can implement are:
- PetscLogHandlerEventBegin() / PetscLogHandlerEventEnd()
- PetscLogHandlerEventSync()
- PetscLogHandlerStagePush() / PetscLogHandlerStagePop()
- PetscLogHandlerObjectCreate() / PetscLogHandlerObjectDestroy()
The remaining operations are PetscObject standard functions,
with the except of `PetscLogHandlerSetState()` / `PetscLogHandlerGetState()`,
which control the `PetscLogState` that the handler can use to
get information about registered things and about the stage stack.
This change only introduces the type, it does not deploy it
to replace anything in the current profiling interface.
When it does, the user-defined log handling that was implemented
by setting callbacks in in PetscLogSet() will be replaced with
user-defined PetscLogHandler instances.
show more ...
|
| 176456b8 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add interface for getting unqiue global logging names
It has been implicit since the beginning that, in default logging
interface, stages, events, and classes have to be registered in the
Profiling: Add interface for getting unqiue global logging names
It has been implicit since the beginning that, in default logging
interface, stages, events, and classes have to be registered in the same
order on every process. This is harder to accomplish with nested
logging, where new events and stages may be created on the fly.
This change uses the newly created PetscLogState interface
to assign each event/stage/class a globally unique id based
on its name. Essentially, each process in turn tells the other
processes the names of things that haven't been assigned ids yet,
and they all agree on their new ids. The global ids are used
to construct global <-> local maps.
In the easy case of all processes having the same things named in the
same order, the process stops after they all confirm that they
have the same names in the same order.
This communication pattern is straight-forward and does not
attempt to be efficient, but log viewing lots and lots of
MPI_Allreduces, so the cost of this will not be noticeable.
show more ...
|
| 6873511f | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add PetscLogState interface to describe events and stages
The design that is being implemented is a separation between the
logging state that is common to multiple log handlers and the lo
Profiling: Add PetscLogState interface to describe events and stages
The design that is being implemented is a separation between the
logging state that is common to multiple log handlers and the log
handlers themselves. This new interface PetscLogState will
become the replacement for PetscStageLog, but this change
only introduces it without deploying it.
Internally, PetscLogState is:
- A registry (PetscLogRegistry) of more-or-less immutable information
about stages, events, and classes that have been registered. The
registry is not exposed, but the registry entries (PetscLogEventInfo,
PetscLogStageInfo, PetscLogClassInfo), that I feel comfortable exposing
them (pass-by-copy) through PetscLogStateXXXGetInfo() functions.
- A stack of stages that have been pushed and popped
- A PetscBT describing the active/inactive state of logging stages
and events. I have kept the existing semantics of
PetscLogStateSetActive / PetscLogEventActivate /
PetscLogEventActivateClass, which is: if the stage is active
and the (event,stage) pair is active, then log handling proceeds.
This logic is encoded in PetscLogStateStageEventIsActive
- For symmetry I have added PetscLogClass that matches PetscLogStage
and PetscLogEvent
- The profiling interface is littered with ad hoc resizable array types
(PetscEventRegLog, PetscEventPerfLog, PetscClassPerfLog,
PetscStageLog). The intention is for these to go away. This commit
introduces a PetscHashMap-style macro constructor for resizable
arrays (PETSC_LOG_RESIZABLE_ARRAY). It is only used in implementing
these arrays: they do not appear in the public interface. Storage
could be switched to a hashmap or other backing storage at some point
in the future.
show more ...
|