| 53e0a2f3 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add PetscLogState and PetscLogHandler to global logging
Without removing the existing global objects (such as petsc_stageLog),
This commit creates a global PetscLogState (petsc_log_state)
Profiling: Add PetscLogState and PetscLogHandler to global logging
Without removing the existing global objects (such as petsc_stageLog),
This commit creates a global PetscLogState (petsc_log_state) and
connects it to PetscLogInitialize() and PetscLogFinalize().
The functions PetscLogHandlerStart() and PetscLogHandlerStop() are added
to connect a general PetscLogHandler() to the global petsc_log_state,
A small array of slots for log handlers, PetscLogHandlers, is
added. It will be used in the macros like `PetscLogEventBegin()`
and `PetscLogEventEnd()`, so we make a wrapper type
`PetscLogHandlerHot` that exposes just as much of the internals
as are necessary to implement the macros.
Because they do not conflict with existing functions,
the following functions that depend on the new infrastructure
are added:
- PetscLogStageGetName(), PetscLogEventGetName(), PetscLogClassGetId(),
and PetscLogClassIdGetName() for completeness and to bring
of the functionality from petsc4pys' custom header into the
library proper.
- PetscLogEventsPause() and PetscLogEventsResume(), which
will replace PetscLogPauseCurrentEvent_Internal() and
PetscLogEventResume_Internal()
- PetscLogPerfstubsBegin() and PetscLogLegacyCallbacksBegin()
show more ...
|
| b9321188 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add nested PetscLogHandler implementation
This log handler has behavior like the behavior of
PetscLogNestedBegin().
Unlike the implementation of PetscLogHandler_Default, I did not try to
Profiling: Add nested PetscLogHandler implementation
This log handler has behavior like the behavior of
PetscLogNestedBegin().
Unlike the implementation of PetscLogHandler_Default, I did not try to
do a faithful translation of the existing nested logging behavior, for
two reasons.
- The existing implementation keeps sorted lists of nested paths ("Event
A;Event B;"), and for each of those paths keeps a sorted list of the
top-level events ("Event C", "Event D") that map that path to other
paths (path + event => other path). It's a complicated data structure,
and the benefit of the sorting (O(log n) lookup) is probably not worth
the more expensive insertion cost each time a new path + event combo
appears. Instead I implemented the path + event => other path
lookup as a hash map, which simplified the implementation
significantly.
- The existing implementation assigns an event to each path, modifying
the event registry in place. This messes things up if you want
to both -log_view and -log_view ::ascii_flamegraph output, the
original motivation for refactoring the profiling infrastructure.
In the new implementation, the nested handler has its own
PetscLogState that it modifies, and an inner copy of
PetscLogHandler_Default the acts on that nested state.
- The PetscLogGlobalNames data structure also simplifies the
construction of a linearized tree of events in the final
viewing stage, although the low-level xml construction
routines are copied almost verbatim from the previous
implementation.
show more ...
|
| 78f1b9b4 | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Implement the default PetscLogHandler
The existing default log handling that uses the PetscStageLog
is translated in the PetscLogHandler_Default implementation
of PetscLogHandler.
- Most
Profiling: Implement the default PetscLogHandler
The existing default log handling that uses the PetscStageLog
is translated in the PetscLogHandler_Default implementation
of PetscLogHandler.
- Most data that is global in the existing log handling
is moved into the struct, with the exception of the
raw counters.
- The registry data structures of PetscStageLog (eventLog,
classLog) are not reproduced because they are now in
the PetscLogState interface.
- PETSC_LOG_RESIZABLE_ARRAY is used to implement
resizable arrays instead of PetscStageInfo,
PetscEventPerfLog, and PetscClassPerfLog
- The tau perfstubs and MPE functionality is not
reproduced because they will become new
handlers on their own.
- The PetscLogGlobalNames provided by the
PetscLogRegistry interface make the logging
robust to differences in which events / stages are
registered on which process and in which order.
Otherwise the functions are relatively faithful translations
of the existing PetscStageLog functions into the new interface.
When reviewing this commit it might help to have a diff
between the PetscStageLog and PetscLogHandler_Default functions
to see what changes between them. I am working on a way to attach
that to the MR associated with this change.
show more ...
|
| 19ef957c | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add PetscLogHandler interface
PetscStageLog tries to handle multiple ways of interpreting profiling
data:
* -log_view (default),
* -log_view ::ascii_xml and -log_view ::ascii_flamegraph
Profiling: Add PetscLogHandler interface
PetscStageLog tries to handle multiple ways of interpreting profiling
data:
* -log_view (default),
* -log_view ::ascii_xml and -log_view ::ascii_flamegraph (nested),
* -log_trace,
* -log_mpe,
* tau perfstubs (implicitly),
* nvidia
They can interact with petsc_logState in incompatible ways.
PetscLogHandler gives each handler its own interface so that
it can have its own data structures. They key operations
that a log handler can implement are:
- PetscLogHandlerEventBegin() / PetscLogHandlerEventEnd()
- PetscLogHandlerEventSync()
- PetscLogHandlerStagePush() / PetscLogHandlerStagePop()
- PetscLogHandlerObjectCreate() / PetscLogHandlerObjectDestroy()
The remaining operations are PetscObject standard functions,
with the except of `PetscLogHandlerSetState()` / `PetscLogHandlerGetState()`,
which control the `PetscLogState` that the handler can use to
get information about registered things and about the stage stack.
This change only introduces the type, it does not deploy it
to replace anything in the current profiling interface.
When it does, the user-defined log handling that was implemented
by setting callbacks in in PetscLogSet() will be replaced with
user-defined PetscLogHandler instances.
show more ...
|
| 6873511f | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: Add PetscLogState interface to describe events and stages
The design that is being implemented is a separation between the
logging state that is common to multiple log handlers and the lo
Profiling: Add PetscLogState interface to describe events and stages
The design that is being implemented is a separation between the
logging state that is common to multiple log handlers and the log
handlers themselves. This new interface PetscLogState will
become the replacement for PetscStageLog, but this change
only introduces it without deploying it.
Internally, PetscLogState is:
- A registry (PetscLogRegistry) of more-or-less immutable information
about stages, events, and classes that have been registered. The
registry is not exposed, but the registry entries (PetscLogEventInfo,
PetscLogStageInfo, PetscLogClassInfo), that I feel comfortable exposing
them (pass-by-copy) through PetscLogStateXXXGetInfo() functions.
- A stack of stages that have been pushed and popped
- A PetscBT describing the active/inactive state of logging stages
and events. I have kept the existing semantics of
PetscLogStateSetActive / PetscLogEventActivate /
PetscLogEventActivateClass, which is: if the stage is active
and the (event,stage) pair is active, then log handling proceeds.
This logic is encoded in PetscLogStateStageEventIsActive
- For symmetry I have added PetscLogClass that matches PetscLogStage
and PetscLogEvent
- The profiling interface is littered with ad hoc resizable array types
(PetscEventRegLog, PetscEventPerfLog, PetscClassPerfLog,
PetscStageLog). The intention is for these to go away. This commit
introduces a PetscHashMap-style macro constructor for resizable
arrays (PETSC_LOG_RESIZABLE_ARRAY). It is only used in implementing
these arrays: they do not appear in the public interface. Storage
could be switched to a hashmap or other backing storage at some point
in the future.
show more ...
|
| 1c1ad86e | 13-Jul-2023 |
Toby Isaac <toby.isaac@gmail.com> |
Profiling: make visibility of logging types align with manpages
- Create file petsclogtypes.h for exposed logging data types. Move types
that will remain through upcoming changes (PetscLogEvent,
Profiling: make visibility of logging types align with manpages
- Create file petsclogtypes.h for exposed logging data types. Move types
that will remain through upcoming changes (PetscLogEvent,
PetscLogStage, PetscIntStack, PetscLogEventPerfInfo).
- Add docstring for PetscLogEventPerfInfo: it is the one important
type carrying profiling information that will remain through upcoming
changes.
- Change the submansec of petsclog.h from Sys to Profiling, better
reflecting the contents of the file.
- As the PetscIntStack methods have had manual pages (and externally
visible symbols) for years, move their declarations from
petsc/private/logimpl.h to petsclog.h. PetscIntStack is a low-level
data-structure similar to PetscBT, and this change puts PetscIntStack on
roughly equal footing with PetscBT.
show more ...
|