Class OutputManagerH5

Defined in File OutputManagerH5.hpp

Inheritance Relationships

Base Type

public io::OutputManager (Class OutputManager)

Class Documentation

class OutputManagerH5 : public io::OutputManager 

HDF5-output specialization of OutputManager.

Unless the linked HDF5 build was itself configured with its (opt-in) thread-safety support – which is not something this codebase can assume of every HDF5 install it might be built against – HDF5 is not safe to call from more than one thread at a time AT ALL, even when two threads only ever touch entirely separate files/objects and every individual call is otherwise guarded by a mutex/critical section of its own: the library’s own internal global state (ID tables, allocators, etc.) is not itself protected, so genuinely concurrent entry from two threads can corrupt it regardless of what each call actually touches. So, every method here that calls into HDF5 – openOutputFile(), closeOutputFile(), and every write* method – shares a single, global OpenMP critical section (see openOutputFile()’s own comment), fully serializing entry into the library across every thread, not just concurrent access to a shared file.

On top of that global serialization, when built with OpenMP each thread also gets its own private HDF5 file (outDir/modelName/thread_NNNN.h5, one per OpenMP thread) rather than sharing one – this doesn’t help with the concurrent-entry problem above (that’s why the critical section exists regardless), but does still avoid a different class of problem: interleaved mutations to one file’s own internal metadata cache (B-tree nodes, free-space managers, …) from calls made by different physical OS threads, which serializing calls in time does not, by itself, guarantee is safe. By default (SimControls::OutputMode::h5), those per-thread files are merged back into a single outDir/modelName.h5 – identical in format to what this class always produced before, except that row order within each dataset is no longer guaranteed, since threads may process trials in any order – once the run completes; see consolidateFiles()’s own comment. SimControls::OutputMode:: h5divided instead leaves the per-thread files as-is, skipping consolidation, for very large runs where merging into one file would itself be costly. Without OpenMP, there is only ever one thread, so this class behaves exactly as it always did: a single outDir/modelName.h5, no per-thread directory, no consolidation step (and the critical sections are all no-ops).

Public Functions

explicit OutputManagerH5(const SimControls &simControls, bool restart = false)

Open the output file(s) and write their header.

simControls is stored by reference, so the object passed in must outlive this OutputManagerH5. See this class’s own header comment for what “the output file(s)” means when built with OpenMP.

If restart is true, calls restartSetup() before openNewOutputFiles(), so checkpointNumber_ (and, via it, openNewOutputFiles()’s own choice of output path) already reflects where the run being restarted left off by the time the first file is actually opened.

Parameters:

simControls – Simulation controls (physics settings and control-flow settings together)
restart – Whether this run is resuming a previous, interrupted run from its most recent checkpoint (see restartSetup()’s own comment); defaults to false

Throws:

std::runtime_error – if restart is true and simControls.outputMode() is OutputMode::ascii – restarting is only ever supported with HDF5 output. In practice this can only be reached by a caller building an OutputManagerH5 directly (e.g. from Python) with an ascii SimControls, since the CLI itself (main.cpp) checks for this same illegal combination before ever constructing an OutputManagerH5, and always builds an OutputManagerAscii instead when outputMode() is ascii – kept here anyway as a second line of defense, mirroring the same defense-in-depth pattern SimControls::initControlFlow()’s own checkpointInterval()/ascii check and OutputManagerAscii::checkpoint()’s own throw already use.
std::runtime_error – if restart is true and simControls.checkpointInterval() is 0. Restarting with checkpointing disabled for this session would still correctly resume trial numbering/counting from the run being restarted (restartSetup() does not depend on this session’s own checkpointInterval()), but this session’s own output would then be written to a fresh, non-checkpointed modelName.h5 – which slug_reader’s own checkpoint discovery silently ignores whenever any modelName_chkNNNNN checkpoint exists (exactly the case here, since one must exist for there to be anything to restart from), making this session’s own data invisible to any later read. Worse, without checkpointing there is also no longer any checkpoint boundary to protect this session’s own progress should it, too, be interrupted before finishing. Rejected here rather than left to silently produce output nothing can read back.

~OutputManagerH5() override: Close the output file.

OutputManagerH5(const OutputManagerH5&) = delete

auto operator=(const OutputManagerH5&) -> OutputManagerH5& = delete

OutputManagerH5(OutputManagerH5&&) = delete

auto operator=(OutputManagerH5&&) -> OutputManagerH5& = delete

virtual void writeCluster(unsigned long trial, const core::Cluster &cluster) override

Write a cluster’s data as a row of the clusters datasets.

If cluster output was not enabled for this simulation, this is a no-op.

Parameters:

trial – Trial number to which this cluster belongs
cluster – The cluster whose data should be written

virtual void writeClusterSpec(unsigned long trial, double time, core::Cluster &cluster) override

Write a cluster’s spectrum as a row of the cluster-spectra datasets.

If spectral synthesis was not enabled for this simulation (the cluster_spectra group does not exist), or the cluster has disrupted, this is a no-op.

Parameters:

trial – Trial number to which this cluster belongs
time – The output time at which the cluster’s spectrum was computed, in yr
cluster – The cluster whose spectrum should be written

virtual void writeClusterPhot(unsigned long trial, double time, core::Cluster &cluster) override

Write a cluster’s photometry as a row of the cluster_phot datasets.

If no filter collection or bolometric luminosity was requested for this simulation (the cluster_phot group does not exist), or the cluster has disrupted, this is a no-op.

Parameters:

trial – Trial number to which this cluster belongs
time – The output time at which the cluster’s photometry was computed, in yr
cluster – The cluster whose photometry should be written

virtual void writeClusterYields(unsigned long trial, double time, core::Cluster &cluster) override

Write a cluster’s nucleosynthetic yields as a row of the cluster_yields datasets.

If no yield channels were requested for this simulation (the cluster_yields group does not exist), this is a no-op – unlike writeClusterPhot()/writeClusterSpec(), a disrupted cluster’s own row is still written (see OutputManager::writeClusterYields()’s own comment for why).

Parameters:

trial – Trial number to which this cluster belongs
time – The output time at which this row was recorded, in yr
cluster – The cluster whose yields should be written

virtual void writeGalaxy(unsigned long trial, double time, core::Galaxy &galaxy) override

Write a galaxy’s data as a row of the galaxy datasets.

If galaxy output was not enabled for this simulation (the galaxy group does not exist), this is a no-op. Otherwise, after writing this row, also calls writeCluster() on every currently-alive (non-disrupted) cluster in galaxy.

Parameters:

trial – Trial number to which this galaxy belongs
time – The output time at which this row was recorded, in yr
galaxy – The galaxy whose data should be written

virtual void writeGalaxySpec(unsigned long trial, double time, core::Galaxy &galaxy) override

Write a galaxy’s spectrum as a row of the galaxy_spectra datasets.

If spectral synthesis was not enabled for this simulation (the galaxy_spectra group does not exist), this is a no-op. Otherwise, after writing this row, also calls writeClusterSpec() on every currently-alive (non-disrupted) cluster in galaxy.

Parameters:

trial – Trial number to which this galaxy belongs
time – The output time at which the galaxy’s spectrum was computed, in yr
galaxy – The galaxy whose spectrum should be written

virtual void writeGalaxyPhot(unsigned long trial, double time, core::Galaxy &galaxy) override

Write a galaxy’s photometry as a row of the galaxy_phot datasets.

If no filter collection or bolometric luminosity was requested for this simulation (the galaxy_phot group does not exist), this is a no-op. Otherwise, after writing this row, also calls writeClusterPhot() on every currently-alive (non-disrupted) cluster in galaxy.

Parameters:

trial – Trial number to which this galaxy belongs
time – The output time at which the galaxy’s photometry was computed, in yr
galaxy – The galaxy whose photometry should be written

virtual void writeGalaxyYields(unsigned long trial, double time, core::Galaxy &galaxy) override

Write a galaxy’s nucleosynthetic yields as a row of the galaxy_yields datasets.

Writing the galaxy_yields row itself is a no-op if the galaxy_yields group does not exist (no yield channels were requested, or output.write_galaxy_yields is false) – writeClusterYields() is still called on every currently-alive (non-disrupted) cluster in galaxy regardless, since output.write_cluster_yields is independently togglable from output.write_galaxy_yields (see SimControls::readYields()’s own sanity check, which only rejects both being false at once); writeClusterYields() itself then no-ops per cluster if the cluster_yields group also does not exist.

Parameters:

trial – Trial number to which this galaxy belongs
time – The output time at which this row was recorded, in yr
galaxy – The galaxy whose yields should be written

virtual void writeClusterFeedback(unsigned long trial, double time, core::Cluster &cluster) override

Write a cluster’s feedback as a row of the cluster_feedback datasets.

If the cluster_feedback group does not exist (output.write_cluster_feedback is false), this is a no-op. Like writeClusterYields(), a disrupted cluster’s own row is still written (see OutputManager::writeClusterFeedback()’s own comment for why).

Parameters:

trial – Trial number to which this cluster belongs
time – The output time at which this row was recorded, in yr
cluster – The cluster whose feedback should be written

virtual void writeGalaxyFeedback(unsigned long trial, double time, core::Galaxy &galaxy) override

Write a galaxy’s feedback as a row of the galaxy_feedback datasets.

Writing the galaxy_feedback row itself is a no-op if the galaxy_feedback group does not exist – writeClusterFeedback() is still called on every currently-alive (non-disrupted) cluster in galaxy regardless, exactly as writeGalaxyYields() does (see its own comment for why).

Parameters:

trial – Trial number to which this galaxy belongs
time – The output time at which this row was recorded, in yr
galaxy – The galaxy whose feedback should be written

virtual void checkpoint(unsigned long trialsCompleted) override

Roll over to a new checkpoint.

Only meaningful if SimControls::checkpointInterval() is non-zero. Closes every thread’s own currently-open output file (closeOutputFile()), increments checkpointNumber_, then opens a fresh one for the new checkpoint for every thread (openNewOutputFiles()) – so the checkpoint just finished is fully flushed to disk (closing an HDF5 file forces this) before any more trials are written, bounding how much work a crash or a walltime kill can lose to at most one checkpoint’s worth. consolidateFiles() is deliberately not called here (only at the very end, from the destructor, once per checkpoint – see its own comment): consolidating after every single checkpoint would slow down the run itself for no benefit, since nothing needs the checkpoints merged until the whole run is over.

Requires the caller to already have ensured that no thread is concurrently writing output when this runs – specifically, that every trial belonging to the checkpoint being closed has already been written in full (not, say, one thread partway through a trial’s own later output times/photometry while another has already moved on and rolled over) – and that this itself is called on exactly one thread, from outside any active parallel region. SimCluster::run()’s/SimGalaxy::run()’s own comment explains how they guarantee that: trials run in batches of checkpointInterval() at a time, each batch its own “#pragma omp parallel for”, so the implicit barrier at the end of that construct – every thread has finished every trial in the batch before any of them proceeds past it – has already elapsed by the time either of them calls this, and neither is still inside a parallel region when it does. This method relies on both of those guarantees rather than re-deriving them itself (e.g. via its own locking): closeOutputFile() below runs once per thread via its own fresh “#pragma omp

parallel” spawn, exactly mirroring the destructor’s own identical pattern, which is only correct because nothing else is concurrently touching any thread’s own handles at that moment.

Parameters:: trialsCompleted – Number of trials completed so far in the run, across every checkpoint including the one just closed – see closeOutputFile()’s own comment for what this is used for

inline virtual auto restartTrialsDone() const -> unsigned long override

Return the number of trials already completed by the run being restarted.

Only meaningful when this OutputManagerH5 was itself constructed with restart = true – SimCluster::run()/ SimGalaxy::run() only ever call this when they were themselves given restart = true, which the CLI (main.cpp) only ever does alongside the same restart argument this object’s own constructor received, so the two stay in sync in practice, but nothing here enforces that a caller constructing these objects directly always pairs them consistently.

Returns:: restartTrialsDone_, as set by restartSetup() when this OutputManagerH5 was constructed with restart = true (0 if it was not, or if restartSetup() found no existing checkpoint to resume from)

inline virtual auto restartMaxTrial() const -> unsigned long override

Return the largest trial number the run being restarted ever actually wrote.

Only meaningful under the same conditions as restartTrialsDone(); see its own comment.

Deliberately distinct from restartTrialsDone()

: under dynamic OpenMP scheduling, a batch that SIGTERM interrupted partway through can finish with a higher-numbered trial (e.g. one a fast thread happened to grab) done while a lower-numbered one (assigned to a slower thread, still mid-flight when the batch’s own implicit barrier released) is not – trialsCompleted_’s own count still correctly reports how many trials completed, but “trials

[0, trialsCompleted_) are done” is not a safe assumption once that can happen. maxTrial_ (see its own comment) exists precisely so a restart can resume numbering new trials at maxTrial_ + 1 – guaranteed never to collide with one already written – independently of how many more trials

restartTrialsDone() says are actually needed to reach SimControls::nTrial(). SimCluster::run()’s/ SimGalaxy::run()’s own comment has the full detail of how the two combine.

Returns:: restartMaxTrial_, as set by restartSetup() when this OutputManagerH5 was constructed with restart = true (0 if it was not, or if restartSetup() found no existing checkpoint to resume from)

inline virtual void notifyEarlyTermination(unsigned long trialsCompleted) override

Record that this run is ending early, having only actually completed trialsCompleted trials.

Stores trialsCompleted into earlyTerminationTrialsCompleted_, for the destructor to write as the currently-open file’s own final “trials_completed”/”restart_uid” attributes instead of SimControls::nTrial()/utils::uniqueID().read() – see the destructor’s own comment, and OutputManager:: notifyEarlyTermination()’s own comment for the full rationale. Does not itself close anything, or roll over to a new checkpoint: SimCluster::run()/SimGalaxy::run() call this and then simply return, leaving the currently-open checkpoint (or, without checkpointing, the run’s own single output file) for the ordinary destructor to close, same as it always does once this object goes out of scope.

Parameters:: trialsCompleted – Number of trials actually completed