.. _thread-manager: Thread Manager ============== Overview -------- The **ThreadManager** provides a reusable and thread-safe framework for managing long-running background worker threads in Telescope Management Controller (TMC). It centralizes thread lifecycle management and provides: * Thread registration * Thread creation using factories * Individual and bulk thread startup * Graceful shutdown using ``threading.Event`` * Thread restart support * Thread status reporting * Thread-safe operations using ``threading.Lock`` The manager is generic and can be reused by any Component that requires long-running monitoring threads. Typical use cases include: * Health monitoring * Healthstate monitoring * Aggregate process monitoring * GPM monitoring * Periodic background tasks Design ------ .. code-block:: text ComponentManager │ ▼ ThreadManager │ │ ├──────────────┐ │ │ ▼ ▼ ThreadInfo ThreadInfo │ │ ▼ ▼ Factory() Factory() │ │ ▼ ▼ Thread Thread │ │ ▼ ▼ Worker Loop Worker Loop │ │ ▼ ▼ stop_event stop_event Each managed worker owns its own ``threading.Event`` used for graceful shutdown. Thread Lifecycle ---------------- .. code-block:: text register() ──► ThreadInfo (stores factory) │ ▼ start() ──► Create stop_event + call factory ──► thread.start() │ ▼ Worker Running │ ▼ stop() ──► stop_event.set() + join() + cleanup │ ▼ restart() ──► stop() + start() (new Thread) Responsibilities ---------------- =========================== ============================================================ Component Responsibility =========================== ============================================================ ThreadManager Manage lifecycle of worker threads ThreadInfo Store metadata for a managed thread Factory Create a new Thread instance threading.Thread Execute background work threading.Event Request graceful worker shutdown threading.Lock Protect shared manager state =========================== ============================================================ Thread Factory -------------- Unlike many thread management implementations, ThreadManager stores **thread factories** rather than ``threading.Thread`` instances. A thread factory is a callable responsible for creating a new ``threading.Thread`` whenever the manager needs one. Example:: def gpm_factory(stop_event): return threading.Thread( target=self._monitor_gpm, args=(stop_event,), name="gpm", ) The ThreadManager stores the factory: :: "gpm" -> gpm_factory rather than storing a permanently constructed Thread. Why use a factory? ^^^^^^^^^^^^^^^^^^ Python Thread objects can only be started once. Once a thread has finished execution, it cannot be restarted. The following is invalid:: thread.start() ... thread.start() # Raises RuntimeError Instead, a completely new Thread object must be created. By storing a factory rather than a Thread, the manager can always request a new worker whenever one is required. .. code-block:: text Factory │ ▼ Thread #1 │ ▼ Running │ ▼ Stopped │ ▼ Factory │ ▼ Thread #2 This design naturally enables restart support without requiring the ThreadManager to understand how individual workers are constructed. Why not create the Thread inside ThreadManager? ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ The ThreadManager is responsible for managing thread lifecycles, not constructing worker threads. Different workers often require different initialization. For example: * GPM monitor * Health monitor * Aggregate process monitoring Each worker may require different constructor arguments such as: * ComponentManager * Logger * DeviceProxy * Configuration Rather than embedding worker-specific knowledge inside ThreadManager, each worker provides its own factory. The manager simply calls:: thread = factory(stop_event) This keeps ThreadManager generic and independent of worker implementation details. Benefits ^^^^^^^^ Using thread factories provides several advantages: * Separates thread creation from thread management. * Allows different workers to construct threads differently. * Supports dependency injection. * Enables restart by creating a new Thread instance. * Makes unit testing easier by allowing custom factories. * Keeps ThreadManager reusable across different ComponentManagers. Public API ---------- register(name, factory) ^^^^^^^^^^^^^^^^^^^^^^^^ Registers a worker thread factory with the manager. Registration stores metadata only. No thread is created or started. Use this method when thread creation requires custom initialization, additional constructor arguments, or dependency injection. Example:: thread_manager.register("gpm", gpm_factory) start(name) ^^^^^^^^^^^ Creates and starts a registered worker. The manager: * Creates a fresh ``threading.Event`` * Calls the registered factory * Starts the new thread * Records thread metadata If the worker is already running, no additional thread is created. start_all() ^^^^^^^^^^^ Starts every registered worker. This is typically called during Component initialization. stop(name) ^^^^^^^^^^ Gracefully stops a worker. The manager: * Signals the worker using ``stop_event.set()`` * Waits for completion using ``join()`` * Cleans up thread metadata Workers are expected to periodically check: :: while not stop_event.is_set(): ... or :: stop_event.wait(timeout) stop_all() ^^^^^^^^^^ Stops every registered worker in reverse registration order. Stopping in reverse order (LIFO) helps cleanly shut down workers that may depend on previously registered workers. restart(name) ^^^^^^^^^^^^^ Restarts a worker. Restart performs: .. code-block:: text stop() │ ▼ restart_count += 1 │ ▼ start() A restart always creates a **new Thread instance**. Since Python threads cannot be started twice, the manager invokes the registered factory to construct a replacement thread. status() ^^^^^^^^ Returns runtime information for every registered worker. Information includes: * Running state * Thread identifier * Daemon status * Uptime * Restart count This API is intended for diagnostics and debugging. get_threads() ^^^^^^^^^^^^^ Returns currently active ``threading.Thread`` objects. Only running workers are returned. This method is useful when another component needs direct access to the active thread objects. Convenience API --------------- start_worker(name, target, daemon=False) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ start_worker() is a convenience wrapper around register() and start(). It is intended for simple worker functions. ``start_worker()`` is a convenience API for workers that use the standard ``threading.Thread`` constructor and only require a ``threading.Event`` for graceful shutdown. Internally this method: * Creates a thread factory * Registers the worker * Starts the worker Equivalent to: :: register(...) start(...) This API reduces boilerplate for simple workers while reusing the same thread lifecycle management provided by ``register()`` and ``start()``. Example:: def monitor(stop_event): while not stop_event.is_set(): ... stop_event.wait(2) thread_manager.start_worker( name="monitor", target=monitor, ) When worker creation requires additional constructor arguments, ``register()`` with an explicit thread factory is generally preferred. Custom factories can inject worker-specific dependencies while keeping ``ThreadManager`` generic. Examples of such dependencies include: * ComponentManager * Logger * DeviceProxy * Event subscriptions * Configuration objects Example:: def gpm_factory(stop_event): return threading.Thread( target=self._monitor_gpm, args=( stop_event, self._logger, self._gpm_proxy, ), name="gpm", ) thread_manager.register("gpm", gpm_factory) Using explicit factories provides greater flexibility while still allowing the ThreadManager to remain completely generic. Choosing Between register() and start_worker() ---------------------------------------------- ============================= ====================================================== Use register() Use start_worker() ============================= ====================================================== Workers requiring custom Simple worker functions thread creation Additional constructor Worker only requires ``stop_event`` arguments Dependency injection Default ``threading.Thread`` creation is required is sufficient Custom thread initialization Convenience API for straightforward workers Maximum flexibility Minimal boilerplate ============================= ====================================================== clear() ^^^^^^^ Stops all workers and removes every registration. This API is intended for unit testing only. Graceful Shutdown ----------------- Workers should periodically check their stop event. Recommended pattern:: def monitor(stop_event): while not stop_event.is_set(): poll_device() stop_event.wait(timeout=2) Using ``stop_event.wait(timeout)`` allows workers to wake immediately when shutdown is requested while still maintaining a polling interval. Best Practices -------------- * Register all workers during ComponentManager initialization. * Use ``start_all()`` during startup. * Use ``stop_all()`` during shutdown. * Prefer ``register()`` with custom thread creation. * Use ``start_worker()`` only for simple worker functions. * Ensure workers periodically check ``stop_event``. * Avoid ``while True`` loops inside worker threads. * Keep worker logic independent from thread lifecycle management. * Never restart an existing ``Thread`` object. * Keep long-running work outside critical sections protected by locks.