Thread Manager

Overview

The ThreadManager provides a reusable and thread-safe framework for managing long-running background worker threads in Telescope Management Controller (TMC).

It centralizes thread lifecycle management and provides:

  • Thread registration

  • Thread creation using factories

  • Individual and bulk thread startup

  • Graceful shutdown using threading.Event

  • Thread restart support

  • Thread status reporting

  • Thread-safe operations using threading.Lock

The manager is generic and can be reused by any Component that requires long-running monitoring threads.

Typical use cases include:

  • Health monitoring

  • Healthstate monitoring

  • Aggregate process monitoring

  • GPM monitoring

  • Periodic background tasks

Design

ComponentManager
        │
        ▼
  ThreadManager
        │
        │
        ├──────────────┐
        │              │
        ▼              ▼
  ThreadInfo      ThreadInfo
        │              │
        ▼              ▼
  Factory()       Factory()
        │              │
        ▼              ▼
    Thread         Thread
        │              │
        ▼              ▼
  Worker Loop    Worker Loop
        │              │
        ▼              ▼
   stop_event     stop_event

Each managed worker owns its own threading.Event used for graceful shutdown.

Thread Lifecycle

register() ──► ThreadInfo (stores factory)
      │
      ▼
start() ──► Create stop_event + call factory ──► thread.start()
      │
      ▼
Worker Running
      │
      ▼
stop() ──► stop_event.set() + join() + cleanup
      │
      ▼
restart() ──► stop() + start() (new Thread)

Responsibilities

Component

Responsibility

ThreadManager

Manage lifecycle of worker threads

ThreadInfo

Store metadata for a managed thread

Factory

Create a new Thread instance

threading.Thread

Execute background work

threading.Event

Request graceful worker shutdown

threading.Lock

Protect shared manager state

Thread Factory

Unlike many thread management implementations, ThreadManager stores thread factories rather than threading.Thread instances.

A thread factory is a callable responsible for creating a new threading.Thread whenever the manager needs one.

Example:

def gpm_factory(stop_event):

    return threading.Thread(
        target=self._monitor_gpm,
        args=(stop_event,),
        name="gpm",
    )

The ThreadManager stores the factory:

"gpm" -> gpm_factory

rather than storing a permanently constructed Thread.

Why use a factory?

Python Thread objects can only be started once.

Once a thread has finished execution, it cannot be restarted.

The following is invalid:

thread.start()

...

thread.start()      # Raises RuntimeError

Instead, a completely new Thread object must be created.

By storing a factory rather than a Thread, the manager can always request a new worker whenever one is required.

Factory
   │
   ▼
Thread #1
   │
   ▼
Running
   │
   ▼
Stopped
   │
   ▼
Factory
   │
   ▼
Thread #2

This design naturally enables restart support without requiring the ThreadManager to understand how individual workers are constructed.

Why not create the Thread inside ThreadManager?

The ThreadManager is responsible for managing thread lifecycles, not constructing worker threads.

Different workers often require different initialization.

For example:

  • GPM monitor

  • Health monitor

  • Aggregate process monitoring

Each worker may require different constructor arguments such as:

  • ComponentManager

  • Logger

  • DeviceProxy

  • Configuration

Rather than embedding worker-specific knowledge inside ThreadManager, each worker provides its own factory.

The manager simply calls:

thread = factory(stop_event)

This keeps ThreadManager generic and independent of worker implementation details.

Benefits

Using thread factories provides several advantages:

  • Separates thread creation from thread management.

  • Allows different workers to construct threads differently.

  • Supports dependency injection.

  • Enables restart by creating a new Thread instance.

  • Makes unit testing easier by allowing custom factories.

  • Keeps ThreadManager reusable across different ComponentManagers.

Public API

register(name, factory)

Registers a worker thread factory with the manager.

Registration stores metadata only.

No thread is created or started.

Use this method when thread creation requires custom initialization, additional constructor arguments, or dependency injection.

Example:

thread_manager.register("gpm", gpm_factory)

start(name)

Creates and starts a registered worker.

The manager:

  • Creates a fresh threading.Event

  • Calls the registered factory

  • Starts the new thread

  • Records thread metadata

If the worker is already running, no additional thread is created.

start_all()

Starts every registered worker.

This is typically called during Component initialization.

stop(name)

Gracefully stops a worker.

The manager:

  • Signals the worker using stop_event.set()

  • Waits for completion using join()

  • Cleans up thread metadata

Workers are expected to periodically check:

while not stop_event.is_set():
    ...

or

stop_event.wait(timeout)

stop_all()

Stops every registered worker in reverse registration order.

Stopping in reverse order (LIFO) helps cleanly shut down workers that may depend on previously registered workers.

restart(name)

Restarts a worker.

Restart performs:

stop()
    │
    ▼
restart_count += 1
    │
    ▼
start()

A restart always creates a new Thread instance.

Since Python threads cannot be started twice, the manager invokes the registered factory to construct a replacement thread.

status()

Returns runtime information for every registered worker.

Information includes:

  • Running state

  • Thread identifier

  • Daemon status

  • Uptime

  • Restart count

This API is intended for diagnostics and debugging.

get_threads()

Returns currently active threading.Thread objects.

Only running workers are returned.

This method is useful when another component needs direct access to the active thread objects.

Convenience API

start_worker(name, target, daemon=False)

start_worker() is a convenience wrapper around register() and start(). It is intended for simple worker functions.

start_worker() is a convenience API for workers that use the standard threading.Thread constructor and only require a threading.Event for graceful shutdown.

Internally this method:

  • Creates a thread factory

  • Registers the worker

  • Starts the worker

Equivalent to:

register(...)
start(...)

This API reduces boilerplate for simple workers while reusing the same thread lifecycle management provided by register() and start().

Example:

def monitor(stop_event):
    while not stop_event.is_set():
        ...
        stop_event.wait(2)

thread_manager.start_worker(
    name="monitor",
    target=monitor,
)

When worker creation requires additional constructor arguments, register() with an explicit thread factory is generally preferred. Custom factories can inject worker-specific dependencies while keeping ThreadManager generic.

Examples of such dependencies include:

  • ComponentManager

  • Logger

  • DeviceProxy

  • Event subscriptions

  • Configuration objects

Example:

def gpm_factory(stop_event):
    return threading.Thread(
        target=self._monitor_gpm,
        args=(
            stop_event,
            self._logger,
            self._gpm_proxy,
        ),
        name="gpm",
    )

thread_manager.register("gpm", gpm_factory)

Using explicit factories provides greater flexibility while still allowing the ThreadManager to remain completely generic.

Choosing Between register() and start_worker()

Use register()

Use start_worker()

Workers requiring custom

Simple worker functions

thread creation

Additional constructor

Worker only requires stop_event

arguments

Dependency injection

Default threading.Thread creation

is required

is sufficient

Custom thread initialization

Convenience API for straightforward workers

Maximum flexibility

Minimal boilerplate

clear()

Stops all workers and removes every registration.

This API is intended for unit testing only.

Graceful Shutdown

Workers should periodically check their stop event.

Recommended pattern:

def monitor(stop_event):

    while not stop_event.is_set():

        poll_device()

        stop_event.wait(timeout=2)

Using stop_event.wait(timeout) allows workers to wake immediately when shutdown is requested while still maintaining a polling interval.

Best Practices

  • Register all workers during ComponentManager initialization.

  • Use start_all() during startup.

  • Use stop_all() during shutdown.

  • Prefer register() with custom thread creation.

  • Use start_worker() only for simple worker functions.

  • Ensure workers periodically check stop_event.

  • Avoid while True loops inside worker threads.

  • Keep worker logic independent from thread lifecycle management.

  • Never restart an existing Thread object.

  • Keep long-running work outside critical sections protected by locks.