.. _sdm_mode:

**************
SDM Mode Guide
**************

This section is intended for running the Batch Pre-processing Pipeline (BPP)
within the SDP production system. It describes how to run BPP in **SDM mode**,
where the pipeline exchanges data with other pipelines via a shared
Science Data Model (SDM) directory.

If you are running BPP as a standalone tool — reading Measurement Sets from
disk and writing outputs to an output directory of your choice — see the
:ref:`quickstart` and :ref:`configuration` instead.

.. note::

   The Demixer step is not available in SDM Mode yet.


Overview
========

In SDM mode, BPP:

- Reads calibration tables and other step inputs **from the SDM directory**.
- Writes logs, QA products and the run configuration **into the SDM
  directory**, under a uniquely-indexed subdirectory of ``<sdm>/logs/``.
- Writes processed Measurement Sets to the ``--output-dir`` as usual.


CLI Invocation
==============

Pass ``--sdm-path`` for SDM mode:

.. code-block:: text

   ska-sdp-batch-preprocess run \
       --config myConfig.yaml \
       --output-dir /path/to/output_dir \
       --sdm-path /path/to/sdm_directory \
       input1.ms input2.ms ...


Output Directory Conventions
=============================

When ``--sdm-path`` is provided, a new indexed subdirectory is created under
``<sdm>/logs/``:

.. code-block:: text

   <sdm>/
   └── logs/
       └── NN-bpp/   # NN = next available index (01, 02, …)
           ├── pipeline.log
           ├── config.yaml
           ├── task-list.json
           ├── dask-report.html
           └── qa/
               ├── <input_stem>_flagging_report.zarr
               └── <input_stem>_flagging_report.png

Processed Measurement Sets are written to ``--output-dir`` as in standalone
mode.


Configuring Steps in SDM Mode
==============================

Applycal
--------

In SDM mode, specify a ``table`` block with ``kind: sdm``, ``field_id`` and
``purpose``. BPP resolves the calibration table at:

.. code-block:: text

   <sdm>/calibration/<purpose>/<field_id>/gaintable.h5parm

Example configuration:

.. code-block:: yaml

   steps:
     - step: applycal
       table:
         kind: sdm
         field_id: target_field
         purpose: bandpass

AOFlagger
---------

Strategy presets should be used when using the AOFlagger step in SDM mode.
See :ref:`pipeline_steps` and :doc:`aoflagger_presets` for details.


Demixer
-------

.. note::

   Sky model lookup from the SDM is not yet implemented.
   Further adjustments to the SDM and sky model schemas must be discussed before
   SDM-mode Demixer is implemented.