Merge upstream/main into feature/integrate-lfq-tmt

Author: hjn0415a

.claude/skills/add-presets.md

@@ -0,0 +1,98 @@
+# Add Parameter Presets
+
+Add or modify parameter presets for a TOPP workflow in `presets.json`.
+
+## Instructions
+
+1. **Ask the user** for:
+   - Which workflow the presets are for
+   - Preset names and descriptions
+   - Parameter values for each preset (TOPP tool parameters and/or custom widget parameters)
+
+2. **Determine the workflow key** from the display name:
+   - Convert to lowercase, replace spaces with hyphens
+   - Example: "TOPP Workflow" -> "topp-workflow"
+   - Example: "Metabolite Analysis" -> "metabolite-analysis"
+
+3. **Read or create `presets.json`** at the repository root. Add entries following this schema:
+
+```json
+{
+  "workflow-name": {
+    "Preset Name": {
+      "_description": "Tooltip text shown on hover over the preset button",
+      "TOPPToolName": {
+        "algorithm:section:param_name": value
+      },
+      "_general": {
+        "custom-widget-key": value
+      }
+    }
+  }
+}
+```
+
+4. **Verify the result** by checking that:
+   - Workflow name key matches the name passed to `WorkflowManager.__init__()` (lowercased, hyphenated)
+   - TOPP tool names match those used in `input_TOPP()` calls
+   - Parameter paths use colon-separated format matching the TOPP tool's .ini structure
+   - `_general` keys match widget keys from `input_widget()` calls
+   - JSON is valid
+
+## Schema Rules
+
+- **Workflow key**: lowercase, hyphens — must match `WorkflowManager("Display Name", ...)` converted
+- **`_description`**: optional tooltip text for the preset button
+- **TOPP tool keys**: dictionary name must exactly match the TOPP tool name (e.g., `"FeatureFinderMetabo"`)
+- **TOPP parameter paths**: colon-separated, e.g., `"algorithm:common:noise_threshold_int"`
+- **`_general`**: overrides for custom `input_widget()` keys (non-TOPP parameters)
+- **Keys starting with `_`** are metadata and not applied as tool parameters
+
+## How Presets Work at Runtime
+
+1. Preset buttons auto-appear in `parameter_section()` via `StreamlitUI.preset_buttons()`
+2. Only presets matching the current workflow name are displayed
+3. Clicking a preset updates `params.json` in the workspace and refreshes the UI
+4. If no `presets.json` exists or no presets match, no buttons are shown
+
+## Reference Files
+
+- Existing presets: `presets.json`
+- Preset loading: `src/workflow/ParameterManager.py`
+- Preset buttons: `src/workflow/StreamlitUI.py` — `preset_buttons()` method
+- Documentation: `docs/build_app.md` (Parameter Presets section)
+
+## Example
+
+For a workflow initialized as `super().__init__("Feature Analysis", ...)`:
+
+```json
+{
+  "feature-analysis": {
+    "High Sensitivity": {
+      "_description": "Optimized for detecting low-abundance features",
+      "FeatureFinderMetabo": {
+        "algorithm:common:noise_threshold_int": 500.0,
+        "algorithm:common:chrom_peak_snr": 2.0
+      }
+    },
+    "Fast Processing": {
+      "_description": "Quick analysis with relaxed thresholds",
+      "FeatureFinderMetabo": {
+        "algorithm:common:noise_threshold_int": 2000.0
+      },
+      "_general": {
+        "run-python-script": false
+      }
+    }
+  }
+}
+```
+
+## Checklist
+
+- [ ] Workflow key is lowercase with hyphens, matching the WorkflowManager name
+- [ ] Each preset has a descriptive `_description`
+- [ ] TOPP tool names match exactly
+- [ ] Parameter paths use colon-separated format
+- [ ] `presets.json` is valid JSON

.claude/skills/add-python-tool.md

@@ -0,0 +1,118 @@
+# Add a Python Analysis Tool
+
+Add a custom Python analysis script for MS data processing to `src/python-tools/` with auto-generated UI.
+
+Python tools handle analysis steps that aren't covered by TOPP tools — e.g., exporting consensus features to DataFrames, computing differential expression statistics, custom metabolite annotation, or post-processing identification results.
+
+## Instructions
+
+1. **Ask the user** for:
+   - Tool name (lowercase, hyphens for spaces)
+   - What the tool does
+   - Input/output file types
+   - Parameters: name, type, default value, constraints
+
+2. **Create the tool script** at `src/python-tools/<name>.py`:
+
+```python
+import json
+import sys
+
+DEFAULTS = [
+    # Hidden I/O parameters (always include these)
+    {"key": "in", "value": [], "help": "Input files.", "hide": True},
+    {"key": "out", "value": [], "help": "Output files.", "hide": True},
+    # Visible parameters — examples of each type:
+    {
+        "key": "threshold",
+        "value": 1000.0,
+        "name": "Intensity Threshold",
+        "help": "Minimum intensity for filtering.",
+        "min": 0.0,
+        "max": 100000.0,
+        "step_size": 100.0,
+    },
+    {
+        "key": "method",
+        "value": "default",
+        "name": "Processing Method",
+        "options": ["default", "advanced", "custom"],
+        "help": "Which algorithm to use.",
+    },
+    {
+        "key": "num-features",
+        "value": 10,
+        "name": "Number of Features",
+        "widget_type": "slider",
+        "min": 1,
+        "max": 100,
+        "step_size": 1,
+        "help": "How many features to report.",
+    },
+    {
+        "key": "advanced-setting",
+        "value": 5,
+        "name": "Advanced Setting",
+        "help": "Only shown in advanced mode.",
+        "advanced": True,
+    },
+    {
+        "key": "enabled",
+        "value": True,
+        "name": "Enable Processing",
+    },
+]
+
+
+def get_params():
+    if len(sys.argv) > 1:
+        with open(sys.argv[1], "r") as f:
+            return json.load(f)
+    else:
+        return {}
+
+
+if __name__ == "__main__":
+    params = get_params()
+    # Tool logic here — read inputs, process, write outputs
+```
+
+3. **Wire into a workflow** by adding to the workflow's `configure()` and `execution()` methods:
+
+```python
+# In configure():
+self.ui.input_python("tool_name")
+
+# In execution():
+self.executor.run_python("tool_name", {"in": input_files})
+```
+
+## DEFAULTS Metadata Keys
+
+| Key | Required | Type | Description |
+|-----|----------|------|-------------|
+| `key` | Yes | str | Unique identifier for the parameter |
+| `value` | Yes | any | Default value (type determines widget: bool=checkbox, str with options=selectbox, number=number_input) |
+| `name` | No | str | Display name in the UI |
+| `help` | No | str | Tooltip text |
+| `hide` | No | bool | If `True`, parameter is not shown in UI (use for in/out files) |
+| `options` | No | list | Valid choices — renders as selectbox |
+| `min` | No | number | Minimum value for numeric inputs |
+| `max` | No | number | Maximum value for numeric inputs |
+| `step_size` | No | number | Step size for numeric inputs |
+| `widget_type` | No | str | Override widget type: `"slider"`, `"textarea"`, `"number"`, `"text"`, etc. |
+| `advanced` | No | bool | If `True`, only shown when user expands advanced parameters |
+
+## Reference Files
+
+- Example tool: `src/python-tools/example.py`
+- Another example: `src/python-tools/export_consensus_feature_df.py`
+- UI generation: `src/workflow/StreamlitUI.py` — `input_python()` method
+- Execution: `src/workflow/CommandExecutor.py` — `run_python()` method
+
+## Checklist
+
+- [ ] Script created in `src/python-tools/` with DEFAULTS list
+- [ ] `get_params()` function and `__main__` block included
+- [ ] `in` and `out` keys in DEFAULTS with `"hide": True`
+- [ ] Wired into workflow via `self.ui.input_python()` and `self.executor.run_python()`

.claude/skills/add-visualization.md

@@ -0,0 +1,134 @@
+# Add MS Data Visualization
+
+Add mass spectrometry data visualizations using pyopenms-viz or OpenMS-Insight components.
+
+MS data has specialized visualization needs: mass spectra (m/z vs intensity stick plots), chromatograms (RT vs intensity), 2D peak maps (RT vs m/z heatmaps), isotope patterns, fragment ion annotations, and statistical plots like volcano plots for differential expression.
+
+## Instructions
+
+1. **Ask the user** for:
+   - What data to visualize (spectra, chromatograms, peak maps, tables, heatmaps)
+   - Data size expectations (small/medium vs. large datasets with millions of points)
+   - Whether interactive cross-component linking is needed
+
+2. **Choose the right library**:
+
+| Use Case | Library | Why |
+|----------|---------|-----|
+| Quick spectrum/chromatogram/peak map plots | **pyopenms-viz** | Simple one-liner API, publication quality |
+| Large datasets (millions of points) | **OpenMS-Insight** | Server-side pagination, intelligent downsampling |
+| Interactive tables with pagination | **OpenMS-Insight** `Table` | Tabulator.js, CSV export, server-side filtering |
+| Cross-linked views (click table row → highlight in plot) | **OpenMS-Insight** | Shared `link_id` across components |
+| Standard Plotly plots (bar, scatter, etc.) | **plotly.express** directly | Already available, use with `show_fig()` |
+
+3. **Implement the visualization** in a page or workflow results section.
+
+## pyopenms-viz Patterns
+
+pyopenms-viz extends pandas DataFrames with MS-specific plot accessors. Always use the `plotly` backend for Streamlit.
+
+```python
+import pyopenms_viz
+from src.common.common import show_fig
+
+# Mass spectrum (stick plot from mz/intensity columns)
+fig = df.plot.ms_spectrum(backend="plotly", title="MS1 Spectrum")
+show_fig(fig, "spectrum-plot")
+
+# 2D peak map (RT vs m/z, colored by intensity)
+fig = df.plot.peak_map(backend="plotly")
+show_fig(fig, "peak-map")
+
+# Chromatogram (RT vs intensity)
+fig = df.plot.chromatogram(backend="plotly")
+show_fig(fig, "chromatogram")
+
+# Mobilogram (ion mobility vs intensity)
+fig = df.plot.mobilogram(backend="plotly")
+show_fig(fig, "mobilogram")
+```
+
+**Key points:**
+- DataFrame must have appropriate columns (e.g., `mz`, `intensity` for spectra)
+- Always use `backend="plotly"` in Streamlit context
+- Use `show_fig()` from `src/common/common.py` for consistent display with download buttons
+- Import `pyopenms_viz` to register the plot accessors (even if not used directly)
+
+## OpenMS-Insight Patterns
+
+OpenMS-Insight provides Vue.js-backed interactive components optimized for large MS datasets.
+
+```python
+from openms_insight import Table, LinePlot, Heatmap, VolcanoPlot, SequenceView
+
+# Interactive table with server-side pagination
+Table(df, key="results-table", page_size=50)
+
+# Stick-style mass spectrum
+LinePlot(df, x="mz", y="intensity", key="spectrum-plot")
+
+# 2D heatmap for large datasets (auto-downsampling)
+Heatmap(df, x="rt", y="mz", z="intensity", key="peak-heatmap")
+
+# Volcano plot for differential expression
+VolcanoPlot(df, x="log2fc", y="neg_log10_pval", key="volcano")
+
+# Peptide sequence with fragment ion annotations
+SequenceView(sequence="PEPTIDER", ions=ion_df, key="sequence")
+```
+
+### Cross-Component Linking
+
+Link components so selections in one update another:
+
+```python
+# Selecting a row in the table highlights the corresponding point in the plot
+Table(df, key="linked-table", link_id="feature_id")
+LinePlot(df, x="mz", y="intensity", key="linked-plot", link_id="feature_id")
+Heatmap(df, x="rt", y="mz", z="intensity", key="linked-heatmap", link_id="feature_id")
+```
+
+All components sharing the same `link_id` column are automatically synchronized.
+
+## Integration in Workflow Results
+
+Add visualization to a workflow's `results()` method:
+
+```python
+@st.fragment
+def results(self) -> None:
+    result_file = Path(self.workflow_dir, "results", "step-name", "output.tsv")
+    if result_file.exists():
+        df = pd.read_csv(result_file, sep="\t")
+
+        # pyopenms-viz for spectrum plots
+        import pyopenms_viz
+        fig = df.plot.ms_spectrum(backend="plotly")
+        show_fig(fig, "results-spectrum")
+
+        # Or OpenMS-Insight for interactive exploration
+        from openms_insight import Table
+        Table(df, key="results-table")
+    else:
+        st.warning("No results found. Please run the workflow first.")
+```
+
+## Reference Files
+
+- Display utilities: `src/common/common.py` — `show_fig()`, `show_table()`
+- Dependencies: `requirements.txt` — `pyopenms-viz`, `openms-insight`
+- Example workflow results: `src/Workflow.py` — `results()` method
+
+## Library Repositories
+
+- **pyopenms-viz**: OpenMS/pyopenms-viz — pandas DataFrame `.plot()` extension with matplotlib/bokeh/plotly backends
+- **OpenMS-Insight**: t0mdavid-m/openms-insight — Vue.js interactive Streamlit components with caching and downsampling
+
+## Checklist
+
+- [ ] Correct library chosen for the use case
+- [ ] Dependencies present in `requirements.txt`
+- [ ] `show_fig()` used for pyopenms-viz plots (consistent download/export behavior)
+- [ ] `backend="plotly"` specified for all pyopenms-viz plots
+- [ ] Unique `key=` values for all OpenMS-Insight components
+- [ ] Cross-component linking set up if multiple views of same data