Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ Use `opensampl config show` to inspect the current resolved configuration.

## CLI

The main CLI exposes `collect`, `config`, `create`, `init`, and `load`.
The main CLI exposes `collect`, `config`, `create`, `init`, `load`, and `sdk`.
Use `opensampl --help` and `opensampl <command> --help` for current options.

If you plan to use the NTP, Microchip TWST, or Microchip TP4100 collectors, install the optional collection dependencies:
Expand Down
65 changes: 42 additions & 23 deletions docs/guides/create_probe_type.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,10 +7,13 @@ probe family inside a local clone of the repository.

1. Clone the repository locally.
2. Install OpenSAMPL in the development environment.
3. Run `opensampl create` to generate the scaffold.
4. include `--collect-mixin` flag if you intend to implement timing collection as well to prefill those functions
4. Fill in the generated parser, metadata model, and any collector mixins you need.
5. Run `opensampl init` or `opensampl create --update-db ...` to create the new tables in the database.
3. Run `opensampl sdk template probe.yaml` to create a starter configuration.
4. Edit `probe.yaml` for the new clock probe type.
5. Run `opensampl sdk create probe.yaml` to generate the scaffold. Include the
`--collect-mixin` flag to prefill the collection functions.
6. Fill in the generated parser, metadata model, and any collector mixins you need.
7. Run `opensampl init` or include `--update-db` when running `opensampl sdk create`
to create the new tables in the database.

```bash
git clone git@github.com:ORNL/OpenSAMPL.git
Expand All @@ -25,7 +28,16 @@ add any required schema or migration updates alongside the generated code.

## Usage

Command: `opensampl create <CONFIG PATH> [OPTIONS]`
Create a starter configuration without replacing an existing file:

`opensampl sdk template <CONFIG PATH>`

Generate a probe scaffold from the edited configuration:

`opensampl sdk create <CONFIG PATH> [OPTIONS]`

The original `opensampl create <CONFIG PATH> [OPTIONS]` command remains available
as a compatibility alias.

Arguments:

Expand All @@ -34,6 +46,7 @@ Arguments:
Options:

* `--update-db` (`-u`): Update the database with the new probe type
* `--collect-mixin` (`-c`): Include a shell for implementing probe collection

## Config File Formatting

Expand All @@ -52,10 +65,10 @@ it is `f'{name.capitalize()}Metadata'`.
`metadata_table`: Optional. The database table name for the metadata table. By default it is
`f'{name.lower()}_metadata'`.

`metadata_fields`: A dictionary of metadata fields that will be provided for your new probe type.
`metadata_fields`: A list of metadata fields that will be provided for your new probe type.

* The keys become column names in the generated metadata table.
* The values are optional SQLAlchemy type names. If omitted, the field defaults to `Text`.
* Each entry has a required `name`, which becomes a column in the generated metadata table.
* Each entry can have an optional `type`, which defaults to `Text` when omitted.


For a concrete example, this is the configuration that would scaffold the existing ADVA
Expand All @@ -67,19 +80,25 @@ parser_module: adva
metadata_orm: AdvaMetadata
metadata_table: adva_metadata
metadata_fields:
type:
start: TIMESTAMP
frequency: Integer
timemultiplier: Integer
multiplier: Integer
title:
adva_probe:
adva_reference:
adva_reference_expected_ql:
adva_source:
adva_direction:
adva_version: Float
adva_status:
adva_mtie_mask:
adva_mask_margin: Integer
- name: type
- name: start
type: TIMESTAMP
- name: frequency
type: Integer
- name: timemultiplier
type: Integer
- name: multiplier
type: Integer
- name: title
- name: adva_probe
- name: adva_reference
- name: adva_reference_expected_ql
- name: adva_source
- name: adva_direction
- name: adva_version
type: Float
- name: adva_status
- name: adva_mtie_mask
- name: adva_mask_margin
type: Integer
```
11 changes: 8 additions & 3 deletions docs/guides/opensampl-cli.md
Original file line number Diff line number Diff line change
Expand Up @@ -103,15 +103,20 @@ Arguments:

## Create
**<mark>Experimental</mark>**
Create a new probe type scaffold from a configuration file. See the
Create a template configuration and use it to scaffold a new probe type. See the
[Create page](create_probe_type.md) for the current workflow and limitations.

Command: `opensampl create <CONFIG PATH> [OPTIONS]` <br>
Commands:

* `opensampl sdk template <CONFIG PATH>`: Write a starter YAML configuration without overwriting an existing file
* `opensampl sdk create <CONFIG PATH> [OPTIONS]`: Create the probe scaffold from the edited configuration
* `opensampl create <CONFIG PATH> [OPTIONS]`: Compatibility alias for `opensampl sdk create`

Arguments:

* `CONFIG PATH`: The path to the config file defining the new probe type

Options:

* `--update-db` (`-u`): Update the database with the new probe type

* `--collect-mixin` (`-c`): Include a shell for implementing probe collection
23 changes: 23 additions & 0 deletions opensampl/cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -267,6 +267,25 @@ def table_load(filepath: dict | list, table_name: str, if_exists: Literal["updat
raise click.Abort() # noqa: RSE102,B904


@cli.group()
def sdk():
"""Develop custom clock probe types for openSAMPL."""


@sdk.command(name="template")
@click.argument("config_path", type=click.Path(exists=False, dir_okay=False, path_type=Path))
def create_config_template_command(config_path: Path):
"""Write an editable probe configuration template to CONFIG_PATH."""
from opensampl.create.create_vendor import write_config_template

try:
write_config_template(config_path)
except OSError as exc:
raise click.ClickException(f"Could not create config template at {config_path}: {exc}") from exc

click.echo(f"Created probe configuration template at {config_path}")


@cli.command(name="create")
@click.argument("config_path", type=click.Path(exists=True, path_type=Path))
@click.option(
Expand All @@ -284,6 +303,7 @@ def table_load(filepath: dict | list, table_name: str, if_exists: Literal["updat
def create_probe_command(config_path: Path, update_db: bool, collect_mixin: bool):
"""Create a new probe type with scaffolding, based on a config file."""
from opensampl.create.create_vendor import VendorConfig

# TODO figure out best way to allow Vendor Config be through cli flags (too complicated nesting for pydanclick)

vendor_config = VendorConfig.from_config_file(config_path)
Expand All @@ -292,6 +312,9 @@ def create_probe_command(config_path: Path, update_db: bool, collect_mixin: bool
create_new_tables()


sdk.add_command(create_probe_command, name="create")


if __name__ == "__main__":
try:
cli()
Expand Down
13 changes: 13 additions & 0 deletions opensampl/create/create_vendor.py
Original file line number Diff line number Diff line change
Expand Up @@ -23,6 +23,19 @@
from opensampl.create.insert_markers import INSERT_MARKERS, InsertMarker
from opensampl.vendors.constants import VendorType

CONFIG_TEMPLATE_PATH = Path(__file__).parent / "templates" / "vendor_config.yaml"


def write_config_template(config_path: str | Path) -> Path:
"""Write an editable vendor configuration template without replacing an existing file."""
if isinstance(config_path, str):
config_path = Path(config_path)

with config_path.open("x", encoding="utf-8") as config_file:
config_file.write(CONFIG_TEMPLATE_PATH.read_text(encoding="utf-8"))

return config_path


class MetadataField(BaseModel):
"""
Expand Down
35 changes: 35 additions & 0 deletions opensampl/create/templates/vendor_config.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# openSAMPL clock probe configuration
#
# Generate a probe scaffold after editing this file:
# opensampl sdk create path/to/config.yaml

# Required: human-readable name for the probe type. The optional identifiers below
# are derived from this value when omitted.
name: My Vendor

# Optional: Python class name for the probe parser.
# Default for "My Vendor": MyVendorProbe
# parser_class: MyVendorProbe

# Optional: Python module path under opensampl/vendors/.
# Default for "My Vendor": my_vendor
# parser_module: my_vendor

# Optional: SQLAlchemy ORM class for the probe metadata table.
# Default for "My Vendor": MyVendorMetadata
# metadata_orm: MyVendorMetadata

# Optional: database table for probe metadata.
# Default for "My Vendor": my_vendor_metadata
# metadata_table: my_vendor_metadata

# Optional: vendor-specific metadata columns. Each entry requires a name and may
# specify a SQLAlchemy column type; omitted types default to Text. Common types
# include Text, String, Integer, Float, Boolean, DateTime, JSONB, and Numeric.
# The generator supplies probe_uuid and additional_metadata automatically.
metadata_fields:
- name: serial_number
type: Text
- name: firmware_version
- name: sample_rate_hz
type: Integer
Comment on lines +30 to +35

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Should probably leave this commented out too, so that users don't end up adding these two three random columns

77 changes: 76 additions & 1 deletion tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,81 @@ def test_cli_create_command(self, runner):

assert result.exit_code == 0

def test_cli_sdk_commands(self, runner):
"""Test the SDK command group and its subcommands."""
result = runner.invoke(cli, ["sdk", "--help"])

assert result.exit_code == 0
assert "create" in result.output
assert "template" in result.output

create_help = runner.invoke(cli, ["sdk", "create", "--help"])

assert create_help.exit_code == 0
assert "--collect-mixin" in create_help.output
assert "--update-db" in create_help.output

def test_cli_sdk_template_creates_valid_config(self, runner, tmp_path):
"""The SDK template command should create a config accepted by VendorConfig."""
from opensampl.create.create_vendor import VendorConfig

config_path = tmp_path / "probe.yaml"

result = runner.invoke(cli, ["sdk", "template", str(config_path)])

assert result.exit_code == 0
assert config_path.is_file()
assert str(config_path) in result.output

config = VendorConfig.from_config_file(config_path)
assert config.name == "My Vendor"
assert config.parser_class == "MyVendorProbe"
assert config.parser_module == "my_vendor"
assert {field.name for field in config.metadata_fields} == {
"serial_number",
"firmware_version",
"sample_rate_hz",
"additional_metadata",
}

def test_cli_sdk_template_does_not_overwrite_existing_file(self, runner, tmp_path):
"""The SDK template command should leave an existing destination untouched."""
config_path = tmp_path / "probe.yaml"
original_content = "user-owned content\n"
config_path.write_text(original_content)

result = runner.invoke(cli, ["sdk", "template", str(config_path)])

assert result.exit_code != 0
assert config_path.read_text() == original_content

def test_cli_sdk_template_does_not_create_parent_directories(self, runner, tmp_path):
"""The SDK template command should fail when the destination parent is missing."""
config_path = tmp_path / "missing" / "probe.yaml"

result = runner.invoke(cli, ["sdk", "template", str(config_path)])

assert result.exit_code != 0
assert "Could not create config template" in result.output
assert not config_path.parent.exists()

@patch("opensampl.create.create_vendor.VendorConfig.from_config_file")
def test_cli_sdk_create_matches_top_level_create(self, mock_from_config, runner, tmp_path):
"""SDK and top-level create commands should invoke the same scaffolding behavior."""
config_path = tmp_path / "probe.yaml"
config_path.write_text("name: Test Probe\nmetadata_fields: []\n")
vendor_config = Mock()
mock_from_config.return_value = vendor_config

root_result = runner.invoke(cli, ["create", str(config_path), "--collect-mixin"])
sdk_result = runner.invoke(cli, ["sdk", "create", str(config_path), "--collect-mixin"])

assert root_result.exit_code == 0
assert sdk_result.exit_code == 0
assert mock_from_config.call_count == 2
assert vendor_config.create.call_count == 2
vendor_config.create.assert_called_with(collect_mixin=True)

def test_cli_config_command(self, runner):
"""Test the config command."""
result = runner.invoke(cli, ['config', '--help'])
Expand Down Expand Up @@ -185,4 +260,4 @@ def test_cli_case_insensitive_commands(self, runner):
result3 = runner.invoke(cli, ['load', 'Table', '--help'])

# All should work the same
assert result1.exit_code == result2.exit_code == result3.exit_code == 0
assert result1.exit_code == result2.exit_code == result3.exit_code == 0