{ "cells": [ { "cell_type": "markdown", "id": "72dc694e", "metadata": {}, "source": [ "# Step 2: Build and Review QC Inventory\n" ] }, { "cell_type": "markdown", "id": "fad9a8a6", "metadata": {}, "source": [ "This notebook creates the QC tables used to decide which records can move into metric calculations. Observed and synthetic records are tracked separately, and a comparison is only eligible when both sides pass for the requested metric, component, and period band.\n" ] }, { "cell_type": "markdown", "id": "bdcca858", "metadata": {}, "source": [ "## Imports\n", "\n", "These helpers build QC rows, create comparison-ready records, export a manual-review queue, launch the QC dashboard, and make QC overview figures.\n" ] }, { "cell_type": "code", "execution_count": 1, "id": "ae2622bf", "metadata": { "execution": { "iopub.execute_input": "2026-06-07T01:49:06.405455Z", "iopub.status.busy": "2026-06-07T01:49:06.405157Z", "iopub.status.idle": "2026-06-07T01:49:08.531314Z", "shell.execute_reply": "2026-06-07T01:49:08.530679Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Run time: 564.8 ms\n" ] } ], "source": [ "from spatial_vtk.config.notebook import notebook_timer, register_svtk_cell_timer\n", "\n", "with notebook_timer():\n", " from spatial_vtk.config import SpatialVTKConfig\n", " from spatial_vtk.config.metrics import metrics_settings_from_config\n", " from spatial_vtk.config.outputs import resolve_output_path\n", " from spatial_vtk.io import load_output_table, waveform_preprocessing_label, write_output_table\n", " from spatial_vtk.qc import (\n", " build_comparison_eligibility,\n", " build_event_station_pair_retention_table,\n", " build_retention_figure_table,\n", " build_metric_qc_summary,\n", " build_post_qc_record_table,\n", " build_qc_waveform_comparison_records,\n", " build_waveform_qc_summary,\n", " export_manual_review_queue,\n", " )\n", " from spatial_vtk.visualize.dashboard import launch_qc_dashboard\n", " from spatial_vtk.visualize.waveforms import plot_event_trace_comparison\n", " from spatial_vtk.visualize.qc import (\n", " plot_event_station_retention_heatmap,\n", " plot_post_qc_station_event_map,\n", " plot_qc_drop_cause_diagnostics,\n", " plot_retention_summary,\n", " )\n", " register_svtk_cell_timer()" ] }, { "cell_type": "markdown", "id": "8aa66e11", "metadata": {}, "source": [ "## Configuration\n", "\n", "Load the tutorial config and set the output folder for this QC step.\n" ] }, { "cell_type": "code", "execution_count": 2, "id": "ace06f88", "metadata": { "execution": { "iopub.execute_input": "2026-06-07T01:49:08.534340Z", "iopub.status.busy": "2026-06-07T01:49:08.534035Z", "iopub.status.idle": "2026-06-07T01:49:08.559113Z", "shell.execute_reply": "2026-06-07T01:49:08.558473Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Run time: 21.2 ms\n" ] } ], "source": [ "from pathlib import Path\n", "\n", "# Use the repository root so paths match the public source checkout.\n", "repo_root = Path.cwd()\n", "config_path = repo_root / \"data/examples/configuration/example_spatial_vtk_config.yaml\"\n", "\n", "# Load the tutorial run scenario and make it the active config for later package calls.\n", "cfg = SpatialVTKConfig.from_file(config_path, run_scenario=\"tutorial\").activate()\n", "\n", "notebook_overrides = {\"qc_dashboard_port\": 8502}" ] }, { "cell_type": "markdown", "id": "ebd32988", "metadata": {}, "source": [ "## Load the Step 1 Files\n", "\n", "Start from the prepared station, event, and event-station tables written by the first notebook." ] }, { "cell_type": "code", "execution_count": 3, "id": "4c561abc", "metadata": { "execution": { "iopub.execute_input": "2026-06-07T01:49:08.561815Z", "iopub.status.busy": "2026-06-07T01:49:08.561554Z", "iopub.status.idle": "2026-06-07T01:49:08.677641Z", "shell.execute_reply": "2026-06-07T01:49:08.677083Z" } }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Prepared stations: 30\n", "Prepared events: 5\n", "Event-station records: 150\n", "Run time: 113.0 ms\n" ] } ], "source": [ "# Read the prepared station table written by Step 1.\n", "stations = load_output_table(\"prepared_stations\")\n", "\n", "# Read the prepared event table written by Step 1.\n", "events = load_output_table(\"prepared_events\")\n", "\n", "# Read the event-station record table written by Step 1.\n", "event_stations = load_output_table(\"event_station_records\")\n", "\n", "print(f\"Prepared stations: {len(stations)}\")\n", "print(f\"Prepared events: {len(events)}\")\n", "print(f\"Event-station records: {len(event_stations)}\")" ] }, { "cell_type": "markdown", "id": "470300a6", "metadata": {}, "source": [ "Use the QC helpers to inspect the observed and synthetic traces, then apply those decisions to the metric-level QC table. Observed and synthetic records are checked separately. A record is available for comparison only when both sides pass." ] }, { "cell_type": "code", "execution_count": 4, "id": "4a9a8148", "metadata": { "execution": { "iopub.execute_input": "2026-06-07T01:49:08.680244Z", "iopub.status.busy": "2026-06-07T01:49:08.679998Z", "iopub.status.idle": "2026-06-07T01:49:21.936755Z", "shell.execute_reply": "2026-06-07T01:49:21.935661Z" } }, "outputs": [ { "data": { "text/html": [ "
| \n", " | source | \n", "event_id | \n", "station | \n", "event_title | \n", "event_lat | \n", "event_lon | \n", "station_lat | \n", "station_lon | \n", "network | \n", "magnitude | \n", "distance_km | \n", "component | \n", "passband | \n", "metric_group | \n", "metric | \n", "period_s | \n", "qc_status | \n", "qc_reason | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "Z | \n", "1-2 sec | \n", "amplitude | \n", "PGA | \n", "NaN | \n", "pass | \n", "\n", " |
| 1 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "Z | \n", "2-3 sec | \n", "amplitude | \n", "PGA | \n", "NaN | \n", "pass | \n", "\n", " |
| 2 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "Z | \n", "1-2 sec | \n", "amplitude | \n", "PGV | \n", "NaN | \n", "pass | \n", "\n", " |
| 3 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "Z | \n", "2-3 sec | \n", "amplitude | \n", "PGV | \n", "NaN | \n", "pass | \n", "\n", " |
| 4 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "Z | \n", "1-2 sec | \n", "amplitude | \n", "PGD | \n", "NaN | \n", "pass | \n", "\n", " |
| \n", " | source_observed | \n", "event_id | \n", "station | \n", "component | \n", "passband | \n", "metric_group | \n", "metric | \n", "period_s | \n", "qc_status_observed | \n", "qc_reason_observed | \n", "... | \n", "qc_status_synthetic | \n", "qc_reason_synthetic | \n", "event_title | \n", "event_lat | \n", "event_lon | \n", "station_lat | \n", "station_lon | \n", "network | \n", "magnitude | \n", "distance_km | \n", "
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| 0 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "Z | \n", "1-2 sec | \n", "amplitude | \n", "PGA | \n", "NaN | \n", "pass | \n", "\n", " | ... | \n", "pass | \n", "\n", " | ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "
| 1 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "Z | \n", "2-3 sec | \n", "amplitude | \n", "PGA | \n", "NaN | \n", "pass | \n", "\n", " | ... | \n", "pass | \n", "\n", " | ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "
| 2 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "Z | \n", "1-2 sec | \n", "amplitude | \n", "PGV | \n", "NaN | \n", "pass | \n", "\n", " | ... | \n", "pass | \n", "\n", " | ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "
| 3 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "Z | \n", "2-3 sec | \n", "amplitude | \n", "PGV | \n", "NaN | \n", "pass | \n", "\n", " | ... | \n", "pass | \n", "\n", " | ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "
| 4 | \n", "observed | \n", "ci38038071 | \n", "BFS | \n", "Z | \n", "1-2 sec | \n", "amplitude | \n", "PGD | \n", "NaN | \n", "pass | \n", "\n", " | ... | \n", "pass | \n", "\n", " | ci38038071 | \n", "34.136333 | \n", "-117.774667 | \n", "34.239 | \n", "-117.659 | \n", "CI | \n", "4.38 | \n", "15.60501 | \n", "
5 rows × 21 columns
\n", "