Streaming NEXRAD Level 2 Chunks from S3#
xradar can now ingest a list of NEXRAD Level 2 chunk byte objects directly, so you can stream real-time radar data from S3 without downloading full volume files first. This notebook demonstrates:
Listing and downloading chunk files from the
unidata-nexrad-level2-chunksbucketOpening a full volume assembled from all chunks
Handling partial volumes with
incomplete_sweep="drop"(default)Handling partial volumes with
incomplete_sweep="pad"Early streaming with just a few chunks
import warnings
import cmweather # noqa: F401 -- registers colormaps
import fsspec
import matplotlib.pyplot as plt
import numpy as np
import xradar as xd
Background: NEXRAD chunk files on S3#
NOAA publishes NEXRAD Level 2 data to two public S3 buckets:
Bucket |
Content |
Latency |
|---|---|---|
|
Complete volume files |
Minutes after scan |
|
Real-time chunk files |
Seconds after scan |
Each radar volume is split into many small chunk files that arrive as the radar scans. A volume directory typically contains:
One S (start) chunk that includes the volume header
Many I (intermediate) chunks with sweep data
One E (end) chunk marking the volume boundary
For example:
KABR/903/KABR20250717_120038_V06_S (start)
KABR/903/KABR20250717_120038_V06_I02 (intermediate)
KABR/903/KABR20250717_120038_V06_I03
...
KABR/903/KABR20250717_120038_V06_E (end)
xradar accepts a list of chunk bytes (or file paths, or file-like objects)
directly via open_nexradlevel2_datatree(). The chunks are concatenated
internally, so you never need to assemble them manually.
Load chunk data#
chunk_paths = []
try:
fs = fsspec.filesystem("s3", anon=True)
volumes = sorted(fs.ls("unidata-nexrad-level2-chunks/KABR/"))
if volumes:
chunk_paths = sorted(fs.ls(volumes[-1]))
except Exception:
pass
if chunk_paths:
print(f"Found {len(chunk_paths)} live S3 chunks")
for p in chunk_paths[:3]:
print(f" {p.split('/')[-1]}")
print(f" ... {chunk_paths[-1].split('/')[-1]}")
else:
print("S3 bucket unreachable, using open-radar-data fixture")
/home/docs/checkouts/readthedocs.org/user_builds/xradar/conda/latest/lib/python3.13/site-packages/fsspec/registry.py:305: UserWarning: Your installed version of s3fs is very old and known to cause
severe performance issues, see also https://github.com/dask/dask/issues/10276
To fix, you should specify a lower version bound on s3fs, or
update the current installation.
warnings.warn(s3_msg)
Found 55 live S3 chunks
20260517-180847-001-S
20260517-180847-002-I
20260517-180847-003-I
... 20260517-180847-055-E
Download / load chunk bytes#
Real-time S3 chunks age out one at a time, so the most recent volume can
be missing the start (S) chunk that carries the AR2V volume header.
open_nexradlevel2_datatree raises ValueError in that case, so we let
it validate the listing for us and fall back to the open-radar-data
fixture when it rejects the bytes.
all_bytes = None
if chunk_paths:
candidate = [fs.open(p, "rb").read() for p in chunk_paths]
try:
xd.io.open_nexradlevel2_datatree(candidate)
all_bytes = candidate
except ValueError as e:
print(f"S3 listing rejected: {e}")
if all_bytes is None:
import tarfile
from pathlib import Path
from open_radar_data import DATASETS
archive = DATASETS.fetch("nexrad_level2_chunks_KLOT.tar.gz")
with tarfile.open(archive) as tar:
tar.extractall("/tmp/nexrad_chunks", filter="data")
chunk_files = sorted(Path("/tmp/nexrad_chunks/nexrad_chunks_KLOT").iterdir())
all_bytes = [f.read_bytes() for f in chunk_files]
total_mb = sum(len(b) for b in all_bytes) / 1e6
print(f"Loaded {len(all_bytes)} chunks ({total_mb:.1f} MB total)")
Loaded 55 chunks (8.7 MB total)
Full volume from all chunks#
When all chunks (S through E) are available, passing the list to
open_nexradlevel2_datatree produces the same result as opening a
complete volume file.
dtree = xd.io.open_nexradlevel2_datatree(all_bytes)
display(dtree)
<xarray.DataTree>
Group: /
│ Dimensions: ()
│ Coordinates:
│ latitude float64 8B 45.46
│ longitude float64 8B -98.41
│ altitude int64 8B 421
│ Data variables:
│ volume_number int64 8B 0
│ platform_type <U5 20B 'fixed'
│ instrument_type <U5 20B 'radar'
│ time_coverage_start <U20 80B '2026-05-17T18:08:47Z'
│ time_coverage_end <U20 80B '2026-05-17T18:12:28Z'
│ Attributes: (12/25)
│ Conventions: None
│ instrument_name: KABR
│ version: None
│ title: None
│ institution: None
│ references: None
│ ... ...
│ avset_enabled: True
│ ebc_enabled: True
│ super_res_status: 2
│ rda_build_number: 2400
│ operational_mode: 4
│ actual_elevation_cuts: 12
├── Group: /sweep_0
│ Dimensions: (azimuth: 720, range: 1832)
│ Coordinates:
│ * azimuth (azimuth) float64 6kB 0.2225 0.7388 1.255 ... 359.2 359.7
│ elevation (azimuth) float64 6kB 0.4395 0.4395 ... 0.4395 0.4395
│ time (azimuth) datetime64[ns] 6kB ...
│ range (range) float32 7kB 2.125e+03 2.375e+03 ... 4.599e+05
│ Data variables:
│ DBZH (azimuth, range) float64 11MB ...
│ ZDR (azimuth, range) float64 11MB ...
│ PHIDP (azimuth, range) float64 11MB ...
│ RHOHV (azimuth, range) float64 11MB ...
│ CCORH (azimuth, range) float64 11MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 0
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 0.4834
│ Attributes:
│ waveform_type: contiguous_surveillance
│ channel_config: sz2_phase_coding
│ super_resolution: 11
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_1
│ Dimensions: (azimuth: 720, range: 1192)
│ Coordinates:
│ * azimuth (azimuth) float64 6kB 0.2225 0.7278 1.225 ... 359.2 359.7
│ elevation (azimuth) float64 6kB 0.4395 0.4395 ... 0.4395 0.4395
│ time (azimuth) datetime64[ns] 6kB ...
│ range (range) float32 5kB 2.125e+03 2.375e+03 ... 2.999e+05
│ Data variables:
│ DBZH (azimuth, range) float64 7MB ...
│ VRADH (azimuth, range) float64 7MB ...
│ WRADH (azimuth, range) float64 7MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 1
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 0.4834
│ Attributes:
│ waveform_type: contiguous_doppler
│ channel_config: sz2_phase_coding
│ super_resolution: 7
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_2
│ Dimensions: (azimuth: 720, range: 1832)
│ Coordinates:
│ * azimuth (azimuth) float64 6kB 0.2417 0.7224 1.252 ... 359.2 359.8
│ elevation (azimuth) float64 6kB 0.835 0.835 0.835 ... 0.835 0.835
│ time (azimuth) datetime64[ns] 6kB ...
│ range (range) float32 7kB 2.125e+03 2.375e+03 ... 4.599e+05
│ Data variables:
│ DBZH (azimuth, range) float64 11MB ...
│ ZDR (azimuth, range) float64 11MB ...
│ PHIDP (azimuth, range) float64 11MB ...
│ RHOHV (azimuth, range) float64 11MB ...
│ CCORH (azimuth, range) float64 11MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 2
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 0.8789
│ Attributes:
│ waveform_type: contiguous_surveillance
│ channel_config: sz2_phase_coding
│ super_resolution: 11
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_3
│ Dimensions: (azimuth: 720, range: 1192)
│ Coordinates:
│ * azimuth (azimuth) float64 6kB 0.228 0.7278 1.211 ... 359.2 359.7
│ elevation (azimuth) float64 6kB 0.835 0.835 0.835 ... 0.835 0.835
│ time (azimuth) datetime64[ns] 6kB ...
│ range (range) float32 5kB 2.125e+03 2.375e+03 ... 2.999e+05
│ Data variables:
│ DBZH (azimuth, range) float64 7MB ...
│ VRADH (azimuth, range) float64 7MB ...
│ WRADH (azimuth, range) float64 7MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 3
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 0.8789
│ Attributes:
│ waveform_type: contiguous_doppler
│ channel_config: sz2_phase_coding
│ super_resolution: 7
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_4
│ Dimensions: (azimuth: 720, range: 1712)
│ Coordinates:
│ * azimuth (azimuth) float64 6kB 0.2692 0.7416 1.269 ... 359.3 359.7
│ elevation (azimuth) float64 6kB 1.23 1.23 1.23 ... 1.23 1.23 1.23
│ time (azimuth) datetime64[ns] 6kB ...
│ range (range) float32 7kB 2.125e+03 2.375e+03 ... 4.299e+05
│ Data variables:
│ DBZH (azimuth, range) float64 10MB ...
│ ZDR (azimuth, range) float64 10MB ...
│ PHIDP (azimuth, range) float64 10MB ...
│ RHOHV (azimuth, range) float64 10MB ...
│ CCORH (azimuth, range) float64 10MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 4
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 1.274
│ Attributes:
│ waveform_type: contiguous_surveillance
│ channel_config: sz2_phase_coding
│ super_resolution: 11
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_5
│ Dimensions: (azimuth: 720, range: 1192)
│ Coordinates:
│ * azimuth (azimuth) float64 6kB 0.2197 0.7306 1.198 ... 359.2 359.7
│ elevation (azimuth) float64 6kB 1.23 1.23 1.23 ... 1.23 1.23 1.23
│ time (azimuth) datetime64[ns] 6kB ...
│ range (range) float32 5kB 2.125e+03 2.375e+03 ... 2.999e+05
│ Data variables:
│ DBZH (azimuth, range) float64 7MB ...
│ VRADH (azimuth, range) float64 7MB ...
│ WRADH (azimuth, range) float64 7MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 5
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 1.274
│ Attributes:
│ waveform_type: contiguous_doppler
│ channel_config: sz2_phase_coding
│ super_resolution: 7
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_6
│ Dimensions: (azimuth: 360, range: 1536)
│ Coordinates:
│ * azimuth (azimuth) float64 3kB 0.596 1.596 2.59 ... 358.6 359.6
│ elevation (azimuth) float64 3kB 1.758 1.758 1.758 ... 1.758 1.758
│ time (azimuth) datetime64[ns] 3kB ...
│ range (range) float32 6kB 2.125e+03 2.375e+03 ... 3.859e+05
│ Data variables:
│ DBZH (azimuth, range) float64 4MB ...
│ VRADH (azimuth, range) float64 4MB ...
│ WRADH (azimuth, range) float64 4MB ...
│ ZDR (azimuth, range) float64 4MB ...
│ PHIDP (azimuth, range) float64 4MB ...
│ RHOHV (azimuth, range) float64 4MB ...
│ CCORH (azimuth, range) float64 4MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 6
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 1.802
│ Attributes:
│ waveform_type: staggered_pulse_pair
│ channel_config: constant_phase
│ super_resolution: 14
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_7
│ Dimensions: (azimuth: 360, range: 1336)
│ Coordinates:
│ * azimuth (azimuth) float64 3kB 0.585 1.596 2.585 ... 358.6 359.6
│ elevation (azimuth) float64 3kB 2.373 2.373 2.373 ... 2.373 2.373
│ time (azimuth) datetime64[ns] 3kB ...
│ range (range) float32 5kB 2.125e+03 2.375e+03 ... 3.359e+05
│ Data variables:
│ DBZH (azimuth, range) float64 4MB ...
│ VRADH (azimuth, range) float64 4MB ...
│ WRADH (azimuth, range) float64 4MB ...
│ ZDR (azimuth, range) float64 4MB ...
│ PHIDP (azimuth, range) float64 4MB ...
│ RHOHV (azimuth, range) float64 4MB ...
│ CCORH (azimuth, range) float64 4MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 7
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 2.417
│ Attributes:
│ waveform_type: staggered_pulse_pair
│ channel_config: constant_phase
│ super_resolution: 14
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_8
│ Dimensions: (azimuth: 360, range: 1168)
│ Coordinates:
│ * azimuth (azimuth) float64 3kB 0.585 1.59 2.59 ... 358.6 359.6
│ elevation (azimuth) float64 3kB 3.032 3.032 3.032 ... 3.032 3.032
│ time (azimuth) datetime64[ns] 3kB ...
│ range (range) float32 5kB 2.125e+03 2.375e+03 ... 2.939e+05
│ Data variables:
│ DBZH (azimuth, range) float64 3MB ...
│ VRADH (azimuth, range) float64 3MB ...
│ WRADH (azimuth, range) float64 3MB ...
│ ZDR (azimuth, range) float64 3MB ...
│ PHIDP (azimuth, range) float64 3MB ...
│ RHOHV (azimuth, range) float64 3MB ...
│ CCORH (azimuth, range) float64 3MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 8
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 3.076
│ Attributes:
│ waveform_type: staggered_pulse_pair
│ channel_config: constant_phase
│ super_resolution: 14
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_9
│ Dimensions: (azimuth: 360, range: 984)
│ Coordinates:
│ * azimuth (azimuth) float64 3kB 0.5878 1.588 2.579 ... 358.6 359.6
│ elevation (azimuth) float64 3kB 3.955 3.955 3.955 ... 3.955 3.955
│ time (azimuth) datetime64[ns] 3kB ...
│ range (range) float32 4kB 2.125e+03 2.375e+03 ... 2.479e+05
│ Data variables:
│ DBZH (azimuth, range) float64 3MB ...
│ VRADH (azimuth, range) float64 3MB ...
│ WRADH (azimuth, range) float64 3MB ...
│ ZDR (azimuth, range) float64 3MB ...
│ PHIDP (azimuth, range) float64 3MB ...
│ RHOHV (azimuth, range) float64 3MB ...
│ CCORH (azimuth, range) float64 3MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 9
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 3.999
│ Attributes:
│ waveform_type: staggered_pulse_pair
│ channel_config: constant_phase
│ super_resolution: 14
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
├── Group: /sweep_10
│ Dimensions: (azimuth: 360, range: 820)
│ Coordinates:
│ * azimuth (azimuth) float64 3kB 0.5768 1.571 2.582 ... 358.6 359.6
│ elevation (azimuth) float64 3kB 5.054 5.054 5.054 ... 5.054 5.054
│ time (azimuth) datetime64[ns] 3kB ...
│ range (range) float32 3kB 2.125e+03 2.375e+03 ... 2.069e+05
│ Data variables:
│ DBZH (azimuth, range) float64 2MB ...
│ VRADH (azimuth, range) float64 2MB ...
│ WRADH (azimuth, range) float64 2MB ...
│ ZDR (azimuth, range) float64 2MB ...
│ PHIDP (azimuth, range) float64 2MB ...
│ RHOHV (azimuth, range) float64 2MB ...
│ CCORH (azimuth, range) float64 2MB ...
│ sweep_mode <U20 80B 'azimuth_surveillance'
│ sweep_number int64 8B 10
│ prt_mode <U7 28B 'not_set'
│ follow_mode <U7 28B 'not_set'
│ sweep_fixed_angle float64 8B 5.098
│ Attributes:
│ waveform_type: staggered_pulse_pair
│ channel_config: constant_phase
│ super_resolution: 14
│ sails_cut: False
│ sails_sequence_number: 0
│ mrle_cut: False
│ mrle_sequence_number: 0
│ mpda_cut: False
│ base_tilt_cut: False
└── Group: /sweep_11
Dimensions: (azimuth: 357, range: 676)
Coordinates:
* azimuth (azimuth) float64 3kB 0.574 1.574 2.598 ... 358.6 359.6
elevation (azimuth) float64 3kB 6.46 6.46 6.46 ... 6.46 6.46 6.46
time (azimuth) datetime64[ns] 3kB ...
range (range) float32 3kB 2.125e+03 2.375e+03 ... 1.709e+05
Data variables:
DBZH (azimuth, range) float64 2MB ...
VRADH (azimuth, range) float64 2MB ...
WRADH (azimuth, range) float64 2MB ...
ZDR (azimuth, range) float64 2MB ...
PHIDP (azimuth, range) float64 2MB ...
RHOHV (azimuth, range) float64 2MB ...
CCORH (azimuth, range) float64 2MB ...
sweep_mode <U20 80B 'azimuth_surveillance'
sweep_number int64 8B 11
prt_mode <U7 28B 'not_set'
follow_mode <U7 28B 'not_set'
sweep_fixed_angle float64 8B 6.416
Attributes:
waveform_type: staggered_pulse_pair
channel_config: constant_phase
super_resolution: 14
sails_cut: False
sails_sequence_number: 0
mrle_cut: False
mrle_sequence_number: 0
mpda_cut: False
base_tilt_cut: Falseds = xd.georeference.get_x_y_z(dtree["sweep_0"].to_dataset(inherit="all_coords"))
fig, ax = plt.subplots(figsize=(6, 5))
ds.DBZH.plot(x="x", y="y", cmap="ChaseSpectral", vmin=-10, vmax=70, ax=ax)
ax.set_title(f"Full volume - sweep_0 ({ds.sweep_fixed_angle.values:.1f} deg)")
ax.set_aspect("equal")
fig.tight_layout()
Partial volume – drop mode (default)#
When only some chunks have arrived, the last sweep is usually incomplete
(fewer rays than a full 360-degree rotation). By default,
incomplete_sweep="drop" excludes these partial sweeps and emits a warning.
This is the safest option for downstream processing that expects complete sweeps.
partial_chunks = all_bytes[:15]
with warnings.catch_warnings(record=True) as w:
warnings.simplefilter("always")
dtree_drop = xd.io.open_nexradlevel2_datatree(
partial_chunks, incomplete_sweep="drop"
)
# Show warnings
for warning in w:
print(f"WARNING: {warning.message}")
sweep_groups = list(dtree_drop.match("sweep_*").keys())
print(f"\nSweeps kept: {sweep_groups}")
WARNING: Dropped 1 incomplete sweep(s): [2]. Use incomplete_sweep='pad' to include them with NaN-filled rays.
Sweeps kept: ['sweep_0', 'sweep_1']
if len(sweep_groups) >= 2:
fig, axes = plt.subplots(1, 2, figsize=(12, 5))
for ax, grp in zip(axes, sweep_groups[:2]):
ds = xd.georeference.get_x_y_z(dtree_drop[grp].to_dataset(inherit="all_coords"))
ds.DBZH.plot(x="x", y="y", cmap="ChaseSpectral", vmin=-10, vmax=70, ax=ax)
ax.set_title(f"{grp} ({ds.sweep_fixed_angle.values:.1f} deg)")
ax.set_aspect("equal")
fig.suptitle("Drop mode: only complete sweeps", y=1.02, fontsize=13)
fig.tight_layout()
elif len(sweep_groups) == 1:
fig, ax = plt.subplots(figsize=(6, 5))
ds = xd.georeference.get_x_y_z(
dtree_drop[sweep_groups[0]].to_dataset(inherit="all_coords")
)
ds.DBZH.plot(x="x", y="y", cmap="ChaseSpectral", vmin=-10, vmax=70, ax=ax)
ax.set_title(f"{sweep_groups[0]} ({ds.sweep_fixed_angle.values:.1f} deg)")
ax.set_aspect("equal")
fig.suptitle("Drop mode: only complete sweeps", y=1.02, fontsize=13)
fig.tight_layout()
else:
print("No complete sweeps in 15 chunks (all dropped).")
Partial volume – pad mode#
With incomplete_sweep="pad", incomplete sweeps are kept and reindexed
to a full azimuth grid. Missing rays are filled with NaN. The angular
resolution (0.5 or 1.0 degree) is auto-detected from the data.
This is useful for visualization and monitoring where you want to see all available data as soon as it arrives.
dtree_pad = xd.io.open_nexradlevel2_datatree(partial_chunks, incomplete_sweep="pad")
sweep_groups_pad = list(dtree_pad.match("sweep_*").keys())
print(f"Sweeps available (pad mode): {sweep_groups_pad}")
# Show NaN percentage in each sweep
for grp in sweep_groups_pad:
ds = dtree_pad[grp].to_dataset()
if "DBZH" in ds:
nan_pct = np.isnan(ds.DBZH.values).mean() * 100
print(f" {grp}: azimuth size={ds.sizes['azimuth']}, DBZH NaN={nan_pct:.1f}%")
Sweeps available (pad mode): ['sweep_0', 'sweep_1', 'sweep_2']
sweep_0: azimuth size=720, DBZH NaN=0.0%
sweep_1: azimuth size=720, DBZH NaN=0.0%
sweep_2: azimuth size=735, DBZH NaN=68.0%
/home/docs/checkouts/readthedocs.org/user_builds/xradar/conda/latest/lib/python3.13/site-packages/xradar/util.py:466: UserWarning: Rays might miss on beginning and/or end of sweep. `a1gate` information is needed to fully recover. We'll assume sweep start at first valid ray.
warnings.warn(
n_sweeps = len(sweep_groups_pad)
fig, axes = plt.subplots(1, n_sweeps, figsize=(6 * n_sweeps, 5))
if n_sweeps == 1:
axes = [axes]
for ax, grp in zip(axes, sweep_groups_pad):
ds = xd.georeference.get_x_y_z(dtree_pad[grp].to_dataset(inherit="all_coords"))
ds.DBZH.plot(x="x", y="y", cmap="ChaseSpectral", vmin=-10, vmax=70, ax=ax)
ax.set_title(f"{grp} ({ds.sweep_fixed_angle.values:.1f} deg)")
ax.set_aspect("equal")
fig.suptitle("Pad mode: incomplete sweeps filled with NaN", y=1.02, fontsize=13)
fig.tight_layout()
Early streaming – few chunks#
Even with only 5 chunks (before the first sweep completes), pad mode shows the partial data that has arrived. The NaN wedge makes it clear which azimuths are still missing.
early_chunks = all_bytes[:5]
dtree_early = xd.io.open_nexradlevel2_datatree(early_chunks, incomplete_sweep="pad")
sweep_groups_early = list(dtree_early.match("sweep_*").keys())
print(f"Sweeps from 5 chunks: {sweep_groups_early}")
if sweep_groups_early:
ds = xd.georeference.get_x_y_z(
dtree_early[sweep_groups_early[0]].to_dataset(inherit="all_coords")
)
nan_pct = np.isnan(ds.DBZH.values).mean() * 100
print(f"DBZH NaN percentage: {nan_pct:.1f}%")
fig, ax = plt.subplots(figsize=(6, 5))
ds.DBZH.plot(x="x", y="y", cmap="ChaseSpectral", vmin=-10, vmax=70, ax=ax)
ax.set_title(
f"Early stream: {sweep_groups_early[0]} "
f"({ds.sweep_fixed_angle.values:.1f} deg) -- {nan_pct:.0f}% NaN"
)
ax.set_aspect("equal")
fig.tight_layout()
else:
print("No sweeps found in 5 chunks.")
/home/docs/checkouts/readthedocs.org/user_builds/xradar/conda/latest/lib/python3.13/site-packages/xradar/util.py:466: UserWarning: Rays might miss on beginning and/or end of sweep. `a1gate` information is needed to fully recover. We'll assume sweep start at first valid ray.
warnings.warn(
Sweeps from 5 chunks: ['sweep_0']
DBZH NaN percentage: 35.5%
Summary#
Scenario |
|
Behavior |
|---|---|---|
Full volume (all chunks) |
|
All sweeps present, no difference |
Partial volume |
|
Incomplete sweeps excluded, warning emitted |
Partial volume |
|
Incomplete sweeps kept, missing rays filled with NaN |
Early stream (few chunks) |
|
Single partial sweep visible with NaN wedge |
Note: Single-file, bytes, and file-like inputs continue to work exactly as
before. The list input and incomplete_sweep parameter are additive features.