When Underwater Acoustic Recognition Fails Across Datasets: A Cross-Dataset Benchmark Revealing Zero-Transfer and Label Shift

1. Introduction

Recognizing vessels from their radiated noise is a central task in passive underwater acoustics, with applications in maritime surveillance, environmental monitoring, and autonomous underwater systems. Deep learning has substantially advanced this task, but progress has been measured almost exclusively within individual public datasets, among which ShipsEar [1], DeepShip [2], Oceanship [3], and QiandaoEar22 [4] are the most widely used. Each dataset is typically split into training and test partitions drawn from the same recordings, the same hydrophone, and the same body of water. It has long been known in computer vision that such within-dataset evaluation overstates real-world generalization [5]: models exploit dataset-specific regularities and degrade sharply on data collected under different conditions. Whether, and how badly, underwater acoustic classifiers fail across datasets has rarely been quantified on public ship-noise corpora.

This gap has direct practical consequences. A recognition system trained on open-ocean recordings (e.g., the Ocean Networks Canada hydrophones underlying Oceanship [3]) will be deployed in waters with different sound-speed profiles, ambient-noise floors, and vessel populations. Conversely, datasets collected in confined freshwater environments such as Qiandao Lake [4] exhibit channel reverberation and traffic composition that have no open-ocean counterpart. A benchmark that measures cross-dataset transfer honestly is therefore a prerequisite for any claim of operational readiness, and it gives domain-adaptation research [6] an empirical basis.

Building such a benchmark is nontrivial because public UATR datasets do not share label taxonomies. Oceanship labels 15 fine-grained AIS vessel types (all surface vessels); QiandaoEar22 labels eight “view” categories of the form {target vessel} × {present/absent} plus background noise; VTUAD adds a 12-class AIS type-and-cargo taxonomy; ShipsEar contributes a 12-class official-registry taxonomy. We show that only two class pairs are semantically mappable across the four taxonomies (one of them approximate). This label shift must be treated as an experimental variable in its own right: any cross-dataset conclusion must be conditioned on an explicit mapping table and on the fraction of each test set the mapping covers. We therefore distinguish two conceptually different obstacles to cross-dataset deployment: (i) label shift, which restricts how much of a target dataset can be evaluated at all and how predictions are to be interpreted, and (ii) acoustic domain shift, which determines how well a model performs on the subset that can be evaluated. Our benchmark quantifies both separately.

In this paper we make five contributions:

(1)

A unified cross-dataset benchmark for UATR. We preprocess Oceanship, QiandaoEar22, VTUAD, and ShipsEar into an identical feature format (16 kHz, 3 s segments with 1.5 s hop, single-channel 128-bin log-mel spectrograms), freeze recording-level-isolated train/validation/test splits, and release the class-mapping table with per-class exclusion rationales and evaluation coverage. The full preprocessing chain, classifier, and weighting scheme are specified mathematically, with every hyperparameter traceable to the released code.

(2)

An empirical demonstration of cross-dataset zero-transfer, with a feature-space mechanism analysis. ResNet-18 classifiers that reach usable in-library accuracy collapse to 0.0% (Oceanship→QiandaoEar22) and 4.1% (QiandaoEar22→Oceanship) accuracy on the shared label space, far below majority-class chance. The two distinct, asymmetric failure modes are characterized at the per-class prediction-distribution level. A centroid-level analysis of the feature space shows that the domain gap is 5–17× larger than the within-dataset class gap. The analysis also reproduces both failure modes geometrically.

(3)

A quantitative study of class-imbalance handling under extreme label skew. On Oceanship’s 4,278:1 imbalance, naive inverse-frequency class weighting collapses training (macro-F1 0.089, near the 15-class random level). Square-root smoothing instead raises macro-F1 to 0.1452 ± 0.0136 (mean ± std over seeds 42/43/44; +24% relative over the unweighted baseline on reference seed 42, +12% on the three-seed mean) at a cost of 3.4 accuracy points on the reference seed. We show analytically and numerically that sqrt smoothing approximates class-balanced (effective-number) weighting [7] with β ≈ 0.995, and we explain the collapse of inverse weighting through a gradient-variance argument.

(4)

Evidence-based motivation for domain adaptation. The single positive transfer signal (speedboat precision 0.679 vs. 0.5563 prior) coincides with the geometrically closest semantically mapped class pair in feature space, and an acoustic-mechanism discussion explains why small-craft signatures transfer while large-vessel signatures do not. These results indicate where domain-adaptive methods should be applied first and provide the baseline against which our subsequent adaptation work is measured.

(5)

A third library that isolates labeling-pipeline shift from water-body shift, with completed cross-library and adaptation measurements. We reconstruct the subscription-gated VTUAD corpus [8] from public ONC archives (the same Strait of Georgia region as Oceanship) with a fully automatic AIS labeling pipeline (525 single-vessel scenario clips, 12 classes, 132,273 unique segments, MMSI-retaining metadata, and frozen clip-level splits), and we evaluate all four new cross-library directions. Zero-transfer holds on every one of them, including the same-water Oceanship→VTUAD pair (0.9% accuracy). This localizes a substantial part of “dataset bias” to labeling provenance and scenario definition, beyond the acoustic channel. A companion UDA study [9] quantifies how much of the gap moment-matching adaptation can close (Section 4.6). A fourth library, ShipsEar (nearshore harbor recordings off the Spanish Atlantic coast, 12 registry classes), extends the cross-library measurement to twelve directions (Section 4.7).

2. Related Work

2.1. Public Datasets for Underwater Acoustic Target Recognition

ShipsEar [1] is an early public database of vessel noise recorded off the Spanish Atlantic coast (90 recordings, 11 vessel types commonly merged into four size-based classes plus background). DeepShip [2] substantially increased scale with 47 h of recordings of 265 vessels in four classes (cargo, passenger, tanker, tug) from Ocean Networks Canada (ONC) hydrophones in the Strait of Georgia. Oceanship [3], also built from ONC data, provides 121 h with 15 AIS-derived vessel types and rich metadata, and was released together with an audio-retrieval baseline (Oceannet). QiandaoEar22 [4] differs in kind. It was recorded with a DigitalHyd SR-1 hydrophone in Qiandao Lake in 2022, and it contains 9 h 28 min of multi-target ship-radiated noise and 21 h 58 min of background noise, organized as target-identification tasks for three specific vessels (SpeedBoat, the passenger ship KaiYuan, and a UUV). Across all four corpora, published evaluations are within-dataset; to our knowledge, cross-dataset transfer between public ship-noise corpora has not been systematically measured under an open, reproducible protocol. The closest cross-domain UATR studies are few-shot and require a small amount of labeled target data. He et al. [10] improve few-shot underwater acoustic recognition with unlabeled target samples alongside a labeled support set. Cui et al. [11] instead drive few-shot recognition with cross-domain contrastive learning. Neither evaluates label-free transfer between public corpora, which is the measurement supplied here.

2.2. Dataset Bias and Cross-Dataset Generalization

Torralba and Efros [5] showed that visual recognition datasets are biased samples of the visual world and that classifiers degrade across dataset boundaries. In their “Name That Dataset” experiment, the dataset identity of an image was itself easily classifiable. Each dataset therefore occupies a distinctive region of feature space. Domain-adaptation methods, such as domain-adversarial training (DANN) [6], were later introduced to close such gaps using unlabeled target data. In underwater acoustics, domain shift is acknowledged anecdotally (accuracy drops when the test water area differs from the training area), but, to our knowledge, no public, quantitative cross-dataset benchmark exists for ship-radiated noise classification. The present work supplies this missing measurement for UATR and provides a feature-space analogue of the “Name That Dataset” experiment (Section 4.4). Following [5], we report the negative result itself as a finding.

2.3. Class Imbalance and Spectrogram Augmentation

Long-tailed recognition is commonly addressed by re-weighting or re-sampling; class-balanced losses based on the effective number of samples [7] formalize the intuition that raw inverse-frequency weights over-correct. Spectrogram augmentation (SpecAugment [12]) is a standard regularizer for audio classifiers. We adopt ResNet-18 [13] with ImageNet pretraining as a conventional, reproducible classifier. We also show empirically that the degree of imbalance qualitatively changes which weighting scheme is viable. At 4,278:1 skew, inverse-frequency weights (ratio 0.0018:7.5) destroy training, while their square-root-smoothed counterparts (0.07:4.6) are beneficial. Section 3.4 develops the weighting schemes formally and relates sqrt smoothing to the effective-number-of-samples weighting of [7].

3. Materials and Methods

3.1. Datasets, Acquisition Conditions, and Composition

Oceanship (fine-grained subset). We use the 15-class AIS vessel-type subset of Oceanship [3]: Anti-pollution equipment, Cargo, Diving ship, Dredging, Fishing, Military ship, Passenger, Pleasure Craft, Port Tender, Sailing, Search and Rescue vessel, Spare, Tanker, Towing, and Tug. The class “Spare” is an AIS reserved type code: these are vessels, not background noise. The processed feature library contains 81,446 log-mel segments (56.3 h of unique audio under the definition of Table 2). The official release totals 107,540 audio samples, the coarse-grained (53,771) and fine-grained (53,769) versions of the same clips counted together [3]. We use the fine-grained version only and re-segment its clips with the unified front-end of Section 3.2 (3 s windows, 1.5 s hop, trailing remainders discarded), yielding the 81,446-segment count used throughout.

QiandaoEar22. We use all eight labels of the QiandaoEar22 task view [4]: kaiyuan_target, speedboat_target, uuv_target (segments with the respective target vessel present), kaiyuan_other, speedboat_other, uuv_other (other vessels plus environment within each sub-view, not pure noise), noise (annotated pure background), and noise_target_target (multi-ship mixtures). KaiYuan is a passenger sightseeing ship; SpeedBoat is a small recreational craft; the UUV is an unmanned underwater vehicle [4]. The extraction manifest contains 36,238 entries. Of these, 597 segments share byte-identical underlying files (duplicate manifest rows, all within the noise class, which is cut from a small number of long background recordings). All evaluation counts unique content by hash, yielding 35,641 segments. This corresponds to 26.1 h of unique audio under the duration definition of Table 2, while the official release totals 31.4 h (9 h 28 min target plus 21 h 58 min background). The ≈5.4 h difference comes from the partial extraction of the background material: about 16.6 h of the 21 h 58 min background enters the feature library (spread across the noise and *_other classes), while the target-side audio is extracted in full. Of the 21 h 58 min of background audio in the official release, the extracted noise class uses 50 source files totaling 4.17 h of unique audio (Table 4).

VTUAD (AIS-auto-labeled reconstruction). As a third library we rebuild VTUAD [8], originally gated behind an IEEE DataPort subscription, from its public upstream source: Ocean Networks Canada hydrophone ICLISTENAF2523 in the Strait of Georgia (the same recording region as Oceanship), jointly archived AIS traffic, and CTD casts. The fully automatic pipeline (AIS parsing, MMSI-level static-table propagation, hydrophone-distance filtering, single-vessel scenario identification, audio–AIS interval matching) is specified rule by rule in the companion data descriptor [14]. It yields 525 labeled single-vessel scenario clips (one deployment, 2016-05–2017-06; inclusion radius 4 km) over 12 classes (passengership, tug, tanker, cargo, background, pleasurecraft, fishing, sailing, pilotvessel, rescue, dredger, other). Segmented with the unified front-end of Section 3.2, the extraction manifest lists 133,784 segment rows. Of these, 1,511 are duplicate rows from eight background/vessel filename collisions (the colliding feature files were overwritten on disk). All counts in this paper use the deduplicated count of 132,273 unique log-mel segments (56.0 h of unique audio), split at the clip level into 369/80/76 train/val/test clips so that no scenario crosses partitions. The split therefore isolates whole clips. The retained MMSI metadata supports hull-level isolation, but the frozen split itself does not enforce hull-disjoint partitions. VTUAD’s role in the benchmark is specific. It shares the water body and observatory region with Oceanship while differing in hydrophone, labeling pipeline, and scenario definition. The Oceanship↔VTUAD pairs therefore isolate labeling-provenance and population shift from channel shift.

ShipsEar (nearshore harbor recordings). As a fourth library we include ShipsEar [1]: 90 single-channel recordings made off the Spanish Atlantic coast (mostly at <50 m source range, 52,734/44,100 Hz), totaling 188.9 min. Its 12 registry-level classes are taken verbatim from the official shipsEar.xlsx registry (passengers, ocean_liner, motorboat, sailboat, tugboat, pilot_ship, dredger, roro, fishboat, trawler, mussel_boat, background). Segmented with the unified front-end, the library contains 7,425 log-mel segments (3.1 h of unique audio; 4,962/1,557/906 train/val/test), split at the recording level with the same frozen protocol (63/15/12 recordings; each WAV is an independent recording). ShipsEar differs from the other three libraries in range geometry and platform (nearshore, short-range, moored-hydrophone recordings). Five of its twelve classes have no test segments, so in-library conclusions for those classes are unreliable. The cross-library evaluation uses only its two mappable shared classes, which correspond to three registry classes with test support (passengers, ocean_liner, motorboat). This leaves four unmappable classes that have test segments.

Acquisition-condition comparison. Table 2 contrasts the four corpora along every axis that can contribute to a domain gap. Oceanship and QiandaoEar22 differ simultaneously in water body (open coastal ocean vs. confined freshwater lake), recording platform (cabled observatory hydrophones vs. a self-contained recorder), sound-speed profile and boundary conditions (deep, range-dependent ocean waveguide vs. shallow, strongly reverberant lake), label provenance (AIS logs vs. per-view annotation), and target set (an open population of transiting vessels vs. three specific, repeatedly recorded craft). VTUAD holds the first three of these (nearly) constant against Oceanship and varies only the last two. No axis is held constant across all four libraries. The benchmark therefore measures the total distribution shift a deployed system would face, with the Oceanship↔VTUAD pairs providing the closest available controlled contrast.

Table 1. The cross-dataset benchmark: sample statistics and label taxonomies. Segment counts are content-deduplicated (VTUAD: 133,784 extraction-manifest rows minus 1,511 filename-collision duplicates; Section 3.1).

Dataset	Classes	Train	Val	Test	Total segments	Label taxonomy
Oceanship (FG)	15	56,998	12,242	12,206	81,446	AIS vessel types (all surface vessels)
QiandaoEar22	8	25,891	5,426	4,324	35,641	{target vessel} × {target/other} view labels
VTUAD (rebuild)	12	93,592	20,479	18,202	132,273	AIS type-and-cargo codes, auto-labeled (this work, after [8])
ShipsEar	12	4,962	1,557	906	7,425	Official registry vessel types (shipsEar.xlsx)

Table 1. The cross-dataset benchmark: sample statistics and label taxonomies. Segment counts are content-deduplicated (VTUAD: 133,784 extraction-manifest rows minus 1,511 filename-collision duplicates; Section 3.1).

Dataset	Classes	Train	Val	Test	Total segments	Label taxonomy
Oceanship (FG)	15	56,998	12,242	12,206	81,446	AIS vessel types (all surface vessels)
QiandaoEar22	8	25,891	5,426	4,324	35,641	{target vessel} × {target/other} view labels
VTUAD (rebuild)	12	93,592	20,479	18,202	132,273	AIS type-and-cargo codes, auto-labeled (this work, after [8])
ShipsEar	12	4,962	1,557	906	7,425	Official registry vessel types (shipsEar.xlsx)

Table 2. Systematic comparison of acquisition conditions. Unique audio duration is measured per source recording as (last segment end − first segment start) under the frozen 3 s / 1.5 s-hop front-end, summed over recordings; it counts the 50% segment overlap once and therefore differs from both the released durations and a segments × 3 s figure. As a rough conversion, unique-audio duration lies between segments × 1.5 s (long recordings, overlap counted once; VTUAD’s 132,273 segments ≈ 55.1 h and ShipsEar’s 7,425 ≈ 3.1 h sit near this bound) and segments × 3 s (one segment per short clip, frequent in Oceanship and QiandaoEar22). Oceanship and QiandaoEar22 sit closer to the segments × 3 s upper bound. In Oceanship this is partly because short source clips are reflect-padded to 3 s (12,569 segments, 15.4%; Section 3.1), and in QiandaoEar22, which uses no padding, it is because 99.8% of its source recordings are short enough to yield a single segment each. “—” = not published in the source release.

Property	Oceanship (FG subset) [3]	QiandaoEar22 [4]	VTUAD (rebuild) [8]	ShipsEar [1]
Water body	Open coastal ocean (ONC observatories, Pacific coast of Canada)	Qiandao Lake (confined freshwater lake, China)	Open coastal ocean (ONC observatory, Strait of Georgia — same region as Oceanship)	Nearshore harbor (Spanish Atlantic coast)
Recording platform	ONC cabled observatory hydrophones	DigitalHyd SR-1 self-contained hydrophone	ONC cabled observatory hydrophone ICLISTENAF2523	Moored hydrophone, near-range (mostly <50 m)
Typical source range	Long-range transiting traffic (fixed observatory geometry)	Short-range passes of three dedicated craft	Single-vessel scenarios within 4 km of the hydrophone	Short-range moored recordings
Label provenance	AIS vessel-type logs (15 types)	Per-view annotation, {target} × {present/other} (8 labels)	AIS type-and-cargo codes via automatic pipeline (12 classes)	Official registry (shipsEar.xlsx), 12 classes
Released duration	121 h (full release); 56.3 h unique audio after segmentation here	9 h 28 min target + 21 h 58 min background (31.4 h full release); 26.1 h unique audio after segmentation and content dedup here	56.0 h unique audio after segmentation here (rebuild, this work)	188.9 min (90 recordings); 3.1 h unique audio after segmentation here
Recording year / season	—	2022	2016-05–2017-06 (single deployment)	—
Class imbalance (train)	4,278:1 (Tug 12,833 vs. Diving ship 3)	≈10:1 (noise 6,567 vs. uuv_target 653)	81.6:1 (tug 29,230 vs. pilotvessel 358)	Long-tailed; 5 of 12 classes absent from test

Table 2. Systematic comparison of acquisition conditions. Unique audio duration is measured per source recording as (last segment end − first segment start) under the frozen 3 s / 1.5 s-hop front-end, summed over recordings; it counts the 50% segment overlap once and therefore differs from both the released durations and a segments × 3 s figure. As a rough conversion, unique-audio duration lies between segments × 1.5 s (long recordings, overlap counted once; VTUAD’s 132,273 segments ≈ 55.1 h and ShipsEar’s 7,425 ≈ 3.1 h sit near this bound) and segments × 3 s (one segment per short clip, frequent in Oceanship and QiandaoEar22). Oceanship and QiandaoEar22 sit closer to the segments × 3 s upper bound. In Oceanship this is partly because short source clips are reflect-padded to 3 s (12,569 segments, 15.4%; Section 3.1), and in QiandaoEar22, which uses no padding, it is because 99.8% of its source recordings are short enough to yield a single segment each. “—” = not published in the source release.

Property	Oceanship (FG subset) [3]	QiandaoEar22 [4]	VTUAD (rebuild) [8]	ShipsEar [1]
Water body	Open coastal ocean (ONC observatories, Pacific coast of Canada)	Qiandao Lake (confined freshwater lake, China)	Open coastal ocean (ONC observatory, Strait of Georgia — same region as Oceanship)	Nearshore harbor (Spanish Atlantic coast)
Recording platform	ONC cabled observatory hydrophones	DigitalHyd SR-1 self-contained hydrophone	ONC cabled observatory hydrophone ICLISTENAF2523	Moored hydrophone, near-range (mostly <50 m)
Typical source range	Long-range transiting traffic (fixed observatory geometry)	Short-range passes of three dedicated craft	Single-vessel scenarios within 4 km of the hydrophone	Short-range moored recordings
Label provenance	AIS vessel-type logs (15 types)	Per-view annotation, {target} × {present/other} (8 labels)	AIS type-and-cargo codes via automatic pipeline (12 classes)	Official registry (shipsEar.xlsx), 12 classes
Released duration	121 h (full release); 56.3 h unique audio after segmentation here	9 h 28 min target + 21 h 58 min background (31.4 h full release); 26.1 h unique audio after segmentation and content dedup here	56.0 h unique audio after segmentation here (rebuild, this work)	188.9 min (90 recordings); 3.1 h unique audio after segmentation here
Recording year / season	—	2022	2016-05–2017-06 (single deployment)	—
Class imbalance (train)	4,278:1 (Tug 12,833 vs. Diving ship 3)	≈10:1 (noise 6,567 vs. uuv_target 653)	81.6:1 (tug 29,230 vs. pilotvessel 358)	Long-tailed; 5 of 12 classes absent from test

Per-class composition. Table 3 and Table 4 give the full per-class composition of both feature libraries, including the number of distinct source recordings and recording groups behind each class. These quantities determine the effective diversity of a class. Two structural asymmetries matter here. First, Oceanship’s head classes are backed by thousands of distinct recordings (Tug: 11,852 files, 8,216 groups), whereas QiandaoEar22’s noise class, though large in segment count (9,353), derives from only 50 source files / 47 recording groups, so background diversity in the lake dataset is intrinsically limited. Second, Oceanship contains a substantial fraction of very short source clips that required reflect-padding to reach the 3 s segment length (12,569 of 81,446 segments, 15.4%, concentrated in the head classes), while QiandaoEar22 required none. We flag every padded segment in the manifest so that it can be audited.

Oceanship is extremely imbalanced: Tug (12,833 training segments) outweighs Diving ship (3) by 4,278:1 (Figure 1). QiandaoEar22 is mildly imbalanced (largest/smallest ≈ 10:1; noise 6,567 vs. uuv_target 653 training segments).

3.2. Unified Feature Extraction

All audio is converted to a single, frozen feature format by one preprocessing chain (preprocess_audio.py, released with the benchmark). Every parameter below is the value actually used in the released feature libraries.

Resampling and segmentation. Each waveform is converted to mono (channel mean where necessary) and resampled to $f_s=\mathrm{16,000}$ Hz. The resampled signal $x[n]$, $n=0,\dots ,L-1$, is cut into segments of $N=3\hspace{0.17em}\mathrm{s}\times f_s=\mathrm{48,000}$ samples with hop $H_{\mathrm{seg}}=\mathrm{24,000}$ samples (1.5 s, 50% overlap):

$$x_s[n]=x[s{H}_{\mathrm{seg}}+n], n=0,\dots ,N-1, s=\mathrm{0,1},\dots$$

A trailing remainder shorter than $N$ is discarded. Recordings that are themselves shorter than 3 s are reflect-padded to $N$ samples and yield exactly one segment, flagged padded=1 in the manifest (12,569 Oceanship segments, 0 QiandaoEar22 segments; Table 3 and Table 4). Reflect padding is used instead of zero padding because it avoids step discontinuities that would inject broadband artifacts into the spectrogram.

Short-time Fourier transform. Each segment is analyzed with a Hann window

$$w[n]=1/2(1-cos{\displaystyle \frac{2\pi n}{M-1}}), M=1024\hspace{0.25em}(64\mathrm{ms}),$$

with hop $H=320$ samples (20 ms, 68.75% window overlap) and center padding of $M/2$ samples at both ends, giving

$$X[k,t]={\displaystyle \sum _{n=0}^{M-1}}w[n]\hspace{0.17em}{x}_s[n+tH], k=0,\dots ,512, t=0,\dots ,150,$$

i.e., $K=513$ frequency bins (bin spacing $f_s/M=15.625$ Hz) and $T=\lfloor N/H\rfloor +1=151$ frames per segment. The power spectrogram is $P[k,t]=|X[k,t]{|}^2$.

Mel filterbank. The power spectrogram is projected onto $B=128$ triangular mel filters. The mel mapping follows the Slaney-style scale used by the implementation [15]: linear below 1 kHz and logarithmic above,

$$m(f)=\{\begin{array}{cc}{\displaystyle \frac{3f}{200}},& f<1000\hspace{0.25em}\mathrm{Hz},\\ 15+{\displaystyle \frac{{log}_{10}(f/1000)}{{log}_{10}(6.4)/27}},& f\ge 1000\hspace{0.25em}\mathrm{Hz},\end{array}$$

a practical approximation of the psychoacoustic pitch scale [16]. The filter vertices are placed at $B+2$ equally spaced mel frequencies between 0 and $f_s/2=8$ kHz and mapped back to hertz. The $m$-th filter is the triangle $G_m[\cdot ]$ peaked at the center frequency ${\widehat{f}}_m$. Each triangle is area-normalized (Slaney normalization) by $2/({\widehat{f}}_{m+1}-{\widehat{f}}_{m-1})$, so that filters of different width contribute comparable energy:

$$S[m,t]={\displaystyle \sum _{k=0}^{K-1}}{G}_m[k]\hspace{0.17em}P[k,t], m=1,\dots ,128.$$

Log compression and normalization. The mel spectrogram is log-compressed per segment with the segment maximum as reference and an 80 dB dynamic-range floor:

$$L[m,t]=max(10{log}_{10}{\displaystyle \frac{max(S[m,t],\hspace{0.17em}{10}^{-10})}{\underset{m^{\text{'}},t^{\text{'}}}{max}S[m^{\text{'}},t^{\text{'}}]}},\hspace{0.33em}-80).$$

The resulting single-channel array $L\in {\mathbb{R}}^{128\times 151}$ (float32) is the stored feature. At training time each feature is standardized with the source training-set mean and standard deviation,

$$\widehat{L}={\displaystyle \frac{L-{\mu }_{\mathrm{src}}}{{\sigma }_{\mathrm{src}}}},$$

estimated from up to 3,000 randomly sampled training segments. The estimated values differ markedly between the two libraries (Oceanship: $\mu =-52.42$ dB, $\sigma =11.53$ dB; QiandaoEar22: $\mu =-28.73$ dB, $\sigma =10.35$ dB), a first, easily measurable symptom of the domain gap that Section 4.4 analyzes in detail. Because the reference in Eq. (6) is the per-segment maximum, the 23.7 dB difference of the means reflects a difference in spectral shape (peak-to-average ratio), not in recording gain. Oceanship spectra are narrowly concentrated around a few dominant low-frequency bins, whereas QiandaoEar22 spectra are substantially flatter.

Implementation note. The released pipeline computes Eq. (5) as an explicitly sparse weighted sum over each filter’s non-zero support instead of a dense matrix product. The two are numerically identical (same filterbank, same power, same log mapping), and the choice does not affect any result in this paper. Table 5 collects all preprocessing parameters.

3.3. Classifier Architecture

We use ResNet-18 [13] with ImageNet-pretrained weights (IMAGENET1K_V1) as the classifier, a standard and reproducible choice. Two adaptations are made. First, the input stem is rebuilt for single-channel input: the pretrained 3-channel $7\times 7$ kernel $W\in {\mathbb{R}}^{64\times 3\times 7\times 7}$ is summed over the channel axis,

$$W_{c,1,i,j}^{\text{'}}={\displaystyle \sum _{c^{\text{'}}=1}^3}{W}_{c,c^{\text{'}},i,j},$$

which preserves the pretrained spatial filters while exactly matching the original response to any grayscale (channel-replicated) input. Second, the classification head is replaced by a linear layer ${\mathbb{R}}^{512}\to {\mathbb{R}}^C$ with $C=15$ (Oceanship) or $C=8$ (QiandaoEar22). Table 6 gives the full architecture for the $1\times 128\times 151$ input. The total parameter count is 11,177,935 ($C=15$) and 11,174,344 ($C=8$), of which 11,170,240 are in the shared backbone.

3.4. Training Objective and Class-Weight Smoothing

Weighted cross-entropy. All models are trained with class-weighted cross-entropy. For a batch of examples $(x_i,y_i)$ with logits $z_i$ and $p_{i,c}=exp(z_{i,c})/\sum _{c^{\text{'}}}exp(z_{i,c^{\text{'}}})$,

$$\mathcal{L}=-{\displaystyle \frac{1}{\left|\mathcal{B}\right|}}\sum _{i\in \mathcal{B}}{w}_{y_i}log{p}_{i,y_i},$$

where the class weight derives from the training count $n_c$ via a power-law smoothing family,

$$w_c={\displaystyle \frac{n_c^{-\alpha }}{Z}}, Z={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c^{\text{'}}=1}^C}{n}_{c^{\text{'}}}^{-\alpha },$$

with $Z$ chosen so the weights average to one. $\alpha =0$ is the unweighted baseline, $\alpha =1$ is inverse-frequency weighting, and $\alpha =0.5$ is the sqrt-smoothed variant.

Relation to effective-number weighting. Cui et al. [7] argue that the marginal benefit of additional samples saturates and propose weighting by the inverse effective number of samples, $w_c\propto (1-\beta )/(1-{\beta }^{n_c})$ with $\beta \in [\mathrm{0.9,0.9999}]$. Two limits connect Eq. (10) to this family. By L’Hôpital’s rule,

$$\underset{\beta \to 1}{lim}{\displaystyle \frac{1-\beta }{1-{\beta }^{n_c}}}={\displaystyle \frac{1}{n_c}},$$

so effective-number weighting recovers inverse frequency as $\beta \to 1$, while for $\beta$ bounded away from 1 the factor $1-{\beta }^{n_c}$ saturates for large $n_c$ and compresses the dynamic range of the weights. Power-law smoothing sits in the same design space. Numerically, on Oceanship’s training counts, solving $(1-{\beta }^3)/(1-{\beta }^{12833})=(3/12833{)}^{1/2}$ for the two extreme classes yields ${\beta }^{*}\approx 0.9949$. At this value the effective-number weights (extreme ratio 65.7:1) are within 1% of the sqrt weights (65.4:1) across all 15 classes. Thus, for this count distribution, sqrt smoothing is numerically equivalent to effective-number weighting with a sensible $\beta$ and inherits the principled basis of that family.

Why inverse weighting collapses at 4,278:1. Table 7 lays out the failure mechanism in numbers. Under $\alpha =1$ the per-sample loss weight spans four orders of magnitude (Diving ship 7.54 vs. Tug 0.0018, a 4,278:1 ratio). In a batch of 64, the three Diving-ship training segments (re-sampled every epoch) each contribute as much gradient as ≈4,200 Tug segments. The gradient direction is therefore dominated by whichever rare class happens to be present in the batch. The per-batch gradient variance scales as $Var\propto \sum _c{p}_c{w}_c^2\propto \sum _{c}1/n_c$, which is controlled almost entirely by the smallest classes. Training therefore does not diverge numerically, but it oscillates. Validation accuracy peaks at epoch 2 (0.241) and immediately degrades, and the best checkpoint retains near-chance per-class behavior (macro-F1 0.089). Sqrt smoothing compresses the weight range to 65:1 (0.07–4.6), which retains the correction direction while bounding single-sample influence. On QiandaoEar22’s mild 10:1 imbalance the inverse weights span only 0.30–2.98. The variance inflation is negligible there, and inverse weighting works as expected (Section 4.1).

Training protocol. Two baseline families are trained per dataset (batch 64, weight decay 1×10⁻⁴, early stopping with patience 10, best checkpoint by validation accuracy). The final configuration of each dataset is run with seeds 42/43/44 and reported as mean ± std (population std, ddof = 0); exploratory configurations remain single-seed and are marked † in Table 10. The cross-dataset evaluations of Section 4.2 and Section 4.6, and 4.7 use the seed-42 checkpoints as source models. The two families are:

• Coarse baseline: lr = 1×10⁻³, no augmentation, no class weighting.
• Tuned: lr = 3×10⁻⁴, SpecAugment [12] (two frequency masks of width ≤ 16 mel bins and two time masks of width ≤ 24 frames, applied after normalization), inverse-frequency class weights ($\alpha =1$

).

For Oceanship we additionally run a sqrt-weighted variant ($\alpha =0.5$), motivated by the training collapse observed with raw inverse weights (Section 4.1). Cross-dataset evaluation uses the sqrt-weighted Oceanship model (its best macro-F1 configuration) and the tuned QiandaoEar22 model. Bracketed accuracies next to source-model names in Tables 11, 14, and 16 are seed-42 checkpoint values; Table 10 reports three-seed means.

3.5. Evaluation Metrics

We report accuracy and macro-F1, formalized as follows. For a test set of $N$ examples with predictions ${\widehat{y}}_i$ and $C$ classes, per-class precision and recall are

$$P_c={\displaystyle \frac{{TP}_c}{{TP}_c+{FP}_c}}, R_c={\displaystyle \frac{{TP}_c}{{TP}_c+{FN}_c}},$$

and

$$Acc={\displaystyle \frac{1}{N}}{\displaystyle \sum _{i=1}^N}1[{\widehat{y}}_i=y_i], F{1}_c={\displaystyle \frac{2P_c{R}_c}{P_c+R_c}}, macro-F1={\displaystyle \frac{1}{C}}{\displaystyle \sum _{c=1}^C}F{1}_c.$$

Under class imbalance, accuracy is dominated by head classes and can be inflated by degenerate predictors. On the Oceanship test set a constant “Tug” predictor scores 22.6% accuracy with macro-F1 ≈ 0.025, and on the QiandaoEar22 test set a constant “noise” predictor scores 27.6% with macro-F1 ≈ 0.054. Macro-F1 weighs every class equally and therefore penalizes the failure mode that extreme imbalance induces (ignoring rare classes), so we adopt it as the primary benchmark metric and report accuracy alongside. For cross-dataset evaluation we additionally report the majority-class chance level of each evaluated subset (the accuracy of always predicting the majority shared label) and the unmapped-prediction rate (Section 3.6), since chance levels change once evaluation is restricted to a mapped subset.

3.6. Cross-Dataset Label Mapping and Evaluation Protocol

Cross-dataset evaluation requires a shared label space. We map the four taxonomies by vessel semantics, with documented exclusion rules (Table 8 and Table 9).

Three exclusion decisions need explicit justification, since a less careful mapping at these points could quietly inflate cross-dataset scores:

• uuv_target vs. Diving ship. A surface diving-support vessel (a crewed ship carrying diving equipment) and an unmanned underwater vehicle are acoustically unrelated; mapping them would manufacture a spurious “underwater” correspondence. Both are excluded.
• The *_other classes are not background. Each *_other class contains other vessels plus environment recorded within the corresponding target’s sub-view; treating them as “noise” would conflate vessel noise with background and contaminate the shared space. They are excluded as mixtures.
• noise vs. “Spare”. Oceanship’s “Spare” is an AIS reserved code attached to real vessels, not background; QiandaoEar22’s noise is annotated pure background. Neither dataset contains the other’s category, so both are excluded. Critically for the interpretation of Section 4.3, there is no legitimate mapping under which the lake-trained model’s habit of calling ocean vessels “noise” could be counted as correct.

This mapping covers 802/4,324 (18.5%) of the QiandaoEar22 test set, 2,736/12,206 (22.4%) of the Oceanship test set, 2,426/18,202 (13.3%) of the VTUAD test set, and 511/906 (56.4%) of the ShipsEar test set. The excluded mass is itself structured. In the Oceanship→QiandaoEar22 direction, the 3,522 excluded segments split into speedboat_other 1,096, noise 1,194, noise_target_target 484, kaiyuan_other 374, uuv_other 274, uuv_target 100. In the reverse direction, the 9,470 excluded segments are dominated by Tug 2,758, Towing 2,583, and Cargo 2,279. On the VTUAD side, the 15,776 excluded test segments split into tug 6,492, background 3,431, cargo 2,706, other 1,434, fishing 1,195, dredger 279, and sailing 239. On the ShipsEar side, the remaining 395 excluded test segments spread over the four unmappable classes that have test support. The fundamental asymmetry of the four taxonomies is preserved and reported, not hidden.

No-adaptation protocol. The source model is applied directly to target-domain features, normalized with the source training-set mean/std (the realistic deployment condition, Eq. (7)). Predictions that fall on source classes outside the mapping are collected into an unmapped_pred column of the confusion matrix and counted as errors. We report accuracy, macro-F1, per-class precision/recall, and the majority-class chance level of each evaluated subset.

Label shift vs. acoustic domain shift. The protocol cleanly separates the two obstacles. Label shift determines the evaluated mass (18.5%/22.4%/13.3%/56.4%) and the interpretation of predictions (the unmapped_pred column). It is a property of the taxonomies, independent of any model. Acoustic domain shift determines performance within the evaluated subset. In particular, the coverage figures above are a property of the label taxonomies alone. They quantify label shift (how much of a test set can be evaluated at all), not acoustic domain shift (how well a model performs on the covered subset). Because every number in Section 4.2 is computed strictly inside the shared label space, a low cross-dataset score there cannot be blamed on label mismatch (the labels are semantically aligned by construction) and must be attributed to the acoustic/conditional distribution shift. Section 4.4 quantifies this shift directly in feature space.

4. Results

4.1. In-Library Baselines and Class-Weight Smoothing

Figure 2 quantifies the class-weighting effect. First, raw inverse-frequency weighting is harmful under extreme imbalance. On Oceanship the weight ratio spans 0.0018:7.5 and training collapses by epoch 2 (validation accuracy peaks at 0.241 and immediately degrades), yielding macro-F1 0.0893, close to the 15-class random level. Square-root smoothing (weight ratio 0.07:4.6) instead lifts macro-F1 to 0.1452 ± 0.0136 across seeds 42/43/44 (seed 42 alone: 0.1616, a 24% relative gain over the coarse baseline; +12% on the three-seed mean), at the cost of about 3 accuracy points. Second, under QiandaoEar22’s mild imbalance (weights 0.30–2.98) inverse weighting has the expected effect: macro-F1 +7.5 points (0.2667 → 0.3412 ± 0.0063), accuracy −7.0 points (0.5453 → 0.4748 ± 0.0105). Third, the accuracy/macro-F1 trade-off is systematic on both original datasets, so we treat macro-F1 as the primary benchmark metric and report accuracy alongside. The VTUAD baseline in Table 10 uses the winning recipe directly (tuned + sqrt-smoothed weights): 0.3620 ± 0.0058 accuracy / 0.2390 ± 0.0141 macro-F1 over 12 classes at 81.6:1 imbalance. The per-class profile (seed 42) is topped by sailing (F1 0.736), passengership (0.610), and tug (0.477), with the three test-absent classes (rescue, tanker, pilotvessel) at F1 0. On ShipsEar the same recipe reaches 0.5894 ± 0.0128 accuracy yet only 0.2767 ± 0.0122 macro-F1. Five of its twelve classes have no test segments, a regime in which accuracy alone would mislead, as Section 4.7 confirms at the cross-library level.

Per-class test metrics of the sqrt-weighted Oceanship model (released as test_metrics.csv) show that the macro-F1 gain comes from mid- and tail-class recall, with head classes contributing little. Cargo reaches F1 0.508, Towing 0.420, Tanker 0.339, and Port Tender 0.253, while the four classes with ≤ 11 test segments (Diving ship, Military ship, Spare, Dredging) remain at F1 0. Weighting cannot create information that 3–11 test segments do not contain. The QiandaoEar22 tuned model reaches F1 0.967 on noise (1,194 test segments) but only 0.163–0.220 on the three uuv_*/*_target tail classes. Its in-library competence is therefore concentrated in background-vs-vessel discrimination. This bias later becomes visible as a failure mode in cross-dataset evaluation.

4.2. Cross-Dataset Transfer: Zero-Transfer in Both Directions

Table 11. Cross-dataset evaluation on the shared label space (no adaptation). Chance = majority-class level of the evaluated subset. Source models are the full within-corpus checkpoints (class counts 15/8/12/12 for Oceanship/QiandaoEar22/VTUAD/ShipsEar; Table 1). Zero-transfer protocol: predictions outside the mapping are counted as errors; the companion UDA study [9] instead retrains two-class models on the shared label space (its Table 10 protocol), so the numbers of the two papers are not directly comparable.

Direction	Source model (in-library acc)	Acc	Macro-F1	Evaluated	Pred. outside mapping	Chance
Oceanship → QiandaoEar22	sqrt-weighted (33.3%)	0.0000	0.0000	802	802/802 (100%)	0.8890
Oceanship → QiandaoEar22	tuned (25.0%)	0.0000	0.0000	802	802/802 (100%)	0.8890
QiandaoEar22 → Oceanship	tuned (46.7%)	0.0409	0.0664	2,736	2,571/2,736 (94.0%)	0.5563

Table 11. Cross-dataset evaluation on the shared label space (no adaptation). Chance = majority-class level of the evaluated subset. Source models are the full within-corpus checkpoints (class counts 15/8/12/12 for Oceanship/QiandaoEar22/VTUAD/ShipsEar; Table 1). Zero-transfer protocol: predictions outside the mapping are counted as errors; the companion UDA study [9] instead retrains two-class models on the shared label space (its Table 10 protocol), so the numbers of the two papers are not directly comparable.

Direction	Source model (in-library acc)	Acc	Macro-F1	Evaluated	Pred. outside mapping	Chance
Oceanship → QiandaoEar22	sqrt-weighted (33.3%)	0.0000	0.0000	802	802/802 (100%)	0.8890
Oceanship → QiandaoEar22	tuned (25.0%)	0.0000	0.0000	802	802/802 (100%)	0.8890
QiandaoEar22 → Oceanship	tuned (46.7%)	0.0409	0.0664	2,736	2,571/2,736 (94.0%)	0.5563

Cross-dataset transfer fails completely and systematically (Figure 3). The Oceanship-trained model does not produce a single correct prediction on the 802 evaluated QiandaoEar22 segments: 100% of predictions land on source classes outside the mapping. The QiandaoEar22-trained model reaches 4.1% accuracy, still an order of magnitude below its 55.6% majority-class chance level, with 94.0% of predictions unmappable. The relation between in-library and cross-library performance is also counterintuitive. The model with the better in-library accuracy (QiandaoEar22 tuned, 46.7%) is the only one that transfers at all, yet it too operates far below chance.

Because evaluation is confined to the shared label space (Section 3.6), this collapse cannot be attributed to label mismatch. The ground-truth labels of all 802 and 2,736 evaluated segments are semantically aligned by construction. A model that had learned vessel-class acoustics, as opposed to dataset-specific regularities, would score far above the respective chance levels even with imperfect features. The measured 0.0%/4.1% accuracies therefore isolate acoustic domain shift as the operative failure cause on the evaluated mass, while label shift separately determines that this evaluated mass is only 18.5%/22.4% of the full test sets and that 94–100% of predictions fall outside the mapped space.

4.3. Failure-Mode Diagnostics

The per-class prediction distributions reveal two asymmetric failure modes.

Ocean → lake: everything becomes a high-energy vessel. The Oceanship model maps virtually all QiandaoEar22 input to Tug, Towing, or Anti-pollution equipment: 90% of the 1,194 pure-background noise segments fall into these three classes; of the 89 kaiyuan_target segments, none is predicted as Passenger; none of the 713 speedboat_target segments is predicted as Pleasure Craft. The ocean-trained Passenger and Pleasure-Craft detectors essentially never fire on lake data. The attractor classes are not arbitrary. Tug and Towing are Oceanship’s two largest and most energetic workboat classes (12,833 and 12,173 training segments), whose high-energy broadband signatures dominate the ocean feature space.

Lake → ocean: everything becomes background noise. The QiandaoEar22 model classifies the overwhelming majority of ocean vessels as noise: 865 of 1,214 Passenger segments and 847 of 1,112 Fishing segments, along with 69% of the 2,583 Towing and 42% of the 2,758 Tug segments. The lake-trained “vessel detector” barely responds to open-ocean ship-radiated noise. This is consistent with the in-library profile noted in Section 4.1. The QiandaoEar22 model’s competence is concentrated in the noise-vs-vessel axis (noise F1 0.967). When confronted with ocean vessel spectra that do not resemble its vessel classes, its default attractor is the background class.

The single positive signal. In the QiandaoEar22→Oceanship direction, the speedboat_target prediction column achieves precision 0.679 (112/165), above the 0.5563 majority prior of the evaluated subset: the small-fast-craft acoustic signature contains a component that transfers across datasets. No transferable signal exists for the large passenger vessel pair (kaiyuan ↔ Passenger: zero correct predictions in both directions). Recall remains low (0.074). The transferred detector is conservative: it fires rarely but with high reliability, an operating point that can serve as an anchor for domain adaptation.

4.4. Feature-Space Analysis of the Domain Gap

To move beyond behavioral symptoms, we quantify the domain gap directly in the 128-dimensional feature space of Section 3.2. For each class $c$ of each dataset we compute its centroid spectrum ${\bar{L}}_c\in {\mathbb{R}}^{128}$, defined as the time-average of $L[m,t]$ per segment, averaged over up to 200 randomly sampled training segments (seed 42). We then measure cosine distances between centroids. Figure 4 visualizes the result, and Table 12 summarizes the spectral-band energy distribution.

Table 12. Linear-power fraction of centroid spectra in three bands (low < 500 Hz, mid 500 Hz–2 kHz, high > 2 kHz), range over classes per dataset.

Dataset	Low (<500 Hz)	Mid (500 Hz–2 kHz)	High (>2 kHz)
Oceanship (15 classes)	0.994–1.000	0.000–0.004	0.000–0.002
QiandaoEar22 (8 classes)	0.513–0.649	0.210–0.404	0.064–0.142

Table 12. Linear-power fraction of centroid spectra in three bands (low < 500 Hz, mid 500 Hz–2 kHz, high > 2 kHz), range over classes per dataset.

Dataset	Low (<500 Hz)	Mid (500 Hz–2 kHz)	High (>2 kHz)
Oceanship (15 classes)	0.994–1.000	0.000–0.004	0.000–0.002
QiandaoEar22 (8 classes)	0.513–0.649	0.210–0.404	0.064–0.142

(i) The two domains occupy differently shaped regions of feature space. The global peak-relative level differs by 24.5 dB between the libraries (recomputed on the sampled centroids: −52.96 dB vs. −28.49 dB, matching the training-set normalization constants of Eq. (7), −52.42 vs. −28.73 dB). Oceanship spectra concentrate 99.4–100% of linear power below 500 Hz in every class, consistent with long-range ocean propagation acting as a strong low-pass filter (frequency-dependent absorption and boundary interaction [17]) combined with observatory-scale source ranges. QiandaoEar22 spectra retain 21–40% of power in 500 Hz–2 kHz and up to 14% above 2 kHz, consistent with short-range recordings in a shallow lake where high-frequency content survives. The class-conditional feature distribution is thus dominated by the channel and geometry, with the vessel itself secondary.

(ii) The domain gap dwarfs the class gap. The mean cross-dataset centroid distance is 0.0171 (median 0.0193), versus a mean within-dataset inter-class distance of 0.0010 for Oceanship and 0.0032 for QiandaoEar22, so the domain gap is 17× and 5.3× the class gap, respectively. This is the acoustic analogue of the “Name That Dataset” experiment of [5]. Dataset identity is the strongest signal in this feature space, and a classifier trained to separate classes within one domain has no reason to carve the other domain along class lines.

(iii) The geometry reproduces both failure modes. Every one of the 15 Oceanship class centroids is closer to the QiandaoEar22 noise centroid than to any QiandaoEar22 vessel centroid (the closest cross-dataset pair overall is Spare↔noise at 0.0036, followed by Diving ship↔noise and Pleasure Craft↔noise). A lake-trained model that partitions its feature space into “vessel-like” and “noise-like” regions therefore receives ocean vessels (whose steeply low-pass spectra resemble the lake’s background more than its close-range vessel recordings) squarely in the noise region. The lake→ocean failure mode of Section 4.3 is thus the geometric default behavior. Conversely, the QiandaoEar22 centroids lie far from the Oceanship Passenger/Pleasure-Craft centroids (see (iv)). The ocean model’s mapped-class detectors therefore never fire, and its predictions collapse onto the dominant workboat attractors.

(iv) The transferable class is the geometrically closest mapped pair. The small_fast_craft pair (Pleasure Craft ↔ speedboat_target) sits at cosine distance 0.0190, the 47th percentile of all 120 cross-dataset pairs, a typical cross-dataset distance. The passenger_ship pair (Passenger ↔ kaiyuan_target) instead sits at 0.0226, the 93rd percentile, among the most distant pairs. The lone positive transfer signal of Section 4.3 (precision 0.679) thus coincides with the one semantically mapped pair that is not geometrically anomalous, and the zero-transfer passenger pair is the pair whose centroids are farthest apart. Feature-space geometry and classifier behavior are consistent.

QiandaoEar22 classes are also intrinsically more heterogeneous. Their intra-class dispersion (mean cosine distance of segments to their class centroid, 0.0037–0.0102) is several times Oceanship’s (0.0006–0.0060), reflecting the multi-target, short-range variability of the lake recordings.

4.5. Why Does the Speedboat Signal Transfer?

The asymmetry between the two mapped pairs has a physical explanation. Large vessels (Passenger, KaiYuan) radiate energy dominated by low-frequency machinery tonals and blade-rate line spectra. These components are the most strongly shaped by propagation (modal filtering, bottom interaction, frequency-dependent absorption [17]) and are recorded at long range in Oceanship but short range in QiandaoEar22. The same vessel type therefore presents genuinely different received spectra in the two corpora (centroid distance 0.0226, 93rd percentile). Small fast craft instead radiate a strong broadband cavitation component at mid-to-high frequencies, produced by high-speed, partially ventilating propellers. This component is distinctive in the QiandaoEar22 speedboat recordings. It is also present in Oceanship’s Pleasure Craft class, whose small recreational craft are recorded at shorter range and keep more high-frequency energy than the large-vessel classes (Figure 4a: the red pair tracks visibly better above ~300 Hz than the blue pair). The transferring signal is thus the component least processed by the channel. Two caveats apply: the mapping is approximate (Pleasure Craft mixes subtypes), and the transferred detector operates at low recall (0.074). The result therefore demonstrates the existence of a transferable component; it does not yield a usable standalone classifier.

4.6. Extension to a Third Library: VTUAD (Same-Water, Independent Labeling)

The VTUAD reconstruction (Section 3.1) adds a third library that shares the water body, recording region, and observatory platform with Oceanship while differing in hydrophone unit, labeling pipeline, and scenario definition. Its per-class composition (Table 13) is long-tailed but far less extreme than Oceanship’s (81.6:1 vs. 4,278:1), and nine of the twelve classes have test support. Trained with the best recipe of Section 4.1 (tuned + sqrt-smoothed weights), the VTUAD in-library baseline reaches 0.3620 ± 0.0058 accuracy / 0.2390 ± 0.0141 macro-F1 over 12 classes (seeds 42/43/44; Table 10), a usable reference model well above its 12-class random level.

We then evaluate all four new cross-library directions under the no-adaptation protocol of Section 3.6 (Table 14).

Three observations extend the zero-transfer finding. First, zero-transfer generalizes to all six cross-library directions (the two original directions of Table 11 plus the four new directions of Table 14): every accuracy in Table 14 sits at least 40 points below its majority-class chance level, and 52.5–99.1% of predictions fall outside the mapped space. Second, the same-water pair collapses too. Oceanship→VTUAD, with identical water body and recording region and differing only in hydrophone unit, AIS labeling pipeline, and scenario population, yields 0.9% accuracy with 99.0% of predictions unmappable. The ocean model’s mapped-class detectors essentially never fire on VTUAD audio either (passenger_ship recall 0.015, though at 0.958 precision). Dataset bias in UATR is therefore not reducible to water-body or channel differences: labeling provenance and scenario definition alone suffice for complete transfer failure. Third, the VTUAD-source directions are the least catastrophic (10–12% accuracy, and a lower unmapped-prediction mass of 52.5–72.8%). This is consistent with VTUAD’s 4 km scenario geometry producing closer-range, higher-bandwidth recordings that partially resemble the other libraries’ mapped classes, but they remain an order of magnitude below chance.

A companion study [9] runs the full UDA protocol on this shared label space, including the four VTUAD directions (source-only, CORAL, multi-kernel MMD, and a supervised target oracle; three seeds each, source-validation model selection). Those UDA numbers are measured on two-class models retrained on the shared label space, a different protocol from the frozen 15/8/12-class source models of Table 11, Table 14 and Table 16. The two sets of numbers are therefore not directly comparable. Table 15 summarizes the target-test macro-F1 of the two moment-matching methods to close the loop opened by the present benchmark; the full four-method results are reported in [9].

Table 15. UDA summary on the VTUAD-related directions (target-test macro-F1, mean ± std over seeds 42/43/44, n = 3; from the companion study [9], identical shared label space and splits). The target oracle is a supervised upper bound and is not counted among the UDA methods. Bold marks the best UDA method per direction (none on VTUAD→QiandaoEar22, where both UDA methods fall below the source-only lower bound). Chance-level macro-F1 of a majority-only predictor is 0 on all directions.

Direction	Source-only	CORAL	MMD	Target oracle (upper bound)
Oceanship → VTUAD	0.4054 ± 0.0356	0.4202 ± 0.0292	0.3421 ± 0.0241	0.8237 ± 0.0085
QiandaoEar22 → VTUAD	0.2741 ± 0.0000	0.4136 ± 0.0540	0.3880 ± 0.0747	0.8237 ± 0.0085
VTUAD → Oceanship	0.4032 ± 0.0136	0.4586 ± 0.0100	0.4685 ± 0.0105	0.7119 ± 0.0069
VTUAD → QiandaoEar22	0.4000 ± 0.0118	0.3119 ± 0.0472	0.2103 ± 0.0203	0.5112 ± 0.0216

Table 15. UDA summary on the VTUAD-related directions (target-test macro-F1, mean ± std over seeds 42/43/44, n = 3; from the companion study [9], identical shared label space and splits). The target oracle is a supervised upper bound and is not counted among the UDA methods. Bold marks the best UDA method per direction (none on VTUAD→QiandaoEar22, where both UDA methods fall below the source-only lower bound). Chance-level macro-F1 of a majority-only predictor is 0 on all directions.

Direction	Source-only	CORAL	MMD	Target oracle (upper bound)
Oceanship → VTUAD	0.4054 ± 0.0356	0.4202 ± 0.0292	0.3421 ± 0.0241	0.8237 ± 0.0085
QiandaoEar22 → VTUAD	0.2741 ± 0.0000	0.4136 ± 0.0540	0.3880 ± 0.0747	0.8237 ± 0.0085
VTUAD → Oceanship	0.4032 ± 0.0136	0.4586 ± 0.0100	0.4685 ± 0.0105	0.7119 ± 0.0069
VTUAD → QiandaoEar22	0.4000 ± 0.0118	0.3119 ± 0.0472	0.2103 ± 0.0203	0.5112 ± 0.0216

Table 16. Cross-library evaluation between ShipsEar and the other three libraries on the shared label space (no adaptation). Chance = majority-class level of the evaluated subset. The evaluated mass on ShipsEar is 511/906 (56.4%) of its test set. ShipsEar’s frozen split, class-mapping table, and per-seed results are released with this paper; the original audio is not redistributed with the release. Source models are the full within-corpus checkpoints (class counts 15/8/12/12 for Oceanship/QiandaoEar22/VTUAD/ShipsEar; Table 1). ShipsEar source rows report the seed-42 checkpoint; the 58.9% ± 1.3 figure describes the three-seed training protocol.

Direction	Source model (in-library acc)	Acc	Macro-F1	Evaluated	Pred. outside mapping	Chance
Oceanship → ShipsEar	sqrt-weighted (33.3%)	0.0000	0.0000	511	511 (100%)	0.8356
QiandaoEar22 → ShipsEar	tuned (46.7%)	0.0196	0.0364	511	411 (80.4%)	0.8356
VTUAD → ShipsEar	tuned + sqrt (36.3%)	0.1605	0.1845	511	363 (71.0%)	0.8356
ShipsEar → Oceanship	tuned + sqrt (58.9% ± 1.3, 3 seeds)	0.1963	0.2022	2,736	1,209 (44.2%)	0.5563
ShipsEar → QiandaoEar22	tuned + sqrt (58.9% ± 1.3, 3 seeds)	0.4788	0.4296	802	84 (10.5%)	0.8890
ShipsEar → VTUAD	tuned + sqrt (58.9% ± 1.3, 3 seeds)	0.2424	0.2248	2,426	1,320 (54.4%)	0.6224

Table 16. Cross-library evaluation between ShipsEar and the other three libraries on the shared label space (no adaptation). Chance = majority-class level of the evaluated subset. The evaluated mass on ShipsEar is 511/906 (56.4%) of its test set. ShipsEar’s frozen split, class-mapping table, and per-seed results are released with this paper; the original audio is not redistributed with the release. Source models are the full within-corpus checkpoints (class counts 15/8/12/12 for Oceanship/QiandaoEar22/VTUAD/ShipsEar; Table 1). ShipsEar source rows report the seed-42 checkpoint; the 58.9% ± 1.3 figure describes the three-seed training protocol.

Direction	Source model (in-library acc)	Acc	Macro-F1	Evaluated	Pred. outside mapping	Chance
Oceanship → ShipsEar	sqrt-weighted (33.3%)	0.0000	0.0000	511	511 (100%)	0.8356
QiandaoEar22 → ShipsEar	tuned (46.7%)	0.0196	0.0364	511	411 (80.4%)	0.8356
VTUAD → ShipsEar	tuned + sqrt (36.3%)	0.1605	0.1845	511	363 (71.0%)	0.8356
ShipsEar → Oceanship	tuned + sqrt (58.9% ± 1.3, 3 seeds)	0.1963	0.2022	2,736	1,209 (44.2%)	0.5563
ShipsEar → QiandaoEar22	tuned + sqrt (58.9% ± 1.3, 3 seeds)	0.4788	0.4296	802	84 (10.5%)	0.8890
ShipsEar → VTUAD	tuned + sqrt (58.9% ± 1.3, 3 seeds)	0.2424	0.2248	2,426	1,320 (54.4%)	0.6224

Read against Table 14, adaptation is doing real work: retraining on the shared space with moment matching lifts macro-F1 from ≈0.0–0.15 to 0.31–0.47. The recovery is partial and method-sensitive, however. CORAL is the most reliable choice across the four directions (never significantly worse than source-only), and MMD turns significantly harmful on VTUAD→QiandaoEar22 (−0.190 macro-F1 vs. source-only, p = 0.008). On VTUAD-target directions the best UDA method reaches only about half of the supervised oracle (0.414/0.824), leaving the largest measured headroom of the six-direction matrix. The zero-transfer baseline documented in this paper is therefore no dead end, but closing it on auto-labeled third domains remains an open problem.

4.7. Extension to a Fourth Library: ShipsEar (Nearshore Harbor, Registry Labels)

The ShipsEar fourth library (Section 3.1) differs from the other three in range geometry and platform. It consists of nearshore, short-range (mostly <50 m), moored-hydrophone recordings off the Spanish Atlantic coast, with labels taken verbatim from the official vessel registry (shipsEar.xlsx). Its two-class mappable subset (passenger_ship = passengers + ocean_liner; small_fast_craft = motorboat) covers 511/906 (56.4%) of its test segments and is 83.6% majority-class, the most extreme evaluated-subset skew in this benchmark. We evaluate all six cross-library directions involving ShipsEar under the no-adaptation protocol of Section 3.6 (Table 16). The source models are the frozen in-library checkpoints of Section 4.2 and Section 4.6 (seed 42), plus the tuned + sqrt-smoothed ShipsEar model (in-library accuracy 0.5894 ± 0.0128 over seeds 42/43/44; Table 10).

Three observations complete the twelve-direction picture. First, zero-transfer now holds on all twelve cross-library directions: every accuracy in Table 16 sits at least 36 points below its majority-class chance level, and 10.5–100% of predictions fall outside the mapped space. Second, the Oceanship→ShipsEar direction fails completely: none of the 511 evaluated segments receives a prediction inside the mapped space. The open-ocean model’s passenger-ship and pleasure-craft detectors never fire on nearshore harbor audio. This is the same detector-silence failure mode as Oceanship→VTUAD (0.9% accuracy, 99.0% unmappable). Third, the ShipsEar-source directions are the least catastrophic of the benchmark (19.6–47.9% accuracy, with only 10.5% unmappable predictions toward QiandaoEar22). This is consistent with near-range, high-bandwidth harbor recordings partially resembling the other libraries’ mapped classes. Even so, the best direction (ShipsEar→QiandaoEar22, 47.9% accuracy / 43.0% macro-F1) remains 41 points below its 88.9% chance level. Because every ShipsEar-target evaluated subset is 83.6% majority-class, accuracy on these directions is especially misleading. Macro-F1 is the only informative metric there. The companion UDA study [9] extends the same frozen two-class protocol to three ShipsEar directions and finds the same direction-dependent asymmetry. The ShipsEar directions thus confirm the direction dependence from the opposite end. When the source domain is nearshore harbor recordings, cross-library transfer is relatively less catastrophic, yet it remains far below chance.

5. Discussion

Zero-transfer is a genuine finding. Three design choices rule out trivial explanations. First, preprocessing is identical for all datasets, and evaluation is restricted to semantically mapped classes. The collapse therefore cannot be attributed to feature-format mismatch or label noise in the mapping. Second, all source models work in-library (33.3%, 46.7%, and 36.3% accuracy on 15, 8, and 12 classes, respectively, seed-42 checkpoints). The Oceanship and QiandaoEar22 models sit well above their majority-class chance levels (22.6% and 27.6%, Section 3.5); the VTUAD model clears its own chance level (35.7% = 6,492/18,202) by only 0.6 points, but its macro-F1 of 0.24 (three-seed mean, Table 10) is over five times the constant-predictor level (0.04), so it too is unambiguously above chance in the class-balanced sense. Third, the failure modes follow a consistent structure: each model defaults to the acoustically dominant attractor of its training domain (high-energy workboat types for the ocean model, background noise for the lake model). Section 4.4 shows that these attractors coincide with where the target-domain features land geometrically. The VTUAD extension adds the strongest control. The same-water Oceanship→VTUAD pair shares the water body and recording region, yet still fails completely (0.9% accuracy, 99.0% unmappable predictions). The bias therefore extends beyond water-body effects. This suggests that the models learned domain-specific decision boundaries (hydrophone response, channel, ambient-noise floor, vessel population, labeling provenance) instead of vessel-class acoustics per se. This is the same dataset-bias phenomenon documented for vision in [5], here measured for underwater acoustics.

Comparison with vision’s dataset-bias results. Torralba and Efros [5] reported that object classifiers trained on one vision dataset typically lose roughly half their performance on another. They also found that dataset identity is itself highly classifiable. Our results reflect the same phenomenon at a more extreme operating point. Transfer accuracy (0.0–4.1% on the original pair) sits one to two orders of magnitude below the corresponding in-library accuracy (33–47%), far beyond the factor-of-two loss reported for vision. The centroid analysis points to a cause: in this acoustic feature space the domain gap is 5–17× the class gap, far wider than in typical vision feature spaces. Two UATR-specific factors may widen the gap further. The physical channel multiplies the source spectrum by a strongly environment-dependent transfer function [17]. The two corpora also differ in recording geometry (fixed long-range observatory vs. close-range dedicated craft), a confound with no direct analogue in consumer-photo datasets. Architectures borrowed unchanged from vision therefore should not be expected to provide vision-scale robustness in underwater acoustics.

Label shift as an experimental variable. Only two class pairs are semantically shared across four taxonomies: a 15-type AIS taxonomy, an 8-label target-presence taxonomy, a 12-class auto-labeled AIS taxonomy, and a 12-class registry taxonomy. One of the two mappings is approximate. Cross-dataset evaluation is therefore restricted to 18.5%/22.4%/13.3%/56.4% of the respective test sets. 10–100% of the predictions fall outside the mapped space (94–100% on the original two-library pairs). Any future cross-dataset UATR study must publish its mapping table and coverage, since conclusions drawn on unmapped subsets are uninterpretable. The exclusion rationales of Section 3.6 (in particular uuv_target ≁ Diving ship, and *_other ≠ background) concern the decisions that silently inflate scores when made carelessly.

Practical guidance on imbalance. Our paired results suggest a concrete rule of thumb. Inverse-frequency class weighting is safe at ≈10:1 imbalance (QiandaoEar22) but destructive at ≈4,000:1 (Oceanship), where square-root-smoothed weights should be the default. The gradient-variance argument of Section 3.4 predicts that the failure threshold is governed by the smallest class counts (through $\sum _{c}1/n_c$) and only secondarily by the max/min ratio. The numerical equivalence of sqrt smoothing with effective-number weighting at β ≈ 0.995 (for this count distribution) connects the rule to the theoretical account of over-correction in [7], and our case study supplies extreme-scale underwater evidence for it.

Implications for domain adaptation. The results set the agenda for adaptation methods [6] in UATR. First, adaptation is necessary, since no-deploy transfer fails in all twelve cross-library directions. Second, the small-fast-craft class is a natural probe class for alignment methods, because a transferable signal already exists (precision 0.679) and its centroids are the least distant mapped pair. Third, adaptation objectives must cope with partial label overlap. The unmapped-prediction mass (10–100%) means that naive global distribution alignment risks aligning semantically disjoint classes, for example aligning ocean vessels to lake noise, which Section 4.4 shows is the geometrically easiest, and semantically wrong, alignment. Fourth, the 24.5 dB peak-relative level offset and the band-energy mismatch suggest that low-level feature standardization (e.g., per-dataset cepstral mean normalization or channel-invariant front-ends) may be a prerequisite before any adversarial alignment can help. The companion study [9] confirms this agenda empirically on the same frozen splits. Moment-matching UDA recovers macro-F1 0.41–0.47 on three of the four VTUAD directions, but only about half of the supervised oracle on VTUAD targets. One moment-matching variant (MMD) turns significantly harmful on VTUAD→QiandaoEar22. Adaptation method choice is direction-dependent even within the same paradigm family. This benchmark provides the frozen splits, mapping tables, and zero-adaptation baselines against which such methods can be measured.

Limitations. (i) We evaluated one conventional architecture (ResNet-18 on log-mel). Larger or self-supervised models may shift absolute numbers, though the qualitative gap (0–48% vs. 56–89% chance across twelve directions) leaves little room for a different conclusion. (ii) The shared label space remains limited to two pairs because of the taxonomies themselves; ShipsEar [1] has now been incorporated (Section 4.7), and extending the benchmark to DeepShip [2], which shares cargo/tanker/tug-type labels with Oceanship, would widen coverage further. (iii) The approximate speedboat ↔ Pleasure-Craft mapping may mix subtypes. Results on the small_fast_craft pair should be read with this caveat. (iv) The feature-space analysis is centroid-based. It characterizes first-order geometry, and a fuller distributional treatment (e.g., MMD or optimal-transport distances) is left to the adaptation study this benchmark motivates. (v) The datasets differ in many acquisition factors simultaneously (Table 2), so the benchmark measures total shift. The Oceanship↔VTUAD pairs control water body and region but still confound hydrophone unit, labeling pipeline, and scenario definition. Full factor attribution requires controlled re-recording campaigns that do not currently exist publicly. (vi) The VTUAD reconstruction covers a single deployment (2016-05–2017-06) and three of its twelve classes have no test segments. Multi-deployment extension is future work. (vii) The exploratory in-library configurations (the coarse baselines and Oceanship’s inverse-weight variant) remain single-seed (seed 42). Only the final per-dataset configurations are run with seeds 42/43/44. (viii) Each of the twelve cross-library directions is evaluated with a single source checkpoint (seed 42). The seed robustness of between-direction comparisons (e.g., the 10–12% accuracy of the VTUAD-source directions versus the 47.9% of ShipsEar→QiandaoEar22) is therefore untested.

6. Conclusions

We built a unified cross-dataset benchmark for underwater acoustic target recognition over Oceanship, QiandaoEar22, an AIS-auto-labeled VTUAD reconstruction, and ShipsEar. The benchmark comprises 256,785 unique log-mel segments with frozen recording-level splits and an explicit, coverage-annotated label mapping. To our knowledge, it provides the first systematic measurement of direct cross-dataset transfer on public ship-noise corpora under an open, reproducible protocol. Transfer fails completely and systematically: 0.0% accuracy Oceanship→QiandaoEar22 and 4.1% QiandaoEar22→Oceanship, both far below majority-class chance. The two failure modes are asymmetric (ocean→lake: all inputs map to high-energy vessel types; lake→ocean: ocean vessels map to background noise). A centroid-level feature-space analysis shows that the cross-dataset gap is 5–17× the within-dataset class gap. All ocean class centroids lie closer to the lake’s noise centroid than to any lake vessel centroid. The sole transferable class pair is the geometrically closest mapped pair, which gives the failure-mode observations a quantitative mechanism. Under Oceanship’s 4,278:1 imbalance, sqrt-smoothed class weights improve macro-F1 from 0.1302 to 0.1452 ± 0.0136 over seeds 42/43/44 (+24% relative over the unweighted baseline on reference seed 42), while naive inverse weighting collapses training. Sqrt smoothing is numerically equivalent to effective-number weighting at β ≈ 0.995. The VTUAD third library (525 AIS-labeled scenario clips, 12 classes, 132,273 unique segments from the same Strait of Georgia region as Oceanship) extends the measurement to six cross-library directions, all of which fail without adaptation (0.0–11.6% accuracy against 55.6–88.9% chance). Critically, the same-water Oceanship→VTUAD pair collapses to 0.9%. This localizes a substantial part of dataset bias to labeling provenance and scenario definition beyond the acoustic channel. The companion adaptation study [9] shows that moment-matching UDA on these splits recovers only about half of the supervised ceiling on VTUAD targets. The ShipsEar fourth library (90 nearshore harbor recordings, 12 registry classes, 7,425 segments) extends the measurement to twelve cross-library directions, all still far below chance (0.0–47.9% accuracy against 55.6–88.9% majority-class levels). Under its 83.6% majority-class skew, its in-library baseline reaches 58.9% accuracy at only 27.7% macro-F1, the regime in which accuracy alone would mislead. The lone positive signal is speedboat precision 0.679 against a 0.5563 prior under the approximate speedboat↔Pleasure-Craft mapping. This identifies small fast craft as the priority target for the domain-adaptation work this benchmark is designed to motivate and measure. The core finding of this benchmark is the systematic failure of cross-dataset transfer. We present it as a reproducible measurement report; it is no defect to be remediated. Code, frozen splits, mapping tables, and evaluation scripts are available in the project repository.

Relation to the companion papers. This benchmark supplies the zero-adaptation baseline for a coordinated series. The companion UDA study [9] measures how much of the gap source-available adaptation closes; a source-free follow-up [18] measures how much target performance survives when source data are no longer accessible; two pilot studies probe adjacent directions, self-supervised pretraining [19] and open-set recognition [20]; the VTUAD data descriptor [14] documents the reconstructed corpus used here. All of them evaluate on the frozen splits and mapping tables released with this paper.