File length calculation with SoundTrap click detector enabled

[JMcCordic]

Hello,
I have found an issue when using Pypam to process recordings collected using the ST300 HF model SoundTrap recorder with the high-frequency click detector enabled.
According to the ST manufacturer website, in this configuration the ST records "continuous audio at a low sample rate, while simultaneously capturing short audio snippets of each click detection at full 576 kHz sample rate". It appears to me that in the xml log files, there are multiple event logs with file type = "wav" in each file, even though one of them is actually for the "snippet" files. Pypam seems to be pulling the sample count listed for the snippets (PROC_EVENT ID = "13") instead of the sample count listed for the 48kHz wavs (PROC_EVENT ID = "5"). This resulted in a calculated wav file duration of ~220s instead of ~14,400s for this continuous recording.

I am not entirely sure of the solution, but I have included images of the jpg output from Pypam as well as an example log file highlighting the discrepancy between the two "wav" events in the xml file in case that is helpful.

Thank you!

[ryjombari]

Hi Jessica. We've never processed data from an ST300 with a click detector enabled, so we've not seen how the output data and metadata are organized. Probably needs a discussion looking at the details together, but a couple of questions for starters: (1) With PBP are you only aiming to process the lower sample rate (48 kHz) continuous data into daily summaries (1-minute hybrid millidecade spectra)? (2) Are the 48 kHz data completely separate from the 576 kHz data in wav files?

[JMcCordic]

Hi John,
Thanks for your response--I understand this is perhaps an uncommon use case. Happy to discuss details further as needed, but to answer your initial questions:
(1) Yes, we are only aiming to use PyPAM/PBP to process the continuous 48kHz sample rate wav files.
(2) Yes, the 48kHz wav files are stored in a separate directory from the click detector files.

[danellecline]

Hi Jessica,

The soundtrap sample count is captured from the first element it finds in "WavFileHandler", so it makes sense that it would be 5 instead of 13 in your XML file.

The best longer-term solution is to avoid using the XML altogether and simply use the wav files.
Another idea is to pass through an option to select the ID.
The shorter-term solution if you need to get up and running quickly is to modify the code starting around line 105 in meta_reader.py using something like (untested)

for proc_event in root.iter("PROC_EVENT"):
    if proc_event.get("ID") == "13":
        for element in proc_event.iter("WavFileHandler"):
            value = element.get("SamplingStartTimeUTC")
            if value:
                wav_start_dt = datetime.strptime(value, "%Y-%m-%dT%H:%M:%S")

            value = element.get("SamplingStopTimeUTC")
            if value:
                wav_stop_dt = datetime.strptime(value, "%Y-%m-%dT%H:%M:%S")

            value = element.get("SampleCount")
            if value:
                sample_count = int(value)

Then reinstall with

poetry build
pip install .

[JMcCordic]

Hi Danelle,

Processing this deployment is not particularly urgent on our end, so if there is a potential solution coming down the line to extract the sample rate and file lengths directly from the wav files, happy to stand by for that.

Thanks!

[danellecline]

@JMcCordic Try to reinstall the latest version 1.7.0 and let us know here if that solution works for you.

pip install mbari-pbp==1.7.0

Here are the release notes on the updates: https://github.com/mbari-org/pbp/blob/main/CHANGELOG.md

[danellecline]

@JMcCordic did this work for you?

[danellecline]

@JMcCordic did this work for you? Can we close this issue?