Concentratord PPS Drift and TOO_EARLY Rejection

faruk 1
Viewed 112

The setup I use:
Raspberry Pi 5, Seeed WM1302, L76K GPS Module
Chirpstack Gateway OS

DISTRIB_ID='ChirpStack Gateway OS'
DISTRIB_RELEASE='4.9.0'
DISTRIB_REVISION='r28959-29397011cc'
DISTRIB_TARGET='bcm27xx/bcm2712'
DISTRIB_ARCH='aarch64_cortex-a76'
DISTRIB_DESCRIPTION='ChirpStack Gateway OS 4.9.0 Base'
DISTRIB_TAINTS='no-all'

Using the Semtech - LoRa(R) CoreCell 868MHz (SX1302CSS868GW1) as shield model because somehow my L76K can emit u-blox messages.

From what I have seen this is a rather niche problem we have been facing:

When the GPS signal goes bad the PPS signal will fall behind, and it will drift away from the internal counter of the SX1302 (INST). However, when the GPS signal is back at healthy levels again the Concentratord can't seem to recover from the previous invalid timing state, which then leads all received downlinks (including the Class-B beacons) to be rejected with TOO_EARLY code because the GPS timestamp of the said downlink is scheduled so so far away from the internal counter of SX1302 (INST). That leaves the restart of the Concentratord as the only option for recovery.

The Semtech's lora_pkt_fwd did not handle this, and it seems like Concentratord does not handle it either so I believe a reliable way to detect this kind of invalid state ((INST - PPS) > Threshold, despite having healthy GPS fix) would at least enable user to react quicker and preemptively reset the internal counter by restarting the Concentratord.

Would love to hear your thoughts or any pointers to be able to implement this and contribute to the Concentratord.

chirpstack-concentratord gateways chirpstack-os
3 Answers

Hey @brocaar, I figured the problem is indeed the GPS module but not the way you think, the userspace GPS output is fully compatible with U-BLOX 7 (haven't verified byte-by-byte but haven't had a parsing issue so far either and Class B comm just works along with location). I believe Seeed is shipping their HATs with modified GPS firmware, because I can see some binary nonsense when I call cat /dev/ttyAMA0 along with the NMEA lines.

The problem however lies within how both modules handle the PPS signal. U-BLOX 7 basically locks to GPS time and maintains the 1 Hz PPS signal even if the signal dies later on, however the Quectel L76K uses the PPS pin to inform the receiver about loss of signal and does not maintain the 1 Hz waves. Which causes difference in INST and PPS within the SX1302. Since U-BLOX is practically not available in where I live, I am thinking about possible solutions to avert this and would love to have your comments on this as well.

faruk1

Interesting discovery. Just to confirm, U-BLOX 7 keeps PPS even when fix is lost, where L76K stops the PPS when fix is lost. When fix is restored, it will resume PPS? Is that understanding correct?

Orne Brocaar•

Either that or it gets degraded/incorrect and messes up the instant timer of SX1302. I figured this oscillator type of PPS generation is called "holdover". You basically use the actual PPS to synch your oscillator drift so in case the GNSS signal gets lossy you can rely on the oscillator until you can synch back again. The surprise is that Seeed selling basically incompatible hardware. This kind of setup they use (L76K) is very fragile in Class B config. I managed to find a U-BLOX NEO M8N and replaced by soldering jumpers and loss on downlinks reduced from ~10% to 0.03% under same conditions.

faruk•

The frequency offset of the external oscillator is measured by the FTS device and communicated
to the host which can then make any corrections necessary. The FTS device also generates a PPS
phase reference internally (with no guarantee of coherence with the external oscillator). During
holdover, the phase of 1PPS signal is maintained using either the primary reference oscillator or
the 1PPS_In signal, according to their respective uncertainty.

from: https://content.u-blox.com/sites/default/files/products/documents/u-blox8-M8_ReceiverDescrProtSpec_UBX-13003221.pdf?utm_content=UBX-13003221

faruk•
Jason Reiss11

We use ublox m8 modules. I would not suggest to use our fork at all. It has changes specific to our hardware. I would suggest making changes to the Semtech reference code.

Jason Reiss•

Thanks a lot, will give it a try. May I ask which GPS module have you been using?

faruk•

I also can't seem to figure which version of gpsd I should be linking against when building your fork, the versions I could find in package managers do not look compatible. Would you be able to give couple tips on this?

faruk•

as shield model because somehow my L76K can emit u-blox messages

This has been mentioned before, but I never managed to make this work. As far as I know, there is no documentation stating that the L76K supports the u-blox.

It might as well be that this is the source of the problem. E.g. it looks like the uBlox binary protocol, but at some point it starts failing.

If you are interested in diving into how the time-sync works:

If you set the log-level to TRACING, you should be able to see this log every second: https://github.com/chirpstack/chirpstack-concentratord/blob/master/chirpstack-concentratord-sx1302/src/handler/gps.rs#L311.

Orne Brocaar576
Related
Concentratord PPS Drift and TOO_EARLY Rejection
3 answers
Chirpstack Gateway OS (Pi4b Base 4.9.0) - Halts GW functions when sending Configuration Command if GNSS is enabled
Setting up a region of concentrator
1 answers
Adding new region configuration
2 answers
HAL error on multicast: Convert downlink protobuf to HAL struct error
1 answers