Bangle.js: Packets occasionally missing when receiving data from watch #3601

New issue

Closed

opened 2024-02-28 17:35:35 +00:00 by Ganblejs · 2 comments

Ganblejs commented

2024-02-28 17:35:35 +00:00

Contributor

Leaving this to keep track of the issue first found here: #3153 (comment).

Please confirm that;

I have checked the website, and I didn't find a solution to my problem / an answer to my question.
I have searched the issues, and I didn't find a solution to my problem / an answer to my question.
I have read and understood the Codeberg Terms of Use for images or other type of content that I included here.

Where did you get Gadgetbridge from?

I built it myself from source code

What is your Gadgetbridge version?

0.79.1-banglejs-6f717344e (includes #3478 and #3153)

What happened?

Reference:

Often (but far from always) when I fetch recorder logs the json received by Gadgetbridge will come in malformed where parts of the string was missing. I'm unsure if the problem lies in Bangle.js or with Gadgetbridge.

#3153 (comment)

[...] honestly this is a strange one. It looks like we've just totally missed a packet (which shouldn't happen!). You can see from:
2024-02-17 20:55:05.775  8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs   D  RX: 119,207.76125986222,
2024-02-17 20:55:05.777  8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs   D  characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x30 0x2c 0x38 0x36 0x2c 0x33 0x2e 0x33 0x32 0x36 0x36 0x36 0x30 0x31 0x35 0x36 0x32 0x34 0x2c 0x66
2024-02-17 20:55:05.778  8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs   D  RX: 0,86,3.32666015624,f
2024-02-17 20:55:05.779  8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs   D  characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x74 0x22 0x3a 0x22 0x61 0x63 0x74 0x54 0x72 0x6b 0x22 0x2c 0x22 0x6c 0x6f 0x67 0x22 0x3a 0x22 0x32
2024-02-17 20:55:05.780  8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs   D  RX: t":"actTrk","log":"2
It just hasn't received anything from BtLEQueue between, where there should have been the end and start of a new object. I don't believe it's a Bangle.js/Espruino issue as when using WebBLE we can back the whole watch up without losing any data. @joserebelo do you recall anyone reporting issues in Gadgetbridge with 'lost' data notifications?

Interestingly fanoush on the forums/GitHub noticed yesterday I'd commented out some code that could have pushed 5x more data through BLE with a note saying that Nordic's Cloud Gateway app lost data if >1 packet was sent per connection interval. I wonder if it's related - it is possible that occasionally we could accidentally send >1 packet per connection interval and then maybe Gadgetbridge could lose it?

#3153 (comment)

I did spot this thread: https://stackoverflow.com/questions/24817107/android-receiving-multiple-ble-packets-per-connection-interval

the API is badly designed, and the characteristic value is a shared object, which may be updated by multiple threads. Each notification may be handled in a different thread and they then call onCharacteristicChanged on one thread (either set in connectGatt(...), or some unspecified thread for older Android versions. If the connection interval is very small, or the device sends many packets in a single interval, it may happen that before you receive the callback with the first notification it will be overwritten by another.

This doesn't entirely make sense since we're not seeing duplicate events with the same value, but could this be related I wonder? On StackOverflow it feels like a bunch of people are complaining about Android dropping notification events, but then Chrome seems to work just fine so whatever they did it works...

#3153 (comment)

Do you have logs?

Yes, shared above in the referenced PR: https://codeberg.org/attachments/f3619b01-8d18-4aa8-a390-3a4d32e3948d .

What gadget do you use?

Bangle.js 2

What is your Android version/manufacturer flavor?

Android 14

Leaving this to keep track of the issue first found here: https://codeberg.org/Freeyourgadget/Gadgetbridge/pulls/3153#issuecomment-1569979. ### Please confirm that; - [x] I have checked the [website](https://gadgetbridge.org), and I didn't find a solution to my problem / an answer to my question. - [x] I have searched the [issues](https://codeberg.org/Freeyourgadget/Gadgetbridge/issues), and I didn't find a solution to my problem / an answer to my question. - [x] I have read and understood the [Codeberg Terms of Use](https://codeberg.org/Codeberg/org/src/branch/main/TermsOfUse.md) for images or other type of content that I included here. ### Where did you get Gadgetbridge from? I built it myself from source code ### What is your Gadgetbridge version? 0.79.1-banglejs-6f717344e (includes #3478 and #3153) ### What happened? Reference: >Often (but far from always) when I fetch recorder logs the json received by Gadgetbridge will come in malformed where parts of the string was missing. I'm unsure if the problem lies in Bangle.js or with Gadgetbridge. *https://codeberg.org/Freeyourgadget/Gadgetbridge/pulls/3153#issuecomment-1569979* > [...] honestly this is a strange one. It looks like we've just totally missed a packet (which shouldn't happen!). You can see from: > > ``` > 2024-02-17 20:55:05.775 8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs D RX: 119,207.76125986222, > 2024-02-17 20:55:05.777 8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs D characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x30 0x2c 0x38 0x36 0x2c 0x33 0x2e 0x33 0x32 0x36 0x36 0x36 0x30 0x31 0x35 0x36 0x32 0x34 0x2c 0x66 > 2024-02-17 20:55:05.778 8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs D RX: 0,86,3.32666015624,f > 2024-02-17 20:55:05.779 8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs D characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x74 0x22 0x3a 0x22 0x61 0x63 0x74 0x54 0x72 0x6b 0x22 0x2c 0x22 0x6c 0x6f 0x67 0x22 0x3a 0x22 0x32 > 2024-02-17 20:55:05.780 8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs D RX: t":"actTrk","log":"2 > ``` > > It just hasn't received anything from BtLEQueue between, where there should have been the end and start of a new object. I don't believe it's a Bangle.js/Espruino issue as when using WebBLE we can back the whole watch up without losing any data. @joserebelo do you recall anyone reporting issues in Gadgetbridge with 'lost' data notifications? > > Interestingly fanoush on the forums/GitHub noticed yesterday I'd commented out some code that could have pushed 5x more data through BLE with a note saying that Nordic's Cloud Gateway app lost data if >1 packet was sent per connection interval. I wonder if it's related - it is possible that occasionally we could accidentally send >1 packet per connection interval and then maybe Gadgetbridge could lose it? *https://codeberg.org/Freeyourgadget/Gadgetbridge/pulls/3153#issuecomment-1574621* > I did spot this thread: https://stackoverflow.com/questions/24817107/android-receiving-multiple-ble-packets-per-connection-interval > > > the API is badly designed, and the characteristic value is a shared object, which may be updated by multiple threads. Each notification may be handled in a different thread and they then call onCharacteristicChanged on one thread (either set in connectGatt(...), or some unspecified thread for older Android versions. If the connection interval is very small, or the device sends many packets in a single interval, it may happen that before you receive the callback with the first notification it will be overwritten by another. > > This doesn't entirely make sense since we're not seeing duplicate events with the same value, but could this be related I wonder? On StackOverflow it feels like a bunch of people are complaining about Android dropping notification events, but then Chrome seems to work just fine so whatever they did it works... *https://codeberg.org/Freeyourgadget/Gadgetbridge/pulls/3153#issuecomment-1574667* ### Do you have logs? Yes, shared above in the referenced PR: https://codeberg.org/attachments/f3619b01-8d18-4aa8-a390-3a4d32e3948d . ### What gadget do you use? Bangle.js 2 ### What is your Android version/manufacturer flavor? Android 14

Ganblejs commented

2024-03-02 14:08:24 +00:00

Author

Contributor

Malformed json mentioned on the espruino forum: https://forum.espruino.com/conversations/370102/

`Malformed json` mentioned on the espruino forum: https://forum.espruino.com/conversations/370102/

Ganblejs commented

2024-04-15 22:13:27 +00:00

Author

Contributor

The problem with missing packets on activity fetching seemed to be solved by changing the logic on Bangle.js side to use an interval instead of a while-loop.

The bug reported on the forum was fixed as well.

Closing this now.

The problem with missing packets on activity fetching seemed to be solved by changing the logic on Bangle.js side to use an interval instead of a while-loop. The bug reported on the forum was fixed as well. Closing this now.