Bangle.js: Add activity tracks support #3153

Merged
joserebelo merged 85 commits from Ganblejs/Gadgetbridge:bangle-activity-tracks into master 2024-03-21 23:28:35 +00:00
Contributor

WIP - I do work on this from time to time. But I don't know when/if I will get it finished. So if you read this and want to help out I'm happy to pass the torch. Feel free to use the code in this PR, but please leave a note so we don't end up working on it in parallel.

Requested on the Bangle.js forum.

Companion PR to the BangleApps repo: https://github.com/espruino/BangleApps/pull/2819

TODO:

  • Fix - sometimes the fetching of csv from Bangle.js goes wrong with "malformed json".
    • (Discard the files written so far, reset last fetched log info to the one before and restart the fetching of the current log)
    • I haven't seen "malformed json" since switching from while to setInterval.
    • If we miss a packet, currently I try to stop the transmission from Bangle - but that does not work yet, logic is wrong somewhere.
  • Make GB UI stuff work right, e.g. the spinner, having the tracks show up when done (currently they show after going out of and back to the activity tracks screen).
  • Improve error handling.
  • think about robustness
    • on Bangle.js side:
      • handle if the bangle is reset in the middle of sync (e.g. user navigates to another app)
  • Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips or if you don't receive new data in ~5 seconds which would mean something broke.

To work on in a future PR:

  • think more about optimizations
    • on Bangle.js side:
      • number of lines of log to send with a bt packet (currently 4)
      • the time used for setInterval (currently 50 ms)
  • Fix the different logic regarding analyzing the data from the csv file once it's arrived in Gadgetbridge storage.
    • Commented out the summary data that didn't get feasible results for now. Can be worked on to present more data that's also reliable.
    • What's left in is still something.
  • consider what debug messages should be left in or removed/commented out.
  • Make code prettier. Comments, break out into functions, etc.
  • would it be useful in some way to be able to export GPX tracks without location info, but only with e.g. hrm info?
  • make the parsing of stored csv async?
**WIP - I do work on this from time to time. But I don't know when/if I will get it finished. So if you read this and want to help out I'm happy to pass the torch. Feel free to use the code in this PR, but please leave a note so we don't end up working on it in parallel.** [Requested on the Bangle.js forum](https://forum.espruino.com/conversations/387110). Companion PR to the BangleApps repo: https://github.com/espruino/BangleApps/pull/2819 ## TODO: - [x] Fix - sometimes the fetching of csv from Bangle.js goes wrong with "malformed json". - (Discard the files written so far, reset last fetched log info to the one before and restart the fetching of the current log) - I haven't seen "malformed json" since switching from `while` to `setInterval`. - If we miss a packet, ~~currently I try to stop the transmission from Bangle - but that does not work yet, logic is wrong somewhere~~. - [x] Make GB UI stuff work right, e.g. the spinner, having the tracks show up when done (currently they show after going out of and back to the activity tracks screen). - [x] Improve error handling. - [x] think about robustness - on Bangle.js side: - handle if the bangle is reset in the middle of sync (e.g. user navigates to another app) - [x] Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips or if you don't receive new data in ~5 seconds which would mean something broke. ### To work on in a future PR: - [ ] think more about optimizations - on Bangle.js side: - number of lines of log to send with a bt packet (currently 4) - the time used for `setInterval` (currently 50 ms) - [ ] Fix the different logic regarding analyzing the data from the csv file once it's arrived in Gadgetbridge storage. - Commented out the summary data that didn't get feasible results for now. Can be worked on to present more data that's also reliable. - What's left in is still something. - [ ] consider what debug messages should be left in or removed/commented out. - [ ] Make code prettier. Comments, break out into functions, etc. - [ ] would it be useful in some way to be able to export GPX tracks without location info, but only with e.g. hrm info? - [ ] make the parsing of stored csv async?
Member

On the Gadgetbridge changes, take a look into these:

You need to add support for TYPE_GPS_TRACKS on onFetchRecordedData:

Which calls the correct operation, fetches the summary + gps tracks and persists the data to the db:

Only Huami devices support this so far. I have attempted to refactor some things in the past to make future devices slightly easier to add, but may have missed a couple of things, feel free to reach out here or on matrix if something looks weird :)

On the Gadgetbridge changes, take a look into these: You need to add support for TYPE_GPS_TRACKS on onFetchRecordedData: - https://codeberg.org/Freeyourgadget/Gadgetbridge/src/commit/0c52f3d3da7946636b19106177e972a4bd2c2e15/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/huami/HuamiSupport.java#L1664 Which calls the correct operation, fetches the summary + gps tracks and persists the data to the db: - https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/huami/operations/FetchSportsSummaryOperation.java - The operation in Huami devices is split into 2 (summary + gps tracks), so the logic is kind of weird - this is Huami-specific and not mandatory. Only Huami devices support this so far. I have attempted to refactor some things in the past to make future devices slightly easier to add, but may have missed a couple of things, feel free to reach out here or on matrix if something looks weird :)
joserebelo added the
Bangle.js
label 2023-06-08 21:23:09 +00:00
Ganblejs force-pushed bangle-activity-tracks from 4fa7b4c66a to 17486aee73 2023-06-10 09:45:33 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from 17486aee73 to fc77724ffb 2023-06-10 09:52:05 +00:00 Compare
Author
Contributor

@joserebelo

Thank you for the pointers!

You need to add support for TYPE_GPS_TRACKS on onFetchRecordedData:

0c52f3d3da/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/huami/HuamiSupport.java (L1664)

I started mimicking in BangleJSDeviceSupport.java. When you get the time please look at the diff and say if I'm on the right track.

I get this error to be solved in some way:
image

Should it be handled by introducing a FetchSportsSummaryOperation class specific to Bangle.js?

@joserebelo Thank you for the pointers! > You need to add support for TYPE_GPS_TRACKS on onFetchRecordedData: > > 0c52f3d3da/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/huami/HuamiSupport.java (L1664) I started mimicking in BangleJSDeviceSupport.java. When you get the time please look at the diff and say if I'm on the right track. I get this error to be solved in some way: ![image](/attachments/0660b6c4-57c4-451d-b299-a34ca65736db) Should it be handled by introducing a `FetchSportsSummaryOperation` class specific to Bangle.js?
Member

Should it be handled by introducing a FetchSportsSummaryOperation class specific to Bangle.js?

Yup, that one is specific to Huami devices. You might not need to follow the structure from Huami specifically - I am not yet aware of how we usually fetch data from the Bangle.

Huami has 2 operations (summary and details), maybe the bangle can provide everything in one, for example.

> Should it be handled by introducing a FetchSportsSummaryOperation class specific to Bangle.js? Yup, that one is specific to Huami devices. You might not need to follow the structure from Huami specifically - I am not yet aware of how we usually fetch data from the Bangle. Huami has 2 operations (summary and details), maybe the bangle can provide everything in one, for example.
Author
Contributor

I just copied the huami files to the corresponding banglejs destination. I did some small tweaks to remove errors in Android studio. But not any substantial changes to logic - so it shouldn't work yet.

I also see in the diff that I've removed some imports - I don't understand when though. That should be undone, but not doing it right now.

EDIT: I guess mainly copying and modifying is unnecessarily complex and it's better to do the bangle FetchSportsSummaryOperation class more from the ground up.

I just copied the huami files to the corresponding banglejs destination. I did some small tweaks to remove errors in Android studio. But not any substantial changes to logic - so it shouldn't work yet. I also see in the diff that I've removed some imports - I don't understand when though. That should be undone, but not doing it right now. EDIT: I guess mainly copying and modifying is unnecessarily complex and it's better to do the bangle FetchSportsSummaryOperation class more from the ground up.
Member

Yup - especially if the bangle logic is being build on the watch side from scratch as well, the Huami implementation is overkill.

Yup - especially if the bangle logic is being build on the watch side from scratch as well, the Huami implementation is overkill.
Author
Contributor

I imagine something along the lines of this:

  1. Gadgetbridge sends a message with contents t:"fetch", data:"recorder" or similar.
  2. In return Bangle.js sends a message with recorder log contents. Something like: Bluetooth.println(JSON.stringify({t:"push", data:"data read from e.g. recorder.log20230610a.csv on Bangle.js storage"}))
  3. Gadgetbridge receives the data and writes it to its database to be presented in the activity tracks page.

The recorder app: https://github.com/espruino/BangleApps/tree/master/apps/recorder

Recorder app stores data in files named like "recorder.log20230610a.csv", where the ending letter is incremented when there are new recordings within a day.

Contents of recorder files:
Time,Latitude,Longitude,Altitude,Heartrate,Confidence,Source,Battery Percentage,Battery Voltage,Charging,Steps,Barometer Temperature,Barometer Pressure,Barometer Altitude 1686404996,,,,61,57,,63,3.32226562499,false,0,32.51630350748,1016.57829926224,-27.67358933984

The types and frequency of samples can be changed by the user.

I imagine something along the lines of this: 1. [Gadgetbridge sends a message](https://github.com/espruino/EspruinoDocs/blob/master/info/Gadgetbridge.md#messages-sent-to-banglejs-from-phone) with contents `t:"fetch", data:"recorder"` or similar. 2. In return [Bangle.js sends a message](https://github.com/espruino/EspruinoDocs/blob/master/info/Gadgetbridge.md#messages-from-banglejs-to-phone) with recorder log contents. Something like: `Bluetooth.println(JSON.stringify({t:"push", data:"data read from e.g. recorder.log20230610a.csv on Bangle.js storage"}))` 3. Gadgetbridge receives the data and writes it to its database to be presented in the activity tracks page. The recorder app: https://github.com/espruino/BangleApps/tree/master/apps/recorder Recorder app stores data in files named like "recorder.log20230610a.csv", where the ending letter is incremented when there are new recordings within a day. Contents of recorder files: `Time,Latitude,Longitude,Altitude,Heartrate,Confidence,Source,Battery Percentage,Battery Voltage,Charging,Steps,Barometer Temperature,Barometer Pressure,Barometer Altitude 1686404996,,,,61,57,,63,3.32226562499,false,0,32.51630350748,1016.57829926224,-27.67358933984` The types and frequency of samples can be changed by the user.
Member

Yup, that is pretty much it.

I think you will have to generate a gpx file somewhere in order to get the map / share button to work, but GB can still keep the raw csv data.

Yup, that is pretty much it. I think you will have to generate a gpx file somewhere in order to get the map / share button to work, but GB can still keep the raw csv data.
Author
Contributor

Started working on the Bangle.js side in this PR: https://github.com/espruino/BangleApps/pull/2819

Started working on the Bangle.js side in this PR: https://github.com/espruino/BangleApps/pull/2819
Member

with contents t:"fetch", data:"recorder" or similar.

One thing I missed: we probably want to send a timestamp of the latest log that GB has, so that the Bangle can start iterating from that day onward.

> with contents t:"fetch", data:"recorder" or similar. One thing I missed: we probably want to send a timestamp of the latest log that GB has, so that the Bangle can start iterating from that day onward.
Ganblejs force-pushed bangle-activity-tracks from 42904286d6 to 419cb515ab 2023-06-17 14:29:34 +00:00 Compare
Author
Contributor

How I imagine it now:

  1. In Gadgetbridge, press the circular arrow button on the "Sports Activities" page prompting GB to send a message GB({t:"listRecs"}) to Bangle.js. (the id of last synced track could be supplied in the message to Bangle.js to shorten the returned array in the next step)
  2. Bangle.js will return an array with recording ids.
  3. Gadgetbridge goes through the list returned from Bangle.js and asks for the new tracks by requesting them, in chronological order, by sending e.g. GB({t:"fetchRec", id:"20230616a"}). (Here I think some logic to wait between tracks is needed)
  4. Bangle.js will return the track logs line by line.
  5. Gadgetbridge will need to handle receiving the logs line by line in some way and storing them in the right way somehow.

For (1.) I need to find out how to hook into the circular arrow button on the "Sports Activities" page.
(2.) is largely solved.
(4.) is largely solved.
(3.) and (5.) still needs work and I'm not sure of the best way to do it. Currently I plan to do it inside the handleUartRxJSON function in BangleJSDeviceSupport.java in the cases case "trksList" and case "actTrk". (see current diff of this PR)

Here's the current code on Bangle.js side: https://github.com/espruino/BangleApps/pull/2819/files

How I imagine it now: 1. In Gadgetbridge, press the circular arrow button on the "Sports Activities" page prompting GB to send a message `GB({t:"listRecs"})` to Bangle.js. (the id of last synced track could be supplied in the message to Bangle.js to shorten the returned array in the next step) 2. Bangle.js will return an array with recording ids. 3. Gadgetbridge goes through the list returned from Bangle.js and asks for the new tracks by requesting them, in chronological order, by sending e.g. `GB({t:"fetchRec", id:"20230616a"})`. (Here I think some logic to wait between tracks is needed) 4. Bangle.js will return the track logs line by line. 5. Gadgetbridge will need to handle receiving the logs line by line in some way and storing them in the right way somehow. For (1.) I need to find out how to hook into the circular arrow button on the "Sports Activities" page. (2.) is largely solved. (4.) is largely solved. (3.) and (5.) still needs work and I'm not sure of the best way to do it. Currently I plan to do it inside the `handleUartRxJSON` function in `BangleJSDeviceSupport.java` in the cases `case "trksList"` and `case "actTrk"`. (see current diff of this PR) Here's the current code on Bangle.js side: https://github.com/espruino/BangleApps/pull/2819/files
Member

For (1.) I need to find out how to hook into the circular arrow button on the "Sports Activities" page.

See: https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/activities/ActivitySummariesActivity.java#L384

tl;dr: It calls onFetchRecordedData on the support class with RecordedDataTypes.TYPE_GPS_TRACKS.

> For (1.) I need to find out how to hook into the circular arrow button on the "Sports Activities" page. See: https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/activities/ActivitySummariesActivity.java#L384 tl;dr: It calls `onFetchRecordedData` on the support class with `RecordedDataTypes.TYPE_GPS_TRACKS`.
Author
Contributor

tl;dr: It calls onFetchRecordedData on the support class with RecordedDataTypes.TYPE_GPS_TRACKS.

Thanks!

I'm inching closer to some naive solution. Gadgetbridge now receives the csv files fine, basically. And also keeps track of the ID of latest synced recorder log.

Next steps are to:

  • On Bangle.js side only send back list of recordings made since last sync. Taking the now transmitted ID of last synced log into consideration.
  • Tweaks here and there in the new GB code.
  • Take the data in the csv files stored by Gadgetbridge and shove it into the database so it shows up on the "Sports Activities" page.
  • Change some naming of different things.
    EDIT:
  • Break code out into functions or classes.
>tl;dr: It calls onFetchRecordedData on the support class with RecordedDataTypes.TYPE_GPS_TRACKS. Thanks! I'm inching closer to some naive solution. Gadgetbridge now receives the csv files fine, basically. And also keeps track of the ID of latest synced recorder log. Next steps are to: - [x] On Bangle.js side only send back list of recordings made since last sync. Taking the now transmitted ID of last synced log into consideration. - [ ] Tweaks here and there in the new GB code. - [ ] Take the data in the csv files stored by Gadgetbridge and shove it into the database so it shows up on the "Sports Activities" page. - [ ] Change some naming of different things. EDIT: - [ ] Break code out into functions or classes.
Ganblejs force-pushed bangle-activity-tracks from 984b6d6301 to de85400d7c 2023-06-25 22:18:02 +00:00 Compare
Author
Contributor

I'm stuck on how to get the data into the database from the received csv files. @joserebelo do you have some suggestion or another pointer? I guess I should be able to work it out by looking at the huamisupport class, but it's not easy for me.

  • Take the data in the csv files stored by Gadgetbridge and shove it into the database so it shows up on the "Sports Activities" page.

And also I don't currently know how to generate gpx files from the gps data.

I think you will have to generate a gpx file somewhere in order to get the map / share button to work, but GB can still keep the raw csv data.

I'm stuck on how to get the data into the database from the received csv files. @joserebelo do you have some suggestion or another pointer? I guess I should be able to work it out by looking at the huamisupport class, but it's not easy for me. > - [ ] Take the data in the csv files stored by Gadgetbridge and shove it into the database so it shows up on the "Sports Activities" page. And also I don't currently know how to generate gpx files from the gps data. >I think you will have to generate a gpx file somewhere in order to get the map / share button to work, but GB can still keep the raw csv data.
Member

@Ganblejs it's a somewhat weird part of the code, it took me a while to make sense of it.

  1. You need to persist a summary to the database. For that, you create a BaseActivitySummary, where you can set some data such as the name, start and end times. I believe the only mandatory fields are start time, end time, and activityKind. This class is auto-generated.

You then must persist it to the database - here is an example on Huami, lines 98 to 109. You can probably ignore the part about the summaryParser and raw summary data / binary data.

  1. This summary can also contain a rawDetailsPath - here is where you should save the full csv fetched from the Bangle. In Huami we do it separately (see here).

  2. On order to generate the gpx, you must populate an ActivityTrack. You can then use a GpxExporter to write it to disk, and set the gpxTrack on the summary. We do it here for Huami.

On Huami devices, step 1 is done separately from 2 and 3 because of how the watches work. For the bangle, you can probably persist everything in one step, since you have all the csv data from the start.

@Ganblejs it's a somewhat weird part of the code, it took me a while to make sense of it. 1. You need to persist a summary to the database. For that, you create a `BaseActivitySummary`, where you can set some data such as the name, start and end times. I believe the only mandatory fields are start time, end time, and activityKind. [This class is auto-generated.](https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/GBDaoGenerator/src/nodomain/freeyourgadget/gadgetbridge/daogen/GBDaoGenerator.java#L727) You then must persist it to the database - [here is an example on Huami, lines 98 to 109](https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/huami/operations/FetchSportsSummaryOperation.java#L98). You can probably ignore the part about the summaryParser and raw summary data / binary data. 2. This summary can also contain a rawDetailsPath - here is where you should save the full csv fetched from the Bangle. In Huami we do it separately (see [here](https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/huami/operations/FetchSportsDetailsOperation.java#L121)). 3. On order to generate the gpx, you must populate an [ActivityTrack](https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/model/ActivityTrack.java). You can then use a [GpxExporter](https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/export/GPXExporter.java) to write it to disk, and set the gpxTrack on the summary. We do it [here](https://codeberg.org/Freeyourgadget/Gadgetbridge/src/commit/c6ebfc490029b70afe557138a4adf637cea99a4e/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/service/devices/huami/operations/FetchSportsDetailsOperation.java#L127) for Huami. On Huami devices, step 1 is done separately from 2 and 3 because of how the watches work. For the bangle, you can probably persist everything in one step, since you have all the csv data from the start.
Author
Contributor

Thanks, that will help me a bunch I think!

Thanks, that will help me a bunch I think!
Ganblejs force-pushed bangle-activity-tracks from de85400d7c to a3db96ffb3 2023-07-04 21:05:57 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from 624a3bd4f1 to 9eb78dcc1a 2023-07-04 22:21:04 +00:00 Compare
Author
Contributor

I now manage to persist an activity to the database. It has the correct date and time set, but I don't think much else at the moment.

I now manage to persist an activity to the database. It has the correct date and time set, but I don't think much else at the moment.
Ganblejs force-pushed bangle-activity-tracks from 74d876eca7 to 7cdbd1ce69 2023-07-06 02:08:26 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from c27aacbde4 to f8451d560a 2023-07-07 00:07:41 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from f8451d560a to 63c6a0a56f 2023-07-07 00:13:22 +00:00 Compare
Author
Contributor

I now have the GPS points show up in the "Sport Activity Detail" page:

image

I now have the GPS points show up in the "Sport Activity Detail" page: ![image](/attachments/2cbbaa28-5b8e-4557-9fe3-0086bca638e4)
Ganblejs force-pushed bangle-activity-tracks from 7e28dc645e to 5967399c43 2023-07-07 11:17:21 +00:00 Compare
Author
Contributor

How do I set details so they show up like in the docs? :

image

I don't really see how I would add most of this (e.g. steps) to a BaseActivitySummary, ActivityTrack or ActivityPoint. I do manage to add gps-data and hr-data to the ActivityPoint, and they then show up in the ActivityTrack and the exported gpx-file - but the hr-data is not displayed on my "Sports Activity Detail" page.

How do I set details so they show up [like in the docs](https://codeberg.org/Freeyourgadget/Gadgetbridge/wiki/Sports-Activities-Workouts)? : ![image](/attachments/735d2d6b-a05d-4276-8ad4-ac0568fbd2ba) I don't really see how I would add most of this (e.g. steps) to a `BaseActivitySummary`, `ActivityTrack` or `ActivityPoint`. I do manage to add gps-data and hr-data to the `ActivityPoint`, and they then show up in the `ActivityTrack` and the exported gpx-file - but the hr-data is not displayed on my "Sports Activity Detail" page.
125 KiB
Member

@Ganblejs you need to set the summaryData object, which is a json string. The object contains a set of keys, each key to an object with a value and a unit. Eg.

{
    "ascentSeconds": {
        "value": 100,
        "unit": "seconds"
    },
    // ...
}

Here are the available keys:

And where we set them on HuamiActivitySummaryParser (see addSummaryData and its usages): https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/devices/huami/HuamiActivitySummaryParser.java#L384

@Ganblejs you need to set the summaryData object, which is a json string. The object contains a set of keys, each key to an object with a value and a unit. Eg. ```json { "ascentSeconds": { "value": 100, "unit": "seconds" }, // ... } ``` Here are the available keys: https://codeberg.org/Freeyourgadget/Gadgetbridge/src/commit/932249f148deffe7be6a132c6d0b2328ebb09ecb/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/model/ActivitySummaryJsonSummary.java#L153 And where we set them on HuamiActivitySummaryParser (see addSummaryData and its usages): https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/devices/huami/HuamiActivitySummaryParser.java#L384
Member

but the hr-data is not displayed on my "Sports Activity Detail" page.

This is one of the things that is on my TODO list for a while: the HR data on activity details pages is not pulled from the activity points, but only from the synced activity data - which from what I see, the Bangle does not yet support.

We should eventually update this details page to use the data from gpx if available, and only then fallback to the activity data.

> but the hr-data is not displayed on my "Sports Activity Detail" page. This is one of the things that is on my TODO list for a while: the HR data on activity details pages is not pulled from the activity points, but only from the synced activity data - which from what I see, the Bangle does not yet support. We should eventually update this details page to use the data from gpx if available, and only then fallback to the activity data.
Author
Contributor

Ok, thanks again!

Ok, thanks again!
Ganblejs changed title from WIP: Bangle.js: Add activity tracks support to Bangle.js: Add activity tracks support 2023-07-08 01:06:20 +00:00
Ganblejs changed title from Bangle.js: Add activity tracks support to Bangle.js: Add activity tracks support 2023-07-08 01:06:21 +00:00
Member

I think we should start considering breaking some functionality to standalone classes, this switch/class is becoming huge.

Something like a BangleActivityTracksService, where we could just handle everything related with tracks in a self-contained manner.

Maybe not on this PR.

I think we should start considering breaking some functionality to standalone classes, this switch/class is becoming huge. Something like a BangleActivityTracksService, where we could just handle everything related with tracks in a self-contained manner. Maybe not on this PR.
Ganblejs changed title from Bangle.js: Add activity tracks support to WIP: Bangle.js: Add activity tracks support 2023-07-08 22:03:07 +00:00
Author
Contributor

I think we should start considering breaking some functionality to standalone classes, this switch/class is becoming huge.

Something like a BangleActivityTracksService, where we could just handle everything related with tracks in a self-contained manner.

Sounds sensible to me. There's already this PR for some refactoring: #3149

> I think we should start considering breaking some functionality to standalone classes, this switch/class is becoming huge. > > Something like a BangleActivityTracksService, where we could just handle everything related with tracks in a self-contained manner. Sounds sensible to me. There's already this PR for some refactoring: https://codeberg.org/Freeyourgadget/Gadgetbridge/pulls/3149
Ganblejs force-pushed bangle-activity-tracks from 5967399c43 to dba4b7c5f0 2023-07-12 22:44:58 +00:00 Compare
joserebelo reviewed 2023-07-12 22:53:05 +00:00
@ -675,0 +872,4 @@
} catch (FileNotFoundException e) {
throw new RuntimeException(e);
} catch (IOException e) {
throw new RuntimeException(e);
Member

This crashes GB :)

We should probably handle it a bit more gracefully.

This crashes GB :) We should probably handle it a bit more gracefully.
Author
Contributor

Thanks! :) Still very much a work in progress :P

Thanks! :) Still very much a work in progress :P
Author
Contributor

I realize I probably haven't fixed this yet. I might have hit it some.

I realize I probably haven't fixed this yet. I might have hit it some.
joserebelo reviewed 2023-07-12 22:55:52 +00:00
@ -1305,6 +1552,29 @@ public class BangleJSDeviceSupport extends AbstractBTLEDeviceSupport {
@Override
public void onFetchRecordedData(int dataTypes) {
if (dataTypes == RecordedDataTypes.TYPE_GPS_TRACKS) {
Member

One thing I forgot to mention: We probably want to mark the bangle as busy as this operation starts (getDevice().setBusyTask("Operation name");), and unmark it as it finishes (getDevice().unsetBusyTask();)

This should show the operation spinner both in the main activity card, as well as a spinner in the activity list, and prevent multiple fetches from being triggered in parallel.

One thing I forgot to mention: We probably want to mark the bangle as busy as this operation starts (`getDevice().setBusyTask("Operation name");`), and unmark it as it finishes (`getDevice().unsetBusyTask();`) This should show the operation spinner both in the main activity card, as well as a spinner in the activity list, and prevent multiple fetches from being triggered in parallel.
Author
Contributor

I just had a go at this with this commit: 14415215fe

... and this one to the companion PR: 20ce944469

Before this I would get a perpetual spinner in the activity list which would go away when I navigated away from that page. With the change I did just now I will see no spinner at all.

I'll not worry too much about this currently. But before a potential merge of this PR the spinner should work.

I just had a go at this with this commit: https://codeberg.org/Freeyourgadget/Gadgetbridge/commit/14415215fe76c3efaecc957f8a93aa1a4ad9a595 ... and this one to the companion PR: https://github.com/espruino/BangleApps/pull/2819/commits/20ce944469b1ccc601ada05e374534a22d8a6e43 Before this I would get a perpetual spinner in the activity list which would go away when I navigated away from that page. With the change I did just now I will see no spinner at all. I'll not worry too much about this currently. But before a potential merge of this PR the spinner should work.
Member

With the change I did just now I will see no spinner at all.

Is it so fast that the spinner does not even have time to show? 🤔

> With the change I did just now I will see no spinner at all. Is it so fast that the spinner does not even have time to show? 🤔
Author
Contributor

No I don't think so. If I clear the data in Gadgetbridge and fetch all my logs again, the logcat shows the process is taking more than 10 seconds.

But lets worry more about this when the new code is closer to being finalized 👍

No I don't think so. If I clear the data in Gadgetbridge and fetch all my logs again, the logcat shows the process is taking more than 10 seconds. But lets worry more about this when the new code is closer to being finalized 👍
Ganblejs force-pushed bangle-activity-tracks from 14415215fe to 94e7b1cdb0 2023-08-09 21:52:49 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from 94e7b1cdb0 to a323776c67 2023-10-04 18:41:35 +00:00 Compare
Author
Contributor

but the hr-data is not displayed on my "Sports Activity Detail" page.

This is one of the things that is on my TODO list for a while: the HR data on activity details pages is not pulled from the activity points, but only from the synced activity data - which from what I see, the Bangle does not yet support.

We should eventually update this details page to use the data from gpx if available, and only then fallback to the activity data.

@joserebelo, am I correct in assuming then that there's no function in Gadgetbridge to calculate distances/speeds/etc from the gpx-file for adding to the summary? I've tried to understand GpxParser.java and I don't think it is used for this?

> > but the hr-data is not displayed on my "Sports Activity Detail" page. > > This is one of the things that is on my TODO list for a while: the HR data on activity details pages is not pulled from the activity points, but only from the synced activity data - which from what I see, the Bangle does not yet support. > > We should eventually update this details page to use the data from gpx if available, and only then fallback to the activity data. @joserebelo, am I correct in assuming then that there's no function in Gadgetbridge to calculate distances/speeds/etc from the gpx-file for adding to the summary? I've tried to understand `GpxParser.java` and I don't think it is used for this?
Member

@Ganblejs yup, you are correct. GpxParser really just reads a gpx file to a GpxFile object, but that's really just a wrapper for the tracks and waypoints, there's currently nothing to build statistics from that :/

@Ganblejs yup, you are correct. GpxParser really just reads a gpx file to a GpxFile object, but that's really just a wrapper for the tracks and waypoints, there's currently nothing to build statistics from that :/
Author
Contributor

@Ganblejs yup, you are correct. GpxParser really just reads a gpx file to a GpxFile object, but that's really just a wrapper for the tracks and waypoints, there's currently nothing to build statistics from that :/

If I were to make a class for extracting statistics from gpx-files, would it be good to use GpxParser inside it? Or is it not suited for that and it's better to build more from the ground up (will be a naive implementation if I make it)?

> @Ganblejs yup, you are correct. GpxParser really just reads a gpx file to a GpxFile object, but that's really just a wrapper for the tracks and waypoints, there's currently nothing to build statistics from that :/ If I were to make a class for extracting statistics from gpx-files, would it be good to use GpxParser inside it? Or is it not suited for that and it's better to build more from the ground up (will be a naive implementation if I make it)?
Member

Either having the GpxParser inside or receive a GpxFile as argument, both should be fine. I tried to make that class reusable for anything gpx-parsing-related, as this :)

Either having the GpxParser inside or receive a GpxFile as argument, both should be fine. I tried to make that class reusable for anything gpx-parsing-related, as this :)
Ganblejs force-pushed bangle-activity-tracks from a323776c67 to 09821065d8 2023-11-02 23:47:46 +00:00 Compare
Author
Contributor

I seem to have broken this PR/fork. How do I mitigate that? I force pushed yeaterday and that has not showed up here.

Edit: Looking at the branch on my fork in the web ui the force push is shown there, but it's not forwarded to this PR.

I suppose I could just close this PR and open a new one.

I seem to have broken this PR/fork. How do I mitigate that? I force pushed yeaterday and that has not showed up here. Edit: Looking at the branch on my fork in the web ui the force push is shown there, but it's not forwarded to this PR. I suppose I could just close this PR and open a new one.
Member

Maybe try to force-push again?

It seems like Codeberg was going through some hiccups, this is not the only PR affected..

Maybe try to force-push again? It seems like Codeberg was going through some hiccups, this is not the only PR affected..
Ganblejs reviewed 2023-11-08 00:52:50 +00:00
@ -695,0 +797,4 @@
JSONArray valueArray2 = new JSONArray();
// Add analytics based on GPS coordinates.
Author
Contributor

Weirdly I get in the logcat:

2023-11-08 01:33:58.070 13830-13919 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs   I  UART RX JSON parse failure: Value  at 13 of type java.lang.String cannot be converted to double
2023-11-08 01:33:58.071 13830-13919 nodomain.f...ge.util.GB com.espruino.gadgetbridge.banglejs   E  Malformed JSON from Bangle.js: Value  at 13 of type java.lang.String cannot be converted to double

Which seems like it should come from the handleUartRxLine function. The error goes away if I comment out the analytics below. So I guess I must indirectly use that function somewhere in these calculations. But I can't understand why that would be. Will have to look at it more. Maybe the whole operation is wrapped inside handleUartRxLine, I guess that could be it...

The Value at 13 of type java.lang.String cannot be converted to double part of the logcat has said stuff about NaN and infinity before, but I seem to have resolved those by turning some zeros to 0.001 and changing a null to 0.

It's cumbersome since it will not give me the line number where the problem occurred...

Weirdly I get in the logcat: ``` 2023-11-08 01:33:58.070 13830-13919 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs I UART RX JSON parse failure: Value at 13 of type java.lang.String cannot be converted to double 2023-11-08 01:33:58.071 13830-13919 nodomain.f...ge.util.GB com.espruino.gadgetbridge.banglejs E Malformed JSON from Bangle.js: Value at 13 of type java.lang.String cannot be converted to double ``` Which seems like it should come from the `handleUartRxLine` function. The error goes away if I comment out the analytics below. So I guess I must indirectly use that function somewhere in these calculations. But I can't understand why that would be. Will have to look at it more. Maybe the whole operation is wrapped inside `handleUartRxLine`, I guess that could be it... The `Value at 13 of type java.lang.String cannot be converted to double` part of the logcat has said stuff about `NaN` and `infinity` before, but I seem to have resolved those by turning some zeros to `0.001` and changing a `null` to `0`. It's cumbersome since it will not give me the line number where the problem occurred...
Author
Contributor

I suspect it is the new sumOfJSONArray function et al that are acting up on bad inputs and throws JSONException.

I suspect it is the new `sumOfJSONArray` function et al that are acting up on bad inputs and throws `JSONException`.
Member

BangleJSDeviceSupport in handleUartRxLine, I would replace the log of UART RX JSON parse failure with:

LOG.error("UART RX JSON parse failure", e);
  • Info does not seem appropriate for an error
  • Logging the stack trace is useful :)
BangleJSDeviceSupport in handleUartRxLine, I would replace the log of `UART RX JSON parse failure` with: ```java LOG.error("UART RX JSON parse failure", e); ``` - Info does not seem appropriate for an error - Logging the stack trace is useful :)
Ganblejs force-pushed bangle-activity-tracks from ec879373d0 to ede5445e21 2023-12-06 22:01:37 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from ede5445e21 to 365bc7b85f 2023-12-10 18:03:28 +00:00 Compare
Author
Contributor

Progress so far:

Edit: added a new screenshot with todays progress.

Progress so far: Edit: added a new screenshot with todays progress.
Ganblejs force-pushed bangle-activity-tracks from cdc3ae8da8 to 72fae57665 2024-01-24 22:15:47 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from 72fae57665 to 458cb14c5d 2024-02-17 12:13:59 +00:00 Compare
Author
Contributor

@gfwilliams @joserebelo Do you have any thoughts on this:

Often (but far from always) when I fetch recorder logs the json received by Gadgetbridge will come in malformed where parts of the string was missing. I'm unsure if the problem lies in Bangle.js or with Gadgetbridge.

If possible I'd like to solve the problem, I tried to change from using a while-block to an interval in bangle but that didn't work well (commented out here: https://github.com/espruino/BangleApps/pull/2819/files).

If I can't solve it I'd have to implement logic to back up and refetch the log currently being synced. If so I'd like to be able to cut short the currently executing while-loop on the bangle to restart it - but I don't know if that will be possible?

Edit: Added a logcat that caught a bunch of "Malformed JSON" errors, search for that in the attached file to look at some examples.

@gfwilliams @joserebelo Do you have any thoughts on this: Often (but far from always) when I fetch recorder logs the json received by Gadgetbridge will come in malformed where parts of the string was missing. I'm unsure if the problem lies in Bangle.js or with Gadgetbridge. If possible I'd like to solve the problem, I tried to change from using a while-block to an interval in bangle but that didn't work well (commented out here: https://github.com/espruino/BangleApps/pull/2819/files). If I can't solve it I'd have to implement logic to back up and refetch the log currently being synced. If so I'd like to be able to cut short the currently executing while-loop on the bangle to restart it - but I don't know if that will be possible? Edit: Added a logcat that caught a bunch of "Malformed JSON" errors, search for that in the attached file to look at some examples.
Ganblejs added 1 commit 2024-02-17 20:23:28 +00:00
Member

@Ganblejs I have seen this happen once or twice in the past, but I attributed it to me doing something wrong. Definitely hasn't been happening enough to cause issues like that :/

@Ganblejs I have seen this happen once or twice in the past, but I attributed it to me doing something wrong. Definitely hasn't been happening enough to cause issues like that :/
Member

Sorry for the delay - honestly this is a strange one. It looks like we've just totally missed a packet (which shouldn't happen!). You can see from:

2024-02-17 20:55:05.775  8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs   D  RX: 119,207.76125986222,
2024-02-17 20:55:05.777  8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs   D  characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x30 0x2c 0x38 0x36 0x2c 0x33 0x2e 0x33 0x32 0x36 0x36 0x36 0x30 0x31 0x35 0x36 0x32 0x34 0x2c 0x66
2024-02-17 20:55:05.778  8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs   D  RX: 0,86,3.32666015624,f
2024-02-17 20:55:05.779  8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs   D  characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x74 0x22 0x3a 0x22 0x61 0x63 0x74 0x54 0x72 0x6b 0x22 0x2c 0x22 0x6c 0x6f 0x67 0x22 0x3a 0x22 0x32
2024-02-17 20:55:05.780  8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs   D  RX: t":"actTrk","log":"2

It just hasn't received anything from BtLEQueue between, where there should have been the end and start of a new object. I don't believe it's a Bangle.js/Espruino issue as when using WebBLE we can back the whole watch up without losing any data. @joserebelo do you recall anyone reporting issues in Gadgetbridge with 'lost' data notifications?

Interestingly fanoush on the forums/GitHub noticed yesterday I'd commented out some code that could have pushed 5x more data through BLE with a note saying that Nordic's Cloud Gateway app lost data if >1 packet was sent per connection interval. I wonder if it's related - it is possible that occasionally we could accidentally send >1 packet per connection interval and then maybe Gadgetbridge could lose it?

Moving from a while loop to an interval would be much better from the point of view of not locking up the Bangle while we're receiving the track data (I don't think users would appreciate the whole watch not working for 1+ minute during the upload), but I don't think that will solve the underlying problem you have there.

But as you're wrapping the CSV file line in JSON, maybe you could add a line counter in there too, and then if you lost a line you could request the transfer restarted from that line on? It wouldn't fix the underlying problem but would work around it.

Also, I could imagine you have a situation where the watch is downloading and then the user wants to switch apps, and that itself ends up cancelling the upload and you have to restart it anyway.

So maybe I'd suggest:

  • Switch to setInterval like you'd done - that looks good, but you want to ensure you store fetchRecInterval and clear it next time fetchRec is called.
  • Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips or if you don't receive new data in ~5 seconds which would mean something broke.
  • Maybe also send multiple lines per actTrk event? It would reduce a bit of overhead and would make the upload faster (as well as not having to interrupt Bangle.js so often)
Sorry for the delay - honestly this is a strange one. It looks like we've just totally missed a packet (which shouldn't happen!). You can see from: ``` 2024-02-17 20:55:05.775 8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs D RX: 119,207.76125986222, 2024-02-17 20:55:05.777 8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs D characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x30 0x2c 0x38 0x36 0x2c 0x33 0x2e 0x33 0x32 0x36 0x36 0x36 0x30 0x31 0x35 0x36 0x32 0x34 0x2c 0x66 2024-02-17 20:55:05.778 8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs D RX: 0,86,3.32666015624,f 2024-02-17 20:55:05.779 8904-12871 nodomain.f....BtLEQueue com.espruino.gadgetbridge.banglejs D characteristic changed: 6e400003-b5a3-f393-e0a9-e50e24dcca9e value: 0x74 0x22 0x3a 0x22 0x61 0x63 0x74 0x54 0x72 0x6b 0x22 0x2c 0x22 0x6c 0x6f 0x67 0x22 0x3a 0x22 0x32 2024-02-17 20:55:05.780 8904-12871 nodomain.f...iceSupport com.espruino.gadgetbridge.banglejs D RX: t":"actTrk","log":"2 ``` It just hasn't received anything from BtLEQueue between, where there should have been the end and start of a new object. I don't believe it's a Bangle.js/Espruino issue as when using WebBLE we can back the whole watch up without losing any data. @joserebelo do you recall anyone reporting issues in Gadgetbridge with 'lost' data notifications? Interestingly fanoush on the forums/GitHub noticed yesterday I'd commented out some code that could have pushed 5x more data through BLE with a note saying that Nordic's Cloud Gateway app lost data if >1 packet was sent per connection interval. I wonder if it's related - it is possible that occasionally we could accidentally send >1 packet per connection interval and then maybe Gadgetbridge could lose it? Moving from a while loop to an interval would be much better from the point of view of not locking up the Bangle while we're receiving the track data (I don't think users would appreciate the whole watch not working for 1+ minute during the upload), but I don't think that will solve the underlying problem you have there. But as you're wrapping the CSV file line in JSON, maybe you could add a line counter in there too, and then if you lost a line you could request the transfer restarted from that line on? It wouldn't fix the underlying problem but would work around it. Also, I could imagine you have a situation where the watch is downloading and then the user wants to switch apps, and that itself ends up cancelling the upload and you have to restart it anyway. So maybe I'd suggest: * Switch to `setInterval` like you'd done - that looks good, but you want to ensure you store fetchRecInterval and clear it next time `fetchRec` is called. * Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips *or* if you don't receive new data in ~5 seconds which would mean something broke. * Maybe also send multiple lines per `actTrk` event? It would reduce a bit of overhead and would make the upload faster (as well as not having to interrupt Bangle.js so often)
Member

Also, about the data loss: I could try doing a build that does push more data out per connection interval, and I guess that might really break things which would help track it down more.

I did spot this thread: https://stackoverflow.com/questions/24817107/android-receiving-multiple-ble-packets-per-connection-interval

the API is badly designed, and the characteristic value is a shared object, which may be updated by multiple threads. Each notification may be handled in a different thread and they then call onCharacteristicChanged on one thread (either set in connectGatt(...), or some unspecified thread for older Android versions. If the connection interval is very small, or the device sends many packets in a single interval, it may happen that before you receive the callback with the first notification it will be overwritten by another.

This doesn't entirely make sense since we're not seeing duplicate events with the same value, but could this be related I wonder? On StackOverflow it feels like a bunch of people are complaining about Android dropping notification events, but then Chrome seems to work just fine so whatever they did it works...

It's also interesting that we only seem to get 20 bytes per event (and Bangle.js could send far more - 128 iirc?) so it looks like we could add:

        if (allowHighMTU && Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) {
            builder.requestMtu(131);
        }

in initializeDevice like is done for PineTime and we could massively increase download/upload speed.

Also, about the data loss: I could try doing a build that does push more data out per connection interval, and I guess that might *really* break things which would help track it down more. I did spot this thread: https://stackoverflow.com/questions/24817107/android-receiving-multiple-ble-packets-per-connection-interval > the API is badly designed, and the characteristic value is a shared object, which may be updated by multiple threads. Each notification may be handled in a different thread and they then call onCharacteristicChanged on one thread (either set in connectGatt(...), or some unspecified thread for older Android versions. If the connection interval is very small, or the device sends many packets in a single interval, it may happen that before you receive the callback with the first notification it will be overwritten by another. This doesn't entirely make sense since we're not seeing duplicate events with the same value, but could this be related I wonder? On StackOverflow it feels like a bunch of people are complaining about Android dropping notification events, but then Chrome seems to work just fine so whatever they did it works... It's also interesting that we only seem to get 20 bytes per event (and Bangle.js could send far more - 128 iirc?) so it looks like we could add: ``` if (allowHighMTU && Build.VERSION.SDK_INT >= Build.VERSION_CODES.LOLLIPOP) { builder.requestMtu(131); } ``` in `initializeDevice` like is done for PineTime and we could massively increase download/upload speed.
Author
Contributor

Thanks for your comments!

  • Switch to setInterval like you'd done - that looks good, but you want to ensure you store fetchRecInterval and clear it next time fetchRec is called.
  • Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips or if you don't receive new data in ~5 seconds which would mean something broke.

I'll look at getting those changed/added!

  • Maybe also send multiple lines per actTrk event? It would reduce a bit of overhead and would make the upload faster (as well as not having to interrupt Bangle.js so often)

I don't know the limit here, how many lines should I expect to be able to send per event?

Also, about the data loss: I could try doing a build that does push more data out per connection interval, and I guess that might really break things which would help track it down more.

That could be good I guess, so I don't have to do repeated fetches hoping to get one that breaks.

... we could massively increase download/upload speed.

I have certainly noticed especially firmware updates takes eons from my phone, where it is relatively quick from my laptop.

Thanks for your comments! > - Switch to setInterval like you'd done - that looks good, but you want to ensure you store fetchRecInterval and clear it next time fetchRec is called. > - Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips or if you don't receive new data in ~5 seconds which would mean something broke. I'll look at getting those changed/added! > - Maybe also send multiple lines per actTrk event? It would reduce a bit of overhead and would make the upload faster (as well as not having to interrupt Bangle.js so often) I don't know the limit here, how many lines should I expect to be able to send per event? > Also, about the data loss: I could try doing a build that does push more data out per connection interval, and I guess that might really break things which would help track it down more. That could be good I guess, so I don't have to do repeated fetches hoping to get one that breaks. > ... we could massively increase download/upload speed. I have certainly noticed especially firmware updates takes eons from my phone, where it is relatively quick from my laptop.
Member

how many lines should I expect to be able to send per event?

It's all a bit of a bit hand-wavey. I'd say it'd be nice if we didn't end up adding more than 10-20% overhead in the JSON that we send, so maybe just send 4 lines at a time?

I have certainly noticed especially firmware updates takes eons from my phone

Yes - I'll see about getting that change put in

edit: it's in now

> how many lines should I expect to be able to send per event? It's all a bit of a bit hand-wavey. I'd say it'd be nice if we didn't end up adding more than 10-20% overhead in the JSON that we send, so maybe just send 4 lines at a time? > I have certainly noticed especially firmware updates takes eons from my phone Yes - I'll see about getting that change put in edit: it's in now
Author
Contributor

Just a note to say that sometimes when I'm connected Bangle.js watch <-> Gadgetbridge Web IDE Remote Access <-> Web IDE on my laptop, uploads will come in with some characters missing, breaking the code.

That seems to be the same bug discussed above here.

Just a note to say that sometimes when I'm connected `Bangle.js watch <-> Gadgetbridge Web IDE Remote Access <-> Web IDE on my laptop`, uploads will come in with some characters missing, breaking the code. That seems to be the same bug discussed above here.
Member

uploads will come in with some characters missing

You mean when sending data to the Bangle? That one's actually the opposite problem :(

> uploads will come in with some characters missing You mean when sending data *to* the Bangle? That one's actually the opposite problem :(
Author
Contributor

You mean when sending data to the Bangle?

Yes. Most recently when uploading the messagegui with the modifications for swipe up/down (edit: from the Web IDE right hand side window).


Found BANGLEJS2, 2v21.25
>
Connected to Android
>
Module messages not found
Module messageicons not found
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
>
Uncaught SyntaxError: Got ID:t expected ','
 at line 1 col 516
...break;case\xd1\tkeep_left:imgGB({"t":"notify","id":1709118388,"src"...
                                     ^
>=\xd1PERmBAACAAOAB+AD+AP+B/+H3+PO+8c8w4wBwADgAHgAPAAfAAfAAfAAfAAeAAeAAcAA8AA4ABwADgA==;¨;¼\xd1\nkeep_right:img=\xd1PERmBAACAAOAA/AD+AP+A//D/fPueeceY4YBwADgAPAAeAB8AHwAfAB8ADwAPAAcAB4ADgAHAAOAAAA==;¨;¼\xd1\nuturn_left:img=\xd1lGRiBAAAH4AAP/AAP/wAPj8APAfAPAHgHgB4DgA8BwAOA4AHAcADsOMB/HPA7zvgd9/gOf/gHH/gDh/gBwfgA4DgAcBgAOAAAHAAADgAABw==;¨;¼\xd1\vuturn_right:img=\xd1lGRiBAAPwAAf+AAf/gAfj4AfAeAPAHgPADwHgA4DgAcBwAOA4AHAcBjhuB5x/A+57gP99wD/84A/8cAP8OAD8HAA4DgAMBwAAA4AAAcAAAA==;¨;¼\xd1\6finish:img=\xd1ŒHhsBAcAAAD/AAAH/wAAPB4AAeA4AAcAcAAYIcAA4cMAA48MAA4cMAAYAcAAcAcAAcA4AAOA4AAOBxjwHBzjwHjj/4Dnn/4B3P/4B+Pj4A8fj8Acfj8AI//8A",8192);
Uncaught SyntaxError: Got '=' expected EOF
 at line 1 col 1
=\xd1PERmBAACAAOAB+AD+AP+B/+H3+PO+8c8w4wBwADgAHgAPAAfAAfAAfA...
^
|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
Uncaught SyntaxError: Got ',' expected EOF
 at line 21 col 3690 in messagegui.app.js
...VLarge,label:distance||""}]},]},{type:"txt",font:"6x8:2",labe...
                               ^
>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
Module messages not found
Module messageicons not found
|____|___|  _|_| |___|_|_|_
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
>
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
>>
{"t":"act","stp":0,"hrm":0,"mov":122}
{"t":"act","stp":0,"hrm":0,"mov":122}
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_
 ____                 _
|  __|___ ___ ___ _ _|_|___ ___
|  __|_ -| . |  _| | | |   | . |
|____|___|  _|_| |___|_|_|_|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
|___|
         |_| espruino.com
 2v21.25 (c) 2023 G.Williams
Disconnected from Android
>>

I managed to upload it on the subsequent try though.

> You mean when sending data to the Bangle? Yes. Most recently when uploading the `messagegui` with the modifications for swipe up/down (edit: from the Web IDE right hand side window). ``` Found BANGLEJS2, 2v21.25 > Connected to Android > Module messages not found Module messageicons not found ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_|___| |_| espruino.com 2v21.25 (c) 2023 G.Williams > Uncaught SyntaxError: Got ID:t expected ',' at line 1 col 516 ...break;case\xd1\tkeep_left:imgGB({"t":"notify","id":1709118388,"src"... ^ >=\xd1PERmBAACAAOAB+AD+AP+B/+H3+PO+8c8w4wBwADgAHgAPAAfAAfAAfAAfAAeAAeAAcAA8AA4ABwADgA==;¨;¼\xd1\nkeep_right:img=\xd1PERmBAACAAOAA/AD+AP+A//D/fPueeceY4YBwADgAPAAeAB8AHwAfAB8ADwAPAAcAB4ADgAHAAOAAAA==;¨;¼\xd1\nuturn_left:img=\xd1lGRiBAAAH4AAP/AAP/wAPj8APAfAPAHgHgB4DgA8BwAOA4AHAcADsOMB/HPA7zvgd9/gOf/gHH/gDh/gBwfgA4DgAcBgAOAAAHAAADgAABw==;¨;¼\xd1\vuturn_right:img=\xd1lGRiBAAPwAAf+AAf/gAfj4AfAeAPAHgPADwHgA4DgAcBwAOA4AHAcBjhuB5x/A+57gP99wD/84A/8cAP8OAD8HAA4DgAMBwAAA4AAAcAAAA==;¨;¼\xd1\6finish:img=\xd1ŒHhsBAcAAAD/AAAH/wAAPB4AAeA4AAcAcAAYIcAA4cMAA48MAA4cMAAYAcAAcAcAAcA4AAOA4AAOBxjwHBzjwHjj/4Dnn/4B3P/4B+Pj4A8fj8Acfj8AI//8A",8192); Uncaught SyntaxError: Got '=' expected EOF at line 1 col 1 =\xd1PERmBAACAAOAB+AD+AP+B/+H3+PO+8c8w4wBwADgAHgAPAAfAAfAAfA... ^ |___| |_| espruino.com 2v21.25 (c) 2023 G.Williams Uncaught SyntaxError: Got ',' expected EOF at line 21 col 3690 in messagegui.app.js ...VLarge,label:distance||""}]},]},{type:"txt",font:"6x8:2",labe... ^ > ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | Module messages not found Module messageicons not found |____|___| _|_| |___|_|_|_ ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_|___| |_| espruino.com 2v21.25 (c) 2023 G.Williams > ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_|___| |_| espruino.com 2v21.25 (c) 2023 G.Williams ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_ ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_|___| |_| espruino.com 2v21.25 (c) 2023 G.Williams |___| |_| espruino.com 2v21.25 (c) 2023 G.Williams >> {"t":"act","stp":0,"hrm":0,"mov":122} {"t":"act","stp":0,"hrm":0,"mov":122} ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_ ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_|___| |_| espruino.com 2v21.25 (c) 2023 G.Williams |___| |_| espruino.com 2v21.25 (c) 2023 G.Williams ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_ ____ _ | __|___ ___ ___ _ _|_|___ ___ | __|_ -| . | _| | | | | . | |____|___| _|_| |___|_|_|_|___| |_| espruino.com 2v21.25 (c) 2023 G.Williams |___| |_| espruino.com 2v21.25 (c) 2023 G.Williams Disconnected from Android >> ``` I managed to upload it on the subsequent try though.
Member

I think you'd need to look at a Gadgetbridge log when it happens - to see whether the requests from the WebView were missing data, or it was the Internal Intent that got lost (which is my guess - I feel like I've seen that before) or whether it was going all the way through to BtLEQueue and then getting lost.

I think you'd need to look at a Gadgetbridge log when it happens - to see whether the requests from the WebView were missing data, or it was the Internal `Intent` that got lost (which is my guess - I feel like I've seen that before) or whether it was going all the way through to BtLEQueue and then getting lost.
Author
Contributor

I've found the section of the log where it happened but haven't gone through to identify where the problem occurs yet (will get to it later). I'll send the log snippet in a message to you @gfwilliams on the espruino forum.

I've found the section of the log where it happened but haven't gone through to identify where the problem occurs yet (will get to it later). I'll send the log snippet in a message to you @gfwilliams on the espruino forum.
Member

Sorry, but I don't really have the bandwidth to look at that one at the moment - if you can figure it out it'd be great but I've just got too much on at the mo

Sorry, but I don't really have the bandwidth to look at that one at the moment - if you can figure it out it'd be great but I've just got too much on at the mo
Member

Just looked a bit closer at the above:

...break;case\xd1\tkeep_left:imgGB({"t":"notify","id":1709118388,"src"...

So what's happened is it was in the middle of the upload, and you received a notification, which sent out GB({"t":"notify","id":1709118388,"src"... right in the middle of the stream of data coming from the IDE.

It could happen with the app loader too.

So it's not actually losing any data anywhere, it's inserting it where it shouldn't!

So we could detect if we'd been asked to send any data by the WebView in the last 5 seconds, and if so could delay sending any notifications until we were sure nothing had been sent for a while.

Or in fact we could just refuse to send any data to the Bangle from Gadgetbridge when the WebView is active, and send it all when it's closed...

Just looked a bit closer at the above: ``` ...break;case\xd1\tkeep_left:imgGB({"t":"notify","id":1709118388,"src"... ``` So what's happened is it was in the middle of the upload, and you received a notification, which sent out `GB({"t":"notify","id":1709118388,"src"...` right in the middle of the stream of data coming from the IDE. It could happen with the app loader too. So it's not actually losing any data anywhere, it's inserting it where it shouldn't! So we could detect if we'd been asked to send any data by the WebView in the last 5 seconds, and if so could delay sending any notifications until we were sure nothing had been sent for a while. Or in fact we could just refuse to send *any* data to the Bangle from Gadgetbridge when the WebView is active, and send it all when it's closed...
Ganblejs force-pushed bangle-activity-tracks from 9b5328c5fe to e02e7aed77 2024-03-01 00:43:57 +00:00 Compare
Ganblejs added 1 commit 2024-03-01 13:56:51 +00:00
Ganblejs added 1 commit 2024-03-01 14:30:18 +00:00
Ganblejs added 1 commit 2024-03-01 14:56:27 +00:00
f8457373ea Bangle.js:actTrk- check if HRM could be exported
... to GPX track file. But commented out to avoid error.
Ganblejs added 1 commit 2024-03-01 18:39:24 +00:00
Ganblejs added 1 commit 2024-03-02 00:58:46 +00:00
Author
Contributor

A screenshot where activity and hrm data is shown to be pulled from the 'general' activity data in is attached. Summary details and a mapping of the gps points also shown.

A screenshot where activity and hrm data is shown to be pulled from the 'general' activity data in is attached. Summary details and a mapping of the gps points also shown.
Ganblejs force-pushed bangle-activity-tracks from a9fce966ec to ffda7b535d 2024-03-02 22:49:56 +00:00 Compare
Ganblejs added 1 commit 2024-03-02 23:06:28 +00:00
Ganblejs added 1 commit 2024-03-03 03:19:26 +00:00
a54600b34b Bangle.js:actTrk:try at packet counting
... might not be necessary. Since I got the fetching to work with
intervals on the the Bangle.js side it's been stable.

Didn't manage to make packet counting work yet.
Ganblejs added 1 commit 2024-03-03 13:09:30 +00:00
Author
Contributor

Fetch recorder logs logic:

@gfwilliams

  • Switch to setInterval like you'd done - that looks good, but you want to ensure you store fetchRecInterval and clear it next time fetchRec is called.
  • Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips or if you don't receive new data in ~5 seconds which would mean something broke.

I'll look at getting those changed/added!

  • Maybe also send multiple lines per actTrk event? It would reduce a bit of overhead and would make the upload faster (as well as not having to interrupt Bangle.js so often)

I don't know the limit here, how many lines should I expect to be able to send per event?

It's all a bit of a bit hand-wavey. I'd say it'd be nice if we didn't end up adding more than 10-20% overhead in the JSON that we send, so maybe just send 4 lines at a time?

  • The setInterval works wonders now - thanks for the tip!

  • Some kind of packet counter implemented. Not sure if I did it like you meant. But it should help in the event we miss a packet. Also - I haven't had the malformed JSON problem since getting setInterval to work even without packet counting.

  • 4 lines a packet works well, didn't try with more.

  • I still haven't added the ~5 sec timeout for if the transmission was interrupted. Currently I don't know how but shouldn't be too hard to work out.

So with that I think we're close to being done with transferring and storing recorder logs!

My biggest problems now are:

@joserebelo if you get the time I will appreciate if you have some thoughts on those :) I might be able to work them out eventually though.

### Fetch `recorder` logs logic: @gfwilliams >>> - Switch to setInterval like you'd done - that looks good, but you want to ensure you store fetchRecInterval and clear it next time fetchRec is called. >>> - Have a 'packet counter' that you increment, and then detect in Gadgetbridge if it skips or if you don't receive new data in ~5 seconds which would mean something broke. >> >>I'll look at getting those changed/added! >> >>> - Maybe also send multiple lines per actTrk event? It would reduce a bit of overhead and would make the upload faster (as well as not having to interrupt Bangle.js so often) >> >>I don't know the limit here, how many lines should I expect to be able to send per event? > >It's all a bit of a bit hand-wavey. I'd say it'd be nice if we didn't end up adding more than 10-20% overhead in the JSON that we send, so maybe just send 4 lines at a time? - The `setInterval` works wonders now - thanks for the tip! - Some kind of packet counter implemented. Not sure if I did it like you meant. But it should help in the event we miss a packet. Also - I haven't had the `malformed JSON` problem since getting `setInterval` to work even without packet counting. - 4 lines a packet works well, didn't try with more. - I still haven't added the ~5 sec timeout for if the transmission was interrupted. Currently I don't know how but shouldn't be too hard to work out. So with that I think we're close to being done with transferring and storing `recorder` logs! ### My biggest problems now are: - How can I make the "Activity Summary" page reload automatically after sync is finished? - How do I make the pull-down spinner stop after sync is finished (probably tied to preivous point)? - How to go about "Reset fetch date"? So far I have just read/written a file to keep track, but that is not user facing. I should hook into the GB logic that exists. - Maybe a Bangle.js extension/version of https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/activities/ActivitySummariesActivity.java#L349 @joserebelo if you get the time I will appreciate if you have some thoughts on those :) I might be able to work them out eventually though.
Ganblejs added 1 commit 2024-03-03 15:02:45 +00:00
Member

(I did not yet managed to look into the current diff of this PR, just going from the questions :))

How can I make the "Activity Summary" page reload automatically after sync is finished?

You can just call device.sendDeviceUpdateIntent, which should reload it:

Although I've noticed on some of my devices that that doesn't work 100% of the time. I'm thinking we should update this to react to GB.signalActivityDataFinish, but did not yet managed to look into that.

How do I make the pull-down spinner stop after sync is finished (probably tied to preivous point)?

Yup, it should go away once you unmark the device as busy + send the previous intent.

How to go about "Reset fetch date"? So far I have just read/written a file to keep track, but that is not user facing. I should hook into the GB logic that exists.

We are lacking this in other devices as well.. The reset happens in

I think we should split this into a reset for activity fetch time and sports fetch time, since it may actually happen that one of these fetches is not idempotent for some devices, and will cause data to become messed up / duplicated.

(I did not yet managed to look into the current diff of this PR, just going from the questions :)) > How can I make the "Activity Summary" page reload automatically after sync is finished? You can just call `device.sendDeviceUpdateIntent`, which should reload it: https://codeberg.org/Freeyourgadget/Gadgetbridge/src/commit/e86912e681fdb33852c33198a6e565d6610713bb/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/activities/ActivitySummariesActivity.java#L93 Although I've noticed on some of my devices that that doesn't work 100% of the time. I'm thinking we should update this to react to `GB.signalActivityDataFinish`, but did not yet managed to look into that. > How do I make the pull-down spinner stop after sync is finished (probably tied to preivous point)? Yup, it should go away once you unmark the device as busy + send the previous intent. > How to go about "Reset fetch date"? So far I have just read/written a file to keep track, but that is not user facing. I should hook into the GB logic that exists. We are lacking this in other devices as well.. The reset happens in https://codeberg.org/Freeyourgadget/Gadgetbridge/src/commit/e86912e681fdb33852c33198a6e565d6610713bb/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/activities/DebugActivity.java#L334 I think we should split this into a reset for activity fetch time and sports fetch time, since it may actually happen that one of these fetches is not idempotent for some devices, and will cause data to become messed up / duplicated.
Author
Contributor

(I did not yet managed to look into the current diff of this PR, just going from the questions :))

Sorry in advance, it's probably a little unwieldy... (however I did finally pull it out to a separate class/file) Don't hesitate to ask questions if you start looking at it - either here or on matrix chat.

How can I make the "Activity Summary" page reload automatically after sync is finished?

You can just call device.sendDeviceUpdateIntent, which should reload it:

Absolutely awesome - thank you!

How do I make the pull-down spinner stop after sync is finished (probably tied to preivous point)?

Yup, it should go away once you unmark the device as busy + send the previous intent.

It did!

How to go about "Reset fetch date"? So far I have just read/written a file to keep track, but that is not user facing. I should hook into the GB logic that exists.

We are lacking this in other devices as well.. The reset happens in

I think we should split this into a reset for activity fetch time and sports fetch time, since it may actually happen that one of these fetches is not idempotent for some devices, and will cause data to become messed up / duplicated.

OK, interesting :)

Thanks again!

> (I did not yet managed to look into the current diff of this PR, just going from the questions :)) Sorry in advance, it's probably a little unwieldy... (however I did finally pull it out to a separate class/file) Don't hesitate to ask questions if you start looking at it - either here or on matrix chat. > > How can I make the "Activity Summary" page reload automatically after sync is finished? > > You can just call `device.sendDeviceUpdateIntent`, which should reload it: https://codeberg.org/Freeyourgadget/Gadgetbridge/src/commit/e86912e681fdb33852c33198a6e565d6610713bb/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/activities/ActivitySummariesActivity.java#L93 Absolutely awesome - thank you! > > How do I make the pull-down spinner stop after sync is finished (probably tied to preivous point)? > > Yup, it should go away once you unmark the device as busy + send the previous intent. It did! > > How to go about "Reset fetch date"? So far I have just read/written a file to keep track, but that is not user facing. I should hook into the GB logic that exists. > > We are lacking this in other devices as well.. The reset happens in https://codeberg.org/Freeyourgadget/Gadgetbridge/src/commit/e86912e681fdb33852c33198a6e565d6610713bb/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/activities/DebugActivity.java#L334 > > I think we should split this into a reset for activity fetch time and sports fetch time, since it may actually happen that one of these fetches is not idempotent for some devices, and will cause data to become messed up / duplicated. OK, interesting :) Thanks again!
Ganblejs added 1 commit 2024-03-03 17:16:18 +00:00
Ganblejs added 1 commit 2024-03-03 17:31:20 +00:00
Member

This looks great - thanks!

If you can, the timeout might be good - now we're using setInterval it's actually pretty likely it would be interrupted - and alarm, incoming message(sometimes) or the user changing app could well interrupt it and we don't want Gadgetbridge to get confused (especially given someone's experiencing problems on https://forum.espruino.com/conversations/370102/#comment17306552 - unrelated, but I guess it shows stuff like that can happen)

This looks great - thanks! If you can, the timeout might be good - now we're using setInterval it's actually pretty likely it would be interrupted - and alarm, incoming message(sometimes) or the user changing app could well interrupt it and we don't want Gadgetbridge to get confused (especially given someone's experiencing problems on https://forum.espruino.com/conversations/370102/#comment17306552 - unrelated, but I guess it shows stuff like that can happen)
Ganblejs added 1 commit 2024-03-04 19:47:55 +00:00
Ganblejs added 2 commits 2024-03-05 01:16:22 +00:00
Ganblejs reviewed 2024-03-05 01:19:33 +00:00
@ -97,3 +97,3 @@
swipeLayout.setRefreshing(true);
} else {
boolean wasBusy = swipeLayout.isRefreshing();
boolean wasBusy = swipeLayout.isRefreshing(); // FIXME: This check of swipeLayout.isRefreshing does not cover the case where fetching was initiated by user clicking the blue button with a circle-arrow. In that case no auto-reload will happen.
Author
Contributor

@joserebelo This check of swipeLayout.isRefreshing does not cover the case where fetching was initiated by user clicking the blue button with a circle-arrow. In that case no auto-reload will happen.

I wonder if it would be better to just not do the check and just refresh() every time we end up in the else-statement?

@joserebelo This check of swipeLayout.isRefreshing does not cover the case where fetching was initiated by user clicking the blue button with a circle-arrow. In that case no auto-reload will happen. I wonder if it would be better to just not do the check and just refresh() every time we end up in the else-statement?
Author
Contributor

resolved by adding swipeLayout.setRefreshing(true) in the fetchTrackData() function.

resolved by adding `swipeLayout.setRefreshing(true)` in the `fetchTrackData()` function.
Ganblejs marked this conversation as resolved
Ganblejs added 1 commit 2024-03-05 01:29:45 +00:00
Ganblejs reviewed 2024-03-05 01:31:45 +00:00
@ -0,0 +66,4 @@
};
private Timer timer = new Timer("Activity Fetching Timeout");
timer. // FIXME: I don't get any hints for timer here, so I must be doing something wrong.
Author
Contributor

I don't get Android Studio to help me with Timer + TimerTask... Do you know how to set up Timer-s in Java/Android @gfwilliams ? I tried reading the Android docs and watching a youtube video on java Timer/TimerTask. Android studio would not provide me the method hints for timer.

I don't get Android Studio to help me with `Timer` + `TimerTask`... Do you know how to set up `Timer`-s in Java/Android @gfwilliams ? I tried reading the Android docs and watching a youtube video on java `Timer`/`TimerTask`. Android studio would not provide me the method hints for `timer`.
Author
Contributor

Just started trying with these instructions: https://examples.javacodegeeks.com/android/core/activity/android-timertask-example/

Hopefully that works.

Just started trying with these instructions: https://examples.javacodegeeks.com/android/core/activity/android-timertask-example/ Hopefully that works.
Member

Sorry I'm a bit late to this - I don't have any experience with timers though I'm afraid. My Android experience is pretty minimal

Sorry I'm a bit late to this - I don't have any experience with timers though I'm afraid. My Android experience is pretty minimal
Author
Contributor

It works now :)

It works now :)
Ganblejs marked this conversation as resolved
Ganblejs added 2 commits 2024-03-05 17:31:06 +00:00
Ganblejs added 1 commit 2024-03-05 17:39:49 +00:00
Ganblejs added 1 commit 2024-03-05 18:02:51 +00:00
Ganblejs added 1 commit 2024-03-05 18:11:22 +00:00
Ganblejs added 5 commits 2024-03-05 19:51:23 +00:00
Ganblejs added 3 commits 2024-03-05 22:19:51 +00:00
Ganblejs added 1 commit 2024-03-05 23:55:31 +00:00
Ganblejs force-pushed bangle-activity-tracks from 698f08bfdf to 1ea3639c36 2024-03-09 03:18:47 +00:00 Compare
Ganblejs changed title from WIP: Bangle.js: Add activity tracks support to Bangle.js: Add activity tracks support 2024-03-09 03:29:52 +00:00
Author
Contributor

@gfwilliams @joserebelo @halemmerich

I feel like this could be good enough to go in now or soon. My testing indicate all UI stuff work now. We could eventually introduce more of the summary data entries - but I don't want to work on it now. What's there is already useful. See attached screenshot.

So if you get the time, please review the code and/or try the functionality and provide feedback! Thanks in advance :)

The required modifications to Android Integration Bangle app can be loaded from my app loader: https://thyttan.github.io/BangleApps/?q=android

@gfwilliams @joserebelo @halemmerich I feel like this could be good enough to go in now or soon. My testing indicate all UI stuff work now. We could eventually introduce more of the summary data entries - but I don't want to work on it now. What's there is already useful. See attached screenshot. So if you get the time, please review the code and/or try the functionality and provide feedback! Thanks in advance :) The required modifications to `Android Integration` Bangle app can be loaded from my app loader: https://thyttan.github.io/BangleApps/?q=android
joserebelo reviewed 2024-03-10 14:22:13 +00:00
@ -90,0 +97,4 @@
* @param dst the file to write to
* @throws IOException
*/
public static void copyStringToFile(String string, File dst, String mode) throws IOException{
Member

If there's an exception, this will leak the file.

This should probably be:

try (BufferedWriter writer = new BufferedWriter(new FileWriter(dst, append)) {
    writer.write(string);
}

ensuring the writer gets closed, but we still propagate the exception.

If there's an exception, this will leak the file. This should probably be: ```java try (BufferedWriter writer = new BufferedWriter(new FileWriter(dst, append)) { writer.write(string); } ``` ensuring the writer gets closed, but we still propagate the exception.
Author
Contributor

I tried refactoring with your suggestion in mind - does it look good now?

I tried refactoring with your suggestion in mind - does it look good now?
Member

It will still leak. For example, if writer.write throws an exception, close will never be called.

If you use the snippet I added in the comment, it will not leak, as the writer will be closed automatically by the try-with-resources call.

It will still leak. For example, if `writer.write` throws an exception, `close` will never be called. If you use the snippet I added in the comment, it will not leak, as the writer will be closed automatically by the try-with-resources call.
Author
Contributor

But Android Studio doesn't like that snippet, marking as error.

Screenshot here: https://codeberg.org/attachments/589c0748-92bb-4107-a9bf-0480e77779ff

But Android Studio doesn't like that snippet, marking as error. Screenshot here: https://codeberg.org/attachments/589c0748-92bb-4107-a9bf-0480e77779ff
Author
Contributor

Sorry -works now I think...

Sorry -works now I think...
Ganblejs marked this conversation as resolved
joserebelo reviewed 2024-03-10 14:31:43 +00:00
joserebelo left a comment
Member

Just a few notes, will take a more in-depth look at the BangleJSActivityTrack during this week and try to test it.

Just a few notes, will take a more in-depth look at the BangleJSActivityTrack during this week and try to test it.
@ -0,0 +44,4 @@
import nodomain.freeyourgadget.gadgetbridge.util.FileUtils;
import nodomain.freeyourgadget.gadgetbridge.util.GB;
class BangleJSActivityTrack extends BangleJSDeviceSupport {
Member

Extending BangleJSDeviceSupport shouldn't be needed - why do we need that?

Extending BangleJSDeviceSupport shouldn't be needed - why do we need that?
Author
Contributor

Removed the extends modifier.

Removed the extends modifier.
Ganblejs marked this conversation as resolved
@ -0,0 +422,4 @@
//summaryData = addSummaryData(summaryData,"caloriesBurnt",3,"mm"); // TODO: Should this be calculated on Gadgetbridge side or be reported by Bangle.js?
//summaryData = addSummaryData(summaryData,"totalStride",3,"mm"); // FIXME: What is this?
if (storedLogObject.has("Heartrate")) {
summaryData = addSummaryData(summaryData, "averageHR", averageOfJSONArray(storedLogObject.getJSONArray("Heartrate")), "bpm");
Member
FYI, I extracted quite a few of these string constants to https://codeberg.org/Freeyourgadget/Gadgetbridge/src/branch/master/app/src/main/java/nodomain/freeyourgadget/gadgetbridge/model/ActivitySummaryEntries.java, for both the data types and units.
Author
Contributor

I changed to reference those instead - thanks!

I changed to reference those instead - thanks!
Ganblejs marked this conversation as resolved
@ -0,0 +782,4 @@
String dayString = String.valueOf(day);
if (day<10) dayString = "0" + dayString;
return yearString + monthString + dayString;
Member

You can simplify the 0-padding:

return String.format("%d%02d%02d", year, month, day);

You can simplify the 0-padding: `return String.format("%d%02d%02d", year, month, day);`
Author
Contributor

Did that - thanks!

Did that - thanks!
Ganblejs marked this conversation as resolved
@ -513,2 +513,4 @@
}
private JSONArray tracksList;
private int packetCount;
Member

I feel like we could make BangleJSActivityTrack keep the state, and move these 2 variables to there, so that all the logic is encapsulated there.

Just a suggestion though, would not block the PR on this.

I feel like we could make BangleJSActivityTrack keep the state, and move these 2 variables to there, so that all the logic is encapsulated there. Just a suggestion though, would not block the PR on this.
Member

As above, but if we're keeping these, renaming might be nice as it's not apparent that packetCount is specifically for tracks?

As above, but if we're keeping these, renaming might be nice as it's not apparent that `packetCount` is specifically for tracks?
Author
Contributor

I think I've achieved most of this now. Will leave a review comment re returning tracksList since uartTxJSON can't be called from static context.

I think I've achieved most of this now. Will leave a review comment re returning tracksList since uartTxJSON can't be called from static context.
Ganblejs marked this conversation as resolved
@ -1387,3 +1406,3 @@
}
if ((dataTypes & RecordedDataTypes.TYPE_DEBUGLOGS) != 0) {
if (dataTypes == RecordedDataTypes.TYPE_GPS_TRACKS) {
Member

dataTypes is supposed to be a bitmask, although I haven't yet checked how well the bangle would handle fetching multiple data types at once.

Still, it might be better to replace this with (dataTypes & RecordedDataTypes.TYPE_GPS_TRACKS) != 0` and return inside the if.

dataTypes is supposed to be a bitmask, although I haven't yet checked how well the bangle would handle fetching multiple data types at once. Still, it might be better to replace this with (dataTypes & RecordedDataTypes.TYPE_GPS_TRACKS) != 0` and return inside the if.
Author
Contributor

I hope I do it the right way now?

I hope I do it the right way now?
joserebelo marked this conversation as resolved
@ -1390,0 +1409,4 @@
JSONObject requestTracksListObj = BangleJSActivityTrack.compileTracksListRequest(getDevice(), getContext());
uartTxJSON("requestActivityTracksList", requestTracksListObj);
}
if (dataTypes == RecordedDataTypes.TYPE_DEBUGLOGS) {
Member

Same here, should be seen as bitmask.

Same here, should be seen as bitmask.
Author
Contributor

Updated this too, is it good now?

Updated this too, is it good now?
Member

Yup, looks good.

Yup, looks good.
joserebelo marked this conversation as resolved
Author
Contributor

Thank you @joserebelo - I'll look at those!

Thank you @joserebelo - I'll look at those!
Member

This looks great! @joserebelo knows Gadgetbridge way better than me and his suggestions looks great - but from the point of view of how this all works and integrates with Bangle.js I'm very happy and with those tweaks suggested above I'd think we should merge?

This looks great! @joserebelo knows Gadgetbridge way better than me and his suggestions looks great - but from the point of view of how this all works and integrates with Bangle.js I'm very happy and with those tweaks suggested above I'd think we should merge?
Ganblejs reviewed 2024-03-11 18:03:40 +00:00
@ -0,0 +636,4 @@
} catch (IOException e) {
throw new RuntimeException(e);
} catch (JSONException e) {
throw new RuntimeException(e);
Author
Contributor

This crashes GB :)

We should probably handle it a bit more gracefully.

#3153 (comment)

Said by @joserebelo

>This crashes GB :) > >We should probably handle it a bit more gracefully. *https://codeberg.org/Freeyourgadget/Gadgetbridge/pulls/3153#issuecomment-981951* Said by @joserebelo
Author
Contributor

Please advice on how I best handle/refactor this.

Please advice on how I best handle/refactor this.
Author
Contributor

I've now changed from throwing RuntimeException to logging the exceptions as errors.

I've now changed from throwing RuntimeException to logging the exceptions as errors.
Ganblejs marked this conversation as resolved
Ganblejs force-pushed bangle-activity-tracks from 1ea3639c36 to a69f8832cf 2024-03-11 21:08:46 +00:00 Compare
Ganblejs added 1 commit 2024-03-11 21:45:51 +00:00
Ganblejs added 2 commits 2024-03-12 00:37:13 +00:00
Ganblejs added 1 commit 2024-03-12 01:36:43 +00:00
Ganblejs reviewed 2024-03-12 01:39:53 +00:00
@ -562,1 +562,4 @@
break;
case "actTrksList":
JSONObject requestTrackObj = BangleJSActivityTrack.handleActTrksList(json, getDevice(), getContext());
uartTxJSON("requestActivityTrackLog", requestTrackObj);
Author
Contributor

If I could instead call uartTxJSON from inside BangleJSActivityTrack.handleActTrksList that would look nicer, and not need the returned requestTrackObj above. But that would need changing of modifiers for uartTxJSON, I believe (E.g. Android Studio would complain on calls of non-static method calls from static contexts). But that trigger a cascade of further needed changes it seems...

What do you think @gfwilliams @joserebelo, do you have suggestions?

If I could instead call `uartTxJSON` from inside `BangleJSActivityTrack.handleActTrksList` that would look nicer, and not need the returned `requestTrackObj` above. But that would need changing of modifiers for `uartTxJSON`, I believe (E.g. Android Studio would complain on calls of non-static method calls from static contexts). But that trigger a cascade of further needed changes it seems... What do you think @gfwilliams @joserebelo, do you have suggestions?
Author
Contributor

I seems to me like it's the switch - case statement that enforces/requires the static modifier. If we changed to if statements maybe it wouldn't be required anymore and uartTxJSON could be called from the new class?

After trying - that was not it.

~~I seems to me like it's the `switch - case` statement that enforces/requires the static modifier. If we changed to `if` statements maybe it wouldn't be required anymore and `uartTxJSON` could be called from the new class?~~ After trying - that was not it.
Member

It's probably easier for you to add the bangle support as an argument on the constructor of the BangleJSActivityTrack, that way you can call back that function directly.

It's not the cleanest way to implement this (it introduces a circular dependency between these 2 classes), but given the current Gadgetbridge architecture I think it might be the simplest.

It's probably easier for you to add the bangle support as an argument on the constructor of the BangleJSActivityTrack, that way you can call back that function directly. It's not the cleanest way to implement this (it introduces a circular dependency between these 2 classes), but given the current Gadgetbridge architecture I think it might be the simplest.
Author
Contributor

It's not the cleanest way to implement this (it introduces a circular dependency between these 2 classes), but given the current Gadgetbridge architecture I think it might be the simplest.

Do you think it's better to just leave it like I've done it in the current state of the PR? Or better to do the change you describe?

> It's not the cleanest way to implement this (it introduces a circular dependency between these 2 classes), but given the current Gadgetbridge architecture I think it might be the simplest. Do you think it's better to just leave it like I've done it in the current state of the PR? Or better to do the change you describe?
Member

I think the current way is fine.

While maybe it doesn't seem ideal, the benefit is it's clear from reading the code that when actTrksList is received something else is transmitted back - if it were hidden in handleActTrksList you might not see that.

If anything it'd be nicer I guess if we made uartTxJSON's first argument actually be the value of t that should be in the object, so then you can tell at a glance what packet is sent - but that's not a change for this PR!

I think the current way is fine. While maybe it doesn't seem ideal, the benefit is it's clear from reading the code that when `actTrksList` is received something else is transmitted back - if it were hidden in handleActTrksList you might not see that. If anything it'd be nicer I guess if we made `uartTxJSON`'s first argument actually be the value of `t` that should be in the object, so then you can tell at a glance what packet is sent - but that's not a change for this PR!
Ganblejs marked this conversation as resolved
Author
Contributor

Just a note to say that the FileUtils and ActivitySummariesActivity related commits should not be squashed together with the Bangle specific stuff. I can do separate PRs for them if preferred, but maybe just eventually have three commits on this PR - Bangle specific, FileUtils, and ActivitySummariesActivity.

Just a note to say that the `FileUtils` and `ActivitySummariesActivity` related commits should not be squashed together with the Bangle specific stuff. I can do separate PRs for them if preferred, but maybe just eventually have three commits on this PR - Bangle specific, FileUtils, and ActivitySummariesActivity.
Ganblejs added 1 commit 2024-03-12 17:38:27 +00:00
Ganblejs force-pushed bangle-activity-tracks from e54fe7957d to 2a45b91a52 2024-03-12 17:43:23 +00:00 Compare
Ganblejs added 1 commit 2024-03-13 00:00:57 +00:00
Ganblejs added 1 commit 2024-03-15 18:18:10 +00:00
joserebelo reviewed 2024-03-17 13:26:50 +00:00
@ -562,1 +562,4 @@
break;
case "actTrksList": {
JSONObject requestTrackObj = BangleJSActivityTrack.handleActTrksList(json, getDevice(), getContext());
uartTxJSON("requestActivityTrackLog", requestTrackObj);
Member

requestTrackObj can be null, which is sending GB(null) and throws an error.

requestTrackObj can be null, which is sending `GB(null)` and throws an error.
Ganblejs marked this conversation as resolved
joserebelo reviewed 2024-03-17 14:11:07 +00:00
@ -0,0 +543,4 @@
boolean hasHRMReading = false;
for (int i = 0; i < storedLogObject.getJSONArray("Time").length(); i++) {
timeOfPoint.setTime(storedLogObject.getJSONArray("Time").getLong(i)*1000L);
point.setTime(timeOfPoint);
Member

All the gpx points are ending up with the same timestamp, because the date object is used directly.

Maybe change this to point.setTime(timeOfPoint.clone());

All the gpx points are ending up with the same timestamp, because the date object is used directly. Maybe change this to `point.setTime(timeOfPoint.clone());`
Ganblejs marked this conversation as resolved
joserebelo reviewed 2024-03-17 14:21:10 +00:00
@ -0,0 +110,4 @@
//GB.toast(context, "actTrk says hi!", Toast.LENGTH_LONG, GB.INFO);
String log = json.getString("log");
LOG.debug(log);
String filename = "recorder.log" + log + ".csv";
Member

The probability of a collision here is higher than on other devices (if the user has multiple bangles), since the resolution is the day + a letter.

Should we consider saving these to a folder with the mac address as the name? (maybe replacing : with _ or similar.

The probability of a collision here is higher than on other devices (if the user has multiple bangles), since the resolution is the day + a letter. Should we consider saving these to a folder with the mac address as the name? (maybe replacing : with _ or similar.
Author
Contributor

Sounds sensible - what does @gfwilliams say?

Maybe we just use the four last characters as it's done on the preinstalled widget?

Sounds sensible - what does @gfwilliams say? Maybe we just use the four last characters as it's done on the preinstalled widget?
Author
Contributor

I now add the four last (excluding :) characters of the mac address to the files. But think I like the the thought of putting them inside a folder instead.

I now add the four last (excluding `:`) characters of the mac address to the files. But think I like the the thought of putting them inside a folder instead.
Author
Contributor

Ok - now I store all files in a device specific subfolder (named with getDevice().getName()) so I removed the four appended mac characters.

Ok - now I store all files in a device specific subfolder (named with `getDevice().getName()`) so I removed the four appended mac characters.
Ganblejs marked this conversation as resolved
joserebelo reviewed 2024-03-17 14:21:46 +00:00
@ -0,0 +115,4 @@
try {
dir = FileUtils.getExternalFilesDir();
} catch (IOException e) {
resetPacketCount();
Member

I think we should log the error here, might make it tricky to troubleshoot if it does fail.

I think we should log the error here, might make it tricky to troubleshoot if it does fail.
Author
Contributor

Added an error, but the string should be referenced from strings.xml - I have not done that now.

Added an error, but the string should be referenced from strings.xml - I have not done that now.
Member

I don't think it should to be honest - I meant just a LOG.error("something"), no need for a toast imo.

I don't think it should to be honest - I meant just a LOG.error("something"), no need for a toast imo.
Ganblejs marked this conversation as resolved
joserebelo reviewed 2024-03-17 14:24:31 +00:00
@ -0,0 +592,4 @@
break;
}
String fileName = FileUtils.makeValidFileName("gadgetbridge-" + trackType.toLowerCase() + "-" + summary.getName() + ".gpx");
Member

Same here - I think there's a somewhat high probability of collision if we use the summary name.

Maybe we could use the timestamp of the first point as the activity date?

Same here - I think there's a somewhat high probability of collision if we use the summary name. Maybe we could use the timestamp of the first point as the activity date?
Author
Contributor

I added that timestamp but also kept the bangle style stamp as well - for now at least. Makes it easier to match it to the csv and name in Gadgetbridge.

I added that timestamp but also kept the bangle style stamp as well - for now at least. Makes it easier to match it to the csv and name in Gadgetbridge.
Author
Contributor

I changed to adding the last four characters of the mac-address (not including :) of the device, which I read from the end of the device name ( getDevice().getName().substring( ... )).

I changed to adding the last four characters of the mac-address (not including `:`) of the device, which I read from the end of the device name ( `getDevice().getName().substring( ... )`).
Author
Contributor

Ok - now I store all files in a device specific subfolder (named with getDevice().getName()) so I removed the four appended mac characters.

Ok - now I store all files in a device specific subfolder (named with `getDevice().getName()`) so I removed the four appended mac characters.
Ganblejs marked this conversation as resolved
Author
Contributor

Thank you again @joserebelo - will take a look!

Thank you again @joserebelo - will take a look!
Ganblejs force-pushed bangle-activity-tracks from 95440ed95e to 12c73954c6 2024-03-18 21:56:30 +00:00 Compare
Ganblejs force-pushed bangle-activity-tracks from 12c73954c6 to b91cef6d6d 2024-03-20 18:56:35 +00:00 Compare
Ganblejs added 2 commits 2024-03-20 20:44:30 +00:00
Author
Contributor

Screenshot showing error in Android Studio from suggested snippet is attached.

Edit: was just a parenthesis mismatch.

Screenshot showing error in Android Studio from suggested snippet is attached. Edit: was just a parenthesis mismatch.
Ganblejs added 1 commit 2024-03-20 21:28:23 +00:00
f986ba57a1 FileUtils: refactor copyStringToFile
With the goal to hinder potential leak of file.
Ganblejs added 1 commit 2024-03-20 21:56:49 +00:00
Author
Contributor

@joserebelo I think I have made changes that resolve your latest review comments (and marked them as such).

@joserebelo I think I have made changes that resolve your latest review comments (and marked them as such).
Ganblejs force-pushed bangle-activity-tracks from 12f5b40baa to 8794878755 2024-03-20 22:29:53 +00:00 Compare
joserebelo merged commit de1faf6f00 into master 2024-03-21 23:28:35 +00:00
Member

@Ganblejs thanks! This was quite a big effort.

@Ganblejs thanks! This was quite a big effort.
Author
Contributor

Thanks again for the guidance and feedback @joserebelo and @gfwilliams !

And sorry for the onslaught of commits that went into master 😅

Thanks again for the guidance and feedback @joserebelo and @gfwilliams ! And sorry for the onslaught of commits that went into master 😅
Member

You had mentioned not to squash everything into a single commit, and I forgot to ask to squash into the 3 commits :')

Anyway, it's fine.

You had mentioned not to squash everything into a single commit, and I forgot to ask to squash into the 3 commits :') Anyway, it's fine.
Author
Contributor

I suppose you don't want to reset master to before the merge? I could squash into three commits now if preferred.

I suppose you don't want to reset master to before the merge? I could squash into three commits now if preferred.
Member

@Ganblejs no need, resetting master is not a very good practice, and the commits messages are clean.

@Ganblejs no need, resetting master is not a very good practice, and the commits messages are clean.
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
3 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: Freeyourgadget/Gadgetbridge#3153
No description provided.