Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Bangle.js and Edge Impulse for machine learning #610
Bangle.js and Edge Impulse for machine learning #610
Changes from all commits
3929ec9
e74e854
9930571
4ec5065
fd192bc
bdf3bfb
ef96ff7
a27f333
0b5e3d1
e5fded1
ae18804
c9a6fe3
6b3608c
fe0002c
3bf0bc2
498c2cd
6c5c2a5
03ab2d2
8484743
c3d49c6
4da76ff
c295976
f591869
f209260
6de58e6
0225039
2af9e7f
ac09a76
575fb09
851c594
29a9434
525a56a
52d2f9b
3829bf5
File filter
Filter by extension
Conversations
Jump to
There are no files selected for viewing
Use Bangle.js and Edge Impulse for Machine Learning
INTRODUCTION
In this tutorial you will learn how to get started with Machine Learning on your Bangle.js watch. Specifically you will build and train a model learning to recognize different movements of your watch hand. The steps include how to collect data, how to use Edge Impulse for the machine learning part and how to finally upload the learned model back to the watch and utilise it there.
Prerequisites
Hardware
Software
Preparation
Gesture Test
on your watch from the Bangle App LoaderCollect gesture samples
This part will guide you how to use your watch to collect multiple samples for one gesture type at a time.
event="left";
event="left";
for twitching your watch hand left and later onevent="right";
for the opposite directionevent="<gesture>";
where<gesture>
is the hand movement you will collect;
!Gesture collecting code:
Transfer .CSV-files from Bangle.js to your computer
This part will guide you how to transfer the .CSV-files from your watch to your computer.
left.1.csv (StorageFile)
Save
(the floppy disc icon) for one file at a time and save the files to a folder of your choice, e.g. toc:\temp
Split .CSV-files using Python
This part will guide you how to split the .CSV-files you've downloaded from your watch into separate .CSV-files. The reason for this is that Edge Impulse requires one .CSV-file per sample.
PATENTS = ...
) with the full path and filename for the first file you want to split. I.e. the file you downloaded in previous steps.'timestamp, x, y, z'
in the original file and for each time (= sample) it finds, create a new file.left.1.csv (StorageFile)-15.csv
where-15
at the end is a running number.Use Edge Impulse for machine learning
In this part you will learn how to upload the sample files you've created earlier, create a machine learning model, train and finally analyse it. This tutorial will only cover the essential steps needed for Bangle.js. To learn more about Edge Impulse, see e.g. getting started and continuous motion recognition.
Log in and create a project
Accelerometer data
when asked for the type of data you are dealing with.Let's get started
Upload sample data
Select
Data acquisition
from the left hand menuClick on the icon labeled
Upload existing data
Click on
Choose files
left.1.csv (StorageFile)-0.csv
.Automatically split between training and testing
andInfer from filename
should both be selectedClick
Begin upload
- this will now quickly upload the files to your project.Done. Files uploaded successful: 85. Files that failed to upload: 0. Job completed
Take a look at a sample by selecting any row
Notice that the labels (
left
andright
in this example) were automatically inferred from the filenames you used.Always strive to get a roughly similar amount of samples for each gesture. You can see the balance in the pie graph on the left.
Also notice that Edge Impulse split the sample files so that approximately 80 % will be used for training and 20 % for testing purposes.
Through the four small icons you can filter your data, select multiple items, upload more data or see a slightly more detailed list view. With the help of these you can e.g. mass delete many files at a time.
Create an impulse
An impulse takes raw data, uses signal processing to extract features, and then uses a learning block to classify new data. These steps will create an impulse.
Click
Create impulse
Change the window size and increase according to the screenshot below.
Add the
Raw Data
processing blockAdd the
Classification (Keras)
learning blockClick
Save Impulse
Note that you often need to tweak one or several of the settings, this is depending on what you want to achieve and the quality & quantity of your data.
Generate features
Raw data
from the left hand menuSave parameters
which will take you to the second tab.Generate features
Feature explorer
. This gives you a 3D view of how well your data can be clustered into different groups. In an ideal situation all similar samples should be clustered into same group with a clear distinction between groups. If that's not the case, no worries at this point, the neural network algorithm will in many cases still be able to do a very good job!Train the neural network
Here you will train the neural network and analyse its performance.
NN Classifier
from the left hand menuNumber of training cycles
to 100. This is another parameter to tweak, the higher this number is, the longer time the training will take, but also the better the network will perform, at least until it can't improve anymore.Start training
Download the trained model
Here you will download the trained model to your computer.
Dashboard
from the left hand menuDownload block output
and click on the icon next toNN Classifier model TensorFlow Lite (int8 quantized)
DEPLOYMENT
Transfer the trained model to Bangle.js from your computer
This part will guide you how to transfer the model file from your computer to Bangle.js.
Upload a file
.tfmodel
and clickOk
left,right
.tfnames
and clickOk
Test the gestures on Bangle.js!
Finally you will be able to test how well the trained model performs in real life! Just a few steps left.
left
orright
, will be shown in the left window in Espruino Web IDE as well as on your watch display.FINAL COMMENTS
First of all, hopefully you with this short tutorial were successful in training and recognising gesture events from your Bangle.js. Hopefully it also inspires you to try to improve the performance, e.g. by collecting more samples, by collecting more event types or by tweaking the different parameters and settings in Edge Impulse.