Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Missing Publish flags in MQTT library #480

Closed
jjok opened this issue Jan 16, 2019 · 24 comments
Closed

Missing Publish flags in MQTT library #480

jjok opened this issue Jan 16, 2019 · 24 comments

Comments

@jjok
Copy link
Contributor

jjok commented Jan 16, 2019

Hi. I'd like to publish an MQTT packet that has the retain flag set, but it looks like only qos is currently implemented.

function mqttPublish(topic, message, qos) {
    var cmd = TYPE.PUBLISH << 4 | (qos << 1);
    // ...
}

It would be great to add the retain and DUP flags. I'll see if I can do it myself.

http://docs.oasis-open.org/mqtt/mqtt/v3.1.1/os/mqtt-v3.1.1-os.html#_Toc398718038

gfwilliams added a commit that referenced this issue Jan 16, 2019
Removed number for qos in 'publish' - now needs a module. This was undocumented, and not the same as the MQTT NPM module
Removed automatic conversion of number to qos for options in subscribe (undocumented, not same as MQTT module)
Removed 'callback' from subscribe. It's undocumented, useless, and DOESN'T do what the MQTT NPM module does, which would cause confusion
@gfwilliams
Copy link
Member

Ok, just added. Can you check it out and see if it works?

... = require("https://raw.githubusercontent.com/espruino/EspruinoDocs/master/modules/MQTT.js");
mqtt.publish(topic, message, {retain:true});

@jjok
Copy link
Contributor Author

jjok commented Jan 16, 2019

Thanks! It certainly looks like it should. I'll give it a go tonight, hopefully.

This would be a breaking change for anyone currently passing qos as the third param to mqtt.publish(). Is that what you're intending or should it check to see if the new opts param is just a number, and if it is, assume it's mean to be the qos?

@gfwilliams
Copy link
Member

Thanks - I generally try to avoid breaking changes, but in this case:

  • it should at least error if it was used
  • it wasn't previously mentioned in the docs
  • it doesn't appear to be used in any code examples
  • the module is based on the API from https://www.npmjs.com/package/mqtt and that doesn't allow it
  • everyone always complains about how big the MQTT module is, so anything we can do to make it smaller/more efficient is a bonus :)

@jjok
Copy link
Contributor Author

jjok commented Jan 16, 2019

OK. Well, it's fine by me : )

I'm not sure there's a lot you can do to make that module smaller, is there? It looks like it implements the full spec. I can't see there's much you could do to make it smaller, unless you removed features or made micro-optimisations to the way the code is written.

@jjok
Copy link
Contributor Author

jjok commented Jan 16, 2019

Hmm... I'm getting an error message. I don't seem to be able to copy it out of the IDE though...

Uncaught Error: Cannot read property 'DEF_QOS' of undefined at line 3 col 43

qos: opts.qos || this.C.DEF_QOS
                       ^
in function called from line 14 col 10

in function "subscribe" called from line 13 col 43.

@jjok
Copy link
Contributor Author

jjok commented Jan 16, 2019

Presumably it isn't this when it's inside the callback.

topics.forEach(function (topic) {
    subs.push({
        topic: topic,
        qos  : opts.qos || this.C.DEF_QOS
    });
});

I can't see another example of where you've had to do that, but I think you should be able to do:

var self = this;
topics.forEach(function (topic) {
    subs.push({
        topic: topic,
        qos  : opts.qos || self.C.DEF_QOS
    });
});

or

topics.forEach(function (topic) {
    subs.push({
        topic: topic,
        qos  : opts.qos || this.C.DEF_QOS
    });
}.bind(this));

@jjok
Copy link
Contributor Author

jjok commented Jan 16, 2019

Yeah, both of those options work. I'll do a PR.

@MaBecker
Copy link
Contributor

Just as hint, there is tinyMQTT module by Olli Phillips

https://github.com/olliephillips/tinyMQTT

@gfwilliams
Copy link
Member

@jjok perfect - thanks for that PR! I think a lot of modules to use either the local var, or bind - there's not one way we use in particular.

@olliephillips are you interested in getting that TinyMQTT module on the Espruino site? I'm happy to add it, and IMO it'd make a lot of sense.

@jjok
Copy link
Contributor Author

jjok commented Jan 17, 2019

Just as hint, there is tinyMQTT module by Olli Phillips

https://github.com/olliephillips/tinyMQTT

Thanks. I hadn't seen that. Although, it doesn't look like it has full retain support. I think all messages are flagged as retain. https://github.com/olliephillips/tinyMQTT/blob/816fbb9c081b156bfadae24d3ff4a0b040e3dc21/tinyMQTT.js#L95

Maybe I'll open a ticket there too... 🤔

@jjok
Copy link
Contributor Author

jjok commented Jan 17, 2019

Thanks @gfwilliams. The retain flag seems to be working, so I'll close this issue.

@jjok jjok closed this as completed Jan 17, 2019
@gfwilliams
Copy link
Member

Thanks! I'll update the version on the website with this soon.

@olliephillips
Copy link
Contributor

@gfwilliams If you think it would add value, absolutely.

@jjok pretty sure retain is the default yes

@gfwilliams
Copy link
Member

gfwilliams commented Jan 17, 2019

@olliephillips great - thanks! Just added it - http://espruino.com/tinyMQTT

A bit of a thread hijack, but was there a reason that you didn't define _q as a module-local variable rather than global? I just did it here and it seems to get the module size down to 1.38k (minify can change it to a single-character if it knows it's not exported)

@olliephillips
Copy link
Contributor

@gfwilliams honestly? probably my inexperience with js. I recall the global was contentious at the time I was working on it, but nobody suggested a different approach. I seem to recall the global helped me make the module smaller than it was. If there's a better way that reduces size again I'm happy to learn about it - and I'll bow to your experience with js for sure :)

@jjok
Copy link
Contributor Author

jjok commented Jan 17, 2019

Also not related to this thread...

Hmm... I'm getting an error message. I don't seem to be able to copy it out of the IDE though...

@gfwilliams Should you be able to copy text out of the left pane of the IDE? It would be pretty useful for copying out error messages.

@gfwilliams
Copy link
Member

I never came across the variable before as a way of saving space, but it's really neat. The change to module scope was just:

var _q; // <--- this
function () {
  ...
  _q = this;
}

but you wrote the module a while back so maybe it didn't work in Espruino then? Not sure :)

Just a FYI - I was frustrated that the whole this token was a waste of space in modules too, so if you turn on pretokenise then Espruino automatically converts it and stores it as a single byte - but that doesn't actually help you if you're using 'save on send' with 'to flash' since the text just goes in direct. I'd considered adding something to the IDE to do it before sending but haven't had time yet.

Should you be able to copy text out of the left pane of the IDE?

@jjok yes - just click and drag. When you stop the drag then it copies. It's a bit odd but it was done that way because Ctrl-C is used to break out of running code/clear the line.

@jjok
Copy link
Contributor Author

jjok commented Jan 17, 2019

When you stop the drag then it copies

Ahh. Thanks. : )

@olliephillips
Copy link
Contributor

olliephillips commented Jan 17, 2019

@gfwilliams I'll look at that and likely change. What minifier do you use, I was using the closure compiler? Also with the docs on Espruino for tinyMQTT, will you pick up changes in say the repo README and reflect on espruino.com, or should I be mindful of this, and update the docs in this repo should the API ever change?

@gfwilliams
Copy link
Member

I was just using closure with simple minification. I wrap in (function(){....})() before sending it over - see https://github.com/espruino/EspruinoDocs/blob/master/bin/minify.js.

I'm afraid there's nothing fancy going on with the new page - so if there are any new changes they'll need to be manually mirrored in https://github.com/espruino/EspruinoDocs/blob/master/modules/tinyMQTT.md

It's an interesting thought - I think if I got to the stage where there were a lot of Espruino modules being brought in from their own repos I'd come up with some kind of script, but as far as I remember yours is the only one at the moment.

@olliephillips
Copy link
Contributor

I never came across the variable before as a way of saving space, but it's really neat. The change to module scope was just:

Just tested. Why that and adding the closure helps minification is certainly counter-intuitive to me, but it definitely works! Almost 20 more jsvars freed for application code. I'll revise the module to reflect.

@MaBecker
Copy link
Contributor

but you wrote the module a while back so maybe it didn't work in Espruino then? Not sure :)

Well Gordon it works perfect from the beginning;-). Just did some small changes to allow receiving 256byte and my latest version can even run from storage after quoting at to sections: first and second.

Hi @olliephillips, I got some mqtt scenarios and can offer to run them for a couple of days, let me know if you like me to jump in.

@olliephillips
Copy link
Contributor

Hi @MaBecker, thanks, that would be good! I've run the module locally tonight and my application seems good - but I'm only publishing. I'll get the revisions into a new branch and you can test away! You'll have to tell me more about running the module from storage too, that's not something I've considered. Hope you are well!

@olliephillips
Copy link
Contributor

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants