Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenWRT package compilation #143

Closed
gfwilliams opened this issue Dec 19, 2013 · 39 comments · Fixed by #594
Closed

OpenWRT package compilation #143

gfwilliams opened this issue Dec 19, 2013 · 39 comments · Fixed by #594

Comments

@gfwilliams
Copy link
Member

We need to create an OpenWRT package. The current package file looks as follows - however the compiler used by OpenWRT seems to dislike the 'gc-sections' flag, as well as having issues compiling against libc.

#
# Copyright (C) 2006-2012 OpenWrt.org
#
# This is free software, licensed under the GNU General Public License v2.
# See /LICENSE for more information.
#

include $(TOPDIR)/rules.mk

PKG_NAME:=espruino
PKG_VERSION:=1.44.0-git
PKG_RELEASE:=1

PKG_SOURCE_PROTO:=git
PKG_SOURCE_URL:=git://github.com/espruino/Espruino.git
PKG_SOURCE_VERSION:=7411dd18d09f1891e5c3c9b90f1c0379c2db9643
PKG_SOURCE_SUBDIR:=$(PKG_NAME)-$(PKG_VERSION)
PKG_SOURCE:=$(PKG_NAME)-$(PKG_VERSION)-$(PKG_SOURCE_VERSION).tar.gz
PKG_MIRROR_MD5SUM:=66a728a3cb8dad0e3773035c4049eb5f

PKG_FIXUP:=autoreconf
PKG_INSTALL:=1

include $(INCLUDE_DIR)/package.mk

define Package/espruino
  SUBMENU:=JavaScript
  SECTION:=lang
  CATEGORY:=Languages
  TITLE:=A compact JavaScript interpreter
  URL:=http://www.espruino.com
  DEPENDS:=
endef

define Package/espruino/description
 Espruino is an extremely compact JavaScript interpreter. Originally
 designed for microcontrollers, it runs on OpenWRT while using a very small
 amount of RAM. It includes a cut-down version of node's HTTP client and
 server, filesystem, as well as GPIO support.
endef

MAKE_FLAGS += \
    RELEASE=1

define Package/espruino/install
    $(INSTALL_DIR) $(1)/usr
    $(CP) \
        $(PKG_INSTALL_DIR)/usr/bin \
        $(PKG_INSTALL_DIR)/usr/share \
        $(1)/usr/
endef

define Build/InstallDev
    $(CP) $(PKG_INSTALL_DIR)/* $(1)/
endef

$(eval $(call BuildPackage,espruino))
@vshymanskyy
Copy link
Contributor

Hi. I have managed to build and run Espruino on OpenWRT (on VoCore v1.0).
I can even control GPIO0 :)
For some reason setInterval() doesn't wait between calls, fires very rapidly.

I have prepared a quick-and-dirty feed package (it patches Makefile):
https://github.com/vshymanskyy/OpenWRT-Espruino-packages
It would be great if someone could check it and integrate more cleanly!

Oh, this package only builds with uClibc.. didn't try to build with musl.

@gfwilliams
Copy link
Member Author

That's awesome, thanks! sorry for not replying to you on YouTube - not sure what happened there.

The package looks good - is there any chance the Makefile could be tweaked (maybe put ifdef CCPREFIX around export CC, and do some other stuff) such that I could pull it back into the main Espruino repo? It'd make the OpenWRT package a bit more maintainable.

About the setInterval - maybe check getTime() and make sure it's incrementing by 1 every second? That's about the only thing I think could be an issue.

The code for it is here - but I'm actually pretty surprised that it could be failing :(

@vshymanskyy
Copy link
Contributor

musl libc also works with minor changes!
let me check other stuff..

@vshymanskyy
Copy link
Contributor

Funny.. getTime is negative :)

>getTime()
=-1710.8534
>getTime()
=-1709.39426
>getTime()
=-1708.23584
>getTime()
=-1707.27872

@gfwilliams
Copy link
Member Author

Ahh, well that'd do it!

Maybe an overflow somewhere? It's this function

@daanpape
Copy link
Contributor

As you may have read on twitter, we at DPTechnics started porting the Espruino runtime to our DPT-Board (OpenWRT, MIPS, AR9331). We don't have the getTime() bug, compiled from the latest code, so which codebase have you used?

I must say we have now used the stlibcpp and not uclibc++, I'm going to try to use uclibc++ tomorrow and post back results.

@vshymanskyy
Copy link
Contributor

Actually I got getTime fixed by changing:

-  return tm.tv_sec*1000000L + tm.tv_usec;
+  return (JsSysTime)(tm.tv_sec)*1000000L + tm.tv_usec;

But it didn't help with the setInterval().
Any clues?

@vshymanskyy
Copy link
Contributor

In the https://github.com/vshymanskyy/OpenWRT-Espruino-packages
I added 1 more patch with more time-related changes.
Still could not get the setInterval() and setTimeout() working as expected.

>setInterval(function(){ console.log(getTime()) }, 1000)
=1
1442878344.16901993751
1442878344.16975998878
1442878344.17050004005
1442878344.17126011848
1442878344.17199993133
1442878344.17273998260
1442878344.17348003387
1442878344.17422008514
1442878344.17495989799
...

@vshymanskyy
Copy link
Contributor

I have traced this problem down to this:

#0  _jswrap_interface_setTimeoutOrInterval (func=0x486240, interval=3.1050924472733959e+231, args=0x0, isTimeout=FALSE) at src/jswrap_interactive.c:408
#1  0x0045250c in jswrap_interface_setInterval (func=0x486240, timeout=3.1050924472733959e+231, args=0x0) at src/jswrap_interactive.c:426

Note, that jswrap_interface_setInterval gets a strange timeout value.
I verified that if I change this value to a correct one during debug session, the timer works as expected. Now we need to fix timeout value somehow..

@vshymanskyy
Copy link
Contributor

OK, got it working...

>setInterval(function(){ console.log(getTime()) }, 1000)
=1
1442888667.43619990348
1442888668.43618011474
1442888669.43614006042
1442888670.43616008758
1442888671.44695997238
1442888672.43616008758
1442888673.43616008758
...

I had to change jsnCallFunction to behave same way as on ARM platforms.

@daanpape
Copy link
Contributor

Glad you got it working, I'm was testing your javascript snippet and got a Segmentation fault:

>setInterval(function(){ console.log(getTime()) }, 1000)
Segmentation fault

I'm now modifying the source to compile without warnings, because I get quite a lot of them. Mostly unused parameters and some casting errors.

@gfwilliams
Copy link
Member Author

@daanpape thanks for posting up - any pull requests you have would be great (I reckon you could hard-code paths for your specific device in BOARD.py so that users get access to SPI1/I2C1/etc without needing the explicit paths).

Also, your segfault is almost certainly the issue @vshymanskyy discovered with the calls. Did you want to do a pull request for that (and your other patches) @vshymanskyy ?

IMO the call handling is actually specific to x86. It'd actually be better to have #ifndef X86 (there must be a preprocessor var set for that?) than to have #if defined(ARM) || defined(XTENSA) || defined(MIPS) || defined(everything_else_not_x86)

@gfwilliams
Copy link
Member Author

@daanpape the thing @vshymanskyy fixed is basically this: fc9b830

But with your processor in there. If someone did a PR changing that to non-x86 it'd be great.

note: whether it works actually depends on how your specific processor handles function calls (which registers it puts arguments in). Luckily pretty much every device apart from x86 is really sensible and puts arguments on the stack (and maybe a few registers) so the code is usually the same for each.

@daanpape
Copy link
Contributor

Ok, I just verified the getTime() being negative is a uClibc issue. When i link against the libstdcpp there are no problems. I'm working on a pull request using the OPENWRT define to make uClibc compilation possible.

@gfwilliams I have now introduced ifdef X86 in jsNative to handle this. I'm preparing the pull request with the timefix from @vshymanskyy and the X86 preprocessor. It also introduces the makefile fixes to build for OpenWRT with uClibc/musl. building goes with OPENWRT=1 RELEASE=1 make.

I don't have a segfault any more but the setInterval is going way to fast just like @vshymanskyy the getTime() function now works correctly. I'm still getting the wrong parameters passed:

Breakpoint 1, jswrap_interface_setInterval (func=0x4896dc, timeout=5.3513725183457637e-315, args=0x489634) at src/jswrap_interactive.c:446
446       return _jswrap_interface_setTimeoutOrInterval(func, timeout, args, false);
>setInterval(function(){ console.log(getTime()) }, 1000)
=1
1443532932.48394298553
1443532932.48554396629
1443532932.48673295974
1443532932.48789501190
1443532932.48905611038
1443532932.49021601676
1443532932.49137997627
1443532932.49254107475

Thanks for all the help allready @gfwilliams and @vshymanskyy

@vshymanskyy
Copy link
Contributor

@daanpape , You just hijacked my fixes, LOL ;)

@daanpape
Copy link
Contributor

@vshymanskyy yes, I wanted to test out your fixes, but I have not inserted them in my pull request because you earn the credits. But at even with your fixes I still have the problem on my platform AR9331. Still working out what the difference is.

@vshymanskyy
Copy link
Contributor

@gfwilliams , after fixing Espruino, I got Blynk JS library running on VoCore, as well as on Pico+ESP8266.
Espruino rocks!
Examples here: https://github.com/vshymanskyy/blynk-library-js/tree/master/examples
BTW, I tried to make the library compatible with Node.js, and it also looks working fine.

@daanpape
Copy link
Contributor

I'm still having issues, and I can't work out how to interpret the stack. So as a test I'm calling the following in espruino cli:

setInterval(function(){ console.log(getTime()) }, 0)

Than I break in gdb on 'jsnCallFunction' this gives the following data:

Breakpoint 6, jswrap_interface_setInterval (func=0x484768, timeout=0, args=0x484570) at src/jswrap_interactive.c:446
446       return _jswrap_interface_setTimeoutOrInterval(func, timeout, args, false);
(gdb) up
#1  0x00418e58 in jsnCallFunction (function=0x452c8c <jswrap_interface_setInterval>,
    argumentSpecifier=JSWAT_FINISH, thisParam=0x0, paramData=0x7fff6638, paramCount=2) at src/jsnative.c:157
157             result = ((uint32_t (*)(size_t,size_t,size_t,size_t,size_t,size_t,size_t,size_t,size_t,size_t,size_t,size_t))function)(argData[0],argData[1],argData[2],argData[3],argData[4],argData[5],argData[6],argData[7],argData[8],argData[9],argData[10],argData[11]);
(gdb) info locals
returnType = JSWAT_JSVAR
argsArray = 0x484570
paramNumber = 3
argCount = 5
argData = {4736872, 4323440, 0, 0, 4736368, 2147444176, 2147444184, 4214132, 2147445156, 4324148, 0, 0}
result = 20343752537370468
(gdb) print/x argData
$1 = {0x484768, 0x41f870, 0x0, 0x0, 0x484570, 0x7fff65d0, 0x7fff65d8, 0x404d74, 0x7fff69a4, 0x41fb34, 0x0, 0x0}

So the timeout variable is argData[1,2,3] why 3x*32bit?

@daanpape
Copy link
Contributor

I just found the problem. I used the commits from @vshymanskyy where the USE_X86_CDECL preprocessor was introduced. But this breaks the code for the MIPS in AR9331 in the file jsnative.c on line 53:

#ifndef USE_X86_CDECL 
    if (JSWAT_IS_64BIT(argType))
      argCount = (argCount+1)&~1;
#endif

This increments the argCount which causes a misalignment on line 101:

    case JSWAT_JSVARFLOAT: { // 64 bit float
      JsVarFloat f = jsvGetFloat(param);
#ifdef USE_X86_CDECL
      doubleData[doubleCount++] = f;
#else
      uint64_t i = *(uint64_t*)&f;
#if defined(__WORDSIZE) &&__WORDSIZE == 64
      argData[argCount++] = (size_t)i;
#else
      argData[argCount++] = (size_t)((i) & 0xFFFFFFFF);
      argData[argCount++] = (size_t)((i>>32) & 0xFFFFFFFF);
#endif

On my AR9331 MIPS platform this extra increment of argCount is invalid. So it is not so easy as saying x86 vs all the rest as it turns out.

@vshymanskyy @gfwilliams: I don't understand how this extra increment could ever work on any platform as it just leaves garbage data in the argData array. How should we fix this?

@gfwilliams
Copy link
Member Author

Thanks for all the PRs! In wasn't around yesterday so wasn't able to help out unfortunately.

I don't understand how this extra increment could ever work on any platform as it just leaves garbage data in the argData array.

I think the point is that 64 bit values stay aligned to 64 bit word boundaries in memory. This commit makes me think I did it because it is required on ARM.

So I think in this case it's probably going to require that there's an ifndef MIPS around it.

@daanpape
Copy link
Contributor

Hello,

I have implemented the proposed fix in my fork: dptechnics@52f252a

Before opening a pull request it may be handy if @vshymanskyy verifies this on his hardware as this is also MIPS. I find it quite confusing the code worked on the VoCore (RaLink) and not the DPT-Board (Qualcomm/Atheros).

@gfwilliams
Copy link
Member Author

Yes, absolutely - I wonder why there's the difference. Could it be a GCC version issue?

It only happens on some functions, but given one of them is setInterval it's a pretty big one :)

I got Blynk JS library running on VoCore

That's awesome - sorry, I missed that before - I had like a million e-mails yesterday. I'd noticed the node-compatibility in it when you'd got it running on the Pico - it's really neat.

I'd really like to get this submitted to OpenWRT when we sort out the function call issue. I know now there's currently a build target specifically for dptechnics, but I wonder whether there's a way we can make the same build work for a variety of boards?

@vshymanskyy
Copy link
Contributor

@gfwilliams, @daanpape:
Here is the updated package for OpenWrt:
https://github.com/vshymanskyy/OpenWRT-Espruino-packages

It's updated to Espruino master, until you release version 81 with current changes.
I did some modifications to build it on master and removed obsolete patches.
Please take a look on the remaining patch to the Makefile and tell what you think.
I checked it on my musl compilation and it works.. Still need to check uclibc++.

Regarding other boards - in the patch I add gpio to all linux boards (we could do it only for OpenWrt-basd..). It looks like the most generic approach. But what for the SPI/I2C support...

About Blynk + JS, here is also an Instructable I finished writing today: http://www.instructables.com/id/Blynk-JavaScript-in-20-minutes-Raspberry-Pi-Edison/

@daanpape
Copy link
Contributor

@vshymanskyy the makefile looks very good, I have tested everything with uclibc (not the ulibc++ because it is not needed and takes up additional flash space) and it works like a charm. I think you can lower the dependencies in your makefile. I'm currently using the following openWRT makefile (off course when released the source must be downloaded from espruino git):

# 
# Copyright (C) 2015 Daan Pape, DPTechnics
#
# This is free software, licensed under the GNU General Public License v2.


include $(TOPDIR)/rules.mk
include $(INCLUDE_DIR)/kernel.mk

PKG_NAME:=espruino
PKG_VERSION:=1.80
PKG_RELEASE:=1

PKG_BUILD_DIR := $(KERNEL_BUILD_DIR)/$(PKG_NAME)
PKG_MAINTAINER := Daan Pape <daan@dptechnics.com>

include $(INCLUDE_DIR)/package.mk

define Package/espruino
  SECTION:=lang
  CATEGORY:=Languages
  TITLE:=Espruino JavaScript interpreter
  URL:=http://www.espruino.com
  DEPENDS:=+libpthread
endef

define Package/espruino/description
 Espruino is an extremely compact JavaScript interpreter. Originally
 designed for microcontrollers, it runs on OpenWRT while using a very small
 amount of RAM. It includes a cut-down version of node's HTTP client and
 server, filesystem, as well as GPIO support.
endef

MAKE_FLAGS += RELEASE=1
# Board specific for GPIO, SPI, I²C bindings
MAKE_FLAGS += DPTBOARD=1
MAKE_FLAGS += LD="$(TARGET_CC)"

define Build/Prepare
    mkdir -p $(PKG_BUILD_DIR)
    $(CP) ./src/* $(PKG_BUILD_DIR)/
endef

define Package/espruino/install
    $(INSTALL_DIR) $(1)/usr/sbin
    $(INSTALL_BIN) $(PKG_BUILD_DIR)/espruino $(1)/usr/sbin/espruino
endef

$(eval $(call BuildPackage,espruino))

I'm working on SPI and I2C, I'l give some updates when I got it it work.

@vshymanskyy is your test platform also MIPS, see the bug above.

@gfwilliams
Copy link
Member Author

With SPI and I2C, I guess the issue is finding out which pins are connected to which SPI/I2C device path? While it could be hard-coded for your board it seems like the kind of thing that you should be able to query the OS for?

@daanpape
Copy link
Contributor

@gfwilliams actually there are kernel modules for SPI and I²C and in OpenWRT one must configure these kernel modules to use the correct pins. I'm looking now to connect to these kernel modules which present devices in /dev which are agnosting from the pin-layout. So it should be quite straight forward.

@vshymanskyy
Copy link
Contributor

@daanpape, thank for you hints, I removed the unneeded dependencies.

@gfwilliams
Copy link
Member Author

Hi - I just added some more hacks into jsnative.c as well as some sanity checks that run at startup (unless you use RELEASE=1 when building): 8fe6edd

Might be helpful? You might be able to see if something like the passing of doubles isn't working.

@vshymanskyy
Copy link
Contributor

Bump.
Updated the OpenWRT package again: https://github.com/vshymanskyy/OpenWRT-Espruino-packages
I will try to test this again, hopefully it works on VoCore & Rasperry Pi

@gfwilliams
Copy link
Member Author

Hi, thanks! How did testing go?

Is it easier if you just contribute the Makefile changes as a pull request, then the patch won't need keeping up to date? Perhaps just as an 'OPENWRT' target?

Did you want to submit the package yourself, or do you want me to do it?

@vshymanskyy
Copy link
Contributor

It works well when compiled with musl.
With uclibc it starts, and then fails on setInterval:

>setInterval(function() { console.log("1") }, 1000)
[  163.110581]
do_page_fault(): sending SIGSEGV to espruino for invalid read access from 00000020
[  163.127879] epc = 00404701 in espruino[400000+44000]
[  163.137888] ra  = 0042ded3 in espruino[400000+44000]
[  163.147935]

For me it's no issue (musl is default and I also use it), and I currently don't have time to investigate.
I think it may fail for other architectures as well - but it actually doesn't matter - it can be fixed in time, with community support ;)

Yes we can just include the changes in next release, then update the package not to apply any patches

@gfwilliams
Copy link
Member Author

Which architecture were you going for? I know x86 (non-64) has some issues with GCC5 as I think they changed the calling convention.

@OwenBrotherwood
Copy link
Contributor

Hi @vshymanskyy
Can you help me with the ultimate how to for OpenWrt and Espruino?
https://github.com/OwenBrotherwood/Espruino/blob/puckdoc/README_Building.md#for-openwrt

@vshymanskyy
Copy link
Contributor

vshymanskyy commented Feb 14, 2017

@OwenBrotherwood , I'm quite busy currently, but hopefully I'll get a chance to play with the VoCore2 I got recently, so I will check if it works and update the instructions.

@vshymanskyy
Copy link
Contributor

@OwenBrotherwood Yes I can confirm my package works on VoCore 2 according to instructions on https://github.com/vshymanskyy/OpenWRT-Espruino-packages

 _____                 _
|   __|___ ___ ___ _ _|_|___ ___
|   __|_ -| . |  _| | | |   | . |
|_____|___|  _|_| |___|_|_|_|___|
          |_| http://espruino.com
 1v92 Copyright 2016 G.Williams

@gfwilliams I also updated the version to 1.92. Could you please review and incorporate my patch if possible? https://github.com/vshymanskyy/OpenWRT-Espruino-packages/blob/master/espruino/espruino/patches/010-makefile-fix.patch

@gfwilliams
Copy link
Member Author

Just done - thanks!

@vshymanskyy
Copy link
Contributor

https://github.com/vshymanskyy/OpenWRT-Espruino-packages
was updated to 1.94, the OpenWRT-specific patch was completely removed.

@gfwilliams
Copy link
Member Author

That's fantastic - thanks for the update!

@gfwilliams
Copy link
Member Author

Closing this - looks like there's not been any interest in the last 6 years.

Shame really as it would have been pretty neat - but I guess it's not too hard to just build it natively for whatever Linux target you want.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants