Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Create a additional ESP8266 board with 4MB:1024/1024 #1110

Closed
MaBecker opened this issue Mar 11, 2017 · 37 comments
Closed

Create a additional ESP8266 board with 4MB:1024/1024 #1110

MaBecker opened this issue Mar 11, 2017 · 37 comments
Labels
ESP8266 This is only a problem on ESP8266 devices implemented

Comments

@MaBecker
Copy link
Contributor

Inspired by @pedro [4MB: 1024/1024](Proposed, and implemented, some enhancements (1Mb + 1Mb firmware, resizable jsvars size, ...)) it is time to create a ESP8266_4MB_BOARD.py to have more space for firmware, modules and flash and keep ESP8266_BOARD.py as is.

  • ESP8266_4M_BOARD.py
  • add Graphics and Filesystem
  • use 1MB for save()
  • 1MB to use with "Flash"
  • 1600 variables
  • and about 9500 freeHeap

@gfwilliams please add label ESP8266

@gfwilliams
Copy link
Member

When you do this, please can you also make a 1M_combined image? It'd save a lot of pain for beginners, as then you just have one single image to upload which also wipes the saved code.

@wilberforce
Copy link
Member

@MaBecker

ESP8266_4MB_BOARD.py

Perhaps should be ESP8266_4MB.py

1mb for save area seems a lot!

@MaBecker
Copy link
Contributor Author

make a 1M_combined image

Good point! Yes of course.
Note: wiflash.sh can only handle single bin files

Perhaps should be ESP8266_4MB.py

ok so it is ESP8266_4MB.py

@gfwilliams gfwilliams added the ESP8266 This is only a problem on ESP8266 devices label Mar 13, 2017
@gfwilliams
Copy link
Member

Good point. 1MB does seem like overkill. I guess even 50k would be more than enough.

Personally I'd say you should pack everything into the first 1MB (including saved code), and then you can leave a full 3MB for the user to do what they want (including 1MB for wifi firmware updates and still 2MB free for user data).

@MaBecker
Copy link
Contributor Author

1mb for save area seems a lot!

1MB does seem like overkill. I guess even 50k would be more than enough.

so we have 13 pages for save

using 4MB: 1024/1024 gives you 2 times 1 MB user flash

  chip['saved_code'] = {
    'address' : 0x201000, # first page is used for wifi save 
    'page_size' : 4096,
    'pages' : 13,
    'flash_available' : 912, # firmware can be up to this size 
   };

and add the rest can be added as free Flash Areas, not sure if it makes sense to have to big areas

  .....
  uint32_t map = system_get_flash_size_map();
  if ( map == 6 ) {
     addFlashArea(jsFreeFlash, 0x214000, 0x200000-0x14000);
     addFlashArea(jsFreeFlash, 0x300000, 0x300000-0x4000);
    return jsFreeFlash;
  }
  .....  

@gfwilliams
Copy link
Member

Just wondering - could we not just create a 1MB image? It would then be compatible with all 1MB devices too - so pretty much everything, it just wouldn't allow OTA firmware updates on 1MB parts?

@MaBecker
Copy link
Contributor Author

ok, fs why not, just have to check that there is enough heap space left.

@gfwilliams
Copy link
Member

fs shouldn't take up heap. I guess if it does, it might be a code page table that does - but it should be easy enough to put in flash.

@MaBecker
Copy link
Contributor Author

MaBecker commented Mar 17, 2017

fs shouldn't take up heap

perfect, so module fs is set too.

@Kyklas
Copy link

Kyklas commented Mar 27, 2017

I am looking for a greater save() size. I use 1183 JSVars and have 9k of code so 12KB of storage is not enough. A save() size of 64KB or 128KB would be nice.

@opichals
Copy link
Contributor

Hi @MaBecker! Would you mind sharing the board file and other required changes before it gets to master?

@MaBecker
Copy link
Contributor Author

MaBecker commented Mar 27, 2017

@opichals check https://github.com/MaBecker/Espruino

FLASH_4MB_C1=1 make
FLASH_4MB_C1=1 make flash

Edit: Sorry it is broken, have to fixit .....

@MaBecker
Copy link
Contributor Author

@Kyklas ok lets start with 16 pages for save()

@MaBecker
Copy link
Contributor Author

@gfwilliams continue working on this issue - please let me know if this is the correct way.

My understanding is that

BOARD=ESP8266_4MB  make 

will create a 4MB: 1024/1024 firmware

  • new ESP8266_4MB.py file with 4MB settings
    ...
    'package' : "4MB",
    ...

  • adds to Makefile

    ...
    else ifeq ($(FAMILY),ESP8266)
    USE_ESP8266=1
    ifeq ($(PACKAGE),4MB)
    FLASH_4MB=1
    endif
    ...

  • make/targets/ESP8266.make

    ...
    ifdef FLASH_4MB
    LD_SCRIPT1 = ./targets/esp8266/eagle.app.v6.new.2048.app1.ld
    LD_SCRIPT2 = ./targets/esp8266/eagle.app.v6.new.2048.app2.ld
    else
    LD_SCRIPT1 = ./targets/esp8266/eagle.app.v6.new.1024.app1.ld
    LD_SCRIPT2 = ./targets/esp8266/eagle.app.v6.new.1024.app2.ld
    endif
    ...

and so on....

@gfwilliams
Copy link
Member

Sounds good - but I'd ditch the change to the Makefile and would put FLASH_4MB=1 straight in the board file, like is done here:

https://github.com/espruino/Espruino/blob/master/boards/PUCKJS.py#L43

@MaBecker
Copy link
Contributor Author

THANKS, that makes it very simple!

@MaBecker
Copy link
Contributor Author

Next steps:

  • adopt flash areas

  • try to create a single 1MB flash file

@Kyklas
Copy link

Kyklas commented Apr 3, 2017

I would like to test a firmware with greater storage.
I looked at the build instruction to compile something myself but I run into issues. Perhaps I can work with someone to go thru the process and update the instructions : https://github.com/espruino/Espruino/blob/master/README_Building.md. Should I open a new ticket ?

@MaBecker if you have a image available I would like to try. I have ESP-12F devices i.e 4MB flash.

I am wondering what the ESP8266 RAM limitations are. How much RAM is available to execute JS ? Is JS Code dynamically loaded from Flash for execution ?

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 3, 2017

Hi @Kyklas,

sorry not now, at the moment I am very busy with other things.

@ghost
Copy link

ghost commented Apr 4, 2017

@MaBecker hi, I follow the method you say, compile, but the system_get_flash_size_map function always returns 4, what is the problem?

@MaBecker
Copy link
Contributor Author

Just wondering - could we not just create a 1MB image?

No because default and blank are not in first partition for a 4MB 1024/1024

bildschirmfoto 2017-04-14 um 08 52 46

but it is possible to add a file to blank save and wifi section, which will have the same effect.

@MaBecker
Copy link
Contributor Author

Or, send a 1MB file plus esp_init_data_default.bin and blank.bin, will try that

@ghost
Copy link

ghost commented Apr 14, 2017

thx,but where are esp_init_data_default.bin and blank.bin downloading?

@MaBecker
Copy link
Contributor Author

@gfwilliams not sure if this is what you have in mind

make flash will always flash a combined file plus addons for a 4MB board like this:

flash: $(ESP_COMBINED)
ifndef COMPORT
	$(error "In order to flash, we need to have the COMPORT variable defined")
endif
	-$(ESPTOOL) --port $(COMPORT) --baud $(FLASH_BAUD) write_flash --flash_freq $(ET_FF) --flash_mode qio --flash_size $(ET_FS) 0x0000 $(ESP_COMBINED) $(ESP_FLASH_ADDONS)

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 14, 2017

Splitting 2x1MB up in 4x256k on third partition and 1x1008k on forth partition - any comments?

@MaBecker
Copy link
Contributor Author

For those who like to sneak in, just committed to my Espruino fork.

@wilberforce
Copy link
Member

@MaBecker - please add:
https://github.com/MaBecker/Espruino/blob/master/boards/ESP8266_4MB.py#L25-L31

     'FILESYSTEM',
     'FLASHFS'	

This will use the last 1Mb, at location 3Mb as a file system.

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 15, 2017

@wilberforce - fyi: the third partition is not completely free on a 4MB 1024/1024, but second is totaly free, so I will change the addFlashArea settings

like this:

addFlashArea(jsFreeFlash, 0x200000, 0x100000);
addFlashArea(jsFreeFlash, 0x300000, 0x40000);
addFlashArea(jsFreeFlash, 0x340000, 0x40000);
addFlashArea(jsFreeFlash, 0x380000, 0x40000);
addFlashArea(jsFreeFlash, 0x3C0000, 0x3C000);

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 15, 2017

adding FILESYSTEM and FLASHFS eats to much heap space, is there optimization potential in this libraries?

'libraries' : ['NET','TELNET','GRAPHICS','NEOPIXEL']

    "cpuFrequency": 160, "freeHeap": 10536, "maxCon": 10, <- nice

'libraries' : ['NET','TELNET','GRAPHICS','NEOPIXEL', 'FILESYSTEM']

    "cpuFrequency": 160, "freeHeap": 8688, "maxCon": 10, <- possible

'libraries' : ['NET','TELNET','GRAPHICS','NEOPIXEL', 'FILESYSTEM','FLASHFS']

    "cpuFrequency": 160, "freeHeap": 5168, "maxCon": 10,  <- this might be too small

'libraries' : ['NET','TELNET','GRAPHICS','CRYPTO','NEOPIXEL', 'FILESYSTEM','FLASHFS']

    "cpuFrequency": 160, "freeHeap": 5040, "maxCon": 10,

@wilberforce
Copy link
Member

@MaBecker

Unfortunately with 4096 flash page size,. it also requires 4096 of RAM as the current buffer for read and write operations. So it will always require at least this additional space. Possibly some of the strings could be moved to ROM, however I'm not sure it is worth the effort.

@MaBecker
Copy link
Contributor Author

MaBecker commented Apr 16, 2017

Summary:

  • new board ESP8266_4M with map mode 6 : 1024/1024
    • use partition 1 and 2 for firmware
    • allow firmware up to 940KB
    • save source code in partition 1
    • save wifi setting in partition 1
    • create combined 1024K firmware image to wipe source and wifi settings
    • add flash areas
      • partition 3: 1 x 1MB
      • partition 4: 3 x 256K and 1 x 240K
    • define 1600 vars
    • define 16 pages for save code
    • add flash_combined and flash_erase to make/targets/ESP8266.make
    • wiflash is breaking save sections, no idea.....
  • additional libraries
    • GRAPHICS, freeHeap: 9512
    • ...
  • build option RELEASE=1
    • GRAPHICS, freeHeap: ?
    • ...

@MaBecker
Copy link
Contributor Author

Have to give up to save wifi and code in first partition, because make wiflash wipes that.

  • Moving to forth partition, to keep third as 1MB.
  • build a 4MB combined image, so everything can be included

@MaBecker
Copy link
Contributor Author

adding 'FILESYSTEM' and 'FLASHFS' make no sense, because it is using to much heap space.

I think Flash storage Module, any js type , web pages, css, js and images or something similar would be nice to have for systems with poor resources like the ESP8266.

@MaBecker
Copy link
Contributor Author

@gfwilliams please take a brief look and give some advice to make sure that this is still going in the right direction - thanks

@MaBecker
Copy link
Contributor Author

#1142

@gfwilliams
Copy link
Member

Looks fine to me...

@Kyklas
Copy link

Kyklas commented May 9, 2017

@MaBecker : Thank you for the release of the 64k flash save memory.

@ALL : I have downloaded and flash the new firmware on my ESP8266 yesterday. Unfortunately I am still facing an "Out of Memory" errors. I think i may have just too much JS Code/Vars for the ESP to handle. I would like to know how much JS Code I can have in the ESP RAM. The IDE reports around 9k of JS Code sent. I am using the MQTT module and using it with so much code leads to errors. I am using close to 700+ JS Vars as well. How is the RAM Memory of the ESP8266 split between JS Code and JS Var ?

Could http://www.espruino.com/EspruinoESP8266 be updated to bring information about the limitation of the ESP8266 ?

I may go for the ESP32 since the ESP8266 limitations are reach with program with moderate complexity.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ESP8266 This is only a problem on ESP8266 devices implemented
Projects
None yet
Development

No branches or pull requests

5 participants