Unfortunately this only works with very small POSTs. Generally one needs to buffer all the data received from data events, and then process them on the end event, which doesn't exist yet. Could you implement it? :)

Currently, I seem to be getting exactly 96 chars when the data event fires. req.dRcv seems to be even worse at 32 chars, didn't need to use substr as per the forum post, the headers seem to have already been removed from it.

Refs:

1. Example of usage
2. http://nodejs.org/api/http.html#http_class_http_clientrequest

There is a close event on the response? Is that not correct, or should there be a close and and end event?

The 96 chars may just be to do with the amount needed to parse the header... At the moment everything is being transferred to/from the CC3000 in 32 byte packets.

Ah, makes sense :)

Unfortunately the close event on the response doesn't help; what if our response depends on the POST data? We have to close the connection to get the data, and then it's too late to send a reply :>

I actually found all the events quite confusing, so I spent some time in the spec and made this:

router.addRoute('/post', function(req, res) {

  req.on('data', function(data) {
    console.log('request data: ' + data);
  });

  req.on('close', function() {
    // Indicates that the underlaying connection was closed.
    // Just like 'end', this event occurs only once per response.
    // BUT... we might need to process data before connection closes!
    console.log('request close (http.incomingMessage)');
  });

  req.on('end', function() {
    // This event fires when no more data will be provided (but connection
    // is still open), and all data was consumed [by data event handlers].
    console.log('request stream end (http.incomingMessage stream)');
  });


  res.on('close', function() {
    // Indicates that the underlying connection was terminated before
    // response.end() was called or able to flush.
    console.log('response close (http.ServerResponse)');
  });
  res.on('finish', function() {
    // Emitted when the response has been sent.
    console.log('response finish (http.serverResponse)');
  });


  // should also test when client disconnects before this!
  setTimeout(function() {
    console.log('res.end()');
    res.end();
  }, 5000);

});

Output (if server ends request):

POST /post
request data: 1 3 5 7 9 11 14 18 20 23 26 29 32 35 38 41 44 47 50 53 55 59 62 65 68 71 74 77 80 803 87 91 93 9
request data: 6 99 102 106 110
res.end()
response close (http.ServerResponse)

We shouldn't actually get the close event in this case :> Anyways, we need the end event on the req stream to know all data is available, before calling res.end(). So can we re-open this issue? We still have no reliable way to get a large POST request if we need it to form our response.

My current workaround relies on knowing the expected lastChar of the body, and it works great, until I get unlucky and user supplied data happens to include this char and be chunked at that point :> So, not urgent for me, and doesn't seem to affect anyone else (yet), but definitely needs to get fixed at some point.

Sure - thanks for the code above - it helps a lot :)

Just a quick one though - how do we know that the client has finished sending? is it as simple as checking against the content-length header? (that might actually be the best workaround for you now?)

By the way, if you do get time and fancy poking around, you can actually run Espruino on Linux so could develop/debug changes for this without having to get a toolchain for ARM sorted out :)

Ooh yeah, in my case that's a much better workaround, thanks! I checked and the requests are indeed sent with the content-length header. But clients are also allowed to not send this header when using transfer-encoding: chunked. Not sure if each sent chunk correlates to what is given to the data callback, or if it still needs to be buffered, but I'll save that for another day (there is a last-chunk sequence though).

Anyway, I had to dig around for the "real" answer, and I hope I got it right since I'm a bit out of my league here, but the http server is a socket server with allowHalfOpen set to true (source). And although I couldn't find a ref, it seems to be standard practice for the client to send a TCP FIN after the POST request (or any request I guess). So I think the correct time to fire the end event is on receiving the FIN from the client on the socket.

Re running on Linux, I saw that this morning! Quite keen to check it out when I get a chance (how would I test HTTP though?). I haven't touched C in some 15 years or so, but I used to be quite proficient :> But yeah, would love to rather send PRs than problems hehe. If I'm unsure, I'll just send a link to the commit on a different branch of my fork.

Thanks!

Well, in Espruino there's no socket implementation right now, so the HTTP server is standalone.

You can test HTTP in Linux just like with the Espruino Board (the HTTP server is part of Espruino rather than the CC3000) - just set the port above 1024 or you'll have to run it as a superuser :)

Hi, just to clarify: when clients send transfer-encoding: chunked they never send content-length; The whole purpose is being able to send a possibly unknown amount of data. So you absolutely have to buffer the data, ideally to SD if available.

You can now do this just fine:

var fileNo = 0;

function onPageRequest(req, res) {
  var d = "";
  var boundary;
  var f;
  req.on('close', function() { 
    if (f) {
      f.close();
      console.log("File uploaded");
    }
  });
  req.on('data', function(data) { 
    d += data;
    if (f) {
      var idx = d.indexOf(boundary);
      /* ensure we always leave some in our buffer
      in case boundary straddles packets */
      if (idx>=0) {
        console.log("Boundary found - starting again");
        f.write(d.substring(0, idx));
        f.close();
        f = undefined;
      } else {
        idx = d.length-boundary.length; 
        f.write(d.substring(0, idx));
      }
      d = d.substring(idx);
    } else {
      var idx = d.indexOf("\r\n\r\n");
      if (idx>=0) {
        var header = d.substring(0,idx).split("\r\n");
        boundary = "\r\n"+header.shift();
        header = header.map(function(x){return x.split(": ");});
        console.log(JSON.stringify(header));  
        // choose a filename
        var fileName = "f"+fileNo+".txt";
        console.log("Writing to "+fileName);
        f = E.openFile(fileName, "w");
        fileNo++;
        d = d.substring(idx+4);
      }
    }
  });
  res.writeHead(200, {'Content-Type': 'text/html'});
  res.write('<html><body><form method="post" enctype="multipart/form-data">');
  res.write('<input type="file" name="file1" id="file"><input type="file" name="file2" id="file"><input type="submit">');
  res.end('</form></body></html>');
}
require('http').createServer(onPageRequest).listen(8080);

However in the example above, the SD card is too slow to cope with ESP8266 and high baud rates. You can still use lower baud rates, other network devices, or do something other than write to SD though.