Data from Arduino over Ethernet and wget, instead of python

Question

The topic of an Arduino serving data via Ethernet seems to have been discussed in a few places:

1) Arduino ethernet communication

and

2) Dumping data over ethernet with an arduino client / python server

The way I like the most is the Arduino WebClient option which was mentioned in the first post:

https://www.arduino.cc/en/Tutorial/WebClient

The second post involves some Python (2.7), but it didn't seem like the problem was solved. I was also wondering if it was easier to do with wget.

If you have an Arduino acting as a simple server giving information:

/*
Simply put out data as a server
*/

#include <SPI.h>
#include <Ethernet.h>

unsigned long current_time;
unsigned long old_time;

// Ethernet stuff
// Enter a MAC address and IP address for your controller below.
// The IP address will be dependent on your local network:
byte mac[] = {
  0xDE, 0xAD, 0xBE, 0xEF, 0x12, 0x34
};

// Initialize the Ethernet server library
// with the IP address and port you want to use
// (port 80 is default for HTTP):
EthernetServer server(80);
boolean notYetConnected;

// IP Address is set here
IPAddress ip(192, 168, 3, 50);

void setup()
///////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////
{
  int i;

  Serial.begin(9600);

  // Ethernet option

  // start the Ethernet connection and the server:
  Ethernet.begin(mac, ip);
  server.begin();
  Serial.print("# For Ethernet connection, server is at ");
  Serial.println(Ethernet.localIP());
  Serial.print("# \n");

}


void loop()
///////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////
///////////////////////////////////////////////////////////////////////////
{

  int i;

  current_time = millis();

  // dump data every 100 ms
  if ((current_time - old_time) > 100)
  {

    // data from sensor spoofed here
    int datavalue = random(0, 100);

    Serial.print(current_time);
    Serial.print(",");

    Serial.print(datavalue);
    Serial.print("\n");

    server.print(current_time);
    server.print(",");
    server.print(datavalue);
    server.print("\n");


    // get delta time
    old_time = current_time;
  }
}

... you can use 'wget 192.168.3.50' to get the data, which dumps to a file (default index.html).

This isn't the typical client/server thing, where a program asks for information, and then it is returned; the server just dumps data out, and you can point a web browser to the IP address, or, as seen above, use wget.

When you 'set and forget' the wget command, data is recorded pretty well. I just did a test for 1.75+ hours and got 60K+ lines (once every 100 ms), and the system works.

I've noticed that if I stop the 'wget' command, and restart it, after a few times, the wget process hangs up, and I have to reset my Arduino.

A full client-server program seems a better way as per:

https://giovanniorgantini.wordpress.com/2015/03/14/getting-data-over-the-internet-with-arduino/

... and I'll be working on this now (the original client is in C, if someone can point me to a simple python-Arduino program, otherwise, I'll be looking at a simple python client), but was wondering:

1) Why would stopping the 'wget' (control-C) cause problems in restarting the wget process, where the system hangs up with:

user@machine:$ wget 192.168.3.50 --2018-02-12 19:58:54-- http://192.168.3.50/ Connecting to 192.168.3.50:80...

One reason to stop the data stream is when you are stopping a test, or, programmatically, to start another data file.

2) Is it possible to parse wget output, so that data can be saved in files every N data points or N seconds?

The client-server method seems like the way to go, but the example above seems to work with only using a web browser, or a single command line function. This might seem a bit easier to use, for some applications.

This is a simple application, just being used to dump data from a set of sensors.

In my research, I've also seen UDP client-server:

http://www.toptechboy.com/tutorial/python-with-arduino-lesson-16-simple-client-server-configuration-over-ethernet/

Didn't know if there was a preferred way of doing this.

Answer 1

If you're throwing stuff over the network with no specific client in mind, I think UDP might be a better option.

As for limiting file size, I suggest using logrotate like this answer suggests - https://unix.stackexchange.com/questions/17209/how-to-limit-log-file-size-using