First stage finished?

docs/readthedocs/chapters/usage/software.rst

@@ -79,16 +79,70 @@ Client side
 Advanced Folder Transfer
 ^^^^^^^^^^^^^^^^^^^^^^^^
 
-*Creator: Jerry Kong
+*Creator: Jerry Kong*
 
 To meet our need of a neat and organized data structure, this script is created.
 It has the capability to transfer the entire folder to another remote desktop, no matter it is on a Windows System or Unix system.
-The script rests in PiCar/src/Logging
+The script rests in ``PiCar/src/Logging``
 To use the script, first setup the ip addresses like in the basic version, change the root variable to the root folder name.
 Place the script at the same level as the root folder. Start the server script and then start the client script. The folder would then be transferred.
 A better protocol could be implemented, since the protocol now being used is not really efficient though fulfill the need of our experiment for now.
 
 
+WIFI Router setting
+^^^^^^^^^^^^^^^^^^^
+
+*Creator: Jerry Kong*
+
+*This section is dedicated to users who are not familiar with WIFI network setting, TCP protocol and wireless connection*
+
+To establish communication between two machine we need to know their address. Moreover, to provide a consistent and save network experience, a machine would have many ports to receive connections with different forms,
+thus we also need to agree on the port that two machine establish the connection on. However, depending on different internet environment and different ways of connection (wifi or ethernet), the IP address would also vary.
+Read through this section, you would get a sense of how this astonishingly complicated system works and hopefully learn how to cope with "Connection fails" error when you are using the script I wrote in the repo or any kind of
+Internet application.
+
+IP
+###
+
+IP's full name is Internet Protocol. It's a scheme that specifies how computers find each other in the pool of Internet. The rules behind it is complicated, but the most important thing is that it is a identification
+for modern computer wired to Internet and is universally used as the synonym of IP address.
+
+IPv4? IPv6?
+###########
+
+As a protocal, IP would have different versions, the latest version is version 6 and thus called IPv6. While IPv6 is stronger and has a larger pool of Internet, the older version IPv4 is not obsoleted.
+The logic behind the two protocals are the same, hence we would now stick with IPv4, since it has a more concise format. (XXX.XX.XX.XXX)
+
+WIFI vs ethernet
+################
+
+You must be familiar with this topic. WIFI is more convenient while wired connection (ethernet) offer steadiness and low latency. However, it is important to note that a computer connected to wifi does not have an IP, or at least, an acknowledged IP.
+Wifi or the router serves as a broadcaster and spread the connection from the ethernet to multiple machines, but they have the same IP address. The router can identify each machine by the IP address it assigns to the machine, but the machine can't use
+that address as the identification on the internet. Conclusively, machines under the same WIFI build up a small Internet where these machines can identify each other by the address they are assigned, but once outside WIFI network they are no longer acknowledged.
+
+See
+`Setup static IP address for RaspberryPi <https://www.raspberrypi.org/forums/viewtopic.php?t=191140>`_
+, so a machine would be assigned the same IP address when connected to the WIFI.
+
+TCP
+###
+
+TCP, transmission Control Protocol, is a higher level protocol that enables data sending via the connection established by IP. Socket, a method based on TCP is prevailing on data transfer.
+
+Port and Port forwarding
+########################
+
+With the knowledge about address in mind we could start the connection once we have the right port. It is easy to do so if both machines are on Internet or under the same WIFI, since they can identify with each other. Just pick up an empty port and they are good to go.
+However, we do want to establish connection between two machines even if one is on WIFI and the other is on Internet. To do so, these smart people invented port forwarding. With port forwarding, a client can find the address of the router and use the port that is forwarded to connect with the machine.
+
+For example, the address of the router is 172.10.10.111, and a machine under the WIFI is assigned static IP 192.168.1.188. The routher and the machine agree on that the connection to the port 30000 of the routher would be forwarded to the port 6000 of the machine and vice versa.
+Thus a laptop could setup a connection with 172.10.10.111 on port 30000 to connect the port 6000 on machine with static IP 192.168.1.188.
+
+See
+`How to setup port forwarding <https://www.howtogeek.com/66214/how-to-forward-ports-on-your-router/>`_
+
+
+
 Sensors(Lidar,IMU) reading and writing
 --------------------------------------
 
@@ -210,11 +264,27 @@ The script is based on delta timing method. A constant value of 0.0007 is subtra
 
 Precision defines the minimum time that the script goes to check the diffrence between the last time and current time and consequently defines within what time difference that measures of LiDar and IMU occur simultaneously.
 
-Using -i command line input, we could run the script in endless mode (i.e. the duration would be set to 1000 seconds, we could stop the program by using KeyboardInterrupt(Ctrl + C), the pictures and logging data would be save up until the stopping point)
 
 **A great part of the codes are from Josh Jin's sensor/camera reading code**
 
+Version Beta (Magnetic reading added to IMU)
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+*Creator : Jerry Kong*
+
+*The code could be found in PiCar/src/pi/IMU_Lidar, the  socket_server_client.py file is a integrated and important part of this data logging script, to learn more about socket folder sending, take a look at* ` socket based file sending <http://picar.readthedocs.io/en/latest/chapters/usage/software.html#advanced-folder-transfer>`_
+
+Endless mode is implemented. User could stop the experiment with KeyboardInterrupt, the logging file and camera file would still be saved
+
+Using -i command line input, we could run the script in endless mode (i.e. the duration would be set to 1000 seconds, we could stop the program by using KeyboardInterrupt(Ctrl + C))
+
+Logging file sending module is integarted into the logging script. After the multiprocessing finished (loggind and filming), the script would start a raw socket server and a client on another computer could use the client side script to receive the logging file.
+
+The script could either be called from the terminal or from other script by calling the funtion getSensorAndCamera.
+
+'-s' command line argument and save parameter for getSensorAndCamera is implemented so that users can decide whether they want the logging file to be saved locally.
 
+For installation and usage see the previous section
 
 Data Analysis
 -------------

src/logging/socket_folder_client.py

@@ -1,19 +1,25 @@
 # client.py
+
+# for picar lab students use command "python3 socket_folder_client.py 172.16.10.208 31418"
 import os
 import sys
 import socket                   # Import socket module
 import time
 s = socket.socket()             # Create a socket object
-host = 'localhost'     # Get local machine name
-port = 60001                    # Reserve a port for your service.
+host = sys.argv[1]     # Get local machine name
+port = int(sys.argv[2])                   # Reserve a port for your service.
 
 s.connect((host, port))
 data=s.recv(1024)
+#print(len(data))
+if len(data)!=1024:
+    sup = s.recv(1024-len(data))
+    data = bytearray(data)+sup[0:]
 l=int.from_bytes(data[0:2],byteorder='big',signed='true')
 flds = data[2:2+l].decode('ascii')
 folders = flds.split("!")
 for folder in folders:
-    if folder!="" and " " not in folder:
+    if folder!="":
         os.mkdir(folder)
 
 file = ""
@@ -22,6 +28,13 @@
     get = s.recv(1024)
     if not get:
         break
+    if len(get)!=1024:
+        sup = s.recv(1024-len(get))
+        while len(get)==0:
+            sup = s.recv(1024-len(get))
+        #print(str(len(sup))+"sup")
+        get = bytearray(get)+sup[0:]
+    #print(len(get))
     ll = int.from_bytes(get[0:2],byteorder='big',signed='true')
     if ll>=0 and file!="":
         f.write(get[2:2+ll])
@@ -39,4 +52,3 @@
 
 s.close()
 print('connection closed')
-

src/logging/socket_folder_server.py

@@ -1,47 +1,49 @@
 import os
+import sys
 import socket                   # Import socket module
 
-port = 60001                    # Reserve a port for your service.
-s = socket.socket()             # Create a socket object
-host = "localhost"     # Get local machine name
-s.bind((host, port))            # Bind to the port
-s.listen(5)                     # Now wait for client connection.
-root = "rt"
-print ("begin")
-folders = []
-files = []
-
-for dirname, dirnames, filenames in os.walk(root):
-    # print path to all subdirectories first.
-    for subdirname in dirnames:
-        folders.append(os.path.join(dirname, subdirname))
-    # print path to all filenames.
-    for filename in filenames:
-        files.append(os.path.join(dirname, filename))
-
-
-conn, addr = s.accept()     # Establish connection with client.
-print ('Got connection from', addr)
-#transfer folder
-folderstream = root+"!"+"".join([folder+"!" for folder in folders])
-fl = len(folderstream)
-conn.send(bytearray((fl).to_bytes(2,byteorder='big',signed='true'))+folderstream.encode('ascii')[0:]+bytearray(1022-fl)[0:])
-
-
-for file in files:
-    length = len(file)
-    conn.send((-length).to_bytes(2,byteorder='big',signed='true'))
-    conn.send(file.encode('ascii'))
-    conn.send(bytes(1022-length))
-    f = open(file,'rb')
-    l = f.read(1022)
-    while (l):
-        k = len(l)
-        conn.send(k.to_bytes(2,byteorder='big',signed='true'))
-        conn.send(l)
-        if 1022-k != 0:
-            conn.send(bytes(1022-k))
-        l = f.read(1022)          #alter this to control data sending rate
-    f.close()
-
-conn.close()
+#pi tarzan 192.168.1.121
+
+def send(host,port,root):
+                            # Reserve a port for your service.
+    s = socket.socket()             # Create a socket object
+                                        # Get local machine name
+    s.bind((host, port))            # Bind to the port
+    s.listen(5)                     # Now wait for client connection.
+    print ("begin")
+    folders = []
+    files = []
+
+    for dirname, dirnames, filenames in os.walk(root):
+        # print path to all subdirectories first.
+        for subdirname in dirnames:
+            folders.append(os.path.join(dirname, subdirname))
+        # print path to all filenames.
+        for filename in filenames:
+            files.append(os.path.join(dirname, filename))
+
+
+    conn, addr = s.accept()     # Establish connection with client.
+    print ('Got connection from', addr)
+    #transfer folder
+    folderstream = root+"!"+"".join([folder+"!" for folder in folders])
+    fl = len(folderstream)
+    conn.send(bytearray((fl).to_bytes(2,byteorder='big',signed='true'))+folderstream.encode('ascii')[0:]+bytearray(1022-fl)[0:])
+
+
+    for file in files:
+        length = len(file)
+        conn.send(bytearray((-length).to_bytes(2,byteorder='big',signed='true'))+file.encode('ascii')[0:]+bytes(1022-length)[0:])
+        f = open(file,'rb')
+        l = f.read(1022)
+        while (l):
+            k = len(l)
+            conn.send(bytearray(k.to_bytes(2,byteorder='big',signed='true'))+l[0:]+bytes(1022-k)[0:])
+            l = f.read(1022)          #alter this to control data sending rate
+        f.close()
+
+    conn.close()
+
+if __name__=='__main__':
+    send(sys.argv[1],int(sys.argv[2]),sys.argv[3])
+