One of the goals for this project was to create a remotely reconfigurable OBD logger. To do this requires being able to accept reconfiguration commands. To this point, the data has only been sent from the device to a server and binned into a database. To move forward, I had to rewrite the server to allow for outbound communications and design a command architecture that would allow for several devices to connect to the server simultaneously. The server would also have to allow for the uploading of varied parameter groupings while still binning them correctly, as well as the individual addressing of connected nodes. This was no easy feat, especially since IP addresses over cellular networks can change quickly. The server would have to be able to locate a vehicle before it could change its mode, and would have to be able to contend with constantly changing data streams that might have network-based concurrency issues.
6.2 Experimental Procedure
I first made a list of revised goals for the server. It would maintain an up-to-date list of which clients are connected and their current IP address, a list of configuration parameters for each device to facilitate mode-setting based on last-used parameters, and would parse incoming data into the appropriate locations in a MySQL database. As a new, secondary purpose, it would also serve as a target for web clients, allowing owners of vehicle informatics hardware to change the parameters requested and vary the update rate of their device remotely. To that end, I developed a new architecture that could be tested using simulated clients.
6.2.1 Command Architecture
With multiple devices connecting to a single server, every vehicle needed a unique identifier. Fortunately, HS-3000 provides us with easy access to such an identifier: the VIN number. The first thing the hardware did when turning on was read the VIN number and status of the check engine light to memory. The device connected to a server which replied and then began communication by sending a string in order to store the current IP of the vehicle in a database. The server replied with either a default
mode command (if the device had never before connected), or the last used command. The mode command started and stopped tasks on the processor.
Once the string was received, the device verified the VIN against the variable stored in RAM and if it matched, it stopped the old tasks and created new tasks sampling sensors, uploading them, and logging them to an SD card. The tasks were identified using headers and each task had a sampling period associated with it. The device uploaded the results from these tasks with headers for parsing and binning to an SQL database on the server side. The tasks continued until they were changed, or until the device was shut off.
To be fully reconfigurable, a client needed to be able to tell the server to change the mode of a given device while in operation. A second change of mode command allowed for this. A client sent the server a mode command, and the server attempted to locate the relevant node. The server did a VIN/IP lookup to determine the most recent IP address for the target vehicle. The server attempted to send the mode command to the device, and set the “last used” mode in the database to the new command. This ensured that if the connection failed due to network errors, the device would switch to the new task set as soon as it reconnected. The use of headers on every uploaded parameter avoided the potential for concurrency issues this structure could cause.
`A0100` VARCHAR(32) DEFAULTNULL, /* AA - PIDS SUPPORTED (01-20) - BIT ENCODED */ `A0101` VARCHAR(32) DEFAULTNULL, /* AI - STATUS SINCE DTCS CLEARED */
`A0102` VARCHAR(64) DEFAULTNULL, /* AJ - DTC THAT CAUSED FREEZE FRAME STORAGE */ `A0103` VARCHAR(16) DEFAULTNULL, /* AK - FUEL SYSTEM STATUS */
`A0104` DOUBLE(5,2) DEFAULTNULL, /* AL - CALCULATED ENGINE LOAD VALUE */ `A0105` INT(3) DEFAULTNULL, /* AM - ENGINE COOLANT TEMPERATURE */
`A0106` DOUBLE(5,2) DEFAULTNULL, /* AN - SHORT-TERM FUEL TRIM (BANK 1) */ `A0107` DOUBLE(5,2) DEFAULTNULL, /* AO - LONG-TERM FUEL TRIM (BANK 1) */ `A010A` INT(3) DEFAULTNULL, /* AR - FUEL PRESSURE */
`A010B` INT(3) DEFAULTNULL, /* AS - INTAKE MANIFOLD ABSOLUTE PRESSURE */ `A010C` DOUBLE(7,2) DEFAULTNULL, /* AT - ENGINE RPM */
`A010D` DOUBLE(5,2) DEFAULTNULL, /* AU - VEHICLE SPEED */ `A010E` DOUBLE(3,1) DEFAULTNULL, /* AV - TIMING ADVANCE */ `A010F` INT(3) DEFAULTNULL, /* AW - INTAKE AIR TEMPERATURE */ `A0110` DOUBLE(5,2) DEFAULTNULL, /* AX - MAF AIR FLOW RATE */ `A0111` DOUBLE(5,2) DEFAULTNULL, /* AY - THROTTLE POSITION */ `A0113` VARCHAR(8) DEFAULTNULL, /* BA - LOCATION OF O2 SENSORS */ `A0114A` DOUBLE(4,3) DEFAULTNULL, /* BB - BANK 1, SENSOR 1 O2 VOLTAGE */
`A0114B` DOUBLE(4,1) DEFAULTNULL, /* BB - BANK 1, SENSOR 1 SHORT TERM FUEL TRIM */ `A011C` VARCHAR(8) DEFAULTNULL, /* BJ - OBD STANDARD VEHICLE CONFORMS TO */ `A011D` VARCHAR(8) DEFAULTNULL, /* BK - OXYGEN SENSORS PRESENT */
`A011F` DOUBLE(6,1) DEFAULTNULL, /* BM - RUNTIME SINCE ENGINE START */ `A0121` DOUBLE(6,1) DEFAULTNULL, /* BN - DISTANCE TRAVELLED WITH MIL ON */ `A0124A` DOUBLE(4,3) DEFAULTNULL, /* BQ - BANK 1, SENSOR 1 O2 EQUIVALENCE RATIO */ `A0124B` DOUBLE(4,3) DEFAULTNULL, /* BQ - BANK 1, SENSOR 1 O2 VOLTAGE */
`A012F` DOUBLE(5,2) DEFAULTNULL, /* CB - FUEL LEVEL INPUT */ `A0130` INT(3) DEFAULTNULL, /* CC - WARM-UPS SINCE CODE CLEARED */
`A0131` INT(5) DEFAULTNULL, /* CD - DISTANCE TRAVELED SINCE CODE CLEARED */ `A0133` INT(3) DEFAULTNULL, /* CF - BAROMETRIC PRESSURE */
`A0134A` DOUBLE(4,3) DEFAULTNULL, /* CG - BANK 1, SENSOR 1 O2 EQUIVALENCE RATIO */ `A0134B` DOUBLE(6,3) DEFAULTNULL, /* CG - BANK 1, SENSOR 1 O2 CURRENT */
`A013C` DOUBLE(5,1) DEFAULTNULL, /* CO - BANK 1, SENSOR 1 CATALYST TEMPERATURE */ `A0142` DOUBLE(5,3) DEFAULTNULL, /* CT - CONTROL MODULE VOLTAGE */
`A0143` DOUBLE(7,2) DEFAULTNULL, /* CU - ABSOLUTE LOAD VALUE */
`A0144` DOUBLE(6,5) DEFAULTNULL, /* CV - COMMANDED EQUIVALENCE RATIO */ `A0145` DOUBLE(5,2) DEFAULTNULL, /* CW - RELATIVE THROTTLE POSITION */ `A0146` INT(3) DEFAULTNULL, /* CX - AMBIENT AIR TEMPERATURE */
`A014D` INT(5) DEFAULTNULL, /* DE - RUN TIME WITH MIL ON */ `A014E` INT(5) DEFAULTNULL, /* DF - TIME SINCE CODES CLEARED */ `LAT` DOUBLE(9,7) DEFAULTNULL, /* LA - LATITUDE */
`LON` DOUBLE(10,7) DEFAULTNULL, /* LO - LONGITUDE */ `ALT` DOUBLE(7,2) DEFAULTNULL, /* ZZ - ALTITUDE */
`LIVE_ACCEL_X` DOUBLE(5,3) DEFAULTNULL, /* IX - REAL-TIME ACCELERATION */ `DTCS` VARCHAR(20) DEFAULTNULL, /* TR - DTCS ACTIVE */
Figure 6: Example data structure and header/parameter mapping from the MySQL create statement. Notice that most of the OBD parameters are emissions related.
With the architecture figured out, I wrote a simulator based on this configuration. The simulator used the socket library in Python as it did not need to be asynchronous. I wanted to test the server’s ability
to manage connections, so rather than keeping a socket open, I opened and closed the socket before and after every transmission. This simulated some of the worst network conditions imaginable. Testing with a few simulators running would stress test the server as though dozens or hundreds of vehicles were attempting to connect simultaneously.
To gather data for the simulations, I utilized data from the Bluetooth logger and reformatted it to fit the new architecture. In some cases, I worked backwards from Google Maps, defining a route and making educated guesses about various parameters at small time steps.
The simulation worked. The server was able to process change of mode requests and send them to the connected client directly if the connection was still live, and worked as expected when the client was not connected at the time of the change command and sent it the command upon the next login. Even simulating concurrency issues, the header parsing worked reliably.
6.3 Results and Discussion
The result of this experiment was an architecture capable of successful two-way communication, proper parsing based on transmission headers, a server capable of handling mode requests, and an easy way to store lots of incoming data to MySQL. The server initially had problems handling parameters that reported two values, but rewriting the dictionary for those parameters alleviated the problem. There was also an issue with processing negative numbers that turned out to be a problem with the regular expression parsing the incoming strings. With that fixed, the server was able to handle twenty-five clients constantly connecting and disconnecting at 1Hz without measurable lag or missed data packets. With the parsing and storage to MySQL working fine, the next experiment could begin – developing a means of displaying the incoming data in a user-friendly format.
7 Experiment Revision 4 (Writing a Login System and Client Application)