Telemetry (Data Acquisition)

Physical Setup

I'm going to use a Raspberry Pi along with an ADC (Analogue to Digital Converter) I purchased to collect temperature data. I've connected directly to the RPi GPIO port using a T-Cobbler Breakout Kit.

Materials List:

1. ADC - Microchip Technology MCP3008-I/P
2. Temperature Sensor - Texas Instruments LM35DZ/LFT1
3. BreadBoard -  Bud Industries BB-32621
4. 24AWG Hookup Wire
5. RaspBerry Pi Mod B
6. T-Cobbler Breakout Kit 

Whoops, accidentally ordered the unassembled T-Cobbler kit so I had to break out the old soldering iron.

Ye olde soldering iron

Test Setup

I mounted the MCP3008 on the breadboard and wired up a rough test set-up using a potentiometer as the analogue voltage input for the ADC. For the wiring I referred to an online tutorial that didn't involve using the Serial Peripheral Interface at first, but eventually found this recipe and decided to learn more about the SPI with Analogue sensors on the Raspberry Pi using an MCP3008

• MCP3008 VDD  3.3V
• MCP3008 VREF  3.3V
• MCP3008 AGND  GROUND
• MCP3008 CLK   GPIO11 (P1-23)
• MCP3008 DOUT  GPIO9  (P1-21)
• MCP3008 DIN   GPIO10 (P1-19)
• MCP3008 CS    GPIO8  (P1-24)
• MCP3008 DGND  GROUND

MCP3008 DIP Pin Out Diagram

And here is the finished product wired entirely in purple. Pin 1 (the white stripe) on the ribbon cable corresponds to the P1 on the RPi mainboard. Now to run some smoke tests.

I started with Matt's code from the blog mentioned above and then added a few minor changes to meet my requirements. I used the Pi WebIDE to write and debug my code; it works very well in tandem with my existing BitBucket account by adding  a simple OAuth consumer to my settings.

Here is a sample of the code I tested my setup with:

	import spidev
	import time
	import os
	import json
	# Open SPI bus
	spi = spidev.SpiDev()
	spi.open(0,0)
	# Function to read SPI data from MCP3008 chip
	# Channel must be an integer 0-7
	def ReadChannel(channel):
	adc = spi.xfer2([1,(8+channel)<<4,0])
	data = ((adc[1]&3) << 8) + adc[2]
	return data
	# Function to convert data to voltage level,
	# rounded to specified number of decimal places.
	def ConvertVolts(data,places):
	volts = (data * 3.3) / float(1023)
	volts = round(volts,places)
	return volts
	# Function to calculate temperature from
	# TMP36 data, rounded to specified
	# number of decimal places.
	def ConvertTemp(data,places):
	# ADC Value
	# (approx) Temp Volts
	# 0 -50 0.00
	# 78 -25 0.25
	# 155 0 0.50
	# 233 25 0.75
	# 310 50 1.00
	# 465 100 1.50
	# 775 200 2.50
	# 1023 280 3.30
	temp = ((data * 330)/float(1023))-50
	temp = round(temp,places)
	return temp
	# Define sensor channels
	pot_channel = 0
	# Define delay between readings
	delay = 5
	# Open file for output
	fileOp = 'a'
	fileName = '/home/pi/projects/sensor_output/temperatureData.json'
	outputFile = open(fileName, fileOp)
	# Limit the loop
	for x in range(0, 3):
	# Get a time stamp
	gmTime = time.gmtime()
	strTime = time.asctime(gmTime)
	timeStamp = time.time()
	# Read the input voltage
	level = ReadChannel(pot_channel)
	volts = ConvertVolts(level,2)
	temp = ConvertTemp(level,2)
	# Print out results
	print("Temp: {} ({}C) {} {}".format(level,temp,strTime,timeStamp))
	#dataArr = [level,volts,temp,currentTime]
	data = { 'level':level, 'volts':volts, 'temp':temp, 'dateTime':strTime, 'timeStamp':timeStamp }
	# Print results to file
	json.dump(data, outputFile)
	# Wait before repeating loop
	time.sleep(delay)

view raw TemperatureSampler.py hosted with ❤ by GitHub

I had originally planned on creating a connector class, to read from the generated (JSON) text file; and this was a useful first step, but I have since decided to send the data directly to a database on the DAQ. I think this will give me a little more flexibility as I further develop the system.

Update

After some testing with the previous setup and fiddling with the code a bit I decided to make a few changes and finish up my circuit. So, as I mentioned before, I added a database to record my sensor data, setup a temperature sensor using the TMP35/LM35, cleaned up my wiring a bit and made some additions to my code.

Changes in this update:

1. Cleaned up my wiring
2. Added the temperature sensor circuit
3. Updated my python script to write directly to SQLite
4. Changed the conversion factors on my input voltage and temperature calculations
5. Added a crontab entry to run my script every hour

Here's an image of the completed circuit:

The main additions to the code for writing to a SQLite table are fairly simple. I made my SQL script in two parts just to make it a little easier to look at; the first string gives the table and variable/column names, followed by the values to insert.

#import the library
import sqlite3

# Create output sql strings
sql_insert = "INSERT INTO {tn} ({tc}, {dc}, {lc}, {vc}, {tmp}) ".\
        format(tn=table_name, tc=time_col, dc=date_col, lc=level_col, vc=volt_col, tmp=tempc_col)
     
sql_values = "VALUES ({tc}, '{dc}', {lc}, {vc}, {tmp})".\
        format(tc=timeStamp, dc=strTime, lc=level, vc=volts, tmp=temp)
     
sql_string = sql_insert + sql_values

# execute the script, commit changes and disconnect
c.execute(sql_string)
conn.commit()
conn.close()

Here is the entire code (hosted in Gist):

	import spidev
	import time
	import os
	import json
	import sqlite3
	import sys
	# Open SPI bus
	spi = spidev.SpiDev()
	spi.open(0,0)
	vMax = 1.25 # full scale voltage in mv
	vRef = 3.30
	# Function to read SPI data from MCP3008 chip
	# Channel must be an integer 0-7
	def ReadChannel(channel):
	adc = spi.xfer2([1,(8+channel)<<4,0])
	data = ((adc[1]&3) << 8) + adc[2]
	return data
	# Function to convert data to voltage level,
	# rounded to specified number of decimal places.
	def ConvertVolts(data,places):
	volts = (data * vRef) / float(1023)
	volts = round(volts,places)
	return volts
	# Function to calculate temperature from
	# TMP35 data, rounded to specified
	# number of decimal places.
	def ConvertTemp(data,places):
	# ADC Value (need to apply ref voltage for TMP35 if we want full
	# scale, highest res)
	# (approx) Temp Volts
	# 0 0 0.00
	# 79 25 0.25
	# 157 50 0.50
	# 236 75 0.75
	# 315 100 1.00
	# 393 125 1.25
	temp = ((data * vRef)/float(1023))*100
	temp = round(temp,places)
	return temp
	# Define sensor channels
	pot_channel = 0
	# Define delay between readings
	delay = 1
	# column names
	id_col = 'ID'
	time_col = 'Time'
	date_col = 'DateTime'
	level_col = 'Level'
	volt_col = 'Volt'
	tempc_col = 'TempC'
	# Open db connection
	sqlite_file = '/home/pi/projects/sqlite1.db'
	table_name = 'Temperatures'
	conn = sqlite3.connect(sqlite_file)
	c = conn.cursor()
	# Get a time stamp
	gmTime = time.gmtime()
	strTime = time.asctime(gmTime)
	timeStamp = time.time()
	# Read the input voltage
	level = ReadChannel(pot_channel)
	volts = ConvertVolts(level,3)
	temp = ConvertTemp(level,2)
	# Create output sql string
	sql_insert = "INSERT INTO {tn} ({tc}, {dc}, {lc}, {vc}, {tmp}) ".\
	format(tn=table_name, tc=time_col, dc=date_col, lc=level_col, vc=volt_col, tmp=tempc_col)
	sql_values = "VALUES ({tc}, '{dc}', {lc}, {vc}, {tmp})".\
	format(tc=timeStamp, dc=strTime, lc=level, vc=volts, tmp=temp)
	sql_string = sql_insert + sql_values
	try:
	c.execute(sql_string)
	conn.commit()
	conn.close()
	except: # catch *all* exceptions
	e = sys.exc_info()[0]
	print("Error:{msg}".format(msg=e))
	# Wait before repeating loop
	time.sleep(delay)

view raw TemperatureSampler.py hosted with ❤ by GitHub

I also needed to change the conversion factors from the original sample code as they had been written for the TMP36 and the TMP35 has a different response range, though it is the same linear factor.

Using a vRef of 3.3V this gives me a range from 0V to 1.25V

        volts = (data * vRef) / float(1023)

Which translates to a temperature range from 0°C to 125 °C

temp = ((data * vRef)/float(1023))*100

The TMP35 is rated most accurate between 10°C and 125°C

As my final step, I added a Cron job to run my code once every hour on the hour; this should give me enough data to work with in creating my API and give me some ideas for the way I want to  query and present the data.

I edited the Crontab on the Raspberry Pi from the command line using:

crontab -e

# my entry
0 * * * * python /home/pi/projects/TemperatureSampler.py

They have a little primer about Crontab here on the RPi website:

Cron And Crontab

Anyway, until I add more sensors (or maybe an IR camera) that's it for the data acquisition side of my project. What I'm working on now is finishing up the skeleton of the Web API and starting to fill it out with a few features... and eventually documenting it for another post.

It's still in progress but you can find the code in my GitHub repository.

Telemetry Web API & Web Page

I created a very simple web page starting with ASP.NET MVC and generated it using the MVC scaffolding. The Plot page has a very simple chart built using JavaScript with a library called Chart.js.

The purpose of the page as it stands now is to provide quick visualization as I work out some querying and filtering functionality but I plan on making this a single page application in the future.

You can view my code as it develops here:
https://github.com/jcere/Telemetry/tree/master/Telemetry.Service/Views

Right now I'm working on a few data sampling methods for my controller. So far I'm thinking of being able to query and alter the display based on: start time, number of samples or time span, and by limiting the frequency.

	/// <summary>
	/// get temperature data starting at time and continuing for span with given period
	/// </summary>
	/// <param name="time">time in seconds</param>
	/// <param name="span">length of span in seconds</param>
	/// <param name="period"></param>
	/// <returns>JsonResult containing view model</returns>
	public JsonResult Get(double time, double span, int period)
	{
	var data = tempRepo.GetSamplesFromSpan(time, span, period);
	model = new TempViewModel(data);
	return GetJsonResult(model);
	}

view raw TempController.cs hosted with ❤ by GitHub

I've been introduced to a few very smart tools while working on this, one really nice tool I used to test these methods is Postman; it's available from the Chrome Web Store. Postman is an Http request builder to 'Build, test, and document your APIs faster.' With Postman you can build a full Http request from a given URL and a number of other settings, and then send the request and display the response.

What's really handy for me is being able to see all of the components of the request, I found being able to view the headers and request body, gave me a much better sense of what is going on behind the scenes during this exchange. Postman allows you to build multiple requests in collections and save them for testing later.

Here you can see I entered my URL with some additional arguments for start time, time span, and period. So my code should return data with a period of 2 (or a frequency of 1/2 samples per hour) and from the JSON in the response body I can see I didn't get that, but that is what I was going for :)

Anyway, very nice tool and easy to use, I'm going to go fix my code now.