Making your own smart ‘machine learning’ thermostat using Arduino, AWS, HBase, Spark, Raspberry PI and XBee

Previous part:
5. Turning the boiler on and off at the right time (Arduino)

6. Using outside temperature and scenarios to control an Arduino from a Raspberry PI

6.1 Temperature scenarios

After completing part 5, the Arduino was a successful normal thermostat. From that moment I removed my old thermostat and let the Arduino control the temperature of our house. Luckily it was a cold spring, so we still needed the house to be heated up through April.

The next step of course was to make the thermostat ‘smart’. In the Arduino thermostat code described in part 5, you see my final result for how long the boiler should be turned on and the difference between the set and actual temperature. This is actually the result of different iterations where previous settings resulted in the rooms not getting warm at all or overshooting the desired temperature so much that you would have to open all the windows to let it cool again. Because this was mainly a trial and error effort I thought that an easy way to turn this project into a ‘learning’ thermostat was to automate adjusting the amount of boiler on time.

To be able to turn the thermostat into a smart thermostat I introduced the concept of temperature scenarios. After each time the boiler turns on, the temperature in the room increases and the difference between the set and actual temperature decreases. As a result the time the boiler goes on at each interval decreases towards the set temperature. Looking at the minutes the boiler goes on after increasing the set temperature in the morning a general pattern for each 10 minute interval in first hour was 6 5 5 4 3 2 (figure 13). In the first 10-minute interval the boiler is on for 6 minutes, the second and third interval 5 minutes, the fourth 4 minutes etc.. For the second hour the patern was mostly 2 2 2 2 2 2 (figure 14). Every 10 minute interval the boiler is on for two minutes.

The downside of these patterns is that in the first hour the house does not get warm as quickly as possible, causing discomfort, and in the second hour the boiler is unnecessarily turned on every interval. Understanding that boiler efficiency increases when the amount of on-off cycles is reduced my Arduino thermostat was not very efficient.

Both comfort and energy efficiency could be improved when the thermostat could learn when to apply a better boiler on-off scenario. For example 6 6 6 2 2 2 (figure 13) to heat-up the house and 3 0 3 0 3 0 (figure 14) to keep the house warm. A regular thermostat does this by checking the ‘request for heat’. This is an abstract term and I do not completely understand how a thermostat determines the ‘request for heat’, but I am assuming that it checks how much the temperature has increased after each 10 minute interval.

increase_runs2
Figure 13: Increase room temperature – regular and alternative scenario

keep_runs2
Figure 14: Maintain room temperature – regular and alternative scenario

To determine the ‘request for heat; I took a different route and used the combination of the difference between the inside temperature and the set temperature and the difference between the outside temperature and the set temperature. This required more complex calculations so the first step was the let the Raspberry PI control the boiler state instead of the Arduino.

6.2 Controlling the boiler from the Rapsberry PI through the Arduino

To turn the boiler on and off, the Raspberry PI sends a message to the Arduino every second. This message contains the boiler state, current room temperature, current time and a checksum of all the previous fields. The message is composed of fixed length integer fields <hour><minute><boiler state><temperature in hundreds degrees of Celsius><checksum>. For example: 1204117231740. The serial port on the Raspberry PI can only be used by on thread. So for sending the message to the XBee of the Arduino I modified the XBee receiving thread at the Raspberry PI, xinsert(), to also send message from the message queue every second. The message queue is implemented as an SQLite table.

The messages are inserted into to the message queue table by a boiler control thread, fcontroll(), on the Raspberry PI. This thread is almost an exact copy of the Arduino thermostat loop and functions. The difference between the Raspberry PI and the Arduino, is that the Raspberry PI gets its current and set temperature from the sensor value database instead of the sensors directly.

For safety reasons a mechanism is built in that it will only create a message to send to the Arduino if the record sensor values are not older than 5 seconds, preventing it from turning the boiler on or off based on too old information. Because older messages are deleted after receiving no data for more than 5 seconds the Rapberry Pi will stop sending information to the Arduino.

addition to xinsert() function

def xinsert():
"""
Receives and sends data from and to the XBee.
Sending and receiving combined in one thread because only one tread can use GPIO port.
Recieving is done as fast as possible.
Sinding only at 1 second interval.
Runs as a sperate thread.
"""
ficurtime = 0
fivepoch = 0
fiprevepcoh = 0
ficursource = '40b5af01' #device id of raspberry pi only used for logging
DEST_ADDR_LONG = "\x00\x13\xA2\x00@\xB5\xAF\x00" #destination adress currently fixed value
while True:
logging.debug("insert started")
logging.debug(datetime.datetime.utcnow())
try:
#send at 1 second interval
if fivepoch > fiprevepcoh:
fiprevepcoh = fivepoch
dbd11 = dbc.cursor()
dbd11.execute("SELECT vsendmes from sendmes where vepoch > %i order by vepoch desc,vsub desc LIMIT 1" % fimintime)
rows = dbd11.fetchall()
for row in rows:
fipayload = row[0]
xbee.tx(dest_addr='\x00\x00',dest_addr_long=DEST_ADDR_LONG,data=str(fipayload),frame_id='\x01') #send trought XBee
fins(ficurtime,ficursource,'rx000B04',int('9'+str(fipayload))) #logging of send message in hsensvals table
dbd11.close()
except Exception:
logging.exception("fxinsert")
time.sleep(0.001)

main boiler control function

def fcontroll():
"""
Main boiler controll function.
Gets current and set temperature
uses 3 run levels to controll boiler.
run level 0 = initial,
run level 1 = determine how many mintues boier shoudl be on,
run level 2 = continuesly check of boier should be on for 10 minutes
Inserts message in sendmes table.
Runs as seprate thread.
"""
runScen = 0
boilerStat = 0
setBoiler = 0
maxScen = (10*60) #maximum run for 1 hour long
scenLength = 0
cursource = '40b5af01' #divice id used for logging purposses
logging.info("time sleep 10 started")
time.sleep(10)
logging.info("time sleep 10 complete")
bvepoch = 9999999999
usettemp = 30000
prevsettemp = 30000
settemptime = (60*60*24*5)
while True:
try:
fscurtime = time.time()
fsvepoch = int(fscurtime)
fsvsub = int(datetime.datetime.fromtimestamp(fscurtime).strftime('%f'))
fkepoch = bvepoch fsvepoch
fsmintime = fsvepoch5
curtemp = 30000 #lower than 0 Kelvin
settemp = 30000 #lower than 0 Kelvin
curtempepoch = (60*60*24*5)
settempepoch = (60*60*24*5)
#get current actual inside temperature form sensvals table
dbd8 = dbc.cursor()
dbd8.execute("SELECT vepoch,vvalue FROM sensvals where vsource = '40b5af01' and vport='rx000A04' order by vepoch desc, vkey asc LIMIT 1")
rows = dbd8.fetchall()
for row in rows:
curtempepoch = row[0]
curtemp = row[1]
dbd8.close()
#get current set temperature form sensvals table
dbd9 = dbc.cursor()
dbd9.execute("SELECT vepoch,vvalue FROM sensvals where vsource = '40b5af01' and vport='rx000A02' order by vepoch desc, vkey asc LIMIT 1")
rows = dbd9.fetchall()
for row in rows:
settempepoch = row[0]
settemp = row[1]
dbd9.close()
#only set usettemp if settemp has not ben changed for 5 seconds
#this is prevent the temperature scenrio to be based on a slightly higher settemp instead of waiting for the final set temp
#to be set by the user
if (prevsettemp != settemp):
settemptime = fsvepoch
elif (fsvepoch settemptime > 5 ):
usettemp = settemp
prevsettemp = settemp
#devault variables
curtemptimediff = fsvepochcurtempepoch #check how many seconds ago last current temperature has been received
settemptimediff = fsvepochsettempepoch #check how many seconds ago last current temperature has been received
tempdif = (usettemp*10) curtemp
outtempdif = (usettemp*10) outtemp
#!!include if settemp is set and the other checsk if temp is actual
if(runScen == 0): #no scenario running
runScen = frunScen(curtemp,usettemp)
elif (runScen == 1): #start scenario
#determine number of seconds boiler should be on using fscenlengh() function
scenLength = fscenLength(curtemp,usettemp)
maxrun = 1
elif(runScen == 2): #run scenario
runCurtime = int(time.time())
if (boilerStat == 1 ):
boilerStat = fboilerStat(startScen,scenLength,runCurtime,curtemp,settemp)
if (boilerStat == 1 ):
setBoiler = 1 #send Arduino 1 for boilerstat
else:
setBoiler = 0 #send Arduino 0 for boilerstat
if(runCurtime startScen > maxScen):
runScen = 0
scurhour = int(datetime.datetime.fromtimestamp(fscurtime).strftime('%H'))
scurminute = int(datetime.datetime.fromtimestamp(fscurtime).strftime('%M'))
#only send message if temperature readings are current and withint normal range
if (curtemp > 30000 and curtemptimediff < 20 and settemp > 30000 and settemptimediff < 20):
vchecksum = scurhour+scurminute+setBoiler+curtemp
sendstr = str(scurhour).zfill(2) + str(scurminute).zfill(2) + str(setBoiler) + str(curtemp).zfill(4)+str(vchecksum).zfill(4)
fins(fscurtime,cursource,'rx000B01',scenLength) #for logging purposses, insert in sensvals table
fins(fscurtime,cursource,'rx000B02',boilerStat) #for logging purposses, insert in sensvals table
fins(fscurtime,cursource,'rx000B03',setBoiler) #for logging purposses, insert in sensvals table
#insert message in sendmes table
dbd10 = dbc.cursor()
dbd10.execute("INSERT INTO sendmes(vepoch, vsub, vsendmes) VALUES(?,?,?)",(fsvepoch,fsvsub,sendstr))
dbd10.close()
#delete older messages, keep messages of last 5 seconds
dbd13 = dbc.cursor()
dbd13.execute("DELETE from sendmes where vepoch < %i" % fsmintime)
dbd13.close()
except Exception:
logging.exception("fcontroll")
time.sleep(1)

boiler functions

def frunScen (actTemp, setTemp):
"""
Determines if runlevel in boiler controll loop should go from run level 0 to run level 1.
Used by fcontroll() function.
"""
if ( (setTemp*10) actTemp > 35):
return 1
else:
return 0
def fscenLength(actTemp, setTemp):
"""
Returns the number of minutes the boiler should be on in a 10 minute intervall.
Used by fcontroll() function.
"""
scnel = 0
if ( (setTemp*10) actTemp > 260):
scnel = (6*60)
elif ( (setTemp*10) actTemp > 160):
scnel = (5*60)
elif ( (setTemp*10) actTemp > 70):
scnel = (4*60)
elif ( (setTemp*10) actTemp > 40):
scnel = (3*60)
else:
scnel = (2*60)
return scnel
def fboilerStat(starts,scenl,cur ,actTemp,setTemp):
"""
Check if boiler should stay on or go off.
Used by fcontroll() function.
"""
if (actTemp (setTemp*10) < 35): #criteria 1: only say on if act temperature is below set temperature + margin
if (cur starts < scenl): #criteria 2: only stay on of boiler has not been on for the number of seconds it should be on this interval
return 1 #stay on
else:
return 0 #go off
else:
return 2 #go off (2 is used to monitor overflow)

6.3 Arduino code for receiving instructions form the Raspberry PI

To have the Arduino receive the message from the Raspberry PI through the XBee was a lot harder than expected. I am using the version 2 also called Zigbee XBee’s. Most examples I found are on using version 1 (DigiMesh) of these devices. The Arduino code to use version 2 differs significantly from version 1. The next struggle was to split the message payload generated by the PI into the different fields and convert them to integers. As mentioned in the introduction of this blog post, I am not an experienced programmer, so writing C code was not easy. In my initial attempts I had a lot of stability issues and values from one variable unexplainably flowing over to another variable. I got it all working using the code shown below.

The Arduino splits the XBee message payload in different character arrays as soon as possible and adds 0 to the array to end the array. Every character array is then converted to an integer using the C atoi() function. Surprisingly I only have to give the first position the character array to the atoi function. For example: vhour = atoi(&chour[0]);. This converts the entire array to an integer.

The Arduino then uses a timer to decide if should stay in ‘remote’ mode or fall back to ‘local’ mode. In remote mode the Raspberry PI controls the boiler. In local mode the Arduino ignores the PI and does everything itself. Every time a message is received, it uses the checksum to make sure the content was correct. If the payload was correct it resets the time to 0. In every cycle of 10 milliseconds it adds 1 to the timer. If the timer has reached 3,000 (+/- 30 seconds) it falls back to local mode. After 3,000 the timer always stays at 3,000 to prevent overflowing the timer integer. I opted to use cycles here instead of the more common millis() to more safely handle the Arduino millis() overflow after 49 days. I am not entirely sure this is necessary, but thought it was better to play it safe, especially because you do not want to accidently start heating you house. If the Arduino is in local mode and heating up the boiler it always finishes the cycle of 10 minutes when the connection between the Arduino and Raspberry PI is restored. This is to prevent the boiler from continously getting turned on and off when the Arduino and PI do not agree on whether the boiler should be on ore off.

XBeeResponse response = XBeeResponse();
ZBRxResponse rx = ZBRxResponse();
//variables for receiving xbee
int mesAvailable = 0;
char chour[3];
char cminute[3];
char cboiler[2];
char crtemp[5];
char cvchecksum[5];
int vhour = 0;
int vminute =0;
int vminuteold =0;
int rvboiler = 0;
int rvtemp = 0;
int rvchecksum=0;
//timer to check if raspberry pi is still sending messages to arduino
int remotemode = 0;
int remotemodeold = 0;
unsigned int rcycle = 3000;
unsigned int maxrcycle = 2000;
void loop() {
//reading xbee message
xbee.readPacket();
if (xbee.getResponse().isAvailable()) {
if (xbee.getResponse().getApiId() == ZB_RX_RESPONSE) {
// got a zb rx packet
// now fill our zb rx class
xbee.getResponse().getZBRxResponse(rx);
//convert xbee message to seperate integer values <hour><minute><boiler state><act temperature><checksum>
chour[0] = rx.getData()[0]; //hour
chour[1] = rx.getData()[1];
chour[2] = 0; //0 is used to end array
vhour = atoi(&chour[0]); //only first position of array is needed to confert string array to int
cminute[0] = rx.getData()[2]; //minute
cminute[1] = rx.getData()[3];
cminute[2] = 0; //0 is used to end array
vminute = atoi(&cminute[0]); //only first position of array is needed to confert string array to int
cboiler[0] = rx.getData()[4]; //boiler state 1 or 0
cboiler[1] = 0; //0 is used to end array
rvboiler = atoi(&cboiler[0]); //only first position of array is needed to confert string array to int
crtemp[0] = rx.getData()[5]; //actual temperature
crtemp[1] = rx.getData()[6];
crtemp[2] = rx.getData()[7];
crtemp[3] = rx.getData()[8];
crtemp[4] = 0; //0 is used to end array
rvtemp = atoi(&crtemp[0]); //only first position of array is needed to confert string array to int
cvchecksum[0] = rx.getData()[9]; //checksum
cvchecksum[1] = rx.getData()[10];
cvchecksum[2] = rx.getData()[11];
cvchecksum[3] = rx.getData()[12];
cvchecksum[4] = 0; //0 is used to end array
rvchecksum = atoi(&cvchecksum[0]); //only first position of array is needed to confert string array to int
mesAvailable = 1;
}
}
//check if correct message is avaliable
if (mesAvailable == 1 && (vhour+vminute+rvboiler+rvtemp) == rvchecksum ){
rcycle = 0;
mesAvailable = 0;
} else {
rcycle = rcycle+1;
mesAvailable = 0;
}
//when larger than 3000 reset tot 3000 to prevent int overflow
if (rcycle > 3000) {
rcycle = 3000;
}
//check if thermostat should be in local or remote mode
if (rcycle < maxrcycle) {
remotemode = 1;
} else {
remotemode = 0;
}
if (remotemode == 1) { //if remote mode is true
avgTemp = rvtemp; //set main actual temperature to the actual temperature send by the raspberry pi
}
}

modified boiler control loop

//main boiler controll loop.
if(runScen == 0) { //no scenario running
if(startup > 0 && vrunl ==2) { //wait for a time delay to make sure there is correctly measured avarage temperature
//if temotemode is true base boiler on/off on message send by raspberry pi
if (remotemode == 1) {
if (rvboiler == 1 ) {
digitalWrite(controllPin, HIGH);
} else {
digitalWrite(controllPin, LOW);
}
//when in local mode check if controll should go to runlevel 1
} else {
runScen = frunScen(avgTemp,sensorValue);
}
} //rest of the control loop remains the same

6.4 Using temperature scenarios to turn the Rapsberry PI into a smart thermostat

The setup so far allows the Rapsberry PI to remotely control the boiler. That however, does not make it a smart thermostat. Several modifications were made to the Rapsberry PI Python code to make it more intelligent.

The first modification was to download the outside temperature to the Raspberry PI. A simple python script on the Amazon Cloud server runs every hour as a cron job and obtains the most recent outside temperature of Amsterdam using an online weather service and inserts the temperature in a HBase table. A current temp thread, fcurtemp(), on the Raspberry PI checks every 10 minutes for a new outside temperature and downloads it to a SQLite table if one is available. In this function you might see my current job coming through in that it first inserts the new value and then deletes the old ones instead of simply updating a value. This is to prevent any form of deadlocking, which is a continuous frustration in my work. Here I delete all values but the current one. In other functions, like the function used in the get scenario thread, I save the three most recent versions just to be safe.

The second modification is to download the different temperature scenarios from the Cloud server. Every scenario contains the difference between inside temperature and the set temperature (tempdif), the difference between the outside temperature and the set temperature (outtempdif), the number of minutes the boiler should be on for six cycles of 10 minutes and the score of the scenario. The score determines how well a particular scenario of on-off cycles performed given the tempdif and outtempdif. How this score is calculated is explained in the next and last part of this blog post.

The third and most important modification is the modifcicatin of the control thread, fcontroll(). The control thread is modified to select the best scenario given the current tempdif and outtempdif. When selecting the temperature scenario, at 50% of the time it takes the scenario with the best score for the current tempdif and outtempdif. The other 50% of the time it chooses a scenario with a lower score. This is to say, half the time it selects an ‘alternative scenario’ .

Finally the ‘used scenario’ and the ‘parameters used to select this scenario’ are uploaded to the ‘husedscenario’ HBase table the Amazon Cloud server. Uploading the used scenarios follows the same mechanism for uploading sensor information explained in part 3 of this blog post.

retrieve outside temperature

def fcurtemp ():
"""
Downloads most recent outside temperature from cloud server.
First inserts new value in hcurtemp table.
Then deletes old values from hcurtemp table.
Runs as a sperate thread.
"""
while True:
try:
dbdc1 = dbc.cursor()
dbdc1.execute("SELECT min(vkey) FROM curtemp")
rows = dbdc1.fetchall()
maxtemp = (rows)[0][0]
if (maxtemp == None):
maxtemp = 9999999999
dbdc1.close()
max1 = str(maxtemp)
with hpool.connection(timeout=3) as connectioncur:
tablefc = connectioncur.table('hcurtemp')
hscan2 = tablefc.scan(row_stop=max1,batch_size=1,limit=1)
dbdc2 = dbc.cursor()
for key, data in hscan2:
dbdc2.execute("INSERT INTO curtemp (vkey, vvalue) VALUES(%i,%i)" % (int(key),int(data['fd:curt']) ) )
logging.info("new curtemp")
logging.info(str(data['fd:curt']))
dbdc2.close()
dbdc3 = dbc.cursor()
dbdc3.execute("DELETE FROM curtemp WHERE vkey NOT IN (SELECT MIN(vkey) FROM curtemp)") #delete old values
dbdc3.close()
except Exception:
logging.exception("fcurtemp")
time.sleep(550)

download temperature scenarios

"""
Downloads most recent temperature scenarios from cloud server.
First inserts new temperature scenarios in actscenario table.
Then deletes old scenarios from actscenario table. Leaves the last 3 versions.
Used by fcontroll() function.
"""
while True:
try:
facurtime = time.time()
favepoch = int(facurtime)
dbdac1 = dbc.cursor()
dbdac1.execute("SELECT min(vkey) FROM actscenario")
rows = dbdac1.fetchall()
maxrow = (rows)[0][0]
if (maxrow == None):
maxrow = '40b5af01'+'_'+'9999999999'+'_'+'9999999'
dbdac1.close()
#logging.info(maxrow)
with hpool.connection(timeout=3) as connectionacts:
tableacts = connectionacts.table('hactscenario')
hscanacts = tableacts.scan(row_stop=maxrow)
dbdac2 = dbc.cursor()
for key, data in hscanacts:
logging.info(str(key))
dbdac2.execute("INSERT INTO actscenario (vkey, vgroup, viepoch, vtempdif, vouttempdif, run0, run1, run2, run3, run4, run5, vscore) VALUES('%s',%i,%i,%i,%i,%i,%i,%i,%i,%i,%i,%i)" % (str(key),int(data['fd:group']),int(data['fd:iepoch']),int(data['fd:tempdif']),int(data['fd:outtempdif']),int(data['fd:run0']),int(data['fd:run1']),int(data['fd:run2']),int(data['fd:run3']),int(data['fd:run4']),int(data['fd:run5']),int(data['fd:score']) ) )
dbdac2.close()
#delete all scenario except for the last 3 versions
dbdac3 = dbc.cursor()
dbdac3.execute("DELETE FROM actscenario WHERE viepoch NOT IN (SELECT distinct viepoch FROM actscenario order by viepoch desc LIMIT 3)")
dbdac3.close()
except Exception:
logging.exception("fgetactscenario")
time.sleep(650)

select temperature scenario to use

def getscen(tempdif,outtempdif):
"""
Returns the number of minutes the boiler should be on for 6 x 10 minute intervalls.
Returns used sceneario key and array of 6 values containing the number of minutes
the boiler should be on each interval. For example: [2,2,2,2,2]
Randomly selects
1) best sceario (lowest score) given a tempdif or outtempdif
2) not the best sceario (not lowest score) given a tempdif or outtempdif
Used by fcontroll() function.
"""
selselmode=[1,2] #1 is best scenario, 2 is not the best scenario
selmode = random.choice(selselmode)
usedkey = 'failed' #returns failed if it is not possible to slect scenario
runminutes=[]
try:
#determine the latest version of the temperature scenario's
#viepoch is used as version number
#the same version is used consistently the rest of the function the stay consistent
#even if the scenario's are updated in the meantime
dbdconsa5 = dbc.cursor()
dbdconsa5.execute("SELECT max(viepoch) from actscenario")
rows = dbdconsa5.fetchall()
for row in rows:
maxiepoch = int(row[0])
dbdconsa5.close()
#case for selecting best scenario
if (selmode == 1):
dbdconsa = dbc.cursor()
dbdconsa.execute("SELECT vkey, run0,run1,run2,run3,run4,run5 from actscenario a1 \
INNER JOIN (SELECT MIN(vscore) as maxvscore, vgroup,vtempdif,vouttempdif FROM actscenario \
WHERE viepoch = %i \
GROUP BY vgroup,vtempdif,vouttempdif) a2 \
ON a1.vgroup = a2.vgroup \
AND a1.vouttempdif = a2.vouttempdif \
AND a1.vtempdif = a2.vtempdif \
AND a1.vscore = a2.maxvscore \
INNER JOIN (SELECT MIN(ABS(vouttempdif – %i)) as minouttempdif FROM actscenario \
WHERE viepoch = %i) a3 \
ON ABS(a1.vouttempdif – %i) = a3.minouttempdif \
WHERE viepoch = %i \
ORDER BY ABS(a1.vtempdif – %i) LIMIT 1" % (maxiepoch,outtempdif,maxiepoch,outtempdif,maxiepoch,tempdif))
rows = dbdconsa.fetchall()
for row in rows:
#print(row)
usedkey = str(row[0])
runminutes = row[1:]
dbdconsa.close()
#case for selecting not the best scenario
else:
dbdconsa2 = dbc.cursor()
dbdconsa2.execute("SELECT a1.vgroup, a1.vscore, a1.vouttempdif from actscenario a1 \
INNER JOIN (SELECT MIN(vscore) as maxvscore, vgroup,vtempdif,vouttempdif FROM actscenario \
WHERE viepoch = %i \
GROUP BY vgroup,vtempdif,vouttempdif) a2 \
ON a1.vgroup = a2.vgroup \
AND a1.vouttempdif = a2.vouttempdif \
AND a1.vtempdif = a2.vtempdif \
AND a1.vscore <> a2.maxvscore \
INNER JOIN (SELECT MIN(ABS(vouttempdif – %i)) as minouttempdif FROM actscenario \
WHERE viepoch = %i ) a3 \
ON ABS(a1.vouttempdif – %i) = a3.minouttempdif \
WHERE viepoch = %i \
ORDER BY ABS(a1.vtempdif – %i) LIMIT 1" % (maxiepoch,outtempdif,maxiepoch,outtempdif,maxiepoch,tempdif))
rows = dbdconsa2.fetchall()
for row in rows:
fselgroup = row[0]
fselscore = row[1]
fouttempdif = row[2]
dbdconsa2.close()
dbdconsa3 = dbc.cursor()
dbdconsa3.execute("SELECT rowid FROM actscenario WHERE vgroup = %i AND vscore <= %i AND vouttempdif = %i " % (fselgroup,fselscore,fouttempdif))
rows = dbdconsa3.fetchall()
selrowid = int(random.choice(rows)[0]) #randomly selects a scenario key from the available alternative scenarios
#select the number of minues given the scenario key selected in the previous step
dbdconsa4 = dbc.cursor()
dbdconsa4.execute("SELECT vkey,run0,run1,run2,run3,run4,run5 from actscenario WHERE rowid = %i" % (selrowid,))
rows = dbdconsa4.fetchall()
for row in rows:
usedkey = str(row[0])
runminutes = row[1:]
return usedkey,runminutes
except:
logging.exception("getscen")
return usedkey,runminutes

modified boiler control function

def fcontroll():
"""
Main boiler controll function.
Gets current and set temperature
uses 3 run levels to controll boiler.
run level 0 = initial,
run level 1 = determine how many mintues boier shoudl be on,
run level 2 = continuesly check of boier should be on for 10 minutes
Inserts message in sendmes table.
Runs as seprate thread.
"""
runScen = 0
boilerStat = 0
setBoiler = 0
maxScen = (10*60) #maximum run for 1 hour long
scenLength = 0
cursource = '40b5af01' #divice id used for logging purposses
logging.info("time sleep 10 started")
time.sleep(10)
logging.info("time sleep 10 complete")
bvepoch = 9999999999
usettemp = 30000
prevsettemp = 30000
settemptime = (60*60*24*5)
while True:
try:
fscurtime = time.time()
fsvepoch = int(fscurtime)
fsvsub = int(datetime.datetime.fromtimestamp(fscurtime).strftime('%f'))
fkepoch = bvepoch fsvepoch
fsmintime = fsvepoch5
curtemp = 30000 #lower than 0 Kelvin
settemp = 30000 #lower than 0 Kelvin
curtempepoch = (60*60*24*5)
settempepoch = (60*60*24*5)
outtemp = 1500 #default outside temperature
#get current actual inside temperature form sensvals table
dbd8 = dbc.cursor()
dbd8.execute("SELECT vepoch,vvalue FROM sensvals where vsource = '40b5af01' and vport='rx000A04' order by vepoch desc, vkey asc LIMIT 1")
rows = dbd8.fetchall()
for row in rows:
curtempepoch = row[0]
curtemp = row[1]
dbd8.close()
#get current set temperature form sensvals table
dbd9 = dbc.cursor()
dbd9.execute("SELECT vepoch,vvalue FROM sensvals where vsource = '40b5af01' and vport='rx000A02' order by vepoch desc, vkey asc LIMIT 1")
rows = dbd9.fetchall()
for row in rows:
settempepoch = row[0]
settemp = row[1]
dbd9.close()
#get current outside temperature form curtemp table
dbcontr1 = dbc.cursor()
dbcontr1.execute("SELECT vvalue FROM curtemp ORDER BY vkey asc LIMIT 1")
rows = dbcontr1.fetchall()
for row in rows:
outtemp = row[0]
dbcontr1.close()
#only set usettemp if settemp has not ben changed for 5 seconds
#this is prevent the temperature scenrio to be based on a slightly higher settemp instead of waiting for the final set temp
#to be set by the user
if (prevsettemp != settemp):
settemptime = fsvepoch
elif (fsvepoch settemptime > 5 ):
usettemp = settemp
prevsettemp = settemp
#devault variables
curtemptimediff = fsvepochcurtempepoch #check how many seconds ago last current temperature has been received
settemptimediff = fsvepochsettempepoch #check how many seconds ago last current temperature has been received
tempdif = (usettemp*10) curtemp
outtempdif = (usettemp*10) outtemp
#!!include if settemp is set and the other checsk if temp is actual
if(runScen == 0): #no scenario running
runScen = frunScen(curtemp,usettemp)
elif (runScen == 1): #start scenario
usedkey = 'failed'
usedkey,runminutes = getscen(tempdif,outtempdif)
if (usedkey <> 'failed'): #only use scenarios if a valid scenario has been retrieved
scenLength = runminutes[0]
maxrun = 6
#insert used scenario and values used to determine this scenario to be uploaded to cloud server for scoring.
usscenpreky = cursource+'_'+str(fkepoch)
uscenkey = usscenpreky+'_'+ usedkey
dbdconsa5 = dbc.cursor()
dbdconsa5.execute("INSERT INTO usedscenario (vkey, vprekey, viepoch, scenariokey, vtempdif, vouttempdif, vtemp, vouttemp, vsettemp) values ('%s','%s',%i,'%s',%i,%i,%i,%i,%i)" % (uscenkey,usscenpreky,fsvepoch,usedkey,tempdif,outtempdif,curtemp,outtemp,settemp))
dbdconsa5.close()
else: #otherwise determine number of seconds boiler should be on using fscenlengh() function
scenLength = fscenLength(curtemp,usettemp)
maxrun = 1
logging.info(usedkey)
runnum = 0
startScen = int(time.time())
runScen = 2
boilerStat = 1
logging.info(str(maxrun))
logging.info(str(runnum))
logging.info(str(scenLength))
elif(runScen == 2): #run scenario
runCurtime = int(time.time())
if (boilerStat == 1 ):
boilerStat = fboilerStat(startScen,scenLength,runCurtime,curtemp,settemp)
if (boilerStat == 1 ):
setBoiler = 1 #send Arduino 1 for boilerstat
else:
setBoiler = 0 #send Arduino 0 for boilerstat
if(runCurtime startScen > maxScen):
runScen = 3
elif(runScen == 3): #check if scenario schould go to next interval of 10 minus or complete after 60 miutes
runnum = runnum + 1
if (runnum < maxrun): #go to next iteration of 10 minutes
runScen = 2
boilerStat = 1
startScen = int(time.time())
scenLength = runminutes[runnum] #get number of minutes to be on from scenario array
logging.info(str(maxrun))
logging.info(str(runnum))
logging.info(str(scenLength))
else: #completed 60 minutes
runScen = 0
scurhour = int(datetime.datetime.fromtimestamp(fscurtime).strftime('%H'))
scurminute = int(datetime.datetime.fromtimestamp(fscurtime).strftime('%M'))
#only send message if temperature readings are current and withint normal range
if (curtemp > 30000 and curtemptimediff < 20 and settemp > 30000 and settemptimediff < 20):
vchecksum = scurhour+scurminute+setBoiler+curtemp
sendstr = str(scurhour).zfill(2) + str(scurminute).zfill(2) + str(setBoiler) + str(curtemp).zfill(4)+str(vchecksum).zfill(4)
fins(fscurtime,cursource,'rx000B01',scenLength) #for logging purposses, insert in sensvals table
fins(fscurtime,cursource,'rx000B02',boilerStat) #for logging purposses, insert in sensvals table
fins(fscurtime,cursource,'rx000B03',setBoiler) #for logging purposses, insert in sensvals table
#insert message in sendmes table
dbd10 = dbc.cursor()
dbd10.execute("INSERT INTO sendmes(vepoch, vsub, vsendmes) VALUES(?,?,?)",(fsvepoch,fsvsub,sendstr))
dbd10.close()
#delete older messages, keep messages of last 5 seconds
dbd13 = dbc.cursor()
dbd13.execute("DELETE from sendmes where vepoch < %i" % fsmintime)
dbd13.close()
except Exception:
logging.exception("fcontroll")
time.sleep(1)

Next part:
7. Learning and adapting temperature scenarios in the Amazon cloud (SPARK)

2 thoughts on “Making your own smart ‘machine learning’ thermostat using Arduino, AWS, HBase, Spark, Raspberry PI and XBee

  1. Pingback: Enabling technologies: how to build your own NEST | SmartDomus

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s