SiteSensor - Plugin for Vera Home Automation Controllers

SiteSensor Device IconSiteSensor is a plugin for Vera home automation controllers that periodically makes requests of a remote server. Based on the response, it sets its "tripped" state (acting as a SecuritySensor device), and can also store parsed values from JSON data. This makes it possible to use a remote RESTful API without writing a plugin.

This has many uses. A trivial use may be to periodically make requests of a remote web site simply to determine if your home internet connection is operating properly. An only slightly-less trivial use would be to probe a remote site to determine that it is operating correctly (a poor man's up/down monitor). More complex, but perhaps still fun, is that you can have SiteSensor query the Twitter API and trigger a scene in your home to flash a light when someone mentions you in a tweet that contains the hashtag #happyhour. See the "Recipes" section below for some practical examples.

If you are unfamiliar with Vera Control, Ltd's home automation systems, you can find more information here.

SiteSensor supports Vera UI7 (only), ALTUI, and openLuup.

Donate/supportMake a donation to support this project.

SiteSensor on the Vera UI7 DashboardInstallation

Installing from the Vera App Store (native UI7 or higher)

SiteSensor has been released to the Vera App Store and is awaiting approval from Vera Control, Ltd. Once approved, it will be visible in the Vera App Store and appear in searches there. To install it, go to Apps > Install Apps, and enter SiteSensor in the search field, then click the "Install" button on the matched entry.

Installing using the Alternate App Store

The Alternate App Store is the best way to get SiteSensor. Use the search or alphabetic index to find SiteSensor, and click the "ALT" button to install it.

Installing from Github

You can install the current stable release or development version of SiteSensor by visiting the SiteSensor GitHub repository. For stable, clone the master branch, or download a release ZIP file from the "releases" page (you'll need to unZIP it). For development version, clone the "develop" branch. Once you have the plugin files, upload each file (except LICENSE and README.md) to your Vera using the Apps > Develop Apps > Luup Files interface. Hint: turn the "Reload Luup" checkbox off for all of the files except the last one. Then, do a hard-refresh on your browser. Finally, create a device (Apps > Develop Apps > Create Device) with the following attributes (copy/paste for this activity is highly recommended):

  • Device Type: urn:schemas-toggledbits-com:device:SiteSensor:1
  • Description: SiteSensor
  • Upnp Device Filename: D_SiteSensor1.xml
  • Upnp Implementation Filename: I_SiteSensor1.xml

Release Notes

Release notes can be found in the GitHub repository file CHANGELOG.md

Basic Configuration

Configuration of SiteSensor is through the Device Settings page.

RequestURL

This is the URL to be queried. It must be HTTP or HTTPS. Only the GET method is currently supported, but other methods (POST, PUT, etc.) are planned.

Request Headers

If the site you are probing requires some kind of additional header(s) (for example, for authentication), you can supply those headers in this field. They must be formatted in the usual HTTP syntax: Header: Value. If you have multiple, list them one per line. Note that any automatic wrapping of lines longer than the field width does not cause a newline to be inserted into the header.

Request Interval

The request interval is the number of seconds between requests to the configured URL. This must be an integer >= 60. The default is 1800 (30 minutes). SiteSensor attempts to "keep a schedule", querying as close to the desired interval as possible. Any interruption of service to the Vera or SiteSensor (e.g. reboot, power failure, etc.) does not affect the timing of queries, unless it is longer than the request interval. For example, let's say we're set up for the default 1800 second (30 minute) interval, and SiteSensor makes a query at 8:02am, but the power then fails at 8:15am. As long as power is restored and the Vera rebooted by 8:32am, the query will happen then. If the event delays the request past 8:32am, then as soon as SiteSensor restarts it will make a query, and further queries will continue on 30 minute intervals from then.

Query Only When Armed

This checkbox, which is on by default, causes SiteSensor to only query the configured URL when it is in "armed" state. When disarmed, SiteSensor will not make queries.

Request Timeout

This is the amount of time (in seconds) with which the server must respond before it is considered unreachable. Any integer > 0 can be entered, but be conscious of the fact that performance of the Internet, your network, and the server being queried, can cause spurious "failure" notices for very small values. Giving your remote server at least 10 seconds to respond is recommended.

Response Type

Currently, SiteSensor can handle responses in two ways: as text, or as JSON data. The default is text. For information on handling JSON responses, see the Advanced Configuration section.

Trigger Type

SiteSensor operates as a sensor class device, so it has a "triggered" property, just as a motion detector or door contact would. Setting the trigger type determines how SiteSensor will control its "triggered" status.

The default trigger type is "URL unreachable or server responds with error." In this case, any problem receiving a valid response from the server causes the SiteSensor device to enter triggered state. A successful query restores it from triggered state. This trigger type is available with any response type.

There are two pattern match states, one for positive match, and one for negative match. These cause SiteSensor to look for a string in the returned result text, and are therefore only available when the response type is text. For a positive match, the SiteSensor device will enter triggered state if the pattern string is found in the response text. For a negative match, the device is only triggered if the pattern is not found in the response text.

Additional trigger types applies to JSON responses, and are documented under Advanced Configuration below.

Advanced Configuration

JSON Responses

SiteSensor can retrieve and parse JSON from the remote server. You can then use expressions to fetch values from the decoded data and store those in SiteSensor state variables (which are visible to Lua, PLEG, etc). Evaluation of an expression using the JSON data can also be used to set the SiteSensor device's tripped state.

To configure SiteSensor for JSON response, set the "Response Type" field to JSON data.

Trip Conditions

By default, SiteSensor's triggering mechanism follows the success of the query. That is, if the query fails, SiteSensor enters triggered state. For JSON queries, the server must respond with a complete, non-error response, and the response must be fully parsed as JSON without error, for the query to be deemed successful. Either failure of the server to provide a response, or failure of the response to parse correctly, will cause the device to be triggered.

When the response type is JSON data, the pattern-matching options for trigger type are disabled, and an additional option is presented: "When the result of an expression is true". This allows the entry of an expression which is evaluated against the response data, and if the expression is (logically) true, the device enters triggered state. The following conditions apply:

  • Boolean true and false mean what they normally mean;
  • For numeric results, integer zero (0) is considered false and non-zero is true;
  • Strings "0" and "1" equate to false and true, respectively;
  • For other strings, the empty string is false and any non-empty (length > 0) string is true;
  • For anything else, Lua nil is false and anything else is true.

The JSON response data is encapsulated beneath a response key. That is, if the JSON data was simply { "key": "medeco" }, the one and only value in this data would be accessed by using the expression response.key.

For a full description of the expression syntax, please see the LuaXP expression documentation. SiteSensor uses LuaXP, a general-purpose expression parser (which I also wrote).

Value Expressions

SiteSensor will evaluate up to eight expressions and store the results. The results are stored in state variables named Value1, Value2, ..., Value8. These variables send events when their value changes, so you can use them as triggers for scenes and Lua. The same expression syntax is used for these value expressions as for the trip expression, above.

Query Status

For JSON responses, in addition to the response container for the data returned by the server, SiteSensor provides a status container with the following fields:

  • valid -- 0 or 1 (false or true, invalid or valid, respectively) to indicate if the response container contains valid data;
  • timestamp -- the Unix timestamp of the server response;
  • httpStatus -- the HTTP status returned by the server (200=OK, etc.);
  • jsonStatus -- the result of the JSON decoding of any data returned by the server ("OK" or an error message).

A simple trip expression, for example, might be status.valid != 1, which would cause the device to enter tripped state any time the server response isn't valid (yes, this is effectively the same as choosing the URL/response error trigger--it's just an example).

JSON Example

Let's say we wanted to turn on a light whenever the exchange rate of Euros dipped below USD 1.25. The free public API at fixer.io provides exchange rate data with a JSON response, so we can use that.

  1. In the Request URL field, enter http://api.fixer.io/latest?base=EUR
  2. Leave the Request Headers blank--we don't need any;
  3. Set the Response Type to JSON data;
  4. Set the Trigger Expression to response.rates.USD < 1.25
  5. So that we have access to the actual value if we want to use it in Lua later, set Expression 1 to response.rates.USD

That's it! SiteSensor will query fixer.io on whatever interval you set, and if the price of Euros goes under USD 1.25, SiteSensor will go to triggered state. When the price is USD 1.25 or more, SiteSensor will go back to untriggered state. The only step remaining would be to create two scenes, one to turn the light on, and one to turn it off, both triggered by SiteSensor.

Here's a list of some other public APIs you can play around with. And another!

Turning on "Log Requests" in your SiteSensor's control panel will log the response data to the scrolling window, as well as show you the success (or failure) of your expressions. This is not just a handy way to debug your expressions, but also a good way to find out what data is available in the response, if it's not documented for you otherwise. Copy and paste the response text into jsonlint.com to formatted the response for human-readability.

Querying More than One Site

You can probe as many different sites as you want. Each SiteSensor plugin device will only query one site, but you can create additional instances of SiteSensor by going to "Apps > My apps" in the Vera dashboard, clicking the "Details" button for SiteSensor, and then clicking the "Create another" button.

SiteSensor In Operation

Dashboard Icons

On the Vera UI7 Dashboard, SiteSensor will display one of four icons.

SiteSensor Dashboard Icons
Disarmed Armed Tripped Failure
Disarmed Armed Tripped Failure

The disarmed icon is shown when is disarmed and not tripped or failed. The armed icon is shown when SiteSensor is armed and not tripped or failed. The tripped icon is shown when SiteSensor is tripped and not failed (armed/disarmed doesn't matter). The failure icon is shown when SiteSensor has failed to complete a query or gotten an error evaluating an expression (SiteSensor continues to indicated tripped/not tripped state if possible).

Control Panel

SiteSensor Control PanelOn the UI7 control panel, SiteSensor has an annunciator panel that shows its current status. If a query or expression evaluation fails, the "Query Failed" "lamp" will be lit on the annunciator; the lamp is dark (gray text) if the last query was successful. The green "ARMED" indicator lights to shown when SiteSensor is armed. The yellow "TRIPPED" indicator lights to show that SiteSensor is currently in tripped state.

Below the annunciator panel are two on/off sliders. The slider at left arms and disarms SiteSensor. The slider at right turns on logging of requests. Request logging will enable a small scrolling window below the annunciator that contains the most recent log messages, which can be very useful for debugging your expressions or seeing if the remote service is responding as expected.

Recipes

If you've got a recipe for something useful, submit it via the Vera Forums!

Super Simple: Is My Internet Connection or Personal Web Site Up?

You can use this simple recipe to see if your own web site is operating properly, or to check if your home Internet connection is running by checking a public web site that's rarely down (like Google).

  • Request URL: https://www.example.com/
  • Request Headers: none
  • Response Type: Generic (text)
  • Response Trigger Type: URL unreachable or server replies with error or Response does not match pattern
  • Response Pattern: see instructions below

Put whatever URL you really want to use in the Request URL field. For the simplest form, just use the "URL unreachable" trigger type. If the web site can't be reached, or it connects but returns a response other than an HTTP 200, SiteSensor will go to triggered state, and reset when the site recovers. If you probe http://www.google.com/, this will do a pretty good job of telling you if your Internet connection is working properly or not. If you want to check the state of personal web site, just use that URL.

Complex web sites sometimes will respond with 200s, but if they take a database error or other runtime exception while generating the page, they are really broken but the 200 response doesn't expose that. To detect this, you can use the "Response does/does not match pattern" trigger type, and set the response pattern to something that the site always generates under normal conditions (or generates under error conditions). For example, long ago, I worked on e-commerce sites, and I would add a magic string in an HTML comment of the footer of dynamically-generated pages, so I had a known string to look at that was nearly the last thing written in a response.

Getting and Storing Your External IP Address

ipify.org will provide your external IP address in a variety of formats. The JSON response is very useful here. Since you IP address probably doesn't change often, a request interval of several hours is probably sufficient.

  • Request URL: https://api.ipify.org/?format=json
  • Request Headers: none
  • Response Type: JSON data
  • Response Trigger Type: URL unreachable or server replies with error
  • Trip Expression: none
  • Expression 1: response.ip

To retrieve the value, get the value of the "Value1" state variable from the SiteSensor device: luup.variable_get("urn:toggledbits-com:serviceId:SiteSensor1", "Value1", devicenum)

If your IP address rarely changes, but you need to know when it does, change the trigger type to "result of an expression," and compare the response IP address to your expected IP address in the trip expression (e.g. response.ip ~= "169.254.0.11"). If the address changes, SiteSensor will go into tripped state, and you can use that fact to trigger a notification or scene.

Air Quality Alarm

AirVisual provides a "community" edition (free as of this writing) of their API allowing up to 10,000 queries per day (that's more than one every five minutes--you don't need that many updates, probably). Register for an account, then generate an API key for their community edition. Provide that API key in the Request URL where indicated below. You'll need to provide your longitude and latitude as well, which you can get from Google Maps or your Vera's location settings page.

  • Request URL: http://api.airvisual.com/v2/nearest_city?lat=Value&lon=Value&rad=250&key=YourAPIKey
  • Request Headers: none
  • Response Type: JSON data
  • Response Trigger Type: The result of an expression is true
  • Trip Expression: response.data.current.pollution.aqius > 100
  • Expression 1: response.data.current.pollution.aqius
  • Expression 2: response.data.city
  • Expression 3: response.data.state

This configuration will trip SiteSensor if the current AQI exceeds 100 (US definition of "unhealthy for sensitive groups"). You can set up a notification and/or trigger scenes based on SiteSensor tripping. The expressions store the current AQI as well as nearest city that was used for the report. Retrieve values using luup.variable_get like this:luup.variable_get("urn:toggledbits-com:serviceId:SiteSensor1", "Value1", devicenum)

Civil Twilight

Sunrise-Sunset.org provides a nice API that will give you the civil twilight times for any day and location. You can use this to make triggers for scenes, like sunrise and sunset, but using the slightly different twilight computation. Many people prefer twilight to sunrise-sunset, because they feel that sunset is too early (there's too much light at sunset) and sunrise is too late (morning light comes earlier than sunset). This recipe will create a trigger that works on civil twilight. You'll need to know your latitude, longitude, and timezone so that the API can compute the correct values.

  • Request URL: https://api.sunrise-sunset.org/json?lat=[latitude]&lng=[longitude]&date=[isodate]T12:00:00[tzoffset]&formatted=0
  • Request Headers: none
  • Request Interval: 28800
  • Response Type: JSON data
  • Response Trigger Type: The result of an expression is true
  • Trip Expression:  ! ((time() >= time(response.results.civil_twilight_begin)) & (time() < time(response.results.civil_twilight_end)))
  • Expression 1: response.results.civil_twilight_begin
  • Expression 2: response.results.civil_twilight_end
  • Expression 3: strftime("%x %X", time(expr[1]))
  • Expression 4: strftime("%x %X", time(expr[2]))
  • Re-Evaluate Expressions: Every minute

Note that this recipe uses substitutions in the Request URL. These are the bracketed words, and SiteSensor will automatically supply the correct values. For example, SiteSensor will automatically substitute [isodate] for today's date in YYYY-MM-DD format. The latitude and longitude substitutions are taken from Vera's location configuration, so make sure you have these set correctly, or it all goes wrong.

Also note that the Trip Expression begins with a bang ("!"). This is important, so don't leave it out. As always, applying the recipe by copy-pasta is recommended.

In this configuration, SiteSensor will be tripped from night twilight (dusk) to the following morning's dawn. The first two result expressions show you what the API is returning, which are times in the UTC timezone. The 3rd and 4th expressions are those times converted to your local timezone. You can use these last two values to confirm that the API is returning sensible numbers for your location and timezone. They will slightly different from sunrise and sunset. If they are wildly different (hours), then you need to check your lat/long and time offset in the request URL.

This recipe demonstrates the use of "Re-Evaluate Trip Expression." Normally, SiteSensor only evaluates the Trip Expression and value expressions when doing an API request. In order to get the trigger to change on the minute, however, one would need to set the Request Interval to 60 seconds, and this would spam the API server with unnecessary requests--the twilight times only change once a day, when tomorrow becomes the new day. To avoid this abuse, We seet the Request Interval very high--28,800 seconds is eight hours. Then, we set Re-Evaluate Trip Expression to 60 seconds, which makes SiteSensor re-evaluate the trip expression (which includes comparison with the current time of day) without making a new remote API request--it just re-runs the expression against the previously retrieved data. So SiteSensor will update its tripped state with one minute resolution to the computed twilight times, without having to make overly-frequent requests of the remote server.

This recipe requires SiteSensor 1.4 or higher.

Using OpenWeatherMap to Get Current Conditions

Many weather services have converted to subscription models for their APIs, so the old Vera plugins for WeatherUnderground and others now no longer work without sign-up and monthly payment to their data sources. But OpenWeatherMap.org is still free, and you can use SiteSensor to get the current weather conditions in your area. You will need to sign up to get an "APPID" (which is basically an API key), but it's free.

  • Request URL: http://api.openweathermap.org/data/2.5/weather?zip=your-ZIP-code&APPID=your-app-id
  • Request Headers: none
  • Request Interval: 7200
  • Response Type: JSON data
  • Response Trigger: The result of an expression is true
  • Trip Expression: response.cod == 200
  • Expression 1: last(response.weather).main
  • Expression 2: response.name
  • Expression 3: round(response.main.temp * 9 / 5 - 459.97,1)
  • Expression 4: response.main.humidity
  • Expression 5: round(response.main.pressure * 0.029529988,2)
  • Re-Evaluate Expressions: Only immediately after requests (default)

Note that in expression 1, I've used the last() function, because response.weather is returned as an array. It only ever has one element that I've seen, so this construction quickly selects that element. There are unit conversions in expressions 3 and 5 from OpenWeatherMap.org's published units to common US units. Expression 3 converts temperature in degrees Kelvin to Fahrenheit, and expression 5 converts pressure in millibars to inches of mercury. One could also convert the response's temperature to Celsius using response.temp - 273.15.

In this example, I've set the query interval to every two hours (7200 seconds); you can use a shorter interval, but be aware of the query quota and your API agreement! 

Scenes and Lua Scripting

This section is under construction!

Scenes

SiteSensor can be used like any other sensor in scenes. As a scene trigger (or with PLEG), you can select tripped, not tripped, or armed-tripped states. You can also trigger on arming or disarming of the sensor. As a controlled device in scenes, you can arm and disarm SiteSensor.

State Variables

When using pattern matching, the last matched pattern is stored in LastMatchValue (service urn:toggledbits-com:serviceId:SiteSensor).

When using JSON responses, the values extracted from the JSON response can be found in Exprn, where n is 1 through 8.

The variables normally associated with service urn:micasaverde-com:serviceId:SecuritySensor1 are available, including Armed, Tripped, ArmedTripped, and LastTrip.

ImperiHome Integration

SiteSensor (as of v1.4) implements the ImperiHome ISS API, which allows ImperiHome to see and control SiteSensor devices. To add your SiteSensors to your ImperiHome configuration:

  1. Open the ImperiHome app;
  2. From the menu, choose My Objects;
  3. Click Add a new object;
  4. Choose ImperiHome Standard System;
  5. In the Local API base URL enter: http://your-vera-ip/port_3480/data_request?id=lr_SiteSensor&action=ISS
  6. Click the Next button.

ImperiHome should then connect to SiteSensor on the Vera, take inventory of all SiteSensor devices, and display them in its interface. SiteSensor devices are shown as door sensors in ImperiHome. It is possible, therefore, to arm and disarm, and see the current trip state. For SiteSensors using JSON response, SiteSensor creates additional generic sensor devices for each expression with the value of the last successful evaluation of the expression.

Getting Help

If you technical support, please post a message on the Vera Forums.