Concept

BlankSpace determines how full a space is and makes that information available via a web application. It works using a thermal camera and secondary sensors.

Motivation

Students often waste a lot of time searching for a place to work. BlankSpace will enable students to quickly check space availability without actually needing to visit the space.

Use cases

Determine how full a space is (is there room for me to study at the library?) without making the journey to check in person.

Examine how a space is utilized over time by observing data recorded by the BlankSpace web app.

Detect unauthorized access to a space.

Competitive analysis

Rita

Rita is another 18-549 project. They also wanted to determine in real time whether a space is too crowded or noisy. They are using Kinect, accelerometers, Grid-Eye, and ambient noise sensors and placed around the room to identify the conditions within. Their installation requirements are much more intensive than ours, i.e. sensors on each chair. Our projects differ significantly in the types of sensors used, as well as the centralized nature of our system.

Asure Software

Asure Software combats a common problem in the workplace where often times a lot of desk and workspace is not utilized efficiently. Asure Software uses sensors to find free space as well as helps the user use their space more efficiently. This project is similar to our project, but it pertains to a different problem space as Asure Software focuses more on objects rather than people. Asure accomplishes this using many different types of sensors that get placed on surfaces.

OccupEye

OccupEye is used to increase efficiency through space utilization. OccupEye is similar to Asure Software, but it also is able to detect the presence of people. This project is similar to our project, but it pertains to a different problem space as OccupEye focuses more on objects rather than people.

Technical specifications

Hardware
FLIR Lepton thermal camera

A 80×60px thermal camera.

$183

UDOO

A powerful miniature computer with built-in Wi-fi and Bluetooth LE.

$60

Software
OpenCV

Computer vision and machine learning library. Link.

Amazon Web Services EC2

Infrastructure as a service. Link.

Protocols
Bluetooth Low-Energy

A low-power communication standard for embedded devices.

Wi-fi

A more power-hungry standard, but very common for local area networks.

I2C

A common serial protocol for embedded systems.

SPI

Another common serial protocol for embedded systems.

Architecture

Setup

When a customer receives a BlankSpace system, they go to a url included with the hardware. This url contains a unique number representing the device, which is associated with the user as soon as they create or sign into their account. From there, they can configure the system with a name, maximum room occupancy, and error correction (the number of hot inanimate objects in view of the camera).

Public experience

The customer and the system (via BLE beacon) advertise the availability of BlankSpace in the room. People visit the website and see how full the room is as a percentage of its maximum occupancy.

Collecting data

The camera unit activates every ten minutes when the Bluetooth unit receives an interrupt from its timer. It sends commands over I2C, using Lepton’s SDK that we ported from Raspberry Pi to run on our Bluetooth unit, that instruct the camera to capture an image, and listens on SPI for the packets containing lines of the image. It buffers these packets and sends them via BLE to the base station.

The UDOO serving as a base station receives the image data, assembles that into a standard image format. The image is thresholded, contoured, and counted using OpenCV. Finally, the base station sends the latest count to the web server.

The Amazon EC2 server receives the update and begins serving that data.

Requirements

Accurate estimation of the free space in a room

We want our sensors to be wide angle to gauge the space available and capture all data relevant to the way people are occupying space in the room. We may be able to use heuristics to determine where people are located in a given space or building by knowing beforehand how much space there is and where it is located, what time of day it is, what events may be going on, etc. By using sensor data about the number of people spread out in one area, we could also deduce the number of people in other areas. For a good product, where we place our sensors should allow us to accurately gauge the space being studied and where people are situated in that space. We met this goal with the machine vision algorithm we developed, and found that it performed well in real spaces, despite challenges such as warm inanimate objects.

Low-power sensor units for easy installation and long life

We want our units to overall have low power consumption rates and have long battery life because they should run on battery (to avoid difficult installation and needing external power sources nearby). It may be a good design choice if the units were able to have a “sleep” mode for when buildings are closed to people. The units could also function as security cameras when buildings are closed, and this functionality could be paired with camera for image processing capabilities if desired. We did not meet this goal, because we were unable to get the camera to interface with the Bluetooth LE unit.

Space availability can be determined from a web app

For people, or ourselves, to get live updates when we check our internet-connected devices about the emptiness of rooms we may want to go to. We want the processed data from our product units to be update data on the website.

Cope with changes in weather and season, plus any other discrepancies in the space

Changes in weather could affect the amount of heat (like in the form of light) that enters a room and thus be noise that sensors like the thermal imager would pick up as well. If people wear more clothing on their bodies, the image of their heat image could also appear different than when they wear lighter layers. Our product should work regardless of physical changes in surrounding and additions to a person’s body to analyze input and output consistent, accurate results.

The thermal imager also detects heat the is reflected off of shiny surfaces such as glass, so body heat reflected off surrounding surfaces could skew the product’s analysis of how many people are in a room. We would have to take this discrepancy of rooms into account as well somehow by accounting for which surfaces are merely thermal reflectors and should not affect the people count.We didn't get to this goal since we were focused on getting our basic system to work.

Don’t creep anyone out

We would like our units to be able to discreetly sit in the corners of room ceilings without disarming anyone in the room. The thermal camera’s resolution is so low that this turned out not to be a problem. A typical person is less than eight pixels, so identifying anyone is nearly impossible.

Possible stretch goal: Android and iOS apps

For the same data that is on the web app to be available as phone apps running on Android and iOS. We did not get to our stretch goals.

Risks & mitigation strategies

In regards to schedule, we were able to build in risk into the schedule as we left the last system demo to debugging and optimization. As a result, we have flexibility as to if we end up falling behind because of specific software or hardware issues, so that we could push all the system demos a bit back and use less time at the debugging and optimization part.

From a design perspective, we were planning to augment our design with beam sensors as well as mobile apps that would support the data that we collect from the thermal camera. As a result, if our thermal camera is less accurate in detecting people within the given space, we would be able to adjust the data collected by the thermal camera in accordance to the supporting data that we plan to collect.

From a resources perspective, the major cost for our project is the thermal camera, so we can only buy up to 2 thermal cameras. Fortunately, the thermal cameras have wide angle vision, so it can capture a wide space very accurately. We would just need to place the cameras in locations that would use their wide angle vision to their advantage, which would require a bit of scouting of the area.

Presentations

Project proposal

We presented our initial idea for the class, professor, and TAs.

Google Slides

Photos

We visited the engineering library at the very start of the project.

It was our initial target space.

One of our Flir Lepton cameras in its breakout board.

Our custom printed circuit board.