by Molly Waggett and Cole Sprague

Have you ever been casually walking down the street and noticed that something in the window next to you was reacting to your every move? Ever wonder how those things work? We did too. In late September 2015, with Halloween quickly approaching, we decided to create a reactive display of our own: a seasonally-appropriate interactive pumpkin carving game that would be displayed in a storefront window in downtown Portland.

Who are we? Thorn, a tech startup made up of two recent Reed College grads with a creative approach to computer programming. Armed with a great idea and a spooky spirit, we started in on the project.

The goal was for passers-by to stand in front of a virtual pumpkin and “carve” it using their gestures. Our first task was figuring out what hardware was necessary to track people’s movement. The Xbox 360 Kinect, which collects position and depth information, ended up being both a simple and affordable solution. A quick trip to the local used video game store and $15 later, we had all the tracking capabilities we needed.

When selecting a Kinect for this kind of project, make sure you select one with an AC adaptor that will allow it to connect to your computer. The default connector looks extremely similar to USB, but it is not. Using an Xbox One Kinect with a Windows adaptor will also work and give you 1080p video instead of the 640p resolution on the Xbox 360 version, but the downside is that it costs significantly more. We did not deem the extra resolution worth it for this project.

After figuring out the hardware needed to track depth, the next step was converting the information into something usable. We settled on Processing, a foreign language to both of us. Processing, a Java-based language, contains built-in functions that allow you to quickly generate digital art. Since one of us had prior experience with Java, Processing’s syntax turned out to be completely familiar and easy to work with. Using the OpenKinect library, which provides the interface between the Kinect and Processing, we were able to convert the raw depth and position data into drawable lines (note: if you use an Xbox One Kinect you will need to use the Kinect4W Processing library instead).

With our hardware and software needs sorted, it was just a matter of making them work together. The hardest part was making sure people could only “carve” within the confines of the pumpkin, which was displayed as a background image. To solve this problem, we estimated the coordinates of the bounds of the pumpkin and wrote a function to determine whether a point falls within the bounds or not:

boolean withinPumpkin(PVector point) {
  //Estimation of pumpkin dimensions:
  //ellipse(285, 280, 390, 375);
  return (pow((point.x-285),2)/pow(195,2)) + (pow((point.y-280),2)/pow(187.5,2))  580) && (v1.y < 60)) {
  String filename = "pumpkin-" + count + ".jpg";
  save(filename);
  count++;
  drawMode = false;
  background(startScreen);
 }

We used a conditional to check whether the person was gesturing within the bounds of the button. Our count variable kept track of how many pumpkins had been drawn so far so that each one could be saved with a unique file name. After an image was saved, we incremented the variable, and reset the background to our instructions page.

Just as important as the technical knowledge we acquired along the way, a timeline is key when turning an idea into a reality. We planned to launch our project in time for Halloween. Meeting regularly and setting small goals for each meeting allowed us to stay focused and task-driven. We were fortunate to have a downtown Portland window available to us, which helped us stay accountable and motivated to deliver a finished product. When we finally got our pumpkin display onto a big screen and ready for the public to interact with, it was extremely gratifying. Public interactive art is a personal passion of ours because it brings art out of the museum, out from behind locked doors, and into a realm where everyone can enjoy it, create it, and be inspired by it. It was exciting to contribute something to Portland’s growing tech-art community.

Over the course of the Halloween weekend, over fifty people engaged with our display. The pumpkins turned out great, and the screen captures of each one are kept in Thorn’s office as proof of our pilot project’s success.

Building your own digital art does not require a ton of experience. All you need is a computer and a dream; having an idea and a search engine can take you further than you might think. We had never used Kinect data or Processing before starting our project, but the coding community is more than willing to share their knowledge with anyone looking to learn. With enough desire and dedication, we believe you can make anything imaginable.

Following the success of our pumpkin carving project, we were eager to tackle something new. With a little more experience under our belts, we settled on a slightly more difficult challenge. This time we decided to make a social media app and we are only days away from our first launch. Be on the look out for more fun projects coming soon!

If you are interested in checking out our code, you can find it at https://github.com/ColeSprague/Pumpkin-Drawing.

Molly Waggett is a junior software developer with a background in formal linguistics. She loves playing games, especially Settlers of Catan and Why-Am-I-Getting-A-NullPointerException.

Cole Sprague is a bike racer, photographer, and maker. They enjoy long rides on the beach and late nights staring at a computer screen wondering what the last line of code they wrote does.