EyespyTV

tl;dr

Client

This project was supported by the Creative Europe project  –          The People’s Smart Sculpture. 

Role

Independent design research exploration supported by a Creative Europe project. 

Designing Cultural Probes • Product Design • High-Fidelity Prototyping (Electronics, 3D Printing, & Code) • Visual Design & Packaging

Timeframe

February 2017 – April 2017

Brief

To explore the different ways in which smart cameras could be present in people’s lives – capturing, interpreting, learning from, and acting on it.

What would be the effects of their presence on people’s privacy, identity, routines, and interactions? 

Outcome

Brief

What are the different ways in which smart cameras could be present in people’s homes and participate in their everyday lives?

How would their presence impact people’s habits, experiences, values, and interactions?

Traditionally, smart cameras were mainly used for ensuring the safety and security of our homes. But now, they are used in a wide variety of different ways – from enabling communication between each other, to automatically capturing special moments and even making fashion recommendations. They capture, interpret, learn from, and act on our lives in new and increasingly complex ways.

This project was motivated by this change in the nature and form of smart cameras; their affect on people’s everyday lives; and finally, ethical issues such as, the impact of their ubiquitous presence on moral values such as privacy, trust, and social identity?.

On its surface, a platform like Facebook seems analogous to the musty glue-bound photo albums of postwar America. […] But the analogy is deeply misleading, because something completely different happens when you share a picture on Facebook than when you bore your neighbors with projected slide shows.

When you put an image on Facebook or other social media, you’re feeding an array of immensely powerful artificial intelligence systems information about how to identify people and how to recognize places and objects, habits and preferences, race, class, and gender identifications, economic statuses, and much more.

– Trevor Paglen (Invisible Images [Your Pictures Are Looking at You])

EyespyTV

EyespyTV

EyespyTV is a portable TV that shows CCTV camera feeds featuring people and places from across the world. The feeds are shown with subtitles that are generated by a machine learning algorithm and describe the activities happening in the video feed. An embedded smart camera tracks the user’s reactions and actions and changes the feed to make it more relevant.

Users can control their exposure to the camera by using a ‘privacy knob’. Turning it higher captures the user at a faster rate and shows them a wide variety of feeds while turning it lower/off repeats the same feed over and over again. The brightness of an antenna like ‘privacy light’, located on top of the camera, highlights the selected privacy level. The privacy knob was a physical representation of the choices smart devices often ask us to make – between privacy and convenience, and between continuous tracking/monitoring and personalised interactions. 

How it works

I used an Intel Compute Stick to run a bespoke software written using openFrameworks, that captured, aggregated, and displayed CCTV camera feeds. I created a list of surveillance camera feed URLs by searching for open camera feeds on Shodan, a search engine for smart technology, and customised queries on internet search platforms like Google and Bing. A textual interpretation of the activity in the aggregated feed was generated using the NeuralTalk2 machine learning algorithm and superimposed on top of the feed. The interpreted camera feed was displayed on a 5 inch HDMI display that was connected to the compute stick. An Arduino Mini, connected to the compute-stick, controlled the privacy knob and light, and the power switch. I housed all the hardware in a custom 3D-printed enclosure that had no cuts or joints besides the base that acted as a cover for the entire unit.

The internals of EyespyTV

Probing possibilities with EyespyTV

Design probes

I included EyespyTV in the design probe to give workshop participants a concrete point to start imagining new and different ways in which smart cameras could participate, interpret, act on, and share everyday lives in the near future.

I believed that having a real and working device that re-positions smart security cameras from being used for safety and security to being used for relaxation or entertainment could inspire participants to think about how smart cameras are and could be a part of everyday lives. EyespyTV’s portable and playful form was designed to suspend disbelief and make it easier to imagine it in and across various domestic use contexts, besides the living room.

EyespyTV presented in a workshop setting

Workshop participants engaging with EyespyTV 

Approach

Triggering the imagination

Conceptualizing activities

The first step in the process was exploring and conceptualising activities and tasks that could simultaneously be engaging and provoke new ideas and reflections amongst participants. I brainstormed many activities and thematically clustered them based on the conceptual ideas and questions that they seemed to be connected with. Four main themes emerged: 

• Short narratives of use (When and where are cameras being used)
• Possible consumers (Who are the most likely users)
• Ecosystems (What are the larger ecosystems that support the devices)
• Communication (How do smart cameras help us connect with one another. How are they positioned/advertised in the market)

It was also important for the probe kit to be designed in an interesting and and compelling way to resonate with the participants and engage them with the activities. I thought that a simple questionnaire format could feel monotonous to the participants and they might lose interest in the workshop. Such a format also wouldn’t provoke open discussion. I also wanted the kit to feel accessible to everyone, irrespective of their technical competence. I created initial drafts of the kit and tested it with my colleagues at the university and used their feedback to refine the activities and the content further.

01/

Brainstorming the design probes

Prototyping

Besides the tasks and activities, I felt that having a functional device to think with and reflect upon, could make the workshop more engaging and promote unstructured discussion. I decided to create a smart device, called EyespyTV, that was based on the Eyespy concept I had designed earlier. I went through many rounds of conceptual exploration and lo-fidelity prototyping to narrow down EyespyTV’s core idea, form, and interface. Similar to the probe kit, I wanted to make the device seem fun and approachable, while also appearing to be mundane and believable than something out of a sci-fi movie.

An important learning from prototyping a portable smart device was to work from the inside out. Because of the code and the machine learning algorithm involved, I had to embed a portable computer (the Intel NUC [Next Unit of Computing]) inside the device along with an Arduino to control the hardware components such as the privacy knob and switches. So, I needed to place the components securely and think about possible heating issues as well. This is why the internals of the device defined the dimensions of the outer shell, atleast at a high-fidelity prototype stage. 

02/

Fabricating the outer shell for EyespyTV and assembling it with the portable computer, Arduino, and display

probes

I packaged the probe activities in separate envelopes for each participant. Each envelope consisted of two activities in a postcard format exploring forms of use, a dot grid template for making an advertisement, and a brainstorming sheet discussing what it was that made EyespyTV smart. 

03/

The final packaging for the probes

Workshop: The cultures of machine participation

The probes were used in a workshop with twelve designers and media artists. The theme of the workshop was ‘the cultures of machine participation’ and I was one of the co-organisers of this workshop along with Prof. Alma Culén. This workshop was conducted as a part of the YoungExpressions project on 3rd April 2017. Participants were divided into four groups to discuss the activities but all of them filled out the probes individually. 

participant responses

Comparing the voyeuristic yet routine nature of interactions with EyespyTV to those with social media, a participant wrote –

I would check [EyespyTV] the same way I check social media lazily scrolling, turning the knob in the morning, and taking a look at other people’s lives. In the evening, I could bring it into the kitchen and point it towards nicely prepared food.

Participants discussing EyespyTV and their responses to the probes

EyespyTV could also be used to reflect on how people change over time. A participant described such a scenario –

This knob [on EyespyTV], can it maybe represent some kind of time-based dimension? If you move it you go back in time, you see yourself one week ago at the same moment when you were getting up, sleepy, walking around. As you see yourself one week ago, you also see other people in that situation.

One of the participants suggested that shared camera feeds could be interpreted and archived and segments from them could be selected based on an algorithmically generated ‘script’ to create a life fiction, similar to reality TV shows that are (presumably) pre-scripted. She described it as –

[EyespyTV] would actually function as a TV, but you [the viewer] could also decide to volunteer or take part in the narrative, by exposing yourself to it. So, when you watch it, this feed that was captured, will be used in another narrative in somebody else’s vision of the story. Or it can change the story you are watching, by interpreting and adding you to the story.

On the other hand, this form of physical sharing of personal spaces and actions, could also be really meaningful and personal. One of the participants commented –

My Dad actually lives on an island and I was thinking, “It would be lovely to have that with him.” And almost like a kind of familiar, you could just have this thing and give it as a gift, “Here you go, here’s access to my life and you can have it.” You know, from blocks of time throughout the day. And when I’m busy or unavailable, switch to some feeds from where I live, something routine and boring, like a sidewalk or a clock tower.

A few participant responses to the probes

The workshop setting

Publication

I discussed EyespyTV and Eyespy design concepts and their process in a peer-reviewed research paper. The paper was presented in the NordiCHI 2018 conference in Oslo, Norway, together with my PhD advisor Prof. Alma Leora Culén.

Research Paper

Pandey, S., & Culén, A. L. (2018). Eyespy: Designing Counterfunctional Smart Surveillance Cameras. Proceedings of the 10th Nordic Conference on Human-Computer Interaction, 838–843.