For my Computers for the Rest of you final, I’ve been writing a Processing program that reads a log of all the user’s typed (thanks to KeyloggerX), analyzes that log for unhappy words, and calls the user to cheer them up if the log proves too sad.

I started with just a push-button-to-call program:

And it works! If you press the square on the left, Asterisk calls my cellphone, says “goodbye” nicely, and hangs up. If you press the button on the right, it does the same to my girlfriend (she found that amusing at first, but after a few accidental calls at 2 a.m. I disabled that button — be nice to me or I’ll make it your number instead!).

After I got that working, I moved on to proper log analysis. Right now, that works too! Every three minutes, the program reads the text log. If you’re sad, it calls, and if not, it leaves you alone. It then trashes the log file and sits patiently for another three minutes.

Sad:

And not sad:

Right now there’s a few things to still work out — notably, that the Processing program craps the bed if it doesn’t find a text log file (that’s easily fixable), there isn’t much cheer-up audio (that’s a quick fix too), and the sad-test algorithm needs to be a bit fancier to find more sad words (not too tough). But, I’m on the right path!

And lest I forget, thanks in huge helpings to the kind assistance from Kate Hartman, Adam Simon and Sinan Ascioglu, who all gave me code snippets, told me about programmy things, and/or generally were nice about the fact I can’t code my way out of a paper bag without help.

The basic idea is that I’ll be running a text logger to record what’s typed and using Processing to watch for an abundance of sad, depressed and/or angry words.  When too many of those words appear, Processing triggers Asterisk to call you and chear you up.  It goes like this:

You’re sitting at your computer, frustrated and sad, typing away.

Then if you type too many sad or depressed words, you’ll get a phone call.

Which will cheer you up and take you a step back from the situation, so you can return to it later, happier and more relaxed.

This is the basic science behind that:

The issues are that I’m not a Processing wizard and I’ve done limited work with Asterisk, but this should be fun either way.

Oh, and thanks a ton to Adam Parrish for volunteering (err, falling to my repeated harassment) to be my model for this.  I promise to do absolutely nothing nice for him as a thank-you.

This morning, I got together with Daniel Soltis for a bit to work on a way to mess with the data, and ran with Daniel’s idea of making mad libs out of my typings for the week.  Fortunately, he’s an infinitely better coder than I, so we actually came up with something amusing.

And here’s as far as we got with the random word generation:

More to come on this.

And here’s an initial mapping of the data:

Of course, there were a few issues. The biggest so far has been the phone side of the equation — we’ve so far had an Arduino Mini reading three photocells (covered by R, G and B gels), pushed over BlueSMiRF RX/TX pipe to a Nokia N80 which is running a parsing/logging program (written by DanO). However, the cell phone end proved to be a little jenky for what we were trying to do, and getting the data off the phone was another adventure entirely. Going forward, we’re going to try dropping the bluetooth/cell phone part and just use a SD logger. I’ll post updates on that as we get a prototype together.

Also, the graph sucks. After trying several things in Processing and Max/MSP/Jitter, we ended up defaulting to Excel (err, Numbers) to get a visualization, and it’s hella boring. In the next week I’m'a try to Max this up niiiice.

I’d started looking at architectural shapes and conceptual designs, especially the ones that looked to me like wave file a

[Wikipedia's list of push-pull meanings]

So then I started thinking about the physical interface itself, and how I’d be shaping things, and I came to the idea of event tents (and pup tents, as pictured here).  If at every point on this the poles could be adjusted, changing an attribute about the sound, it’d be a great way to work with pieces of audio — and ideally, I could also have the form assume the shape of incoming audio.

Here’s the presentation I gave in class — far more to come, of course.

In the above photo, you can see that the phone is logging the photocell values (although at the moment, there’s just one cell — still waiting on parts, of course), which we can then shove over Bluetooth to a computer, which can add them to a running log via a PHP script.  If either of us had non-Verizon phones (or weren’t broke) we could probably do this by standard data plans and internets, but hey, you make do with what ya got.

Our next plan is to put two more photocells on the board and cover each in a red, green or blue gel so as to get accurate light values, and wire up a mic to output numerical values of the sound levels in the area.  If we ever get it, we’ll also wire up an accelerometer to indicate position and activity of the person wearing it.  Updates to follow.

Part of our work in the Computers for the Rest of You class is to study that which we experience every day in ways that we aren’t fully paying attention to, studying what’s missing — that which we’re conscious for, in the (generally misused) sense of being awake, but aren’t conscious of. We’re supposed to look at creating a mini-us who would live on our shoulders (also see: Homunculus — pic and linked Wikipedia article below) — objectively watching our daily life, catching what we miss as we go through our day.

We, of course, are going to be nerds and do this via some sort of data logging technology (a video camera, an audio recorder, a microcontroller, whatever). The class-sponsored way (and best fit for my plans, as well as I can tell) is to use a bunch of sensors, read them with a microcontroller, send that data by Bluetooth (Blue SMiRF) to a phone, and log it that way. NYU has loaned me a Nokia N80 phone, and Scott has loaned me a Blue SMiRF, so the experimentation is ON!

So far I’ve gotten the phone to accept the java logger applet that DanO wrote, and started testing it. It’s working thus far, though I haven’t tried sending it any actual data. That’s next on the planner.

I’m continuously fascinated with the way people use music in their daily lives — not in the sense of what their particular tastes cater to, but rather how they use it to preserve or change their mental states, block things out, facilitate experiences, and so on. I know that I personally use my headphones as a barrier — when I put them on, they close out the world, both in terms of changing what I hear and in terms of sending a message to those around me. Standing on a subway, you can see an army of people doing this, especially during the rush hour times when people are packed in like sardines. As a Jersey City resident, taking an 8 a.m. train is very different from 11 a.m. — the 8 is packed solid, dead quiet, and usually filled with people providing their own music for themselves. The 11 is usually open, and the people seem less likely to have headphones on. The streets seem about the same during those times.

I took particular note of what was being played when I went into stores and restaurants, and how it affected people’s behaviors while there. Upbeat music at the corner store kept me moving quickly and out the door, unobtrusive radio rock music at the pizza place did the same. Bad dance remixes at Apple Bar made me feel awkward, while the great D.C.-sound mix at Revival really got me in the mood to talk to people. Radiohead covers at Think made me put my headphones back in.

I also choose what I’m listening to according to the task I’m facing. If I’m going to be working on something that requires writing, I’ll put on instrumental and electronic music; if I’m going to be soldering and building, it’s usually energetic breakbeat or rock with strong drums; if I’m happy and it’s nice out, it might be reggae or acoustic music; if I’m hung over and sleepy, it’ll likely be sleepy-sounding bands as well. This week I pretty much followed this pattern: I wrote blog entries playing Pele and Mogwai, I worked on my Conflux installation while blasting Evil Nine and Hybrid, I played Third World and Easy Star All-Stars on the weekend and The National early Sunday morning.

I’ve personally never been a person who could listen to music while trying to sleep (I always end up paying attention to the songs too much) or wake up to music (I hate mornings, and don’t like cross-referencing music with being grumpy), but I know many people who do. One of my roommates when I was an undergrad would fall asleep with the stereo on every night — usually blasting something really heavy and rhythmic. When I asked him about it, he noted the bands that he’d play: Tool, A Perfect Circle, Deftones — all very drum-heavy. For him, it put him in a proper mental pattern to sleep. I wish it worked for me.

But what I’m most amused by is the symphony I participate in on my way in and out of the city each day.  It’s not just when the music in your ears syncs with the city (and I swear, the whole world locked in with Enon’s Hocus Pocus one day the week before last), but how you adjust your footsteps to lock in with music you hear, how the cars passing blast songs that blend into one (good or bad) soup of sound, how the construction gives a beat to the dog walker’s daily route.  I really hope to explore that more this semester.

However, back to point, what I’m very interested in at the moment is what we’re letting ourselves miss — what we’re blocking out as we cruise through the city, how we’re walling ourselves against the noise. To address this, firstly I have the wonderful fortune to show off a project of mine which explores this at Conflux this weekend (all the blah about that here), which essentially seeks to call attention to the sounds you’re missing at night. Secondly I’m working with Tom Jenkins to study the levels and quality of sound that you ignore during the day, record the data (the levels, not the actual audio) for a few days, and then try to visualize it, especially with the addition of additional sensors (photocells, accelerometers, heartrate sensors, etc). I think that’ll bring up a lot of interesting points.