ICM Final Semi-Proposal

Pixel Sort (of)

Alternate Working Title(s):

  • “Sort the World!”
  • “Harry Potter and the Sort-serer’s Stone”

“Understand the fundamental structure of screens and our perception of them enables us to completely mess with that perception. This is as much a research project into how screens work as it is a programming assignment, me thinks.”

What is it:

A novel pixel sorting program.

Visual References :

Sorting Theory References:

Previous Exploration:

I’ll Now Take Questions:


Visual Reading with Strings and Data

 As Long as We’re Visualizing Data…Productivity_DataVizGraph

String difficult = “difficult”;

This week was difficult for a few reasons.

I found that strings got tricky once they were broken down into characters and it comes time to reassemble them. I can’t really fathom doing anything besides just counting them or showing them on screen. Like animating…although that’s certainly the next step.

And data…actually, data isn’t so bad. It’s only a problem whenever you’re trying to use or work with it. I’ve never read anything that’s made less sense then some of the API documentation I looked at this week.

Ok. I’m done now.

Visual Reading

What is it? 

It’s something I just made up right now. It’s a way of reading a text using images that match strings in the text.

How’s it work?

The sketch displays the text of Arthur Conan Doyle’s A Study in Scarlet on screen, one word at a time. It’s also checking the individual words against a CSV database for matching strings. When it finds a matching string (i.e. “sherlock”) it loads an image from an online image search and displays the image.*


Pixel Experiment 1

Starting with the Basics

I spent some time today with Katie trying to figure where I went wrong last time. She doesn’t have experience with this type of stuff either and so it was a solid hour of us going back and forth with ideas and breaking the sketch. We ended up with something that resembles a basic sort.

The problem is that we’re sorting in an arbitrary way. I.e. not by any interesting intrinsic to the pixels in the picture. I think, as it stands we’re just grabbing and sorting a pixel by the number associated with the way the color is being stored in the computer. Rather than by the value (or values) of the color itself.



Sorted (I guess)

Screen Shot 2014-10-23 at 7.45.46 PM

 Next Steps

This whole pixel sorting thing goes pretty deep as evidenced by this dude, Kim, who apparently coined the term. It’s clear there’s a lot more to learn.

These are, I think, some of the next steps to understand the process before I can create some semblance of direction. Most of these bullets should have a “How do I” or “Figure out how to” in front of them and a “?” at the end.

  • Grab some value associated with a pixel
  • Keep pixel information intact
  • Sort by that value
  • Sort in an interesting way (most unique to least unique, difference from average color, etc)
  • Something I’m forgetting/missing
  • ?

Processing Pixels

This week we took a look at manipulating images and video in Processing. Because our medium is a screen, we’re dealing with pixels. It’s a seemingly obvious fact that I take for granted. Getting into a mind set dealing with a myriad of points instead of a whole shape or larger system of shapes has been difficult but rewarding. I can feel my brain bending.

(Not) Pixel Sorting

A week after being introduced to the concept, I’m still trying to wrap my mind around manipulating pixels. So instead of working with an entire image I figured that a good way to discover the nature of pixels was to look at them on an individual scale. (Nick Bratton actually sent an email to the listserv about pixel sorting that kicked off this path of exploration.)


This is the image I’ve been starting with.

I took this on a Canonet QL17 while in Puerto Rico.

The Algorithm

It’s terrible. Just a poor example of programming all around.

That’s because it’s mostly a random construction designed to allow me to mess with it.

It takes forever  to make a pass through the image and I’ve got no idea how to optimize it. (I tried right shifting the colors (>>), because I read that it was a faster way to retrieve color info, to no avail.)

The Results

The variables I’ve focused on are  s, r, g, b, i, j, t and c.

  •  r, g, b being individual RGB values of the pixel array.
  • s being a random settable value.
  • being the squared factor of s.
  • i and j being the variables determining the size of the rectangle matrix I’m creating to help visualize what’s happening.

Depending on the  combination of variables I get images returned that look like this.


I had a hunch that what I thought was going on wasn’t actually going on.  So I took out the background image. And like I thought I got virtually the same pixel pattern…as long as the window size stayed the same size as the initial image. 

A quick sidebar:

When I removed the image completely and used the same code (I think with a different window size) I got this result that updated for some reason. Maybe the other program runs too slow and I’m not patient enough to wait for it?


That means I’d been altering the pixels of the window and the image remained an untouched layer underneath. These pixel/color patterns and relationships are based on the mathematics of the window and the size of the rectangles not the image information.

My image is merely being drawn behind the color patterns.

I’d bet that every one of these results is reproducible on a separate machine simply by inputting the same window size and integers for variables. The image information probably isn’t used at all except for background fill.

I need to create a relationship between the image colors and the pixel/pattern algorithm…but that’s where I’m getting stuck. I can’t quite figure out how to retrieve and use information from the image’s pixel array. It seems to be sitting right there, but I can’t grasp at it. Yet.

More pixel sorting experiments to come.

Array Lists

Flexible Arrays

Last week we were introduced to arrays. Or…I was introduced to arrays.



We don’t know. In any case the idea of an array was introduced to some conscious being/s. One of whom may or may not have been me. Or You.

One characteristic of these lists of items called arrays is that they have a fixed length. If you initialize an array with 8 slots (indexed 0 – 7), then it will always have 8 slots. If you want to change the length, you find where you initialized the array and change the length. Then you have to change that number everywhere you’re calling the array. Which is time consuming and kind of a pain in the ass. The solution is to use the syntax arrayname.length to refer to…you guessed it…the array length, but you still have to change the hardcoded length.

An ArrayList allows you to call functions that change the length of the array dynamically. On the fly. In the field. On a whim. Should the mood strike you.

For example, in the video below I’ve made a watery, ring object thing and then created an ArrayList of such watery, ring object things. Now I can create a new ring that’s part of the array every time I click the mouse. There are a whole bunch of functions that I can’t remember off hand, but the idea is that they give you the ability to change the number of index slots in the array at will.

Classes and Arrays

Sharing (Classes) is Caring

Part of the concept of code is that it’s modular. Blocks of code should be reusable and pass-around-able.

In this week’s assignment, Nick and I independently created classes of objects and shared them with each other. In this post I’ll show you my class, in the next one I’ll try and break down Nick’s and then show you how the two did or didn’t mesh.


I created a simple particle class that changes its fill to a random color when it hits a boundary. You can check the code here. (Working with git through the terminal was another mini project unto itself. But I’m getting way more comfortable navigating and using git. It’s starting to stick.)

I was keeping the object simple this week in order to add layers of complexity in the way the code was written (i.e. creating classes and an array of objects). The initial idea was to create particles that bounced off walls and each other, changing to a random color when they hit a wall, and passing a color on when they collided with each other. I thought that visually it might be interesting and create a swarm-looking system without the more complex physics simulation code. The color transfer would be enough information that the objects would look like they where somehow connected.

I wasn’t able to crack the collision/color exchange. But that’s a definite next step.

I had a ton of trouble with the acceleration variables. I couldn’t get it to work properly without being sort of self-relegated to one quadrant of the window (as you can see in the video below, which I was used for visual de-bugging). Or quickly ramping up to an infinite speed.

For the sake of my sanity I decided to drop acceleration all together and just work with a velocity for the particles.

ICM Week 4: Functions

This week I was experimenting with functions. I obviously wanted to complete the assignment but it seems like a bit of a step back after I was fairly successful in my quest to orient objects last week.

In any case, I created a simple program to make flag-like thingys. Each flag has parameters to determine the area of the blocks, the colors, and location on screen. There’s some really simple math going on in the color variables in order to give each flag a range of similar, or at least complimentary, colors.

Running the sketch looks like this…

Screen Shot 2014-09-30 at 11.04.05 AM

ICM Week 3: Objects and Control

Prologue: The Sensei and the Student

This week we were assigned to create a program with “rule-based animation, motion, and interaction.” Though I, admittedly, spent most of my time solving another problem that was bugging me too much to let go: how to create classes.

Last week I created a sweet little satellite ellipse that orbited a center point in a circle. Cool. This week I thought that I’d like to create some GUI elements to control different characteristics of that satellite.

At went to Shiffman to talk about using vectors to create motion, but I quickly realized that I wasn’t really asking him about code, I was asking him about basic math. Info which, I could have sat in on a seventh grade class to get. Sorry, Shiffman. 

Anyway he casually mentioned using classes but quickly dismissed on account of it coming up in class in a week or two. But Shiffman and I, we speak a different language. Another code if you will. It consists of fleeting cues, subtle hints, and the occasional wink*. While he was talking about basic vectors and acceleration, he was actually issuing me a challenge. As any good sensei would**.

*Not at all true.  Strictly for dramatic effect.

**Dan Shiffman is not my sensei. He has not agreed to be my sensei, nor have I asked him to be***.

***Although if he wants to be my sensei, I am in the market…

So I spent a bunch of time figuring out how to create a Satellite class.

Chapter I: Actually Doing the Assignment

One of the characteristics of the satellite that I thought would be cool to control is the size. It’s obvious and you’d see the effect immediately. I wanted to do this by programming a horizontal slider as the control element. The slider would move left to right and as it was dragged to the right the size of the slider and the satellite would grow correspondently.

The slider was fairly straight forward to program. I created the slider, then just make sure I got the whole class-thing, I created a Slider class.

Boom. Slider class. Satellite class. Great. Except they weren’t talking. The slider would slide and the satellite would be a satellite, but there was not communication between the two. I knew there must be a simple solution.  Getting classes to talk to each other seems like it should be a basic function. And, good news: it is. I just couldn’t figure it out on my own.

So I talked to a resident: the venerable Ryan Bartley****. He showed me the basics of setters and getters in about 10 minutes. He’s super.

Now the slider works and it’s size corresponds exactly to the size of the satellite.

****Also not my sensei. In any official, or really, unofficial, capacity. Just a smart dude with office hours.

Chapter II: The Sketch



Week 2: Animation in Processing

Circular Trajectory / Orbit

My initial idea was to have a series of small ellipses traveling around a large circular trajectory at different speeds, so as to create patterns around the trajectory.

I thought a elliptical orbit was a simple enough concept that I could knock out fairly quickly and then spend time creating interesting patterns and interactions.

Not the case.

rotate() v Parametric Equations

Initially I used rotate() to move the ellipses. But this was limiting. It seemed that this would rotate everything in the canvas as I added new elements.

The solution I came up with was to define the rotation of the orbital ellipses  around the circumference of a circle. To do this I would need an equation that would map every point around the circumference of a given circle. I did some research and discovered parametric equations.

These are the complementary equations I used :

x = (cx)+xr*cos(a)
y = (cy)+yr*sin(a)

  • x: the x-cooridinate of a given circumference point
  • y: the y-coordinate of the same point
  • cx: the x-coordinate of the ellipse’s origin point 
  • cy: the y-coordinate of the ellipse’s origin point 
  • yr/xr: radius of the given ellipse
  • a: measured angle

Here is the final sketch:

This theoretically allows me to have objects orbit independently of each other; i.e. I don’t have to rotate the entire canvas to create an orbital motion. So in the future I’ll be exploring implementing multiple independent orbits, relational orbits, and parametric equations that describe other shapes besides a ellipse/circle.

Untitled 2 (Self Portrait in Processing)

At It Again

That last self portrait was too static for my liking. I wanted something more dynamic. Something that would capture the true power and underline the essence of my hypnotic gaze.

Screen Shot 2014-09-06 at 12.02.31 AM
I explored several different versions of me shooting things out of my eyes.

Combining my learnings heretofore I was able to use variables and the random call to create the illusion of a colorful and everlasting stream of energy pouring out of my eyes. (I couldn’t figure out how to get it to restart automatically upon reaching the limits of the canvas. Or to get said energy to shoot out of my eyes in real life. But I was able to get it to restart on mouse click or key tap.)

Prepare yourself.