Tuesday, 20 March 2012

More on Minim

I have been trying to find a way to control sounds using a mouse. This is because hopefully the xbox can substitute peoples coordinates in place of the MouseX/Y. The first issue I ran into was how to control the volume of the input from the Audioplayer. Volume() or setVolume was not working for the processing platform I am using and after sometime researching I discovered that I was not alone. The Answer to this issue to this problem was given to me by Zachary Seldess from this forum

http://processing.org/discourse/yabb2/YaBB.pl?num=1224778221

he suggested altering the gain() of the audio track using setGain(). This worked a treat.
The next step was to figure out how to use the mouse x/y's to manipulate the gain amount. I borrowed the principles of the Color  Transparencies  with sound sketch which used the map() function to remap the mouseY position into something useful for the gain value.

http://www.openprocessing.org/sketch/15427

The next task was to split the area (400 x 400 sketch) into zones. As the concept is still to get people to trigger different instruments I need to use the mouseX/Y's to achieve this initially. Using the principles again from the Color Transparencies with sound code again I discovered that -80db is inaudible. - so the easy thing to do was to tell the program to make the track inaudible when in the wrong zone. ie, if you are in Track 1's Zone Track 2's gain is set to -80 ..etc The issue that arose from this was an annoying clipping sound, like a scratched CD. I went back to the forum where Zachary Seldess helped us out and searched untill I wfound a solution to this problem - mute() & unmute how simple. Muting the track will allow me to keep the tracks coherent whilst stopping the sound from emiting.

the code to date is as follows:

// import minim libraries
import ddf.minim.*;
import ddf.minim.signals.*;
import ddf.minim.analysis.*;
import ddf.minim.effects.*;

//------------------------------------------------------------------------

//set up audio functions
AudioPlayer player1;
AudioPlayer player2;
Minim minim;


// setup varables
int x = 10;
int y = 0;
int rectWidth = 400;
int rectHeight = 400;

//------------------------------------------------------------------------

void setup() {

  size(400, 400);

  minim = new Minim(this); // telling the sketch that this object is infact the Minim Function.

// these are the two tracks being loaded into the functions
  player1 = minim.loadFile("08 - Samba Tranquille.mp3", 2048);// Thievory Corporation
//feel free to add your favorite songs in place - as you can see they can bee mp3 or wav
  player2 = minim.loadFile("13 Aqueous Transmission.wav", 2048); // Incubus
}

//------------------------------------------------------------------------

void draw () {

// playing the Audio
  if (mousePressed == true) {
    player1.play();
    player2.play();
  }


// using the mouseY to set the gain of the tracks, map() helps re - map so to fit with
// the db range
  float vol =  map(((y+rectHeight)-mouseY), y, (y+rectHeight), -40, 0); //variable vol is created with the map() as its value
  player1.setGain(vol); // set the gain of the player1 function
  player2.setGain(vol); // set the gain of the player2 function

  println(player1.gain());// have a look to see how everything is (or isn't) working
  println(player2.gain());

//this is suposed to mute the tracks when the mouse goes out of the window - so far doesn't work
 if ((mouseX < 0 || mouseX > rectWidth) && (mouseY < 0 || mouseY > rectHeight)) {
    player1.mute(); // stop the sound of the audio but keep the function running.
    player2.mute();
  }


// zoning the window
  if (mouseX > 200) {// right hand side if the window
    player1.mute();// silence Incubus
    player2.unmute(); // unsilence Thievory Corporation
  }

  if (mouseX < 200) {
    player2.mute();// silence Thievory Corporation
    player1.unmute(); // unsilence Incubus
  }


}

//------------------------------------------------------------------------

//yet to discover a good explanation for this function (apart from the obvious)
  void stop() {
    // always close Minim audio classes when you are done with them
    player1.close();
    minim.stop();
    super.stop();
  }

please comment - would like to hear of possible improvements.

My USB interface and home recording software is all set up and ready to start laying down some tracks. I am using Reaper recording software which is incredibly intuitive, and I am using m-audio Fast Track as my interface to input into Reaper. The next step is to get some dummy tracks set up and add a swarm to the sketch window to test the responsiveness of the sketch.

Thursday, 1 March 2012

Proposal

Human Sketch Pad

Synopsis -

The lobby space is to be turned into a series of zones. Each zone will represent an element of an audio track. For example, zone one is a bass track, zone two a drum track and zone 3 a treble tack. The tracks will play simultaneously but will have "0" volume until activated. Each zone is activated by a person walking into it. The tone of the track will be controlled by the position of the people in the zone. This will create a full piece of music when the space is completely occupied. This project creates an interactive piece of digital design using audio which responds to the environmental, social and ontological aspects of the brief.


Background and Context -

People have been making sounds collaboratively for thousands of years. An individual starts to play a beat in a street somewhere, then another brings a guitar and then someone starts to sing and suddenly everyone on the street is creating music. Digitally this can be achieved with triggers to initiate sounds and speakers to play back the audio. Interactive digital design based audio has already been achieved in forms of the "Reactable" by Reactable Systems and "Maschine" by Native Instrument.

  • The "Maschine"  technology relies on physical triggers in the form of impact pads to initiate and audio clip stored either on a computer or it's own internal memory. The onboard memory also allows the user to loop the sounds they create, which is one of the most powerful parts of this type of equipment. Maschine has the ability to put effects on the audio clips through a series of switches and dials.

  • "Reactable" is more complex system and behaves more like an instrument than a machine. It relies on images on the bottom of a tile to be read by a camera underneath the table. The images proximity to one another, location, rotation and order to the centre of the table allow different sounds/effects to occur. It is a revolutionary digital instrument.

 These two examples are great for the individuals however are too complex for a mass of people. This is because it relies on people to work together succinctly and has no flexibility for errors to occur (especially the Maschine). "Piano Stairs" and "the world's largest bin" by funtheory.com (Volkswagen), are a great way to create simple sounds using a large mass of people.

  • The Piano stairs are initiative to get people using the stairs. It uses a series of impact sensors to trigger the corresponding notes of the piano. This is a neat concept as it also responds to our brief quite well. The idea is that it is more fun to walk up the interactive stair and potentially coordinate with other people to make music.

  • The worlds larges bin uses a motion sensor to activate a long falling sound. This is an initiative to get people using the bin more often instead of littering. This technique is closer to the type of set up I would like to use for my project - a motion sensor which initiates a noise.

It is clear that there are many types of audio systems which are controlled by digital mediums, however as demonstrated in "the worlds greatest puppet show",


the Xbox kinect system is probably going to be the most effective way of achieve the desired outcome for this project. The kinect is able to track movement in a 3D environment and use the movement of people as an input to control the output - image or sound.

References, I have and will continue to studied, that are helping me to generate ideas and concepts are listed below:

Processing+minim - Video sonification

Sekvenser - Processing+minim
also good

Vanessa Wang - Mumble Bubble

How does this address the brief? -

There are four insinuated parts to the brief:
  1. Site Specific
  2. Sustainable
  3. Social
  4. Ontological

1. The project will meet this criterion by having to use the specific volume (m³ - area times height) of the exhibition space. The volumes dimensions will need to be accurately calculated to efficiently control the sound which is initiated by the people in the space. The music will also be a response to the demographic, sophistication and history of the site. This means that the tracks will be a modern jazz fusion with a focus on drum and bass. The music will also be tailored to inhance the atmospher of the site as the day turns into night.

2. Unfortunately in the digital world we need power, but the sustainable factor of this project is that it can potentially create a fun atmosphere for free. This is economically sustainable as it means that people can create music without having to pay for instruments or lessons. It is also socially sustainable as it creates interaction between individuals

3. As stated previously, the ability to connect people is the social component of this project. The idea that the people who are individually enjoying the visual exhibition, must then coordinate in order to enjoy an audible exhibition. This subtle interaction maybe the ice breaker people need in this setting to verbally interact.

4. The project will allow people to redefine themselves as an instrument, or an augmentor of sound. As one person initiates a bass sound and another a drum, they redefine themselves and each other. This abstraction of self is one of the strengths of this project as it allows people to immerse themselves in the exhibition space.




Description -

The initial step of the project is to understand the tools needed for the project:                  
·         motion sensor (xbox)
·         speakers (provided by Dave)
·         computer (with program)
·         audio tracks (provided by Maurice Timbers, Payton Giles, myself and others)
·         exhibition space (Lobby space of the newActon "east" building)

Next is to understand and investigate the different ways of dealing with sound using processing. Minim is an easy way of dealing with sound through processing. It allows for the programming of multiple inputs, volume control and "bandpassfilter", which alters the sound depending upon X and Y positions. This comes with many more options, which can open up an array of opportunities to expand the project. This will be readjusted and experimented with till the day we present the project.

A week will be set aside for creating a 2 - 4 hour song which is suitable for the program. The song whilst having all the parts recorded together will be split into separate tracks. This allows the tracks to be independently controlled by the different zones. This will be done in a week as it should not be the focus of the project however must still be to a high standard.

The next thing to do will be figuring out how to incorporate the motion sensor which will be the Xbox kinect. Sketches which react to position are easily made in 2D however a lot of investigation time will be spent figuring how to create a 3D position sensitive sketch and then how to incorporate the xbox as the input. The range of the Xbox will also need to be determind to inform the scope of the work. This will take up the first 8 weeks of the semester as it will be challenging.

The outcomes expected are for there to be alot of trickery in getting the tracks to play on time with each other and for the xbox to do its job properly.  I expect the Minim library to provide the control over the audio output and I expect the xbox to handle the programs 3D space input. The logic behind the sketch is quite simple however the coding involved could be rather complex.