Week 11 - Libraries and Video Capture

Today:

  • Review: Autonomous agents (arrays of objects)
  • Review: History of computation
  • Libraries
  • Video

Libraries

What is a Library?

Out of the box, Processing comes with lots of great functionality to draw images and shapes on screen, to manipulate colors, create interactivity, and many more built-in functions.

When you hit the play button, the Processing IDE (Integrated Development Environment) compiles your code. It does this by translating your Processing code into machine code.

If you need additional functionality to write a program that can do things not built into the core of Processing, you need to use a Library. A library is a collection of code (essentially, variables, functions and classes) that are pre-made that you can use in your own programs to extend the functionality of the Processing. Examples of this additional functionality include: working with sound, working with video, using machine vision, or working with motors.

These additional libraries are not built into the core part of Processing because it would slow down Processing if it had everything and the kitchen sink included from the start. The big idea is that the core of Processing should get you pretty far, but if you need more functionality, you can add a library to get more.

Processing’s libraries are created by the people behind Processing. But also, anyone can make and distribute their own Processing library.

When you want to use a library, you need to tell Processing to import all of the new functionality into Processing. You do this with an import statement at the top of your program.

import processing.video.*

The * says that you want to import ALL from the Processing Video library.

Installing Libraries

There are libraries created by the Processing Foundation, and there are contributed libraries. Contributed libraries are created by third party individuals.

The Video library is created by the Processing Foundation.

All libraries can be imported to be used.

  1. Navigate to the Sketch Menu drop-down.
  2. Select Import Library » Add Library
  3. Type in the name of a library to search for it, or browse, and click Install if it’s not already installed (Video is pre-installed).

Using an installed library

Once a library is installed, to use it in a sketch just navigate to Import Library in the SKetch menu and click on the name of the library. You’ll see an import statement with the library name added to the top of your sketch.

Video Library

1 Import the video library.

import processing.video.*;

2 Declare a capture object.

Capture video;

3 Initialize the capture object

  • The video library has two classes: Capture - for live video, and Movie - for recorded video file playback.

You initialize this like other objects, using new.

video = new Capture();

You could also pass in arguments.

video = new Capture(this, 320, 240);
//creates a video capture 320 pixels wide by 240 high

4 Start the camera running.

video.start(); //do this once! in setup

5 Read the image from camera

void draw(){
  if (video.available()){
    video.read();
  }
}

Alternatively, you could use a special capture event.

void captureEvent(Capture video){
  video.read();
}

6 Display Video

void draw(){
  image(video, 0, 0);
}

All-together:

import processing.video.*;

Capture video;

void setup() {
  size(320, 240);  
  video = new Capture(this, 320, 240); 
  video.start();
}

void captureEvent(Capture video) {  
  video.read();
}

void draw() {  
  background(255);  
  tint(mouseX, mouseY, 255);  
  translate(width/2, height/2);  
  imageMode(CENTER);  
  rotate(PI/4);  
  image(video, 0, 0, mouseX, mouseY);
}

Working with pixels

import processing.video.*;

// Size of each cell in the grid, ratio of window size to video size
int videoScale = 50;
// Number of columns and rows in our system
int cols, rows;
// Variable to hold onto Capture object
Capture video;

void setup(){
  
  size(800, 600);
  background(255);
  
  // Set up columns and rows
  cols = width/videoScale;
  rows = height/videoScale;
  video = new Capture(this, width, height);
  video.start();
}

void captureEvent(Capture video) {
  video.read();
}

void draw(){
  if (video.available()) {
    video.read();
  }
  
  video.loadPixels();
  
  background(50);
  
  for (int i = 0; i < cols; i++) {
    for (int j = 0; j < rows; j++) {
      int x = i*videoScale;
      int y = j*videoScale;
      // Looking up the appropriate color in the pixel array
      color c = video.pixels[x + y*video.width];
      //background(c);
      fill(c);
      stroke(10,0);
      ellipse(x,y,videoScale,videoScale);
    }
  }
}
  
void keyPressed(){
  if (key == ' ') {
    //save to data folder
      saveFrame("screenshot-###.png");
      println("Shot taken.");
    }
}

Homework:

Watch:

Read:

  • Chapter 12, Libraries
  • Read Chapter 16, on Video

Code:

  • Create your own new camera software

Two potential options:

A. Create a pixel grid project

OR

B. Create a filter or frame

Pixel grid project:

Develop an interesting pattern that covers an entire window using the camera and pattern generation.

Write your own camera software using a pixel array to manipulate and create your own effect. The camera should be a compelling design and have a logic to it.

You must have a SAVE button with a keypress or onscreen button to take a screenshot and save it.

Filter or Frame:

Create a program that takes camera input and displays that, but then adds a filter or a frame to your image. There should be an underlying concept to the work. A meme camera that adds text, a birthday card maker, a special sidekick that pops-up and appears in the picture as well, or something else.

You must have a SAVE button with a keypress or onscreen button to take a screenshot and save it.