Dimensionality Reduction with Spark - Machine Learning with Spark - page 306

scale pixel data will be our features. We will form the vectors by flattening out the two-di-
mensional pixel matrix. The BufferedImage class provides a utility method to do just
this, which we will use in our function:
def getPixelsFromImage(image: BufferedImage): Array[Double]
= {
val width = image.getWidth
val height = image.getHeight
val pixels = Array.ofDim[Double](width * height)
image.getData.getPixels(0, 0, width, height, pixels)
}
We can then combine these three functions into one utility function that takes a file loca-
tion together with the desired image's width and height and returns the raw Ar-
ray[Double] value that contains the pixel data:
def extractPixels(path: String, width: Int, height: Int):
Array[Double] = {
val raw = loadImageFromFile(path)
val processed = processImage(raw, width, height)
getPixelsFromImage(processed)
}
Applying this function to each element of the RDD that contains all the image file paths
will give us a new RDD that contains the pixel data for each image. Let's do this and in-
spect the first few elements:
val pixels = files.map(f => extractPixels(f, 50, 50))
println(pixels.take(10).map(_.take(10).mkString("", ",", ",
...")).mkString("\n"))
You should see output similar to this:
0.0,0.0,0.0,0.0,0.0,0.0,1.0,1.0,0.0,0.0, ...
241.0,243.0,245.0,244.0,231.0,205.0,177.0,160.0,150.0,147.0,
...
253.0,253.0,253.0,253.0,253.0,253.0,254.0,254.0,253.0,253.0,
...
244.0,244.0,243.0,242.0,241.0,240.0,239.0,239.0,237.0,236.0,
...