Database Reference
In-Depth Information
The
generateRandomArray
function creates an array of the specified size where the
entries are randomly generated from a normal distribution. We will use this function ini-
tially to generate our known weight vector,
w
, which will be fixed throughout the life of
the producer. We will also create a random
intercept
value that will also be fixed. The
weight vector and
intercept
will be used to generate each data point in our stream:
/** Function to generate a normally distributed dense
vector */
def generateRandomArray(n: Int) = Array.tabulate(n)(_
=> random.nextGaussian())
// Generate a fixed random model weight vector
val w = new
DenseVector(generateRandomArray(NumFeatures))
val intercept = random.nextGaussian() * 10
We will also need a function to generate a specified number of random data points. Each
event is made up of a random feature vector and the target that we get from computing the
dot product of our known weight vector with the random feature vector and adding the
intercept
value:
/** Generate a number of random data events*/
def generateNoisyData(n: Int) = {
(1 to n).map { i =>
val x = new
DenseVector(generateRandomArray(NumFeatures))
val y: Double = w.dot(x)
val noisy = y + intercept
(noisy, x)
}
}
Finally, we will use code similar to our previous producer to instantiate a network connec-
tion and send a random number of data points (between 0 and 100) in text format over the
network each second:
// create a network producer
val listener = new ServerSocket(9999)
println("Listening on port: 9999")
