Dealing with Large Files - HBase Design Patterns - page 54

This code allows each client to generate a key and to store the data in HBase using
this key, without worrying about possible key collisions with the other clients who
are generating their keys.
That is, in the case of storing user information, if you are storing files, you will
probably use the following code:
byte[] bytedata = Files.toByteArray(file);
byte [] key = UUID.randomUUID().toString();
Put put = new Put (key);
put.add(Bytes.toBytes(familyName), Bytes.toBytes("phone"),
bytedata);
This code generates the key in the same way; then, it reads all of the file's bytes
and stores them in an HBase cell.
What to do when your binary files
grow larger
As mentioned, we are going to progress from simple to more complex solutions. The
code we just saw is good for small binary files. Most developers, as well as the HBase
documentation, will tell you that here we are talking about megabytes (not even 10
MB to 50 MB). You will have to experiment with your solution and vary the data
size. However, what if you find the database to be too slow?
There are a few approaches that you can take.
Using Google Blobstore to store large files
Google BlobStore is described at https://developers.google.com/appengine/
docs/java/blobstore/overview . This is a service that Google gives you, the
developer. It is called the Blobstore API. It allows your application to serve data
objects, called blobs , that are much larger than the size allowed for objects in the
datastore service. Blobs are useful for serving large files, such as video files or image
files, and for allowing users to upload large data files. Naturally, you can use it only
if you are allowed to use Google App Engine as part of your solution.