Database Reference
In-Depth Information
Because the namenode holds filesystem metadata in memory, the limit to the number
of files in a filesystem is governed by the amount of memory on the namenode. As a
rule of thumb, each file, directory, and block takes about 150 bytes. So, for example, if
you had one million files, each taking one block, you would need at least 300 MB of
memory. Although storing millions of files is feasible, billions is beyond the capability
of current hardware.
[
28
]
Multiple writers, arbitrary file modifications
Files in HDFS may be written to by a single writer. Writes are always made at the end
of the file, in append-only fashion. There is no support for multiple writers or for
modifications at arbitrary offsets in the file. (These might be supported in the future,
but they are likely to be relatively inefficient.)
