Database Reference
In-Depth Information
<property>
<name>hive.metastore.client.socket.timeout</name>
<value>3600</value>
<description>MetaStore Client socket timeout in seconds</description>
</property>
Then, I make the following changes to Impala's copy of the hdfs-site.xml file:
<!-- changes to impala -->
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
<property>
<name>dfs.domain.socket.path</name>
<value>/var/run/hdfs-sockets/dn</value>
</property>
<property>
<name>dfs.client.file-block-storage-locations.timeout</name>
<value>4000</value>
</property>
<property>
<name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
<value>true</value>
</property>
These changes boost performance with HDFS by bypassing (short-circuiting) the data node and accessing the
files directly. They specify a domain socket path and a block storage location timeout. These changes, including the
core-site.xml change that follows, provide a real performance boost to Impala's operation. So, I make the following
changes to the Impala's copy of the core-site.xml file:
<!-- impala changes -->
<property>
<name>dfs.client.read.shortcircuit</name>
<value>true</value>
</property>
If the directory /var/run/hadoop-hdfs/ is group writeable, I make sure that the group is root. To check this, I use
the Linux ls command to get a long listing of the directory:
[root@hc1r1m1 ~]# ls -ld /var/run/hadoop-hdfs/
drwxr-xr-x. 2 hdfs hdfs 4096 Sep 7 09:21 /var/run/hadoop-hdfs/
 
Search WWH ::




Custom Search