Reporting with Hadoop - Big Data Made Easy: A Working Guide to the Complete Hadoop Toolset - page 334

I found the report search logs by running the Linux find command from the Hunk install directory /usr/local/
hunk. I knew that the search log file was named search.log, based on forum answers on splunk.com . The error was
caused by the fact that the user running Hunk did not have access to the Splunk working directory /user/hadoop/
hunk/workdir.
A search log generated this second error, as well:
10-29-2014 19:14:07.385 ERROR ERP.cdh5 - SplunkMR - Failed to create a virtual index filesystem
connection: java.net.UnknownHostException: hc2nn.semtech-solutions.co.nz. Advice: Verify that your
vix.fs.default.name is correct and available.
The error was caused because HDFS was not running. The Hunk report had been run before the Hadoop servers
completed their startup sequence.
This error, again from the search log, seems to be an indication of another underlying problem:
10-30-2014 18:22:55.398 ERROR SearchOperator:stdin - Cannot consume data with unset stream_type
10-30-2014 18:22:55.453 ERROR ExternalResultProvider - Error in 'SearchOperator:stdin': Cannot
consume data with unset stream_type
In this case, I had incorrectly created the configuration for my CSV file processing in the configuration files.
Although from Splunk forum entries I understand that it should not be necessary, adding entries to the files props.
conf and transforms.conf solved this problem.
I initially tried running Hunk from a remote Linux server—remote from the Hadoop cluster that it was trying to
connect to. The following test uses the HDFS file system ls command to examine the / directory on the cluster via the
HDFS-based URI hdfs://hc2nn:8020/:
[hadoop@hc1r1m1 ~]$ hdfs dfs -ls hdfs://hc2nn:8020/
ls: Call From hc1r1m1/192.168.1.104 to hc2nn:8020 failed on connection exception: java.net.
ConnectException: Connection refused; For more details see: http://wiki.apache.org/hadoop/
ConnectionRefused
The access failed, so before attempting to use Hunk, it is important to check that the Hunk Linux user on the
server where it is installed has access to Hadoop.
The following errors were displayed when Hunk was started in the Linux command window:
Checking conf files for problems...
Invalid key in stanza [source::/data/hunk/rdbms/...] in /usr/local/hunk/etc/system/
local/props.conf, line 3: DELIMS
(value: ", ")
Invalid key in stanza [source::/data/hunk/rdbms/...] in /usr/local/hunk/etc/system/
local/props.conf, line 4: FIELDS
(value: f1,f2,f3,f4,f5,f6,f7,f8,f9,f10,f11,f12,f13,f14)
Your indexes and inputs configurations are not internally consistent. For more
information, run 'splunk btool check --
debug' Done
All preliminary checks passed.