Reporting with Hadoop - Big Data Made Easy: A Working Guide to the Complete Hadoop Toolset - page 351

Figure 11-29. Muliple-column report
These reports are fine for a simple check of the table column data, but what about generating and using more
complex rules for data quality that check data values and attributes? Well, that is where SQL and regular expression-
based rules can help.
Reports Based on SQL Rules
Reports on data quality are created when the options Table Analysis and then Business Rule Analysis are chosen when
creating a report. For example, I click Next and enter a report name in the form. I click Next again and expand the
database Hive connection until I can select the appropriate table. I click Next, and I select the SQL-based rule that was
created earlier. Finally, I click Finish to generate the report. The resulting report, shown in Figure 11-30 , uses the
SQL-based rule that states amount < 10000 and shows that more that 93 percent of the data values in the “amount”
column fail this data-quality rule.