Tech Tidbit #1 — Write Hive data to CSV file using Shell Script

This script basically consumes a databasename and tablename as an argument and gets the hive data of the table into a CSV in a local file.

Tidbit:

Explanation:

Line 1–2 : Getting command line arguments for database name and table name.

Line 3 : Getting the username of the current session so that we can write the CSV to the user’s local folder later.

Line 5–6 : Making sure we have received both database name and table name from the user.

Line 7–8 : Cleaning up existing HDFS folders and local folders from previous writes.

Line 11 : Executing an hive query to write all table data to a particular HDFS directory as part files.

Line 12 : Merging the part files on HDFS and writing it out to the local directory as a CSV file.

Line 15 : Usage error printed if one of the arguments is missing.

Sample run:

Sample output: