Drill-on-YARN Command-Line Tool
Drill provides the drill-on-yarn command-line tool to start, stop, resize and check the status of your Drill cluster. The tool is located in:
$DRILL_HOME/bin/drill-on-yarn.sh site $DRILL_SITE command
Where command is one of those described below.
Start the Drill Cluster
Start your drill cluster with the start command:
The command shows the startup status followed by a summary of the application:
Launching DrillonYARN... Application ID: application_1462842354064_0001 Application State: ACCEPTED Starting...... Application State: RUNNING Tracking URL: http://10.250.50.31:8088/proxy/application_1462842354064_0001/ Application Master URL: http://10.250.50.31:8048/
The first line confirms which cluster is starting by displaying the cluster name from your configuration file. The next line shows YARN’s application ID and tracks the job status from Accepted to Running. Once the job starts, you’ll see YARN’s job tracking URL along with Drill-on-YARN’s web UI url. Use this URL to visit the web UI described below. Once the application starts, Drill-on-YARN writes an “appid” file into your master directory:
ls $MASTER_DIR … drillbits1.appid
The file name is the same as your Drill cluster ID. The file contains the id if the Drill-on-YARN application for use by the other commands described below. You can run only one Drill AM at a time. If you attempt to start as second one from the same client machine on which you started the first, the client command will complain that the appid file already exists. If you attempt to start the cluster from a different node, then the second AM will detect the conflict and will shut down again.
Drill Cluster Status
You can retrieve basic information about the Drill cluster as follows:
You will see output something like the following:
Application ID: application_1462842354064_0001 Application State: RUNNING Host: yosemite/10.250.50.31 Tracking URL: http://10.250.50.31:8088/proxy/application_1462842354064_0001/ Queue: default User: drilluser Start Time: 20160509 16:56:40 Application Name: DrillonYARN AM State: LIVE Target Drillbit Count: 1 Live Drillbit Count: 1 For more information, visit: http://10.250.50.31:8048/
The first two several lines give you information about YARN’s state: the application ID, the application state and YARN’s tracking URL for the application. Next is the host on which the Drill AM is running, the queue on which the application was placed and the user who submitted the application. The start time tells you when YARN started the application. The next few lines are specific to Drill: the name of the application (which you configured in the Drill-on-YARN configuration file), the Drill application master URL, the number of Drillbits you requested to run and the number actually running. Finally, the last line gives you the URL to use to access the Drill-on-YARN web UI described below.
Stop the Drill Cluster
You can stop the Drill cluster from the command line:
Note that this command is “forceful”, it kills any in-flight queries. The output tracks the shutdown and displays the final YARN application status:
Stopping Application ID: application_1462842354064_0001 Stopping... Stopped. Final status: SUCCEEDED
Resize the Drill Cluster
You can add or remove nodes to your Drill cluster while the cluster runs using the re-size command. You can specify the change either by giving the number of nodes you want to run:
$DRILL_HOME/bin/drill-on-yarn.sh resize 10
Or by specifying the change in node count: + for increase, for decrease. To add two nodes:
$DRILL_HOME/bin/drill-on-yarn.sh resize +2
To remove three nodes:
$DRILL_HOME/bin/drill-on-yarn.sh resize 3
Drill will add nodes only if additional nodes are available from YARN. If you request to stop more nodes than are running, Drill stops all the running nodes. Note that in the current version of Drill, stopping nodes is a forceful operation: any in-flight queries will fail.
Clean the DFS Files
If you run Drill-on-YARN for a temporary cluster, Drill will leave the Drill software archive in your designated DFS directory. You can remove those files with the following:
Specifically, the first start uploads your Drill archive to DFS. Stop leaves the archive in DFS. Subsequent start commands reuse the cached archive if it is the same size as the version on the local disk. Clean removes the cached file, forcing Drill to upload a fresh copy if you again restart the Drill cluster.