If you are configuring an Airflow cluster in QDS, here are some things to check while troubleshooting:
Airflow installation - Airflow is installed inside a virtual environment location at “/usr/lib/python27/virtualenv”. Airflow requires python 2.7 which is available only in this virtualenv inside Qubole Cluster AMIs. Activate the virtualenv before invoking the airflow cli.
Location of service logs - Logs of Airflow services e.g scheduler, webserver, celery etc. are available at “/media/ephemeral0/logs/airflow”. These services are instantiated during cluster bringup, so these logs can be referred for any troubleshooting while bringing up the cluster.
Airflow Home - An environment variable $AIRFLOW_HOME is permanently set to “/usr/lib/airflow” for all machine users. Airflow configuration file (airflow.cfg), dags ( “/dags” folder), logs (“ /logs” folder) are present inside AIRFLOW_HOME folder. Please note that logs of the jobs triggered by airflow are available at $AIRFLOW_HOME/logs
- Restarting Airflow scheduler -
- Become root - sudo su
- Activate Virtualenv - source /usr/lib/virtualenv/python27/bin/activate
- Stop Scheduler Process - /usr/lib/hustler/bin/airflow/monit.sh stop /usr/lib/airflow/scheduler.pid
This may give error if process does not exist
- Start Scheduler Process - /usr/lib/hustler/bin/airflow/monit.sh start /usr/lib/airflow/scheduler.pid