Release Version: 50.6.0
Qubole provides Qubole Data Service (QDS) on Amazon Web Service (AWS), Microsoft Azure, and Oracle Cloud Infrastructure (OCI).
For details of what has changed in this version of QDS, see:
- What is New in QDS-on-Microsoft Azure
- List of Changes and Bug Fixes in QDS-on-Azure
What is New in QDS-on-Microsoft Azure
QDS-on-Microsoft Azure Supports Airflow
A new cluster type, Airflow is supported on QDS with 1.82 as the only version supported.
New Enhancements in Hive
This is a Hive enhancement in this release:
- QDS UI supports adding a personal hive bootstrap in Control Panel > Hive Bootstrap. For more information, see the documentation.
New Enhancements in Notebooks
These are the new enhancements in the notebooks:
- Simplified Spark interpreter properties by removing unnecessary default interpreter properties. This feature is reflected only on new Spark interpreters and is enabled by default.
New Enhancements in Dashboards
- Dashboards are generally available with this release. For more information, see the documentation.
- A user can change the Dashboard theme.
- A user can rename an open notebook or a dashboard from its UI header.
- QDS provides Scheduler support on the Dashboards UI.
New Enhancements in UI
These are the new enhancements in the UI:
- The navigation to the main pages that is the drop-down of the QDS UI main page along with the UI skin color of the top horizontal bar have changed in this release.
- QDS UI supports adding a personal hive bootstrap in Control Panel > Hive Bootstrap.
Qubole Supports Package Management on Spark Clusters
QDS UI supports creating an environment to easily install and upgrade R and Python packages on Spark clusters. It also supports seamless installation/upgrade of packages on a running Spark cluster. This feature is not enabled by default. Create a ticket with Qubole Support to enable this on your account. For more information, see this documentation.
List of Changes and Bug Fixes in QDS-on-Microsoft Azure
Change AIR-52: Airflow version 1.82 is supported on Azure now. It is a new cluster type that is supported on QDS-on-Azure.
Azure Cluster Management
Change ACM-1907: The cluster start-sequence is now more resilient to delays from the cloud provider in setting a hostname.
Change AD-316: QDS UI supports adding a personal hive bootstrap in Control Panel > Hive Bootstrap.
Change PRES-1371: The Postgres jar version has been downgraded back to 9.3-1102-jdbc41 in 0.180 to maintain compatibility with Redshift.
Change MUL-585: QDS-on-Azure supports creating custom roles for more restrictive IAM policy.
Change UI-5383: The navigation to the main pages that is the drop-down of the QDS UI main page along with the UI skin color of the top horizontal bar have changed in this release.
Change SPAR-2077: The Package Management UI feature is supported as a beta feature on Azure. Create a ticket with Qubole Support to enable this feature on the QDS account.
Change SPAR-2078: Proper formatting is done in the Spark History Server response in case the application that is looked for does not exist.
Fix ZEP-581: Fixed the issue in which it was unable to highlight the comments and code formatting for paragraphs in pyspark.
Fix ZEP-601: Fixed the issue in which pyspark syntax highlighting was not working (instead it seemed to use scala).
Change ZEP-1488: QDS provides Scheduler support on the Dashboards UI.
Change ZEP-1552: This fixes issues such as interpreter.json getting malformed and interpreters/repositories getting lost by making the Zeppelin service come up early.
Change ZEP-1700: A user can view a notebook/dashboard information by hovering over the info icon on the side panel.