Qubole Release Notes for QDS-on-Azure Version R49 30-Nov-2017

Release Version: 49.2.0

Qubole provides Qubole Data Service (QDS) on Amazon Web Service (AWS), Microsoft Azure, and Oracle Cloud Infrastructure (OCI).

For details of what has changed in this version of QDS, see:

  • What is New in QDS-on-Microsoft Azure
  • List of Changes and Bug Fixes in QDS-on-Azure


What is New in QDS-on-Microsoft Azure

New Enhancements in Presto

These are the new enhancements in Presto:

  • The default value of query.max-memory has been changed to 100TB. If you want to limit the maximum memory a query can use, you can override this to a lower value either by setting this as a Presto override or setting the query_max_memory session property.
  • QDS supports ADLS on Presto clusters that are on 0.150 and 0.180 versions.


New Enhancements in Spark

This is a new enhancement in Spark:

  • Loading Hive tables from Spark is more robust to eventual consistency errors generally expected from cloud object stores. It can be enabled per job by passing the following command line option:

        --conf spark.hadoop.hive.qubole.consistent.loadpartition=true

 

New Enhancements in Notebooks

These are the new enhancements in Notebooks:

  • A new option to configure Notebook Dashboards has been added.
  • You can use the TAB key to autocomplete non-markdown paragraphs in a notebook.
  • QDS displays the opened Dashboard in the browser tab. 

 

List of Changes and Bug Fixes in QDS-on-Microsoft Azure

Azure Cluster Management

 

Bug Fixes 

Fix   ACM-1424: A timeout has been added while deleting resources of Azure so that processes do not wait indefinitely.

 

PRESTO

 

Bug Fixes 

Fix    PRES-1284: QDS supports ADLS on Presto clusters that are on 0.150 and 0.180 versions.

 

Enhancements

Change    PRES-735: The default value of query.max-memory has been changed to 100TB. If you want to limit the maximum memory a query can use, you can override this to a lower value either by setting this as a Presto override or setting the query_max_memory session property.


SCHEDULER

 

Enhancements

Change    SCHED-142: The issue in which the scheduler notifications used to have 10-12 minutes delivery time has been resolved. Now, the notification should be delivered within 5 minutes.
Change    EAM-286: Create Public Preview Query API has been added to a new Scheduler API call. After entering the parameters in a POST Scheduler API call, you can view the preview of it by using the endpoint https://azure.qubole.com/api/v1.2/scheduler/preview_macro.

 

SPARK

 

Bug Fixes 

Fix    SPAR-1640: The issue in which drop database/table was failing with Error getting object from metastore for Object has been resolved.
Fix    SPAR-1656: With Spark Hive Authorization, the issue in which a user could create a view from a table on which the user does not have any privileges, has been fixed.
Fix    SPAR-2033: The issue in which Spark eventlogs were not being synced to object store in the cloud has been resolved.

 

Enhancements

Change    HIVE-2020: Loading Hive tables from Spark is more robust to eventual consistency errors generally expected from cloud object stores. It can be enabled per job by passing the following command line option:

--conf spark.hadoop.hive.qubole.consistent.loadpartition=true

 

ZEPPELIN/NOTEBOOKS

 

New Features

New     ZEP-1608: A new option to configure Notebook Dashboards has been added.

 

Bug Fixes 

Fix    ZEP-39: A detailed progress of Spark paragraphs run on notebooks, will be offered when zeppelin.enable_detailed_spark_progress is set to true. This feature is enabled by default.


Enhancements

Change    ZEP-405: You can use the TAB key to autocomplete non-markdown paragraphs in a notebook.
Change    ZEP-1607: QDS displays the opened Dashboard in the browser tab.

Have more questions? Submit a request

Comments

Powered by Zendesk