Qubole Release Notes 17-Nov-2016

Release Version: 38.1.0

For details of what has changed in this version of QDS, see What is New and List of Changes and Bug Fixes in this Release.


What is New

Spark Notebook Enhancements
There are a few enhancements in the notebook in this QDS release, which are listed below:

 

  • Qubole now supports running and scheduling Spark notebooks through a REST API call.  For more information, see this description.
  • You can also run and schedule a Spark notebook using the Analyze query composer and the Scheduler UI. For more information, see this description.
  • Deleting a notebook implicitly deletes it from its AWS S3 location.
  • Qubole has removed the create, clone, and delete links within a notebook/zeppelin’s interface but you can perform these operations using the gear box against a notebook in the Notebooks UI’s left pane.

 


UI Enhancements
Qubole has renamed the Repo tab on the Analyze page as Workspace with a few enhancements:

  • Default workspace for new users
  • Logs and results are displayed in the Workspace tab
  • Saved queries can be archived

For more information, see this description.


New Beta Features


ACL for Folders in Notebooks


Notebooks now support folders with access control list, which helps in managing access and limiting access to the desired users.


Note: Currently, Qubole allows a user to run paragraphs or change interpreter settings even without having the notebook update permission. A user can still open notebooks without read permissions from the native Notebooks UI.


Notebook folders and its access control list are available for beta access. Contact help@qubole.com to enable this feature for the account.


List of Changes and Bug Fixes in this Release

HADOOP 2


Fix   HADTWO-643: Fixed the error in S3A FileSystem while fetching credentials from roles for IAM-Roles based accounts.
Fix   HADTWO-647: Making S3A FileSystem compatible with NativeS3FileSystem for empty folders scenario.
Fix   HADTWO-688: yarn.nodemanager.resource.memory-mb is now removed from Recommended Hadoop Settings and is instead calculated and set at the instance boot time. This helps in optimum utilization of instance capacity in the heterogeneous clusters.

 

QDS


New   UI-2829 and UI-4230: Access Control List for folders in Notebook is available now.
Note: Currently, Qubole allows a user to run paragraphs or change interpreter settings even without having the notebook update permission.

However, this feature is available for beta access. Contact help@qubole.com to enable this feature for the account.

New   UI-3952: Qubole has renamed the Repo tab on the Analyze page as Workspace with a few enhancements:

  • Default workspace for new users
  • Logs and results are displayed in the Workspace tab
  • Saved queries can be archived

Fix   SQOOP-68: Qubole supports blob and binary column types while creating Hive tables.
Fix   UI-4652: The Help Center dialog will now only close when you click on the close icon on the top right corner of the dialog.
Fix   UI-4663: Fixed Table search in the Analyze page
Fix   UI-4667: Fixed an issue with the Heterogeneous Cluster option, where a user could not set a node weight with more than one decimal places.
Change   MW-98: Qubole supports running a Spark notebook as a Spark command REST API call.
Change   MW-102: Qubole has done a few enhancements in the Spark Job Server's support to notebooks.
Change   MW-128: Qubole now supports using the Scheduler API to schedule the notebook by setting the SparkCommand type and setting notebook as the language.
Change   UI-4571: New notebook command type supported as part of the Spark command in the Analyze query composer.


SPARK

Fix   SPAR-1128: With this fix, the Spark driver crashes properly when it goes out of memory in Notebooks and does not get hung.
Change   SPAR-1301: Partition scanning on Hive tables backed by Parquet/ORC files with lots of partitions are a lot faster now.


ZEPPELIN/Notebooks

Fix   ZEP-538: Faster loading of large-size notebooks by configuring an output size limit at the account level. Outputs in all clusters above the provided size will be removed and a fetch output link will be provided to users to fetch it if needed.

Change   ZEP-608: Qubole has removed Notebook’s create, clone, and delete links from the Zeppelin Interface but you can perform these operations using the gear box against a notebook in the Notebooks UI’s left pane.
Change   ZEP-613: Deleting a notebook now deletes it even from the Amazon S3 location.


List of Hotfixes Since 25th October 2016

Fix   ACM-703: Qubole has removed an AWS API call made while running a command on a running cluster. This helps commands which fail because of AWS API limits and make the commands run a little faster.
Fix   ACM-727: Qubole has upgraded Kernel in Qubole AMIs for ALAS-2016-757.
Fix   QBOL-5900: Fixed the issue related to Notebook folders when just bucket name is used as the account's default location.
New   UI-4014: Qubole has introduced folders in Notebooks. However, this feature is available for beta access. Contact help@qubole.com to enable this feature for the account.



List of Hotfixes After 17th October 2016


Fix  
ACM-762: Fixed an issue where a heterogeneous cluster could not be terminated.

Change   ZEP-489: Access Control for Notebook Folders. Folders in Notebooks is available for beta access. Contact help@qubole.com to enable this feature for the account.

 

 

Have more questions? Submit a request

Comments

Powered by Zendesk