Qubole Release Notes - 29-Oct-2015

Spark Job Server API

Spark Job Server API. Now users can submit Spark snippets to already running Spark apps.

Non-default Schema Support for Postgres and Redshift Data Stores

Qubole now supports exploring non-default schema in addition to the default schema in Postgres and Redshift data stores. See this description for more information.

Other minor changes and bug fixes are as listed below.

 

HADOOP

  • HIVE-715: Added support for bulk deletion of S3 files during insert overwrites.

 

HADOOP-2

  • MUL-132: Added support for Hadoop2 on GCE
  • HIVE-715: Added support for bulk deletion of S3 files during insert overwrites.

 

HIVE-13

  • HIVE-715: Added support for bulk deletion of S3 files during insert overwrites.
  • HIVE-1078: Just logging the exception message while calculating md5sum for avro schema url
  • HIVE-1013: Include hive-jdbc and beeline jars
  • HIVE-1099: Removing unnecessary references to qubole-hive from cli's pom
  • HIVE-923: Change the “decimal” column type to decimal(10,4)
  • HIVE-792: Function to know the current version of Hive

PIG-11

  • Removes HBase jars from pig11/bin

 

PRESTO

  • PRES-356: Make max cluster size pushable
  • PRES-489: Show views from MySQL and prestogres in ‘SHOW TABLES’ query
  • PRES-478: Support multiple volumes for caching via cluster configuration
  • Use lesser memory for each group in array_agg during initialization to support high cardinality GBY

 

QDS

  • UI-2459: Notebook Integration Phase I (beta)
  • SCHED-70: Support Fair Scheduler in Qubole Scheduler
  • UI-2288: Whitelist controls in control panel
  • HIVE-1028: Hive on cluster master capability and related fixes
  • Reporting Qubole Metrics on OpenTSDB/SignalFX
  • Send cluster node status to tapp using ops api for HBASE.
  • UI-2414: Show the corresponding command ID with an executed schedule job
  • UI-1208: Add support for feedback option under help widget
  • Displaying node status for hbase clusters
  • QBOL-4741: Fixing cpl for roles enabled accounts
  • Ability to have spare dj slots
  • UI-2571: Issue with dbimport not selecting dbtap table name for permalinks
  • UI-2039: Fix for showing appropriate message on Canonical hive reports page
  • QBOL-4709: Cleanup sessions for user terminated clusters
  • HIVE-1138: Picking a hive-tier node should not look at the hive-version
  • UI-2532: Start button on cluster does not work after listing all clusters
  • QBOL-4236: Roles groups support for saved queries.
  • Reading the compute secret key from original cluster if not provided in the API (only for aws role accounts)
  • UI-2494: Show secret key for only qubole managed storage
  • QBOL-3617: Adding support for macros in Pig commands
  • UI-2332: Templates - Fix loading icon keeps on showing
  • UI-2345 Templates - The table in instances page is not formatted properly
  • UI-2491: Show a warning when changing defloc
  • QBOL-3731: Adding support of macros in shell commands
  • Changes to support Qubole Stack in eu-central region
  • UI-2348: Templates: History TAB functioning is not consistent
  • Fix unit tests to reflect hbase snapshots executor change
  • Add support for special characters in custom EC2 tagsƒ
  • Reading the compute secret key from original cluster if not provided in the API (only for AWS role accounts)
  • ACM-16: Fix for connectivity issue with VPC cluster from Qubole tier
  • Removed bug so that refreshing manage/backup cluster page should redirect to the same page.
  • Using Array#join instead of to_s for ruby2 compatibility
  • QBOL-4584: Fix issue with 500 error while deleting dbtab
  • UI-2067: Add non default schema support for Redshift and Postgres Data Stores in Explore
  • UI-1158: Scheduler - While creating a clone job, Scheduler name appears as parent name.
  • UI-2359: Scheduler - Fix rerunning instance message is shown in yellow and is cluttered
  • UI-2428: Fixed account name overlapping
  • UI-2333: Templates - Clone/kill/edit toolbar is misaligned fix
  • UI-2376: Account clone - Account name input field does not get clear, after cloning is done.
  • UI-2341: Templates - The cluster label name is cut off fix
  • Fixes the failure when trying to update cluster settings by label.
  • QBOL-4590: Exception handling in file move
  • Fix for sending correct private-dns value and showing grey icon for NULL status
  • PRES-252: Support custom metastores with non-default ports in Presto
  • QBOL-4554: fixes cluster_config returning different results for cluster id and cluster label.
  • UI-1378: Explore - Refresh option remembers last selected items
  • NEZ-29: Add APIs to access Nezha Metadata.
  • Make HBase Backup/restore executable anywhere
  • Deprecating the use of update-tracker code in getlog flow
  • SPAR-577: Create, Clone, Edit, Delete of Spark notes  
  • QBOL-4377: Performance improvement in fetching results in UI
  • Collecting metrics after query hist is marked as done to make small queries faster
  • QPIG-21: IAM Roles support for pig on hadoop1

 

SPARK

  • SPAR-740: Remove extra jars from spark-examples jar.
  • Allow S3 package name to be specified for any given spark version.
  • SPAR-712: Avoid spark hang by calling exception exit hook.
  • SPAR-475: Kill application in accepted state when stop() is called in driver.
  • SPAR-658: Add kinesis-assembly into list of modules.
  • SPAR-659: Use open-source hive-1.2 for spark-1.5.x. Fix classpath issues related to hive.
  • SPAR-694: Fix ganglia for spark-1.3.1
  • ACM-25: Increase retries for clusters during cluster start.
  • SPAR-646: Avoid messing around with classpath at multiple places and only use the SPARK_DIST_CLASSPATH to control it.
  • SPAR-546:Remove jets3 from kafka assembly
  • SPAR-502: Create kinesis assembly

 

 ZEPPELIN

  • Notebooks as First Class Citizens in QDS
  • SPAR-589 Fix the parquet issue with zeppelin running on Spark 1.4.0
Have more questions? Submit a request

Comments

Powered by Zendesk