我需要在Hive Metastore中注册Delta表,以便能够使用连接到ThriftServer的外部报告工具进行查询
PySpark API运作良好,我能够创建DeltaTable对象
ordersDeltaTable = DeltaTable.forPath(spark,delta_path)
当我运行SQL命令
%%sql
CREATE TABLE orders_delta
USING DELTA
LOCATION '/tmp/orders/delta/'
或
spark.sql("CREATE TABLE orders_delta USING delta LOCATION '/tmp/orders/delta/'")
或这个
%%sql
CREATE TABLE orders_delta
USING delta
AS SELECT *
FROM csv.`s3://orders/raw/public/order_items/`
我总是有相同的例外情况
An error was encountered:
u'Can not create a Path from an empty string'
Traceback (most recent call last):
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/session.py",line 767,in sql
return DataFrame(self._jsparkSession.sql(sqlQuery),self._wrapped)
File "/usr/lib/spark/python/lib/py4j-0.10.7-
src.zip/py4j/java_gateway.py",line 1257,in __call__
answer,self.gateway_client,self.target_id,self.name)
File "/usr/lib/spark/python/lib/pyspark.zip/pyspark/sql/utils.py",line 79,in deco
raise IllegalArgumentException(s.split(': ',1)[1],stackTrace)
IllegalArgumentException: u'Can not create a Path from an empty
string'
我正在使用
EMR 5.27.0 星火2.4.4 蜂巢2.3.5 delta-core_2.11-0.4.0.jar 使用Jupyter Notebook。
还有其他方法可以在Hive Metastore中注册DeltaTable吗?