Required field ‘client_protocol’ is unset!
beeline 出现 org.apache.thrift.TApplicationException: Required field ‘client_protocol’ is unset! Struct:TOpenSessionReq(client_protocol:null,configuration:set:hiveconf:hive.server2.thrift.resultset.default.fetch.size=1000,use:database=default})| 原值 | 替换值 |
|---|---|
| hive/bin 下的 beeline | spark/bin 下的 beeline |
spark 不使用hive的源数据表的问题
对 spark2-hive-site-override 做修改
| 原值 | 替换值 |
|---|---|
| metastore.catalog.default=spark | metastore.catalog.default=hive |
Table is marked as a managed table but is not transactional.
AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: MetaException(message:Table default.partition_test failed strict managed table checks due to the following reason: Table is marked as a managed table but is not transactional.)对hive-site.xml做修改
| 原值 | 替换值 |
|---|---|
| hive.strict.managed.tables=true | hive.strict.managed.tables=false |
file is not owned by hive and load data is also not ran as hive
org.apache.spark.sql.AnalysisException: org.apache.hadoop.hive.ql.metadata.HiveException: org.apache.hadoop.hive.ql.metadata.HiveException: Load Data failed for hdfs://***:8020/warehouse/tablespace/managed/hive/***/.hive-staging_hive_2019-07-02_18-17-08_028_419193115114639265-1/-ext-10000/part-00000-1f0e8f19-6a12-448f-ba18-a2319711c0aa-c000 as the file is not owned by hive and load data is also not ran as hive;spark hive-site.xml 添加 hive.load.data.owner=spark (具体执行用户)
AnalysisException: java.lang.NullPointerException: null
org.apache.spark.sql.AnalysisException: java.lang.NullPointerException: null;at org.apache.spark.sql.hive.HiveExternalCatalog.withClient(HiveExternalCatalog.scala:106)spark sql 不支持 hive 的 OrcInputFormat 格式
