python与hive通信交互
一、python与hive1通信
#!/usr/bin/python2.7 #hive --service hiveserver >/dev/null 2>/dev/null& #/usr/lib/hive/lib/py import sys from hive_service import ThriftHive from hive_service.ttypes import HiveServerException from thrift import Thrift from thrift.transport import TSocket from thrift.transport import TTransport from thrift.protocol import TBinaryProtocol def hiveExe(sql): try: transport = TSocket.TSocket('*.*.*.*', 10000) transport = TTransport.TBufferedTransport(transport) protocol = TBinaryProtocol.TBinaryProtocol(transport) client = ThriftHive.Client(protocol) transport.open() client.execute(sql) print "The return value is : " print client.fetchAll() print "............" transport.close() except Thrift.TException, tx: print '%s' % (tx.message) if __name__ == '__main__': hiveExe("select * from project.table")
python和hive1通讯是一件非常容易的事情,因为python所需要的依赖包从/usr/lib/hive/lib/py获取即可,导入到hive的扩张中就可以应用了。
二、python与hive2的通信
#!/usr/bin/python2.7 #hive --service hiveserver2 >/dev/null 2>/dev/null& #install pyhs2,first install cyrus-sasl-devel,gcc,libxml2-devel,libxslt-devel #hiveserver2 is different from hiveserver on authority import pyhs2 conn = pyhs2.connect(host='*.*.*.*',port=10000,authMechanism="PLAIN", user='hive', password='', database='project') cur = conn.cursor() cur.execute("select * from table limit 10") for i in cur.fetch(): print i cur.close() conn.close()
python与hive2通信比较费劲,需要安装的依赖比较多(install pyhs2,first install cyrus-sasl-devel,gcc,libxml2-devel,libxslt-devel)。但是安装完成后编程还是很容易的。
两种通讯有一个共同点,就是必须启动hive服务器。
相关推荐
archive 2020-07-30
成长之路 2020-07-28
eternityzzy 2020-07-19
taisenki 2020-07-05
tugangkai 2020-07-05
SignalDu 2020-07-05
zlsdmx 2020-07-05
tomson 2020-07-05
tugangkai 2020-07-04
tomson 2020-07-05
Zhangdragonfly 2020-06-28
genshengxiao 2020-06-26
成长之路 2020-06-26
tomson 2020-06-26
蜗牛之窝 2020-06-26
成长之路 2020-06-25