当前位置：首页 > article >正文

Tez 0.10.1安装

article 2025/1/31 3:50:37

个人博客地址：Tez 0.10.1安装 | 一张假钞的真实世界

具体安装步骤参照官网安装手册即可。此处只对官网手册进行补充。

从官网下载apache-tez-0.10.1-bin.tar.gz进行安装未成功，出现下面的异常。最终按照官网源代码编译的方式安装测试成功。

环境

Hadoop 3.2.2
Tez 0.10.1

二进制包安装异常

异常1：

2021-08-18 18:51:15,362 [INFO] [main] |service.AbstractService|: Service org.apache.tez.dag.app.DAGAppMaster failed in state STARTED
org.apache.hadoop.service.ServiceStateException: java.lang.NoClassDefFoundError: org/eclipse/jetty/servlet/ServletContextHandler
  at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
  at org.apache.tez.dag.app.DAGAppMaster.startServices(DAGAppMaster.java:1879)
  at org.apache.tez.dag.app.DAGAppMaster.serviceStart(DAGAppMaster.java:1943)
  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
  at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2626)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:422)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1762)
  at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2622)
  at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2409)
Caused by: java.lang.NoClassDefFoundError: org/eclipse/jetty/servlet/ServletContextHandler
  at org.apache.hadoop.yarn.webapp.WebApps.$for(WebApps.java:515)
  at org.apache.hadoop.yarn.webapp.WebApps.$for(WebApps.java:521)
  at org.apache.hadoop.yarn.webapp.WebApps.$for(WebApps.java:526)
  at org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:92)
  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
  at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1800)
  at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1821)
Caused by: java.lang.ClassNotFoundException: org.eclipse.jetty.servlet.ServletContextHandler
  at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
  at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
  at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
  ... 7 more

异常2：

2021-08-18 16:18:36,244 [INFO] [main] |service.AbstractService|: Service org.apache.tez.dag.app.DAGAppMaster failed in state STARTED
org.apache.hadoop.service.ServiceStateException: java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.getSessionManager()Lorg/eclipse/jetty/server/SessionManager;
  at org.apache.hadoop.service.ServiceStateException.convert(ServiceStateException.java:105)
  at org.apache.tez.dag.app.DAGAppMaster.startServices(DAGAppMaster.java:1879)
  at org.apache.tez.dag.app.DAGAppMaster.serviceStart(DAGAppMaster.java:1943)
  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
  at org.apache.tez.dag.app.DAGAppMaster$9.run(DAGAppMaster.java:2626)
  at java.security.AccessController.doPrivileged(Native Method)
  at javax.security.auth.Subject.doAs(Subject.java:422)
  at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1729)
  at org.apache.tez.dag.app.DAGAppMaster.initAndStartAppMaster(DAGAppMaster.java:2622)
  at org.apache.tez.dag.app.DAGAppMaster.main(DAGAppMaster.java:2409)
Caused by: java.lang.NoSuchMethodError: org.eclipse.jetty.server.session.SessionHandler.getSessionManager()Lorg/eclipse/jetty/server/SessionManager;
  at org.apache.hadoop.http.HttpServer2.initializeWebServer(HttpServer2.java:569)
  at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:550)
  at org.apache.hadoop.http.HttpServer2.<init>(HttpServer2.java:117)
  at org.apache.hadoop.http.HttpServer2$Builder.build(HttpServer2.java:425)
  at org.apache.hadoop.yarn.webapp.WebApps$Builder.build(WebApps.java:341)
  at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:432)
  at org.apache.hadoop.yarn.webapp.WebApps$Builder.start(WebApps.java:428)
  at org.apache.tez.dag.app.web.WebUIService.serviceStart(WebUIService.java:94)
  at org.apache.hadoop.service.AbstractService.start(AbstractService.java:194)
  at org.apache.tez.dag.app.DAGAppMaster$ServiceWithDependency.start(DAGAppMaster.java:1800)
  at org.apache.tez.dag.app.DAGAppMaster$ServiceThread.run(DAGAppMaster.java:1821)

安装说明

pom.xml

修改以下配置与使用的Hadoop版本一致：

<hadoop.version>3.2.2</hadoop.version>

修改以下配置与${HADOOP_HOME}/share/hadoop/common/lib/guava-27.0-jre.jar版本一致。

<guava.version>27.0-jre</guava.version>

tez-site.xml

需要注意两点：

与官网不同，tez.use.cluster.hadoop-libs配置为true。
该配置文件需要分发到Hadoop集群的所有节点。位置：${HADOOP_HOME}/etc/hadoop目录下。

完整内容如下：

<?xml version="1.0" encoding="UTF-8" standalone="no"?>
<?xml-stylesheet type="text/xsl" href="configuration.xsl"?>
<configuration>
  <property>
    <name>tez.lib.uris</name>
    <value>${fs.defaultFS}/lib/tez/tez-0.10.1.tar.gz</value>
    <type>string</type>
  </property>
  <property>
    <name>tez.use.cluster.hadoop-libs</name>
    <value>true</value>
  </property>
</configuration>

查看全文

http://www.kler.cn/a/525449.html