创新互联ApachePig教程：ApachePigExplain运算符
explain 运算符用于显示关系的逻辑，物理和MapReduce执行计划。
成都创新互联是一家集网站建设,蒙自企业网站建设,蒙自品牌网站建设,网站定制,蒙自网站建设报价,网络营销,网络优化,蒙自网站推广为一体的创新建站企业，帮助传统企业提升企业形象加强企业竞争力。可充分满足这一群体相比中小企业更为丰富、高端、多元的互联网需求。同时我们时刻保持专业、时尚、前沿，时刻以成就客户成长自我，坚持不断学习、思考、沉淀、净化自己，让我们为更多的企业打造出实用型网站。
语法
下面给出了 explain 运算符的语法。
grunt> explain Relation_name;
例
假设在HDFS中有一个包含以下内容的文件 student_data.txt 。
001,Rajiv,Reddy,9848022337,Hyderabad 002,siddarth,Battacharya,9848022338,Kolkata 003,Rajesh,Khanna,9848022339,Delhi 004,Preethi,Agarwal,9848022330,Pune 005,Trupthi,Mohanthy,9848022336,Bhuwaneshwar 006,Archana,Mishra,9848022335,Chennai.
使用LOAD运算符将它读入关系 student ，如下所示。
grunt> student = LOAD 'hdfs://localhost:9000/pig_data/student_data.txt' USING PigStorage(',')
as ( id:int, firstname:chararray, lastname:chararray, phone:chararray, city:chararray );
现在，让我们使用 explain 运算符解释名为student的关系，如下所示。
grunt> explain student;
输出
它将产生以下输出。
$ explain student;
2015-10-05 11:32:43,660 [main]
2015-10-05 11:32:43,660 [main] INFO org.apache.pig.newplan.logical.optimizer
.LogicalPlanOptimizer -
{RULES_ENABLED=[AddForEach, ColumnMapKeyPrune, ConstantCalculator,
GroupByConstParallelSetter, LimitOptimizer, LoadTypeCastInserter, MergeFilter,
MergeForEach, PartitionFilterOptimizer, PredicatePushdownOptimizer,
PushDownForEachFlatten, PushUpFilter, SplitFilter, StreamTypeCastInserter]}
#-----------------------------------------------
# New Logical Plan:
#-----------------------------------------------
student: (Name: LOStore Schema:
id#31:int,firstname#32:chararray,lastname#33:chararray,phone#34:chararray,city#
35:chararray)
|
|---student: (Name: LOForEach Schema:
id#31:int,firstname#32:chararray,lastname#33:chararray,phone#34:chararray,city#
35:chararray)
| |
| (Name: LOGenerate[false,false,false,false,false] Schema:
id#31:int,firstname#32:chararray,lastname#33:chararray,phone#34:chararray,city#
35:chararray)ColumnPrune:InputUids=[34, 35, 32, 33,
31]ColumnPrune:OutputUids=[34, 35, 32, 33, 31]
| | |
| | (Name: Cast Type: int Uid: 31)
| | | | | |---id:(Name: Project Type: bytearray Uid: 31 Input: 0 Column: (*))
| | |
| | (Name: Cast Type: chararray Uid: 32)
| | |
| | |---firstname:(Name: Project Type: bytearray Uid: 32 Input: 1
Column: (*))
| | |
| | (Name: Cast Type: chararray Uid: 33)
| | |
| | |---lastname:(Name: Project Type: bytearray Uid: 33 Input: 2
Column: (*))
| | |
| | (Name: Cast Type: chararray Uid: 34)
| | |
| | |---phone:(Name: Project Type: bytearray Uid: 34 Input: 3 Column:
(*))
| | |
| | (Name: Cast Type: chararray Uid: 35)
| | |
| | |---city:(Name: Project Type: bytearray Uid: 35 Input: 4 Column:
(*))
| |
| |---(Name: LOInnerLoad[0] Schema: id#31:bytearray)
| |
| |---(Name: LOInnerLoad[1] Schema: firstname#32:bytearray)
| |
| |---(Name: LOInnerLoad[2] Schema: lastname#33:bytearray)
| |
| |---(Name: LOInnerLoad[3] Schema: phone#34:bytearray)
| |
| |---(Name: LOInnerLoad[4] Schema: city#35:bytearray)
|
|---student: (Name: LOLoad Schema:
id#31:bytearray,firstname#32:bytearray,lastname#33:bytearray,phone#34:bytearray
,city#35:bytearray)RequiredFields:null
#-----------------------------------------------
# Physical Plan: #-----------------------------------------------
student: Store(fakefile:org.apache.pig.builtin.PigStorage) - scope-36
|
|---student: New For Each(false,false,false,false,false)[bag] - scope-35
| |
| Cast[int] - scope-21
| |
| |---Project[bytearray][0] - scope-20
| |
| Cast[chararray] - scope-24
| |
| |---Project[bytearray][1] - scope-23
| |
| Cast[chararray] - scope-27
| |
| |---Project[bytearray][2] - scope-26
| |
| Cast[chararray] - scope-30
| |
| |---Project[bytearray][3] - scope-29
| |
| Cast[chararray] - scope-33
| |
| |---Project[bytearray][4] - scope-32
|
|---student: Load(hdfs://localhost:9000/pig_data/student_data.txt:PigStorage(',')) - scope19
2015-10-05 11:32:43,682 [main]
INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MRCompiler -
File concatenation threshold: 100 optimistic? false
2015-10-05 11:32:43,684 [main]
INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.MultiQueryOp timizer -
MR plan size before optimization: 1 2015-10-05 11:32:43,685 [main]
INFO org.apache.pig.backend.hadoop.executionengine.mapReduceLayer.
MultiQueryOp timizer - MR plan size after optimization: 1
#--------------------------------------------------
# Map Reduce Plan
#--------------------------------------------------
MapReduce node scope-37
Map Plan
student: Store(fakefile:org.apache.pig.builtin.PigStorage) - scope-36
|
|---student: New For Each(false,false,false,false,false)[bag] - scope-35
| |
| Cast[int] - scope-21
| |
| |---Project[bytearray][0] - scope-20
| |
| Cast[chararray] - scope-24
| |
| |---Project[bytearray][1] - scope-23
| |
| Cast[chararray] - scope-27
| |
| |---Project[bytearray][2] - scope-26
| |
| Cast[chararray] - scope-30
| |
| |---Project[bytearray][3] - scope-29
| |
| Cast[chararray] - scope-33
| |
| |---Project[bytearray][4] - scope-32
|
|---student:
Load(hdfs://localhost:9000/pig_data/student_data.txt:PigStorage(',')) - scope
19-------- Global sort: false
----------------
分享文章：创新互联ApachePig教程：ApachePigExplain运算符
文章起源：http://www.shufengxianlan.com/qtweb/news2/405502.html
网站建设、网络推广公司-创新互联，是专注品牌与效果的网站制作，网络营销seo公司；服务项目有等
声明：本网站发布的内容（图片、视频和文字）以用户投稿、用户转载内容为主，如果涉及侵权请尽快告知，我们将会在第一时间删除。文章观点不代表本网站立场，如需处理请联系客服。电话：028-86922220；邮箱：631063699@qq.com。内容未经允许不得转载，或转载时需注明来源： 创新互联