码迷,mamicode.com
首页 > 其他好文 > 详细

搭建部署Hadoop 之Yarn

时间:2018-01-31 22:03:30      阅读:442      评论:0      收藏:0      [点我收藏+]

标签:session   bst   bzip   like   read   执行   hid   prepare   mpi   

Yarn 集群资源管理系统


Yarn 角色及概念

?Yarn 是 Hadoop 的一个通用的资源管理系统

? Yarn 角色

    – Resourcemanager

    – Nodemanager


    – ApplicationMaster

    – Container

    – Client

? ResourceManager

    – 处理客户端请求

    – 启动 / 监控 ApplicationMaster

    – 监控 NodeManager

    – 资源分配与调度

? NodeManager

    – 单个节点上的资源管理

    – 处理来自 ResourceManager 的命令

    – 处理来自 ApplicationMaster 的命令

? Container

    – 对任务运行行环境的抽象,封装了 CPU 、内存等

    – 多维资源以及环境变量、启动命令等任务运行相关的信息资源分配与调度

? ApplicationMaster

    – 数据切分

    – 为应用程序申请资源,并分配给内部任务

    – 任务监控与容错

? Client

    – 用户与 YARN 交互的客户端程序

    – 提交应用程序、监控应用程序状态,杀死应用程序等

Yarn 结构

技术分享图片



? YARN 的核心思想

? 将 JobTracker 和 TaskTacker 进行分离,它由下面几大构成组件:

    – ResourceManager 一个全局的资源管理器

    – NodeManager 每个节点(RM)代理

    – ApplicationMaster 表示每个应用

    – 每一个 ApplicationMaster 有多个 Container 在NodeManager 上运行


系统规划

主机                                      角色                                    软件

192.168.4.1    master       Resource Manager         YARN

192.168.4.2    node1        Node Manager                YARN

192.168.4.3    node2        Node Manager                YARN

192.168.4.4    node3        Node Manager                YARN


Yarn 安装与配置        

具体实验准备 可以参考 http://blog.51cto.com/13558754/2066708


# ssh 192.168.4.1

# cd /usr/local/hadoop/

# cd etc/hadoop/

# cp mapred-site.xml.template mapred-site.xml

# vim mapred-site.xml

<configuration>

  <property>

    <name>mapreduce.framework.name</name>

    <value>yarn</value>                        // 配置使用yarn 资源管理系统

  </property>

</configuration>


# vim yarn-site.xml

<configuration>

  <property>

    <name>yarn.resourcemanager.hostname</name>

    <value>master</value>                //配置 Resource Manager 角色 

  </property>

  <property>

    <name>yarn.nodemanager.aux-services</name>

    <value>mapreduce_shuffle</value>            // 一个 java 的类 真实环境与开发人员沟通

  </property>

</configuration>

配置完成以后

# for i in node{1..3}                    //将配置文件同步到所有主机

> do

> rsync -azSH --delete /usr/local/hadoop/etc/hadoop/ ${i}:/usr/local/hadoop/etc/hadoop -e 'ssh'

> done


# cd /usr/local/hadoop/

启动 yarn 服务

# ./sbin/start-yarn.sh 

在所有主机上执行 jps, 查看是否启动成功

# for i in master node{1..3}

> do

> echo ${i}

> ssh ${i} "jps"

> done

master

3312 Jps

3005 ResourceManager

node1

3284 Jps

3162 NodeManager

node2

2882 NodeManager

3004 Jps

node3

2961 Jps

2831 NodeManager

 显示所有可用的计算节点

# ./bin/yarn node -list

18/01/31 06:41:56 INFO client.RMProxy: Connecting to ResourceManager at master/192.168.4.1:8032

Total Nodes:3

         Node-Id     Node-StateNode-Http-AddressNumber-of-Running-Containers

     node3:46007        RUNNING       node3:8042                           0

     node2:54895        RUNNING       node2:8042                           0

     node1:51087        RUNNING       node1:8042  


resourcemanager

技术分享图片

nodemangager

技术分享图片

验证 Yarn

 

# bin/hadoop fs -ls /input

Found 3 items

-rw-r--r--   2 root supergroup      84854 2018-01-29 21:37 /input/LICENSE.txt

-rw-r--r--   2 root supergroup      14978 2018-01-29 21:37 /input/NOTICE.txt

-rw-r--r--   2 root supergroup       1366 2018-01-29 21:37 /input/README.txt

使用yarn 统计 样本文件中单词出现频率

# ./bin/hadoop jar share/hadoop/mapreduce/hadoop-mapreduce-examples-2.7.3.jar wordcount hdfs://master:9000/input  hdfs://master:9000/output

查看结果

# ./bin/hadoop fs -cat hdfs://master:9000/output/*

""AS	2
"AS	17
"COPYRIGHTS	1
"Contribution"	2
"Contributor"	2
"Derivative	1
"GCC	1
"Legal	1
"License"	1
"License");	2
"Licensed	1
"Licensor"	1
"Losses")	1
"NOTICE"	1
"Not	1
"Object"	1
"Program"	1
"Recipient"	1
"Software"),	5
"Source"	1
"Work"	1
"You"	1
"Your")	1
"[]"	1
"control"	1
"printed	1
"submitted"	1
&	1
'Aalto	1
'Apache	4
'ArrayDeque',	1
'Bouncy	1
'Caliper',	1
'Compress-LZF',	1
'HPACK',	1
'JBoss	1
'JCTools',	1
'JZlib',	1
'Protocol	1
'SLF4J',	1
'Snappy',	1
'Webbit',	1
'jbzip2',	1
'jfastlz',	1
'libdivsufsort',	1
'license'	1
'lz4',	1
'lzma-java',	1
'originates'	1
("AGREEMENT").	1
("Commercial	1
("Indemnified	1
(1)	3
(1) Modifications	1
(1) for	2
(2)	3
(2) for	2
(2) the	1
(3)	1
(3) under	1
(50%)	3
(ASL2	1
(Apache	11
(BIS),	1
(BSD	2
(C)	3
(CDDL)	1
(CDDL)Version	1
(Don't	1
(ECCN)	1
(GNU	1
(INCLUDING	15
(INCLUDING,	11
(June	2
(MIT	5
(MIT)	1
(NOT	2
(New	3
(Oct.	2
(Public	4
(Sept.	2
(TSU)	1
(We	1
(a)	11
(a) rename	1
(a) the	1
(an	1
(and	3
(as	2
(b)	13
(b) otherwise	1
(b) ownership	1
(based	1
(c)	18
(collectively	1
(d)	5
(except	7
(excluding	5
(http://code.google.com/p/leveldb/),	1
(http://code.google.com/p/snappy/),	1
(http://www.apache.org/).	2
(http://www.one-lab.org)	1
(http://www.opensource.org/licenses/bsd-license.php)	1
(https://github.com/dain/leveldb),	1
(https://github.com/fusesource/hawtjni),	1
(https://github.com/fusesource/leveldbjni),	1
(hypertext	1
(i)	4
(i) the	1
(i) third	1
(if	4
(ii)	4
(ii) the	2
(iii)	1
(including	7
(kevina@gnu.org)	1
(meaning	2
(or	17
(other	4
(see	1
(see:	1
(such	2
(the	10
*	174
*/	3
+	2
,	1
-	16
------	8
---------------------------------------------------------------------	1
-------------------------------------------------------------------------------	1
--------------------------------------------------------------------------------	14
/*	1
/**	2
0.1.42	1
0.52	1
0.9.94	1
034819	1
1	1
1.	5
1.0	4
1.0.	1
1.0.4.1,	1
1.1	1
1.1.	1
1.1. Contributor means	1
1.10.	1
1.10. Original	1
1.11.	1
1.11. Patent	1
1.12.	1
1.12. Source	1
1.13.	1
1.13. You	1
1.2.	1
1.2.17,	1
1.2. Contributor	1
1.3	1
1.3.	1
1.3.04	1
1.3. Covered	1
1.4,	1
1.4.	1
1.4. Executable means	1
1.5.	1
1.5. Initial	1
1.6.	1
1.6. Larger	1
1.7.	1
1.7.10	1
1.7. License means	1
1.8	1
1.8.	1
1.8.5	1
1.8. Licensable means	1
1.9	1
1.9.	1
1.9. Modifications means	1
1. Definitions. 	1
10.	2
12.212	3
1995),	4
1995).	2
1995-2000	1
1995-2016	1
1996	1
1999,	2
1999-2007	1
1999.	1
2(a)	1
2(b)	2
2-Clause	1
2.	6
2.0	5
2.0)	10
2.0,	2
2.0.0	1
2.1	3
2.1(a)	1
2.1(b)	1
2.1)	1
2.1.	2
2.101	1
2.2	3
2.2(a)	1
2.2(b)	3
2.2.	2
2.2.2	1
2.2.3	1
2.2.4,	1
2.3	1
2.5	1
2.5.0	1
2.6,	1
2.9.1,	1
2001	1
2001-2016,	1
2002	2
2002-2012	1
2004	2
2005,	1
2005-2006	1
2008	1
2008,2009,2010	1
2008-2016,	1
2010	1
2011	4
2011,	1
2011-2014,	1
2014	2
2014,	1
227.7202-1	2
227.7202-4	2
252.227-7014(a)(1))	1
3-clause	1
3.	4
3.0.0	1
3.1	4
3.1.	2
3.2	1
3.2.	2
3.3.	2
3.4.	2
3.5.	2
3.6.	2
30	2
4(d)	1
4.	4
4.1.	2
4.11	1
4.2.	2
4.3,	2
4.3.	2
48	8
48 C.F.R. 12.212	1
48 C.F.R. 2.101	1
5.	4
5D002.C.1,	1
6.	4
6.1	1
6.1.	2
6.1.26,	1
6.2	2
6.2.	2
6.3.	2
6.4.	1
60	6
7.	4
740.13)	1
8.	3
9	1
9.	3
:	3
<COPYRIGHT	1
<dain@iq80.com>	1
<http://www.wassenaar.org/>	1
<organization>	1
<provos@citi.umich.edu>	1
==	6
====	2
=================	1
==============================================================	2
=========================================================================	2
===============================================================================	1
A	22
A.	2
ACCEPTANCE	1
ACCOMPANYING	1
ACTION	5
ADVISED	12
AGREEMENT,	2
AGREEMENT.	1
AIX/POWER,	1
ALL	3
ALLOW	2
ALPN	2
AN	10
AND	63
ANY	94
APACHE	1
API	4
API,	2
APIs	1
APIs'	1
APPENDIX:	1
APPLICABLE	2
APPLY	4
ARE	11
ARISING	17
AS	5
ASL2	4
ASL2.	1
ASM	1
ASSUME	2
AUTHOR	2
AUTHORIZED	2
AUTHORS	5
AbstractNodeQueue,	1
Accepting	1
Additional	3
Administration	1
Adrien	1
Agreement	21
Agreement,	7
Agreement.	3
Agreement;	2
Aki	1
Alec	1
Aleksander	1
Alexander	1
All	12
Also,	1
America.	1
And	1
Any	13
Apache	20
Appendix	1
Application	2
April	1
As	3
Assorted	1
Atkinson	1
Atomicops	2
Authors.	1
Availability	2
B.	2
BASIS,	6
BE	18
BEEN	2
BEFORE	1
BIS	1
BSD	6
BSD-style	1
BUSINESS	11
BUT	27
BY	11
Base64	1
Based	1
Bay	2
Beams	1
Beider	1
Bloch	1
Bloomberg	1
Bozhanov	1
Broeke,	1
Buffer	2
Buffers	1
Buffers',	1
Bureau	1
Burrows-Wheeler	1
By	1
C	1
C.	2
C.F.R.	8
CAUSED	12
CDDL	3
CHARACTER	2
CIRCUMSTANCES	2
CLAIM,	5
CLAIMS.	2
COMMERCIAL	3
COMMON	2
COMPUTER	2
CONDITIONS	7
CONNECTION	5
CONSEQUENTIAL	16
CONSTITUTES	3
CONTRACT,	19
CONTRIBUTOR)	2
CONTRIBUTOR,	2
CONTRIBUTORS	20
COPYRIGHT	22
CORRECTION.	2
COST	2
COVERED	12
CRC	1
Castle	1
Catholique	1
Chris	1
Claims	6
Claims means	1
Claims”	1
Code	17
Code.	3
Code. 	1
Codec	1
Code means	1
Code”	1
Collections	1
Collet.	1
Commerce,	1
Commercial	10
Commission	1
Commodity	1
Commons	5
Components	1
Concurrency	1
Conditioned	4
Consistent	2
Consulting	2
Container	2
Contracts	2
Contribution	10
Contribution(s)	3
Contribution,	2
Contribution.	2
Contribution."	1
Contributions	3
Contributions)	3
Contributions.	2
Contributor	91
Contributor")	2
Contributor's	2
Contributor,	6
Contributor.	8
Contributor. 	2
Contributor:	1
Contributors	9
Contributors,	2
Contributors.	2
Control	1
Convention	2
Copyright	26
Core	4
Corp.	4
Corporation	2
Corporation.,	1
Covered	46
Crypto	1
D3	1
DAMAGE.	11
DAMAGES	23
DAMAGES,	2
DAMAGES.	3
DATA,	11
DEALINGS	5
DEATH	2
DEFECTIVE	2
DEFECTS,	2
DEFINITIONS	1
DEVELOPER	2
DEVELOPER,	2
DEVELOPMENT	3
DFAR,	2
DIRECT,	12
DISCLAIMED.	11
DISCLAIMER	5
DISCLAIMER.	1
DISCLAIMER. 	1
DISTRIBUTION	6
DISTRIBUTOR	2
DO	2
Dain	1
Daniel	1
Database	1
Decoder,	1
Definitions.	2
Department	1
Derivative	17
Developer	34
Developer)	2
Developer,	2
Developer.	1
Developer. 	1
Developer means	1
Developer”	1
Development	3
Deville.	1
Disclaimer	1
Distribution	4
Dmitriy	1
Domain	1
Domain)	4
Doug	1
Dumbster	1
ECLIPSE	1
EG,	1
EITHER	3
ENC	1
END	3
ENTIRE	2
EPL	2
EPL.	1
ESSENTIAL	2
EVEN	14
EVENT	16
EXCEPT	4
EXCLUSION	4
EXEMPLARY,	12
EXERCISE	1
EXPRESS	17
EXPRESSED	2
EXPRESSLY	2
EXTENT	2
Each	8
Eclipse	5
Effect	2
Encoder	1
End	2
Eng	1
Entity	3
Entity"	1
European	1
Everyone	1
Except	3
Exception	1
Exception"	1
Executable	16
Export	2
External	1
FAILURE	2
FAR,	2
FIT	2
FITNESS	18
FOR	46
FORTH	2
FREE	2
FROM	2
FROM,	5
Fast	1
FastLZ	1
Fennell	1
File	1
Finance	1
FindBugs-jsr305	1
For	18
For:	7
Foundation	9
Framework	1
Framework,under	1
Francis.	1
FreeBSD	1
FuseSource	4
GENERAL	1
GOODS	11
GOODWILL,	2
GOVERNMENT	2
GPL	1
GPLv2	2
GRANT	1
GRANTED	1
GROUP,	2
GlassFish	1
Goods	2
Google	7
Google's	2
Google,	1
Government	7
Grand.	1
Grant	2
Grant.	4
Grants.	2
Greene:	1
Group	3
Group.	1
Group."	1
Group:	2
Gson	1
HADOOP	1
HAVE	3
HEREUNDER	2
HEREUNDER,	1
HOLDER>	1
HOLDERS	15
HOMEPAGE:	24
HOWEVER	12
HPACK	1
HSQL	4
HSQLDB	1
HSQLDB.ORG,	1
HTTP	1
HTTP/2	2
HYPERSONIC	1
Hadoop	2
Hadoop,	1
Hamcrest	1
Harder's	1
Harmony',	1
Hat	1
Header	1
Hoeller,	1
How	1
However,	4
Hypersonic	4
IBM	2
IF	14
IMPLIED	23
IMPLIED,	7
IN	59
INCIDENTAL	2
INCIDENTAL,	14
INCLUDING	5
INCLUDING,	16
INDIRECT,	14
INFORMED	2
INITIAL	4
INJURY	2
INTERRUPTION)	11
IS	27
IS"	12
IS""	2
IS",	5
IS''	1
IS”	1
If	17
In	7
Inc,	1
Inc.	7
Inc.,	1
Inc:	1
Indemnified	3
Industry	1
Initial	40
Institute	1
International	2
Iris	1
It	8
Ivan	1
J.	1
JAXB	2
JLine	1
JSR-166	1
JSch	1
JURISDICTIONS	2
JUnit	1
JVM	1
JVM,	1
January	1
Jason	2
Java	16
Java,	2
Jersey	1
Jetty	6
Josh	1
Juergen	1
KIND,	11
Kevin	1
Kinney.	1
Kitchen	1
Kuhn's	1
LAW	2
LEGAL	2
LGPL	1
LIABILITY	6
LIABILITY,	29
LIABILITY.	2
LIABLE	18
LICENSE	6
LICENSE.	2
LICENSE.<component>.txt	1
LICENSE:	25
LICENSES	1
LIMITATION	9
LIMITATION,	5
LIMITATION.	2
LIMITED	27
LOSS	13
LOSSES,	2
LOST	2
LP.	1
LZ	1
LZ4	4
LZF	1
LZMA	1
Laddad,	1
Lang	1
Lang',	1
Larger	10
Lea,	1
Legal	3
LevelDB	1
Liability.	2
Library	2
Licensable	6
License	71
License)	12
License);	2
License,	12
License.	33
License. 	9
License;	2
Licensed	3
Licensor	8
Licensor,	1
Limitation	1
Log4J',	1
Log4j	1
Logging',	1
Losses	1
Louvain	1
Ltd	1
Ltd.	2
MALFUNCTION,	2
MAY	2
MERCHANTABILITY	12
MERCHANTABILITY,	6
MERCHANTABLE,	2
MISCELLANEOUS.	2
MIT	1
MPSC	1
Marshalling',	1
Massachusetts	1
Matthew	1
Microsystems,	1
Microsystems.,	1
Mockito	1
Modification.	2
Modification;	2
Modifications	23
Modifications,	8
Modifications.	3
Modifications. 	1
Modifications;	2
Modified	2
Mori.	1
Morse.	1
Mort	2
MortBay	1
Mortbay	1
NECESSARY	2
NEGLIGENCE	14
NEGLIGENCE),	2
NEITHER	1
NO	23
NON-INFRINGEMENT,	2
NON-INFRINGING.	2
NONINFRINGEMENT.	5
NOR	1
NOT	33
NOTICE	7
Nations	2
Neither	8
Netty	4
New	5
Niels	1
Nitsan	1
No	3
Nothing	2
Notice	2
Notices.	2
Notwithstanding	5
Number	1
OF	161
ON	15
OR	162
OTHER	16
OTHERWISE)	12
OTHERWISE,	7
OUT	17
OW2	2
OWNER	7
Object	4
Obligations.	1
Obligations. 	1
OneLab	1
OpenJDK	2
Oracle	3
Original	37
Otherwise,	2
P.	1
PART	2
PARTICULAR	20
PARTIES,	2
PARTY	2
PARTY'S	1
PARTYS	1
PERFORMANCE	2
PERSON	2
PERSONAL	2
POSSIBILITY	14
PROCUREMENT	11
PROFITS),	1
PROFITS,	1
PROFITS;	11
PROGRAM	4
PROHIBITS	2
PROVE	2
PROVIDED	20
PUBLIC	1
PURPOSE	18
PURPOSE.	2
ParaNamer	1
Parser	1
Participant	15
Participant)	3
Participant,	2
Participant.	1
Participant. 	1
Patent	7
Patents	1
Patents"	1
Patents.	1
Paul	2
Performance	1
Permission	7
Please	1
Portions	1
Product	2
Program	22
Program,	3
Program.	6
Program;	1
Progress	1
Project	3
Project.	1
Protocol	3
Providing	1
Provisions	2
Provos	1
Pty	2
Public	3
PureJavaCrc32C	1
QUALITY	2
RECIPIENT	1
RECIPIENT'S	1
REPAIR	2
REPRODUCTION	1
REPRODUCTION,	1
REQUIREMENTS	1
RESPECT,	2
RESPONSIBILITY	2
RESULTING	2
RI	1
RIGHTS	2
RISK	2
RUntime	1
Ramnivas	1
Recipient	11
Recipient's	6
Recipients	1
Red	1
Redistribution	11
Redistribution.	1
Redistributions	22
Regulations,	1
Required	2
ResolverUtil.java	1
Rights	2
Robert	1
Roland	1
SE,	1
SERVICES;	11
SERVICING,	2
SET	2
SHALL	23
SHOULD	2
SLF4J	1
SMTP	1
SO	2
SOFTWARE	31
SOFTWARE,	13
SOFTWARE.	5
SOME	2
SPECIAL,	14
SQL	5
SSL	1
STOPPAGE,	2
STRICT	12
SUBCOMPONENTS:	1
SUBSTITUTE	11
SUCH	22
SUPPLIER	2
Sale	2
Saloranta	1
Saloranta.	1
Section	10
Sections	7
Sections 2.1	1
Sections 2.1(a)	1
Sections 2.2(a)	1
Sections 6.1	1
Section 2.1(b)	1
Section 2.2(b)	1
Security	1
See	3
Short	1
Snappy	1
Snappy:	1
Software	79
Software,	21
Software.	13
Software. 	2
Software means	2
Software”	2
Source	28
Spring	1
SpryMedia	1
State	1
States	3
Stephen	1
Steward	3
Steward.	1
Streaming	1
StringUtils.containsWhitespace())	1
Subject	4
Submission	1
Sun	2
Sundstrom	1
T.	1
TERMINATION.	1
TERMINATION. 	1
TERMS	3
THAT	2
THE	111
THEORY	12
THEORY,	2
THIS	38
TITLE,	2
TO	15
TO,	22
TORT	19
Tatu	2
Technology	1
Technology.	1
Terms.	2
That	1
The	76
Therefore,	1
These	1
This	54
Tim	1
To	1
Tomcat	1
Tools	1
Trademarks.	1
Twitter.	1
TypeUtil.java	1
U.S.	7
UCL	1
UNDER	9
URIs	1
USE	19
USE,	13
USERS.	2
Under	1
United	5
University	1
Unix	1
UnixCrypt	1
UnixCrypt.java	1
Unless	4
Unrestricted	1
Use	1
Users	2
V3	1
Van	1
Version	10
Version)	2
Version,	2
Version;	2
Versions	2
Versions.	8
Version means	1
Version”	1
Vyukov's	1
W3C	1
WARRANTIES	23
WARRANTIES,	11
WARRANTY	10
WARRANTY.	2
WAY	12
WHETHER	19
WITH	7
WITHOUT	17
WORK	2
Wakart's	1
Warranty	1
Warranty.	1
We	1
Web	2
WebSocket	1
When	3
While	2
William	1
Williams	1
Work	24
Work,	4
Work.	1
Work;	4
Works	12
Works"	1
Works,	2
Works.	2
Works;	3
Work means	1
Work”	1
Wysoker	1
X,	1
X.	1
X.509	1
XML	4
XML',	1
Xerces	2
Xerces2	1
YOU	2
YOU,	2
YOU.	3
YOU. 	1
Yann	1
York	1
Yoshida,	1
You	125
You.	2
You:	2
Your	27
Your) means	1
Yuta	1
[name	1
[yyyy]	1
``AS	1
a	157
a)	5
about	1
above	28
above,	8
above.	1
absence	2
absolutely	4
accept	1
acceptance	1
accepting	2
accordance	1
according	1
account	1
acquire	3
acquired,	4
act	1
acting	2
action	3
actions	1
actions)	2
acts	1
acts)	1
actual	1
add	2
added	2
addendum	1
addition	3
addition,	1
additional	4
additions	4
addresses	2
administration	2
admission	2
advised	1
affect	1
affiliates.	1
after	3
against	14
against,	1
agree	10
agreed	4
agreement	5
agreement,	2
agreement:	1
agrees	2
algorithm	2
algorithms.	1
all	31
alleged	1
alleging	5
allow	2
allowed	1
allows	1
alone	7
alone,	4
alone.	1
along	1
alongside	1
alphabet	1
also	5
alter	5
alternative	1
alters	2
although	1
always	3
amount	1
an	22
and	281
and/or	37
and 	2
annotations,	1
another	1
any	129
any),	2
any,	6
any.	1
anyone	2
apache-hadoop-common	1
apparatus	2
appear.	1
appears	1
applicable	10
application	2
applies	3
apply	7
appropriate	1
appropriateness	2
archives.	1
are	70
arising	2
arising,	2
arose.	1
array	1
artifacts	10
as	54
assert	5
asserted	1
assign	1
associated	9
assume	1
assumes	2
assurances	1
asymmetric	1
at	17
at:	26
attach	1
attached	1
attempt	2
attorneys	1
attorneys'	1
attribution	6
author	1
authorized	2
authors	1
authors.	1
authorship,	2
authorship.	1
automatically	4
available	32
available.	2
avoid	1
aware	3
b)	5
based	5
basis,	2
basis.	2
be	73
becoming	3
been	6
before	1
behalf	11
behalf,	2
behalf.	1
being	2
believe	2
below	4
below).	1
below:	1
benchmark	1
benchmarking	1
beneficial	3
between	2
beyond	2
binaries	12
binary	42
bind	1
body	1
boilerplate	1
both	3
brackets	1
brackets!)	1
breach	2
breach.	2
bring	1
brought	2
bundle	1
bundles	20
business	1
but	7
by	133
by,	5
by:	4
bzip2	1
c)	1
calculation	1
can	27
cannot	1
carry	1
case	5
case,	2
cause	5
caused	5
causes	1
cease	1
certain	1
certificate	1
changed	1
changes	4
character	1
charge	3
charge,	5
check	1
choice,	2
choose	6
claim	9
claim(s),	2
claim,	1
claims	10
claims,	8
class	1
classes	1
classes.	1
classes:	1
classified	1
classpath	3
clause	4
clear	6
code	39
code,	2
code.	3
code. 	1
combination	15
combination),	2
combination).	1
combination). 	1
combinations	2
combined	1
combines	2
combining	2
comment	1
commercial	10
common	5
communication	3
compiled	1
compiler	1
complete	2
compliance	11
complies	2
comply	4
complying	2
components	1
compression	6
computer	11
concerning	3
conclusions	1
condition	1
conditions	45
conditions,	1
conditions.	3
conditions:	6
configuration	1
conflict-of-law	2
conjunction	1
connection	1
connections,	1
consequential	2
consistent	1
consisting	2
consists	2
conspicuously	1
constant-size	1
constitute	2
constitutes	1
construct	1
construed	3
contact	1
contain	2
containd	1
contained	9
containing	6
contains	19
content	2
contents	3
continue	3
contract	6
contract,	1
contribute	2
contributed	2
contributes	2
contribution	1
contributions	3
contributors	8
contributors,	1
contributory	1
control	7
control,	2
controlled	3
controls,	2
conversions	1
conveyed	4
cooperate	1
copies	13
copies./	1
copy	16
copy,	7
copying	1
copyright	46
copyright,	3
copyright:Copyright	1
copyrighted	3
corresponding	1
costs	4
costs,	2
counterclaim	2
countries)	2
country	1
country's	1
country,	1
court	3
courts	2
covered	2
create	9
created	2
creates	2
creation	2
creation(s)	2
cross-claim	2
cryptographic	3
cryptography	1
cure	3
currently	1
customarily	3
customary	1
d)	1
damage	1
damages	7
damages,	3
damages.	2
data	3
data,	1
date	7
day	4
days	4
de	1
deal	5
declaratory	2
decoding	1
decompression	4
deemed	2
defend	2
defend,	1
defense	1
defined	5
definition,	4
delete	2
deleted	2
deletion	2
deliberate	1
den	1
dependencies	8
depends	13
derivative	5
derived	10
derived.)	1
describing	1
description	1
descriptive	2
designated	1
details	1
details,	1
determining	3
developed	10
devices.	1
devices. 	1
devices;	2
differ	6
different	5
differs	2
direct	4
direct,	2
direction	3
directly	7
directories	1
directory	1
disclaimer	11
disclaimer.	11
disclaimers,	1
disclaims	2
discussing	1
display	1
display,	6
dispose	4
distinguishing	3
distribute	34
distribute,	7
distributed	15
distributes	5
distributing	2
distribution	29
distribution,	1
distribution.	12
distributor	2
distributor)	2
distributors	1
do	13
document.	2
document. 	1
documentation	23
documentation,	3
documentation”	1
documented	1
does	7
downloaded	1
drafter	2
dual	1
each	21
easier	1
ecj-4.3.1.jar	1
editorial	1
effect	2
effective	4
effectively	2
either	11
elaborations,	1
elect	1
electronic	1
electronic,	1
eligible	1
enclosed	2
encoding	1
encryption	3
end	3
endorse	8
enforceability	1
enforceable.	3
entities	1
entities,	2
entity	14
entity,	3
entity.	5
entity. 	1
equipment,	1
equitable	2
equivalent	1
errors,	1
estoppel	1
even	1
event	4
every	7
exact	1
example	1
example,	2
except	5
exception	1
exception.	2
exceptions.	1
exchange.	2
exchange. 	1
excluded.	2
excludes	1
excluding	5
executed	1
exercise	2
exercising	4
exist,	1
expense.	1
expenses.	2
expiration	2
explicitly	1
export	5
express	4
expressed	1
expressly	4
expressly,	1
extensions	1
extent	8
external	1
externally	1
facade	1
facilitate	1
fail	4
fails	1
failure	2
fee	5
fees	2
fields	1
fifty	3
file	20
file,	3
file.	2
filed.	2
files	11
files,	1
files.	1
files;	1
first	6
fitness	1
following	77
following:	4
for	74
for,	3
form	51
form,	4
form.	3
form:	1
format,	2
format.	1
forms,	11
forth	6
forum	1
found	1
framework,	3
free	5
from	51
from)	1
from,	1
fulfilled	2
full	1
functionality.	1
functions	1
furnished	5
further	1
gcc,	1
generate	1
generated	3
generating	1
generic	1
github	1
give	1
given	3
giving	2
goodwill,	1
governed	10
governing	2
grant	6
grant,	2
granted	20
granted,	6
granted:	4
granting	1
grantor.	1
grantor. 	1
grants	9
grossly	1
hadoop-hdfs-project/hadoop-hdfs-native-client/src/main/native/fuse-dfs/util/tree.h	1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.0.2	1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-full-2.0.0.min.js	1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-helpers-1.1.1.min.js	1
hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery-1.10.2.min.js	1
hadoop-tools/hadoop-sls/src/main/html/css/bootstrap-responsive.min.css	1
hadoop-tools/hadoop-sls/src/main/html/css/bootstrap.min.css	1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js	1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3-LICENSE	1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js	1
hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js	1
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/dt-1.9.4/	1
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jquery	1
hadoop-yarn-project/hadoop-yarn/hadoop-yarn-common/src/main/resources/webapps/static/jt/jquery.jstree.js.gz	1
hardware	1
hardware)	1
harmless	1
has	20
have	14
having	2
held	2
hereafter	2
hereby	20
herein	5
herein,	1
herein.	3
herein. 	1
hereof.	2
hereto,	1
hereunder	4
hereunder,	1
hereunder.	3
history	1
history,	1
hold	1
hosted	1
how	3
http://archive.apache.org/dist/harmony/	1
http://asm.ow2.org/license.html	1
http://aspell.net/test/orig/batch0.tab.Copyright	1
http://code.google.com/p/caliper/	1
http://code.google.com/p/lz4/	1
http://code.google.com/p/protobuf/	1
http://code.google.com/p/snappy/	2
http://commons.apache.org/logging/	1
http://fusesource.com	2
http://gcc.gnu.org/onlinedocs/libstdc++/manual/license.html	1
http://gee.cs.oswego.edu/cgi-bin/viewcvs.cgi/jsr166/	1
http://hadoop.apache.org/	1
http://hadoop.apache.org/core/	1
http://iharder.sourceforge.net/current/java/base64/	1
http://logging.apache.org/log4j/	1
http://netty.io/	1
http://openjdk.java.net/legal/gplv2+ce.html	1
http://stevemorse.org/phoneticinfo.htm	1
http://viewvc.jboss.org/cgi-bin/viewvc.cgi/jbosscache/experimental/jsr166/	1
http://wiki.apache.org/hadoop/	1
http://wiki.fasterxml.com/AaltoHome	1
http://www.apache.org/licenses/	1
http://www.apache.org/licenses/LICENSE-2.0	2
http://www.apache.org/licenses/LICENSE-2.0.html	1
http://www.bouncycastle.org/	1
http://www.eclipse.org/legal/epl-v10.html	1
http://www.ibm.com.	1
http://www.jboss.org/jbossmarshalling	1
http://www.jcraft.com/jzlib/	1
http://www.slf4j.org/	1
http://www.sun.com.	1
https://code.google.com/p/jbzip2/	1
https://code.google.com/p/jfastlz/	1
https://code.google.com/p/libdivsufsort/	1
https://commons.apache.org/proper/commons-lang/	1
https://github.com/JCTools/JCTools	1
https://github.com/akka/akka/blob/wip-2.2.3-for-scala-2.11/akka-actor/src/main/java/akka/dispatch/AbstractNodeQueue.java	1
https://github.com/joewalnes/webbit	1
https://github.com/jponge/lzma-java	1
https://github.com/jpountz/lz4-java	1
https://github.com/jquery/jquery	1
https://github.com/ning/compress	1
https://github.com/twitter/hpack	1
https://glassfish.dev.java.net/nonav/public/CDDL+GPL.html	2
https://groups.google.com/forum/#!forum/lz4c	1
https://jquery.org/	1
i)	2
iClick,	1
identification	1
identifies	2
identify	2
identifying	1
if	23
if,	1
ii)	2
iii)	1
implementation	2
implements	1
implication,	1
implied	1
implied,	3
implied.	2
import	1
import,	3
impose	2
improvement	1
improving	1
in	171
in,	1
inability	1
incidental	1
incidental,	1
include	11
included	13
includes	17
including	20
including,	3
inclusion	3
inconsistency	1
incorporated	2
incurred	5
indemnify	5
indemnify,	1
indemnity	6
indemnity,	1
indicated	1
indirect,	5
indirectly	5
indirectly,	2
individual	9
individuals	1
individuals.	1
inform	2
information	2
information.	2
information:	1
informational	1
informs	1
infringe	1
infringed	8
infringement	5
infringement,	1
infringement.	1
infringements	4
infringes	4
initial	7
initiation	1
input	2
institute	1
institutes	1
intellectual	15
intended	3
intentionally	2
interchange	1
interfaces	1
interpreted	1
interruption	1
into	1
invalid	1
irrevocable	2
is	100
issue	1
it	20
it.	2
item,	1
item,”	1
its	31
itself	4
iv)	1
jQuery	1
jars,	1
java.lang	1
java.sun.security.ssl	1
javax.annotation:javax.annotation-api	1
javax.servlet:javax.servlet-api	1
javax.transaction:javax.transaction-api	1
javax.websocket:javax.websocket-api	1
jsp-api	1
judgment	2
jurisdiction	6
jurisdiction's	1
jurisdictions	1
jury	1
knowledge	1
language	4
latest	1
law	8
law,	3
laws	4
laws,	2
lawsuit)	2
lawsuits	1
least	1
legal	7
leveldb	1
leveldbjni	1
leveldbjni-all	1
liability	15
liability.	2
liability. 	1
liable	1
libraries	2
library	10
library,	1
libstdc++.	1
licensable	2
license	46
license)	1
license,	2
license.	4
license/LICENSE.aalto-xml.txt	1
license/LICENSE.abstractnodequeue.txt	1
license/LICENSE.base64.txt	1
license/LICENSE.bouncycastle.txt	1
license/LICENSE.caliper.txt	1
license/LICENSE.commons-lang.txt	1
license/LICENSE.commons-logging.txt	1
license/LICENSE.compress-lzf.txt	1
license/LICENSE.deque.txt	1
license/LICENSE.harmony.txt	1
license/LICENSE.hpack.txt	1
license/LICENSE.jboss-marshalling.txt	1
license/LICENSE.jbzip2.txt	1
license/LICENSE.jctools.txt	1
license/LICENSE.jfastlz.txt	1
license/LICENSE.jsr166y.txt	1
license/LICENSE.jzlib.txt	1
license/LICENSE.libdivsufsort.txt	1
license/LICENSE.log4j.txt	1
license/LICENSE.lz4.txt	1
license/LICENSE.lzma-java.txt	1
license/LICENSE.protobuf.txt	1
license/LICENSE.slf4j.txt	1
license/LICENSE.snappy.txt	1
license/LICENSE.webbit.txt	1
license:	13
license: 	2
licensed	3
licensed.	3
licensee)	1
licensees	1
licenses	15
licenses.	1
licenses;	1
licensing	2
lieu	2
like.	1
limit	2
limitation	5
limitation,	5
limitations	2
limited	5
link	1
linked	2
list	22
lists,	1
litigation	6
litigation,	1
litigation.	1
located	6
location	2
logging	3
losing	2
loss	2
losses),	1
losses,	1
lost	1
made	26
made,	5
made.	1
mailing	1
maintained	1
make	23
make,	6
makes	8
making	1
making,	4
malfunction,	1
managed	1
management	3
manner	6
manner.	1
many	2
marked	1
marks,	1
material	1
materials	11
matter	2
maximum	2
may	53
mean	11
means	20
means:	1
mechanical	1
media	1
medium	3
medium,	1
meet	1
memory	1
merchantability	1
merely	1
merge,	5
met:	11
method,	2
micro-	1
might	1
minimum	1
modification	4
modification,	12
modifications	8
modifications,	3
modifications:	1
modified	19
modififications	1
modify	6
modify,	10
modifying	1
module	1
modules	1
more	9
mortbay.org.	1
must	50
must:	1
name	11
name)	1
names	9
names,	1
native	1
nature,	2
necessarily	2
necessary	3
needed,	1
negligence),	1
negligent	1
negotiate	1
negotiations.	1
new	12
no	10
no-charge,	2
node_modules	1
non-blocking	1
non-commercial	1
non-exclusive	4
non-exclusive,	4
non-infringement,	1
non-intrusive	1
noncompliance.	1
none	1
nor	8
normally	1
not	45
note	2
noted.	1
nothing	1
notice	25
notice,	23
notices	14
notices,	1
notices:	9
notify	1
now	2
number.	3
object	5
obligation	2
obligations	5
obligations,	1
obtain	5
obtained	23
obtaining	5
obtaining,	1
of	445
of,	8
of:	2
offer	8
offer,	3
offer.	3
offer. 	1
offered	5
offering	1
offering,	2
offering.	1
offers	1
official	1
omissions	1
on	56
on.	1
one	8
only	13
open	1
operations.	1
optionally	12
or	276
or,	1
order	2
org.apache.commons.codec.language.bm	1
org.apache.hadoop.util.bloom.*	1
org.apache.taglibs:taglibs-standard-impl	1
org.apache.taglibs:taglibs-standard-spec	1
org.apache.tomcat:tomcat-api	1
org.apache.tomcat:tomcat-el-api	2
org.apache.tomcat:tomcat-jasper	1
org.apache.tomcat:tomcat-jasper-el	2
org.apache.tomcat:tomcat-jsp-api	1
org.apache.tomcat:tomcat-juli	1
org.apache.tomcat:tomcat-util	1
org.apache.tomcat:tomcat-util-scan	1
org.eclipse.jetty.orbit:javax.mail.glassfish	1
org.eclipse.jetty.orbit:javax.security.auth.message	1
org.eclipse.jetty.orbit:org.eclipse.jdt.core	1
org.eclipse.jetty.toolchain:jetty-schemas	1
org.fusesource.hawtjni:hawtjni-runtime	1
org.iq80.leveldb:leveldb-api	1
org.mortbay.jasper:apache-el	1
org.mortbay.jasper:apache-jsp	1
org.ow2.asm:asm	1
org.ow2.asm:asm-commons	1
origin	1
original	7
originally	9
originate	1
originator	2
other	52
others.	1
otherwise	28
otherwise),	2
otherwise,	5
otherwise.	2
our	2
out	3
outstanding	3
own	10
owned	3
owner	5
owner.	1
owner]	1
ownership	5
package	1
page"	1
part	12
participate	1
particular	4
parties	1
partners	1
parts	2
party	16
party.	2
party;	1
password	1
patent	31
patent(s),	1
patent,	3
pay	2
payment	1
per	1
percent	3
perform,	6
performance	4
performing	1
period	3
period,	2
permission	7
permission.	8
permissions	4
permit	5
permitted	12
permitted)	1
permitted.	1
perpetual,	2
person	6
persons	5
pertain	2
php	1
places:	1
please	3
policies	1
policies,	1
port	1
portion	10
portions	23
possession,	2
possibility	1
possible,	2
potential	1
power,	3
practicable.	1
practice,	2
pre-existing	1
preferred	1
preferred,	1
prepare	2
previous	4
prior	13
process,	2
processor,	1
produced	1
product	59
product.	2
products	8
profits;	1
program	1
programs	1
prohibiting	2
project	3
prominent	1
promote	8
promptly	1
property	15
prospectively	2
protection.	1
provide	3
provided	35
provides	8
provision	9
provisions	1
provisions.	2
public	1
publicly	4
publish	3
publish,	5
published	2
published,	1
pure	1
purpose	3
purpose;	1
purposes	7
pursuant	2
qualify,	1
queue.	1
re-export	2
re-implementation	1
read	1
read,	1
readable	1
reason	1
reasonable	8
reasonably	2
received	5
received.	1
receives	2
recipient's	1
recipients	7
recipients'	1
recommend	3
redistributed	2
redistributing	2
refer	1
references	2
referred	2
reformed	3
regarding	1
regulation	4
regulations	3
related	3
relating	4
released	2
remain	3
remainder	1
remove	5
rename	1
replace/modify	1
replaced	1
repository	1
represent	2
represent,	1
representatives,	1
representing	1
represents	3
reproduce	12
reproduce,	6
reproducing	1
reproduction,	3
required	6
requirements	2
requires	2
reserved.	10
reserves	1
reside	1
resolved	1
respect	3
responsibilities	1
responsibility	6
responsibility,	1
responsible	8
restriction,	5
restrictions	1
restricts	2
result	5
result,	1
resulting	2
results	2
retain	11
retain,	1
revised	2
revision	1
revisions)	1
revisions,	1
right	6
rights	55
risks	3
royalty	1
royalty-free	2
royalty-free,	6
sale	1
sale,	4
same	1
se	1
section	2
section)	1
section,	1
secure	1
security	1
see	2
see:	1
selected	1
selected.	1
self-signed	1
sell	5
sell,	8
selling	4
sent	1
separable	1
separate	4
serialization	1
serve	1
server	1
server,	1
service	1
servlet-api	1
set	6
settlement	1
settlement)	1
shall	44
shall,	2
shares	2
shares,	1
should	4
simple	2
single	2
site	1
slicing-by-8	1
snappy	1
so	3
so,	5
software	56
software,	2
software.	2
software:	1
software”	1
sole	2
solely	2
soon	1
source	35
source,	1
special,	2
specific	10
specified	4
src/google/protobuf/stubs/atomicops_internals_generic_gcc.h.	1
src/google/protobuf/stubs/atomicops_internals_power.h.	1
src/main/native/src/org/apache/hadoop/io/compress/lz4/{lz4.h,lz4.c,lz4hc.h,lz4hc.c},	1
src/main/native/src/org/apache/hadoop/util:	1
src/test/org/apache/commons/codec/language/DoubleMetaphoneTest.javacontains	1
standalone	1
state	1
stated	3
statement	1
states	2
statically	1
stating	1
steward	6
steward.	1
steward. 	1
stoppage,	1
string	2
style	1
subcomponents	2
subject	15
sublicense	5
sublicense,	6
submit	1
submitted	2
submitted.	1
subsequent	6
subsequently	3
subsidiaries	1
substantial	5
such	87
sufficient	3
suffix	1
suitable	1
suite	1
supersede	1
supersedes,	2
support	4
support,	7
sure	2
survive	2
survive.	2
survive. 	1
syntax	1
systems	2
systems,	1
taken	1
temporary	1
term	4
terminate	7
terminate,	1
termination	6
termination.	1
termination. 	1
terms	61
terms.	1
test	2
text	8
than	12
that	118
that:	1
the	693
their	7
them,	1
then	9
theory,	1
thereof	3
thereof),	4
thereof).	1
thereof);	3
thereof,	3
thereof.	2
thereof. 	1
these	9
they	2
third	11
third-party	2
this	181
those	9
through	6
time	7
time.	3
title	1
to	253
to,	2
tort	1
tracking	1
trade	1
trademark	2
trademark)	4
trademark,	1
trademarks,	1
transfer	2
transformation	1
transformed	1
translated	1
translation	1
trial	1
types.	1
ultra-high	1
unavailability	1
under	94
under,	2
understands	1
understood,	1
unenforceable	1
unenforceable,	2
unilaterally	2
union	1
unless	4
unmodified	2
upon	6
usage	1
use	23
use,	27
used	19
user	2
users	1
users,	1
uses	1
using	7
using,	2
utilities	1
utilization	2
v	1
v2	1
valid	1
validity	1
validly	2
value	2
venue	2
verbal,	1
version	25
version.	1
versions	3
viewable	1
views	1
visit	2
voluntary	3
waives	1
want	2
warranties	6
warranties,	1
warranty	1
warranty,	7
was	3
way	1
we	1
web	1
website	1
were	3
when	6
where	7
wherever	1
whether	9
which	61
which,	2
which:	1
who	2
whole,	2
whom	8
wiki,	1
will	7
with	103
withdraw	2
within	20
without	42
work	11
work,	3
work.	2
works	4
works,	1
world-wide,	4
worldwide,	4
would	1
writing	2
writing,	4
written	19
xmlenc	1
year	1
you	12
your	5
zlib	1
 252.227-7014(a)(1))	1
§	1
“AS	1
“Contributor	1
“Contributor”	1
“Covered	1
“Executable”	1
“Initial	1
“Larger	1
“Licensable”	1
“License”	1
“Modifications”	1
“Original	1
“Participant”)	1
“Patent	1
“Source	1
“Your”)	1
“You”	2
“commercial	3
“control”	1




搭建部署Hadoop 之Yarn

标签:session   bst   bzip   like   read   执行   hid   prepare   mpi   

原文地址:http://blog.51cto.com/13558754/2067497

(0)
(0)
   
举报
评论 一句话评论(0
登录后才能评论!
© 2014 mamicode.com 版权所有  联系我们:gaon5@hotmail.com
迷上了代码!