标签:过滤 lock wget 重命名 配置 soft out 查看 centos
选择版本和type,这里为 spark-2.4.4-bin-without-hadoop.tgz
1 cd ~/software 2 wget http://apache.communilink.net/spark/spark-2.4.4/spark-2.4.4-bin-without-hadoop.tgz
1 tar -xzvf spark-2.4.4-bin-without-hadoop.tgz 2 mv spark-2.4.4-bin-without-hadoop.tgz spark
1 export SPARK_HOME=/software/spark 2 export PATH=$PATH:$SPARK_HOME/bin
1 cd /software/spark/conf 2 # 复制并重命名 3 cp spark-env.sh.template spark-env.sh 4 vim spark-env.sh
1 # hadoop java scala等须安装好并配置好 环境变量 2 export JAVA_HOME=/usr/local/java/jdk1.8 3 export SCALA_HOME=/software/scala 4 export SPARK_HOME=/software/spark 5 # 单机版配置为本机 ip 6 export SPARK_MASTER_IP=127.0.0.1 7 export SPARK_EXECUTOR_MEMORY=1G 8 export SPARK_DIST_CLASSPATH=$(hadoop classpath)
1 cd /software/spark/conf 2 # 复制并重命名 3 cp slaves.template slaves 4 vim slaves
1 localhost
在 $SPARK_HOME/examples/src/main 目录下有一些 Spark 的示例程序,有 Scala、Java、Python、R 等语言的版本。我们可以先运行一个示例程序 SparkPi(即计算 π 的近似值),执行如下命令:
1 [root@VM_0_8_centos spark]# ./bin/run-example SparkPi 2>&1 | grep "Pi is roughly" 2 Pi is roughly 3.1369756848784243 3 [root@VM_0_8_centos spark]#
1 # 启动shell 2 spark-shell 3 4 # 关闭 5 ctrl+d
标签:过滤 lock wget 重命名 配置 soft out 查看 centos
原文地址:https://www.cnblogs.com/dintalk/p/12234718.html