You can connect it using the spark-shell command as shown below: spark-shell You can also connect the Spark server using the command-line. You should see the Spark worker process in the following screen: Working with Spark Shell Now, go to the Spark dashboard and refresh the screen. You should see the following output: starting .worker.Worker, logging to /opt/spark/logs/. So you can use this address to start the Spark worker process using the following command: start-slave.sh spark://your-server-ip:7077 Now, open your web browser and access the Spark web interface using the URL You should see the following screen: Start Spark Worker ProcessĪs you can see, Spark master service is running on spark://your-server-ip:7077. You can check it using the following command: ss -tpln | grep 8080 You should see the following output: starting .master.Master, logging to /opt/spark/logs/.īy default, Spark is listening on port 8080. Save and close the file then activate the environment with the following command: bashrc file: nano ~/.bashrcĪdd the following lines at the end of the file: export SPARK_HOME=/opt/sparkĮxport PATH=$PATH:$SPARK_HOME/bin:$SPARK_HOME/sbin Next, you will need to configure Spark environment so you can easily run Spark commands. Next, rename the extracted directory to spark as shown below: mv spark-2.4.6-bin-hadoop2.7 spark Once downloaded, extract the downloaded file with the following command: tar -xvzf spark-2.4.6-bin-hadoop2.7.tgz
#Aws linux install apache spark download
You can download it to the /opt directory with the following command: cd /opt
At the time of writing this tutorial, the latest version of Apache Spark is 2.4.6. You should get the following output: Hitesh Jethvaįirst, you will need to download the latest version of Apache Spark from its official website. Now, test the Scala with the following command: scala> println("Hitesh Jethva") You should get the following output: Welcome to Scala 2.11.12 (OpenJDK 64-Bit Server VM, Java 11.0.8).
Now, connect to the Scala interface with the following command: scala
#Aws linux install apache spark code
Scala code runner version 2.11.12 - Copyright 2002-2017, LAMP/EPFL
0 kommentar(er)
