How To: Running Giraph

Buiding Giraph:

To create the jar files, here are the steps:

    export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64
    wget http://apache.mirrors.tds.net/maven/maven-3/3.2.2/binaries/apache-maven-3.2.2-bin.tar.gz
    tar -xvf apache-maven-3.2.2-bin.tar.gz
    cd giraph

    mvn install -DskipTests -X

     
Use Already Built JAR:

create a node_bootstrap.sh

mkdir -p /media/ephemeral0/
cd /media/ephemeral0/
hadoop dfs -get s3://paid-qubole/giraph/giraph-qubole.tar.gz ./    
tar -xvf giraph-qubole.tar.gz

 

Running Jobs:

1.Sample Shortest Path job:

Run as a shell command :

hadoop dfs -put /media/ephemeral0/giraph/tiny_text.txt /tmp
export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-1.7.0.45.x86_64
hadoop jar /media/ephemeral0/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.GiraphRunner org.apache.giraph.examples.SimpleShortestPathsComputation -vif org.apache.giraph.io.formats.JsonLongDoubleFloatDoubleVertexInputFormat -vip /tmp/tiny_text.txt -vof org.apache.giraph.io.formats.IdWithValueTextOutputFormat -op /tmp/out1 -w 1    


2. Page Ranker benchmark job:

hadoop jar /media/ephemeral0/giraph/giraph-examples/target/giraph-examples-1.1.0-SNAPSHOT-for-hadoop-1.2.1-jar-with-dependencies.jar org.apache.giraph.benchmark.PageRankBenchmark  -e 1 -s 3 -v -V 5000 -w 1

 

Have more questions? Submit a request

Comments

Powered by Zendesk