How should I use Spark version of datagen to access LiveGraph

[ggssh]

#!/bin/bash
DEFAULT_HADOOP_HOME=$HOME/hadoop-2.6.0 #change to your hadoop folder
DEFAULT_LDBC_SNB_DATAGEN_HOME=`pwd` #change to your ldbc_snb_datagen folder

# allow overriding configuration from outside via environment variables
# i.e. you can do
#     HADOOP_HOME=/foo/bar LDBC_SNB_DATAGEN_HOME=/baz/quux ./run.sh
# instead of changing the contents of this file
HADOOP_HOME=${HADOOP_HOME:-$DEFAULT_HADOOP_HOME}
LDBC_SNB_DATAGEN_HOME=${LDBC_SNB_DATAGEN_HOME:-$DEFAULT_LDBC_SNB_DATAGEN_HOME}

export HADOOP_HOME
export LDBC_SNB_DATAGEN_HOME
export HADOOP_CLIENT_OPTS="-Xmx64G"

echo ===============================================================================
echo Running generator with the following parameters:
echo -------------------------------------------------------------------------------
echo LDBC_SNB_DATAGEN_HOME: $LDBC_SNB_DATAGEN_HOME
echo HADOOP_HOME: $HADOOP_HOME
echo HADOOP_CLIENT_OPTS: $HADOOP_CLIENT_OPTS
echo ===============================================================================

#  --- for OSX, add this line ---
# zip -d $LDBC_SNB_DATAGEN_HOME/target/ldbc_snb_datagen-0.2.7-jar-with-dependencies.jar META-INF/LICENSE

$HADOOP_HOME/bin/hadoop jar ldbc_snb_datagen-0.2.7-jar-with-dependencies.jar $LDBC_SNB_DATAGEN_HOME/params-foreign-key.ini

rm -f m*personFactors*
rm -f .m*personFactors*
rm -f m*activityFactors*
rm -f .m*activityFactors*
rm -f m0friendList*
rm -f .m0friendList*

The current version used in run.sh is Hadoop. How should I modify it?

[jiguanglizipao]

The experiments used Hadoop version. I think spark version can generate the same dataset if it works.2023年5月29日 20:19，Yizhe Yuan ***@***.***>写道： #!/bin/bash DEFAULT_HADOOP_HOME=$HOME/hadoop-2.6.0 #change to your hadoop folder DEFAULT_LDBC_SNB_DATAGEN_HOME=`pwd` #change to your ldbc_snb_datagen folder # allow overriding configuration from outside via environment variables # i.e. you can do # HADOOP_HOME=/foo/bar LDBC_SNB_DATAGEN_HOME=/baz/quux ./run.sh # instead of changing the contents of this file HADOOP_HOME=${HADOOP_HOME:-$DEFAULT_HADOOP_HOME} LDBC_SNB_DATAGEN_HOME=${LDBC_SNB_DATAGEN_HOME:-$DEFAULT_LDBC_SNB_DATAGEN_HOME} export HADOOP_HOME export LDBC_SNB_DATAGEN_HOME export HADOOP_CLIENT_OPTS="-Xmx64G" echo =============================================================================== echo Running generator with the following parameters: echo ------------------------------------------------------------------------------- echo LDBC_SNB_DATAGEN_HOME: $LDBC_SNB_DATAGEN_HOME echo HADOOP_HOME: $HADOOP_HOME echo HADOOP_CLIENT_OPTS: $HADOOP_CLIENT_OPTS echo =============================================================================== # --- for OSX, add this line --- # zip -d $LDBC_SNB_DATAGEN_HOME/target/ldbc_snb_datagen-0.2.7-jar-with-dependencies.jar META-INF/LICENSE $HADOOP_HOME/bin/hadoop jar ldbc_snb_datagen-0.2.7-jar-with-dependencies.jar $LDBC_SNB_DATAGEN_HOME/params-foreign-key.ini rm -f m*personFactors* rm -f .m*personFactors* rm -f m*activityFactors* rm -f .m*activityFactors* rm -f m0friendList* rm -f .m0friendList* The current version used in run.sh is Hadoop. How should I modify it? —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>