Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

path and to.dfs with R launched from two different directories #185

Open
nturenne opened this issue Nov 16, 2017 · 0 comments
Open

path and to.dfs with R launched from two different directories #185

nturenne opened this issue Nov 16, 2017 · 0 comments

Comments

@nturenne
Copy link

Hi !

i use rmr2 and hadoop as a newbie on windows 7 with hadoop 2.7.1 and R 3.3.2

This are my parameters :

#init env R
Sys.setenv(HADOOP_CMD="D:/Soft/hadoop/bin/hadoop")
Sys.setenv(HADOOP_HOME="D:/Soft/hadoop/")
Sys.setenv("HADOOP_PREFIX"="D:/Soft/hadoop/")
Sys.setenv(HADOOP_STREAMING="D:/Soft/hadoop/share/hadoop/tools/lib/hadoop-streaming-2.7.1.jar")

Sys.setenv(TMP = 'D:\Soft\hadoop')
library(rhdfs)
library(rmr2)
library(ravro)
.jinit()
hdfs.init()

At this point all is ok

if i run with the path on which R is installed all is ok :

getwd()
[1] "C:/Program Files/R/R-3.3.2/bin/x64"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
17/11/16 17:16:08 WARN zlib.ZlibFactory: Failed to load/initialize native-zlib l
ibrary
17/11/16 17:16:08 INFO compress.CodecPool: Got brand-new compressor [.deflate]

BUT if i run on another path there a path problem somewhere and i dont know why

getwd()
[1] "D:/Utilisateurs/turenn"
rmr.options( backend = "hadoop" )
NULL
ints = to.dfs(1:10)
Le chemin d'accès spécifié est introuvable.
Warning message:
l'exécution de la commande 'D:/Soft/hadoop/bin/hadoop jar D:/Soft/hadoop/share/h
adoop/tools/lib/hadoop-streaming-2.7.1.jar loadtb /tmp/file30a0419d6021 < /Temp/
RtmpAL2rCq/file30a0240f658e' renvoie un statut 1

why such problem about path when calling the same Rterm from two different directories
did i miss something ?

thank you for help
Nico

@nturenne nturenne changed the title path and to.dhs with R launched from two different directories path and to.dfs with R launched from two different directories Nov 16, 2017
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant