java - Setting hadoop.tmp.dir on Windows gives error: URI has an authority component -

i'm trying specify base directory hdfs files in hdfs-site.xml under windows 7 (hadoop 2.7.1 built source, using java sdk 1.8.0_45 , windows sdk 7.1). can't figure how provide path specifies drive.

my hdfs-site.xml looks this:

<configuration> <property> <name>dfs.replication</name> <value>1</value> </property> <property> <name>hadoop.tmp.dir</name> <value>xxx</value> </property> </configuration>

and tried various values xxx, tested hdfs namenode -format, leading 1 of these 2 errors:

• xxx=d:/tmp/hdp: 15/07/10 23:38:33 error namenode.namenode: failed start namenode. java.lang.illegalargumentexception: uri has authority component @ java.io.file.<init>(file.java:423) @ org.apache.hadoop.hdfs.server.namenode.nnstorage.getstoragedirectory(nnstorage.java:329)
• xxx=d:\tmp\hdp: error common.util: syntax error in uri file://d:\tmp\hdp/dfs/name

other variants gave similar errors: file:///d:/tmp/hdp (from http://hortonworks.com/community/forums/topic/hadoop-configuration-files-issues/), file://d:/tmp/hdp, d:\\tmp\\hdp

and if use /d/tmp/hdp not crash, goes d folder on current drive.

i'm out of ideas, suggestion? (nb: besides using cygwin, not option me)

you can specify drive spec in hadoop.tmp.dir in core-site.xml prepending '/' in front of absolute path, , using '/' path separator instead of '\' path elements. example, if desired absolute path d:\tmp\hdp, this:

<property>     <name>hadoop.tmp.dir</name>     <value>/d:/tmp/hdp</value> </property> 

the reason works default values many of hdfs directories configured file://${hadoop.tmp.dir}/suffix. see default definitions of dfs.namenode.name.dir, dfs.datanode.data.dir , dfs.namenode.checkpoint.dir here:

http://hadoop.apache.org/docs/r2.7.1/hadoop-project-dist/hadoop-hdfs/hdfs-default.xml

substituting above value hadoop.tmp.dir yields valid file: uri drive spec , no authority, satisfies requirements hdfs configuration. it's important use '/' instead of '\', because bare unencoded '\' character not valid in url syntax.

http://www.ietf.org/rfc/rfc1738.txt

if prefer not rely on substitution behavior, it's valid override configuration properties make use of hadoop.tmp.dir within hdfs-site.xml file. each value must full file: uri. example:

<property>     <name>dfs.namenode.name.dir</name>     <value>file:///d:/tmp/hdp/dfs/name</value> </property>  <property>     <name>dfs.datanode.data.dir</name>     <value>file:///d:/tmp/hdp/dfs/data</value> </property>  <property>     <name>dfs.namenode.checkpoint.dir</name>     <value>file:///d:/tmp/hdp/dfs/namesecondary</value> </property> 

you might find more readable overall.