Posts

Hive - URISyntaxException: Relative path in absolute URI

Problem
I encountered the following exception when I tried to start up Hive

Exception in thread "main" java.lang.IllegalArgumentException: java.net.URISyntaxException: Relative path in absolute URI: ${system:java.io.tmpdir%7D/$%7Bsystem:user.name%7D

Solution
This is configuration in hive-site.xml. Open up hive-site.xml and look for ${system:java.io.tmpdir}/${system:user.name}. If found, replace them with a proper value, e.g. /tmp/somedir. After that, run Hive again.

Hive - Hive metastore database is not initialized

Problem
I encountered the following error when I tried to start up Hive

Exception in thread "main" java.lang.RuntimeException: Hive metastore database is not initialized. Please use schematool (e.g. ./schematool -initSchema -dbType ...) to create the schema. If needed, don't forget to include the option to auto-create the underlying database in your JDBC connection string (e.g. ?createDatabaseIfNotExist=true for mysql)
Solution
a. Delete the existing $DERBY_HOME/data/metastore_db with rm command

b. Stop the Derby server

c. Run schematool -initSchema -dbType derby

Hive - Permission denied when trying to start Hive with other users

Problem
I had installed Hive and the Hiver CLI work with the user who installed Hive. However, when I try to start Hive CLI with another user, it failed with

Exception in thread "main" java.lang.RuntimeException: java.lang.RuntimeException: java.io.IOException: Permission denied
    at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:522)
    at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:677)
    at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:621)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:606)
    at org.apache.hadoop.util.RunJar.run(RunJar.java:221)
    at org.apache.hadoop.util.RunJar.main(RunJar.java:136)
Caused by: java.lang.RuntimeException: java.io.IOException: Permission…

Hadoop - Installing Hive with Derby

Image
Summary
This is my personal experience and summary steps to install Hive and Derby

Pre-requisite
Download HIVE from Apache (I am using Hive 2.0.0)Download Derby from Apache (I am using Derby 10.12.1.1)Make sure Java 1.7 is installedHadoop is configured and working (I am using Hadoop 2.7.1)
Installing Hive
1. Move the Hive installer to a directory. For this example, I had create a folder /usr/local/hive for Hive

cp apache-hive-2.0.0-bin.tar.gz /usr/local/hive
2. Unpackage Hive

tar -xzvf apache-hive-2.0.0-bin.tar.gz
3. Set Hive environment variable

You will need to set the following in the environment

export HIVE_HOME=/usr/local/hive/apache-hive-2.0.0-bin
export PATH=$HIVE_HOME/bin:$PATH
So, you can do the following to export Hive variable to the environment when user log in

a. Create a  /etc/profile.d/hive.sh

sudo vi /etc/profile.d/hive.sh
b. Add the following in /etc/profile.d/hive.sh

export HIVE_HOME=/usr/local/hive/apache-hive-2.0.0-bin
export PATH=$HIVE_HOME/bin:$PATH
c. Source thi…

Linux - Locking a file

You could use the following command to lock a Linux file


#Open myfile as file descriptor 2
$ exec 2>myfile
#lock the file descriptor 2
$ flock -x 2
And release the lock with the following

$ exec 2>&-
Then, you could show file descriptor holding with the following command

lsof | grep myfile
or

lsof myfile
It could return something like

bash        132  userX    2wW   REG    8,2    0 135612 myfile
The following is the man page from lsof


FD
is the File Descriptor number of the file or:
cwd current working directory; Lnn library references (AIX); err FD information error (see NAME column); jld jail directory (FreeBSD); ltx shared library text (code and data); Mxx hex memory-mapped type number xx. m86 DOS Merge mapped file; mem memory-mapped file; mmap memory-mapped device; pd parent directory; rtd root directory; tr kernel trace file (OpenBSD); txt program text (code and data); v86 VP/ix mapped file;
FD is followed by one of these characters, describing the mode under which the…

Oracle - Unable to extend index SYS.I_OBJ1 by 8 in tablespace SYSTEM

Image
If you see the following similar error message

unable to extend index SYS.I_OBJ1 by 8 in tablespace SYSTEM
It means that your tablespace is out of space and you should increase it if possible.

You can issue the following command to check your current size of the tablespace

select * from dba_data_files where tablespace_name='SYSTEM';
It will return the path and the current size of the tablespace


The above should that I have only about 650M tablespace.
Then, depend on your current size, you could issue the following 
alter database datafile '/u01/app/oracle/oradata/XE/system.dbf' resize 1024M
Note: The above example is increasing to 1024M to my path. You should modify the path and size according to your need.
Reference:
1. http://www.markcallen.com/oracle/ora-01654-unable-to-extend-index/