Tuesday, January 6, 2015

CPNTools Troubleshoot: Beta execution aborted Reference is none.

Long time no see. Installing CPNTools for Windows is easy, but today I have a problem when installing cpntools on a Windows Server 2008. Actually the problem was not within the installation process but I can't run the CPNTools because there was an error that keep saying "CreateWindow failed. Error: 0. Create Child Window failed. Error 1406. # Beta execution aborted: Reference is none. # Error during dump: Access violation. Abortion." and creating a quite long dump file. 

I did searching for the error, and followed some guides like:
  • reinstalled the JDK/JRE.
  • made sure the system environment variable JAVA_HOME points to the installation location (e.g., c:\Program Files\Java\jre8)
  • made sure my PATH includes the bin directory under JAVA_HOME (in the example, it would have to include c:\Program Files\Java\jre8\bin).

But none of them worked. So I keep searching, but still no answer. And suddenly I remembered that there is a feature which is called Data Execution Prevention (DEP). DEP is a security feature that can help prevent damage to your computer from viruses, harmful programs, and other security threats. But sometimes DEP can also prevent your programs from being executed, and this also happened to my CPNTools. So to solve this I needed to add my cpntools.exe

You can solve it by following this step: 
Control Panel -> System and Security -> System -> Advanced System Settings.
It will open the System Properties Dialog and move to the Advanced tab and click the "Settings" button. Another dialog will be opened, and you need to go to "Data Execution Prevention" tab and add the cpntools.exe to the whitelist. And now my CPNTools works nicely. :D

======================================================
Bellow is the complete content of my cpntools.dump:

Beta execution aborted: Reference is none.

Call chain: (nti_ms)

  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
  item <*> in ~beta/guienv/private/winnt/guienv_ntibody
    -- * in ~beta/guienv/guienv
  item <*> in ~beta/guienv/guienv
    -- frame# in ~beta/guienv/guienv
  item <frame#> in ~beta/guienv/guienv
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
  item <*> in ~beta/guienv/guienv
    -- size# in ~beta/guienv/guienv
  item <size#> in ~beta/guienv/guienv
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
  item <create#>create# in ~beta/guienv/guienv
    -- open#open#open# in ~beta/postwimp/foundation/guisurface
  item create#<create#> in ~beta/guienv/guienv
    -- open#open#open# in ~beta/postwimp/foundation/guisurface
  item <*> in ~beta/guienv/private/winnt/guienv_ntibody
    -- open#open#open# in ~beta/postwimp/foundation/guisurface
  item open#open#<open#> in ~beta/guienv/guienv
    -- main#window#interfaceObject# in ~beta/postwimp/foundation/guisurface
  item <init#>init# in ~beta/postwimp/foundation/guisurface
    -- GUISurface#Surface# in ~beta/postwimp/foundation/guisurface
  item init#<init#> in ~beta/postwimp/foundation/abstractsurface
    -- GUISurface#Surface# in ~beta/postwimp/foundation/guisurface
  item <*> in ~beta/postwimp/foundation/private/worldbody_nti
    -- * in ~beta/postwimp/foundation/private/worldbody_nti
  item <*> in ~beta/postwimp/foundation/private/worldbody_nti
    -- platformSetWindowEnv# in ~beta/postwimp/foundation/private/worldbody
  item <platformSetWindowEnv#> in ~beta/postwimp/foundation/private/worldbody
    -- PROGRAM-~CPNWorkSpace#Workspace#World#SystemEnv#SysHead# in C:/cpn2000/cpntools/cpntools
  item <setWindowEnv#> in ~beta/postwimp/foundation/world
    -- PROGRAM-~CPNWorkSpace#Workspace#World#SystemEnv#SysHead# in C:/cpn2000/cpntools/cpntools
  item <start#> in ~beta/guienv/private/winnt/guienvntsystemenvbody
    -- PIOPRIVATE-~ in ~beta/guienv/private/winnt/guienvntsystemenvbody
  comp <BASICSCHEDULER-~>SysHead# in ~beta/guienv/private/winnt/guienvntsystemenvbody
    -- PROGRAM-~CPNWorkSpace#Workspace#World#SystemEnv#SysHead# in C:/cpn2000/cpntools/cpntools
  comp BASICSCHEDULER-~<SysHead#> in ~beta/basiclib/basicsystemenv
    -- BETAENV-~ in ~beta/basiclib/betaenv

  item <*> in ~beta/basiclib/betaenv
    -- BETAENV-~ in ~beta/basiclib/betaenv
  basic component in ~beta/basiclib/betaenv


Low level information:

Call chain: (nti_ms)

  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97Warning! nm(DumpSectionTable): Couldn't find section number of .text
 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <handleframeChanged#> in ~beta/guienv/private/winnt/guienv_ntiprivate
  { PC  0x49cb97 < +0x2ecb96>, object 0x2977938, proto 0x9ae2e2 handleframeChanged# [ast: 0x0008,0x0003] < +0x7fe2e1> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <*> in ~beta/guienv/private/winnt/guienv_ntibody
  { PC  0x4a0d9b < +0x2f0d9a>, object 0x29777b8, proto 0x9a8262 * [ast: 0x0069,0x0002] < +0x7f8261> }
    -- * in ~beta/guienv/guienv
    { Surrounding object 0x2977748, proto 0x9bad8e * [ast: 0x0001,0x071f] < +0x80ad8d> }
  item <*> in ~beta/guienv/guienv
  { PC  0x4a091f < +0x2f091e>, object 0x2977748, proto 0x9bad8e * [ast: 0x0001,0x071f] < +0x80ad8d> }
    -- frame# in ~beta/guienv/guienv
    { Surrounding object 0x2977718, proto 0x9bad56 frame# [ast: 0x0001,0x0717] < +0x80ad55> }
  item <frame#> in ~beta/guienv/guienv
  { PC  0x4a1425 < +0x2f1424>, object 0x2977718, proto 0x9bad56 frame# [ast: 0x0001,0x0717] < +0x80ad55> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <*> in ~beta/guienv/guienv
  { PC  0x4a1235 < +0x2f1234>, object 0x2977628, proto 0x9baecc * [ast: 0x0001,0x075f] < +0x80aecb> }
    -- size# in ~beta/guienv/guienv
    { Surrounding object 0x29775f0, proto 0x9bae9e size# [ast: 0x0001,0x0757] < +0x80ae9d> }
  item <size#> in ~beta/guienv/guienv
  { PC  0x444681 < +0x294680>, object 0x29775f0, proto 0x9bae9e size# [ast: 0x0001,0x0757] < +0x80ae9d> }
    -- windowContents#canvas#windowitem#interfaceObject# in ~beta/guienv/private/winnt/guienv_ntiprivate
    { Surrounding object 0x2974020, proto 0x9afe0a windowContents# [ast: 0x0007,0x003f] < +0x7ffe09> }
  item <create#>create# in ~beta/guienv/guienv
  { PC  0x443ab3 < +0x293ab2>, object 0x2976558, proto 0x9bbeea create# [ast: 0x0001,0x0a0d] < +0x80bee9> }
    -- open#open#open# in ~beta/postwimp/foundation/guisurface
    { Surrounding object 0x2976518, proto 0x9a40f2 open# [ast: 0x0001,0x08ae] < +0x7f40f1> }
  item create#<create#> in ~beta/guienv/guienv
  { PC  0x49864a < +0x2e8649>, object 0x2976558, proto 0x9b8f46 create# [ast: 0x0001,0x01a5] < +0x808f45> }
    -- open#open#open# in ~beta/postwimp/foundation/guisurface
    { Surrounding object 0x2976518, proto 0x9a40f2 open# [ast: 0x0001,0x08ae] < +0x7f40f1> }
  item <*> in ~beta/guienv/private/winnt/guienv_ntibody
  { PC  0x4986ce < +0x2e86cd>, object 0x2976548, proto 0x9a6e34 * [ast: 0x0021,0x0002] < +0x7f6e33> }
    -- open#open#open# in ~beta/postwimp/foundation/guisurface
    { Surrounding object 0x2976518, proto 0x9a40f2 open# [ast: 0x0001,0x08ae] < +0x7f40f1> }
  item open#open#<open#> in ~beta/guienv/guienv
  { PC  0x498676 < +0x2e8675>, object 0x2976518, proto 0x9b8f14 open# [ast: 0x0001,0x01a1] < +0x808f13> }
    -- main#window#interfaceObject# in ~beta/postwimp/foundation/guisurface
    { Surrounding object 0x29735c8, proto 0x9a4090 main# [ast: 0x0001,0x08a8] < +0x7f408f> }
  item <init#>init# in ~beta/postwimp/foundation/guisurface
  { PC  0x48ed08 < +0x2ded07>, object 0x2976458, proto 0x9a46a8 init# [ast: 0x0001,0x0b59] < +0x7f46a7> }
    -- GUISurface#Surface# in ~beta/postwimp/foundation/guisurface
    { Surrounding object 0x2973258, proto 0x9a3236 GUISurface# [ast: 0x0001,0x0048] < +0x7f3235> }
  item init#<init#> in ~beta/postwimp/foundation/abstractsurface
  { PC  0x739047 < +0x589046>, object 0x2976458, proto 0xad4702 init# [ast: 0x0001,0x0b45] < +0x924701> }
    -- GUISurface#Surface# in ~beta/postwimp/foundation/guisurface
    { Surrounding object 0x2973258, proto 0x9a3236 GUISurface# [ast: 0x0001,0x0048] < +0x7f3235> }
  item <*> in ~beta/postwimp/foundation/private/worldbody_nti
  { PC  0x48d8a3 < +0x2dd8a2>, object 0x2973248, proto 0x9a2cec * [ast: 0x0008,0x0039] < +0x7f2ceb> }
    -- * in ~beta/postwimp/foundation/private/worldbody_nti
    { Surrounding object 0x2972f38, proto 0x9a2cc0 * [ast: 0x0008,0x0002] < +0x7f2cbf> }
  item <*> in ~beta/postwimp/foundation/private/worldbody_nti
  { PC  0x48d792 < +0x2dd791>, object 0x2972f38, proto 0x9a2cc0 * [ast: 0x0008,0x0002] < +0x7f2cbf> }
    -- platformSetWindowEnv# in ~beta/postwimp/foundation/private/worldbody
    { Surrounding object 0x2972f08, proto 0xa11aaa platformSetWindowEnv# [ast: 0x0003,0x0003] < +0x861aa9> }
  item <platformSetWindowEnv#> in ~beta/postwimp/foundation/private/worldbody
  { PC  0x48d627 < +0x2dd626>, object 0x2972f08, proto 0xa11aaa platformSetWindowEnv# [ast: 0x0003,0x0003] < +0x861aa9> }
    -- PROGRAM-~CPNWorkSpace#Workspace#World#SystemEnv#SysHead# in C:/cpn2000/cpntools/cpntools
    { Surrounding object 0x2970c10, proto 0xa0e0a0 PROGRAM-~ [ast: 0x0001,0x0001] < +0x85e09f> }
  item <setWindowEnv#> in ~beta/postwimp/foundation/world
  { PC  0x631d6b < +0x481d6a>, object 0x2972ef8, proto 0xacf3ac setWindowEnv# [ast: 0x0001,0x1da3] < +0x91f3ab> }
    -- PROGRAM-~CPNWorkSpace#Workspace#World#SystemEnv#SysHead# in C:/cpn2000/cpntools/cpntools
    { Surrounding object 0x2970c10, proto 0xa0e0a0 PROGRAM-~ [ast: 0x0001,0x0001] < +0x85e09f> }
  item <start#> in ~beta/guienv/private/winnt/guienvntsystemenvbody
  { PC  0x44c4b7 < +0x29c4b6>, object 0x2972ee8, proto 0x96ddc4 start# [ast: 0x0005,0x01aa] < +0x7bddc3> }
    -- PIOPRIVATE-~ in ~beta/guienv/private/winnt/guienvntsystemenvbody
    { Surrounding object 0x2971f30, proto 0x96db8e PIOPRIVATE-~ [ast: 0x0005,0x0001] < +0x7bdb8d> }
  comp <BASICSCHEDULER-~>SysHead# in ~beta/guienv/private/winnt/guienvntsystemenvbody
  { PC  0x44ba90 < +0x29ba8f>, object 0x2972dc0, proto 0x96dade BASICSCHEDULER-~ [ast: 0x0002,0x0001] < +0x7bdadd> }
    -- PROGRAM-~CPNWorkSpace#Workspace#World#SystemEnv#SysHead# in C:/cpn2000/cpntools/cpntools
    { Surrounding object 0x2970c10, proto 0xa0e0a0 PROGRAM-~ [ast: 0x0001,0x0001] < +0x85e09f> }
  comp BASICSCHEDULER-~<SysHead#> in ~beta/basiclib/basicsystemenv
  { PC  0x740b4b < +0x590b4a>, object 0x2972dc0, proto 0xadb516 SysHead# [ast: 0x0001,0x0310] < +0x92b515> }
    -- BETAENV-~ in ~beta/basiclib/betaenv
    { Surrounding object 0x2970060, proto 0x94e3a0 BETAENV-~ [ast: 0x0001,0x0001] < +0x79e39f> }

  item <*> in ~beta/basiclib/betaenv
  { PC  0x48b24c < +0x2db24b>, object 0x2970b48, proto 0x9509aa * [ast: 0x0001,0x0c3c] < +0x7a09a9> }
    -- BETAENV-~ in ~beta/basiclib/betaenv
    { Surrounding object 0x2970060, proto 0x94e3a0 BETAENV-~ [ast: 0x0001,0x0001] < +0x79e39f> }
  basic component in ~beta/basiclib/betaenv

Legend:

The above dumps show the dynamic call stack of invoked objects.
Each dump starts at the object that was the current object when
the error occurred and continues down towards the basic component.
The descriptions have the following meaning:
1. Items are shown in two lines, like this:
      item <name#>pname1#pname2#pname3 in ifile
        -- sname#spname1#spname2 in sfile
   meaning that the item is an instance of the pattern "name" which
   has prefix "pname1" which has prefix "pname2", etc.
   This item is defined in the file "ifile".
   The part of the prefix chain enclosed in "<" and ">" indicates
   where in the action sequence the error occurred.
   The line beginning with "--" shows the textually surrounding
   pattern using the same notation.
2. The descriptor names used in the above description will normally
   have one or more "meta characters" appended. These mean:
      #  The descriptor belongs to a pattern, e.g. P: (# ... #)
      ~  Singular named descriptor, e.g. X: @(# ... #)
      *  Singular unnamed descriptor, e.g. ... ; (# ... #) ; ...
      -  Descriptor SLOT.
3. Components are shown using a notation similar to that of items:
      comp <name#>pname1#pname2#pname3 in cfile
4. The bottommost component corresponding to the basic environment is
   shown like an ordinary component, but indicated with
      basic component <name#>pname1#pname2#pname3 in cfile
5. In case the error occurred in some external code called from BETA,
   the top of the call stack is shown as
      [ EXTERNAL ACTIVATION PART ]
6. In case the BETA code has called some external code which has in
   turn called back into the BETA code, and the callback is still
   active at the point of the error, the intermediate call stack part
   is shown as
      [ EXTERNAL ACTIVATION PART ]
7. The section labeled "Low level information" can be avoided by
      set BETART=SimpleDump
   before subsequent executions of the program.

Sunday, November 30, 2014

Dictionary: Remote Procedure Call

The concept of Remote Procedure Call (RPC) was developed by Bruce Jay Nelson, a Xerox Parc researcher, in 1981. In 1984, he and his fellow researcher from Xerox PARC published a paper which entitled "Implementing Remote Calls". This paper describes of implementation of RPC that is used in the Cedar project.

Remote Procedure Call (RPC) is an important concept in a distributed system. It was developed by observation of procedure calls that are well-known mechanism for transfer of control and data within a program running on a single computer. As the networked system grown into popular there is a need to transfer control and data across the network in a simple way. When calling a remote procedure, the calling (caller) environment is suspended, the parameter are passed across the network to the environment where the procedure is executed (callee), and desired procedure will be executed there and then send the result back to the caller.

So in a simple way, RPC is a concept of computer programming for letting a computer program to execute a method or sub-routine in a remote computer and get the result of that execution easily, simpe, and straight-forward as calling a local function. RPC is also known as remote invocation or remote method invocation in a object oriented principles.

What processes happen when a client calls a RPC method or function?

    1. Create a message buffer (contigious array of bytes of some size).
    2. Pack the needed information into the message buffer. The information includes identifier of called function and function arguments. This process often called as message serialization or marshaling the argument.
    3. Wait for the reply, because the function calls are usually synchronous, the call will wait for its completion.
    4. Unpack return code and other arguments. This unpacking process often called as unmarshaling or deserialization.
    5. Return to the caller and the caller can continue for more processing.

Sources:
  1. http://en.wikipedia.org/wiki/Remote_procedure_call
  2. Birrell, A. D.; Nelson, B. J. (1984). "Implementing remote procedure calls". ACM Transactions on Computer Systems 2: 39. doi:10.1145/2080.357392
  3. Arpaci-Dusseau, Remzi H.; Arpaci-Dusseau, Andrea C. (2014), Introduction to Distributed Systems

Tuesday, November 25, 2014

Ubuntu Troubleshoot: Ubuntu can't change display device brightness

After installing my Ubuntu 14.04 all control works fine except for the brightness controller. The brightness button on my keyboard is just fine and able to show the brightness level is changing, but the display brightness is not changing. To fix this I must add a new file named 20-intel.conf in the /usr/share/X11/xorg.conf.d/ directory and enter the below configuration as the content .

Section "Device"
Identifier  "card0"
Driver      "intel"
Option      "Backlight"  "intel_backlight"
BusID       "PCI:0:2:0"EndSection

and now my brightness controller works as I expect.


Sunday, November 23, 2014

Hadoop Tips: Change default namenode and datanode directories

When we start Hadoop in psudo-distributed mode using the sbin/start-all.sh command for the first time, The default directory will be created in /tmp/ directory. The problem arises if you restart your machine the created directories will be deleted and you can't start your hadoop again.

To solve this problem you can change the default directory using configuration file that you can find in your hadoop's etc/hadoop/hdfs-site.xml. Add these configuration properties:

    <!--for namenode-->
    <property>
        <name>dfs.namenode.name.dir</name>
        <value>file:///path/to/your/namenode</value>
    </property>

    <!--for datanode-->
    <property>
        <name>dfs.datanode.data.dir</name>
        <value>file:///path/to/your/datanode/</value>
    </property>

But please make sure, your namenode and datanode directories exist. And don't forget to set the properties value using correct URI (started with file://) format just like in the example. After that you can format your namenode using this command:

$ bin/hdfs namenode -format

and start your hadoop again using:

$ sbin/start-all.sh

If the namenonde or datanode still not working, you can check log files to see the problem.
  
Hope this tips could help you. If you find other problem related to this, please leave a comment below. Cheers! :)

Tuesday, November 18, 2014

Hadoop Troubleshoot: Hadoop build error related to findbugs, Eclipse configuration, protobuf, and AvroRecord

Last week, I was trying to build Hadoop 2.5.0 from source code. I tried several ways to build the source code, first one is using Maven in terminal, and the second one is using Eclipse as my IDE.

1. Related to findbugs (using maven in terminal)

I read the BUILDING.txt that you can find in the root of Hadoop source code directory. And I ran this command to build me a hadoop package:

$ mvn package -Pdist,native,docs,src -DskipTests -Dtar

And some how in the middle of long building time, there is an error related to FINDBUGS_HOME environment variable as you can see in this specifice error message:

hadoop-common-project/hadoop-common/${env.FINDBUGS_HOME}/src/xsl/default.xsl

I already have the findbugs installed and then I tried to set the FINDBUGS_HOME to /usr/bin/findbugs using this command:

$ export FINDBUGS_HOME=/usr/bin/findbugs

But still no use, the error was still there. So I downloaded the findbugs source code from sourceforge and set once again the FINDBUGS_HOME to the findbugs source code root directory.

$ export FINDBUGS_HOME=/path/to/your/<sourcecode>/findbugs

I tried to run build command again and it went well this time. :)


2. Build path error (Eclipse configuration)

When you're trying to import the Hadoop projects into your Eclipse workspace, and try to build all the projects, probably you will many kind of errors, but you could see this specific error message too:

Unbound classpath variable: 'M2_REPO/asm/asm/3.2/asm-3.2.jar' in project 'hadoop-tools-dist' hadoop-tools-dist

This error related to M2_REPO Classpath variable in Eclipse. To solve this problem you can open Classpath Variable configuration  from Eclipse menu:

"Windows -> Preferences". It will open Preferences dialog. After that, in that dialog you can go to:

"Java -> Build Path -> Classpath Variable". You can add new Classpath Variable and give name to the new variable M2_REPO  and fill the path with: 

/home/<username>/.m2/repository  

Try to rebuild all you projects. You won't see those kind of error again after that. But there are still many error in your project.

3. hadoop-streaming build path error

If you are lucky (or not), you will find thiserror message related to hadoop-streaming build path error:

Project 'hadoop-streaming' is missing required source folder: '/hadoop-2.5.0-working/hadoop-yarn-project/hadoop-yarn/hadoop-yarn-server/hadoop-yarn-server-resourcemanager/conf'

This error is quite strange. Because the build path can't be changed although you already edited it (I tried this many times and it was quite annoying). But the solution more strange, just remove the build path configuration and it will disappear. You can do it this way:

To open the configuration, you can right click to your hadoop-streaming project to open the context menu. And then choose "Build Path -> Configure Build Path". It will open "Properties for hadoop-streaming" dialog. From that dialog, you can choose "Source" tab and select the problematic source folder and then press the "Remove" button. After that just rebuild the projects. And the remaining error you see is related to protobuf and avro missing files.

4. Protobuf and Avro missing files

The remaining errors you might find are related to missing files in hadoop-common project. The problematic packages in the project are org.apache.hadoop.io.serializer and org.apache.hadoop.ipc.protobuf (you can see the protobuf directory is empty). I have done searching about the empty protobuf directory but it only tells us to rebuild the project, and it will automatically generate the file for you, using maven with these commands: 

$ mvn clean
$ mvn compile -Pnative
$ mvn package
$ mvn compile findbugs:findbugs
$ mvn install
$ mvn package -Pdist,docs,src,native -Dtar

I don't know if this one works for you, but it didn't work in my project. These errors messages still linger in my project:

AvroRecord cannot be resolved to a type
The import org.apache.hadoop.ipc.protobuf.TestRpcServiceProtos cannot be resolved
The import org.apache.hadoop.ipc.protobuf.TestProtos cannot be resolved
TestProtobufRpcProto cannot be resolved to a type
TestProtobufRpc2Proto cannot be resolved to a type
EmptyRequestProto cannot be resolved to a type
EchoResponseProto cannot be resolved to a type

EchoRequestProto cannot be resolved to a type

After I found this very good site called grepcode, and I was able to find all classes and files that I needed (you even can get from other version of hadoop!).
For example you can download the AvroRecord.java file here, and you can put it in the directory respectively.


That's all. If you find this post is useful, please leave a comment below. See you in my next post.

Thursday, November 13, 2014

MLj Tips: MLj set up and installation

After some trials and errors I managed to run my MLj in my machine. A lot of mistakes I made because I can't find any good tutorial out there. So, I write one for myself. You can read about MLj here.

Here are some steps that I did when setting up MLj in my machine:

1. Get the MLj for Linux

http://www.dcs.ed.ac.uk/home/mlj/dist/downloads/mlj0.2c-linux.tar.gz

2. Install the SMLNJ

You can install the SMLNJ using this command:
$ sudo apt-get install smlnj

3. Extract mlj0.2c-linux.tar.gz

You can extract your mlj package somewhere in your drive. It will contain mlj as the root directory.

4. Go to your mlj/bin directory

Open and go to your mlj/bin directory. You'll find 4 files there. They are mlj, mlj.bat, mlj-jdk1.1.1.x86-linux, and run.x86-linux.

5. Try to run the mlj

Use this command to run the mlj:
$ ./mlj    

Probably it will raise this kind of error:
./mlj: 1: ./mlj: .arch-n-opsys: not found
 mlj: unable to determine architecture/operating system

to fix this, you can set your PATH environment variable using this command:
$ export PATH=$PATH:<your mlj/bin path>

or

you can open and edit your mlj file and change this part of code:
ARCH_N_OPSYS=`.arch-n-opsys`
to    
ARCH_N_OPSYS=`./.arch-n-opsys`

Try run ./mlj or mlj once again. and the error is still there. Because the .arch-n-opsys file is not compatible with current desktop architecture environment. It's okay, we will get rid of it in the next step. 

mlj: unable to determine architecture/operating system

6. Copy .arch-n-opsys file from your smlnj

You may create backup of your current .arch-n-opsys file if you want and then copy the new one from the smlnj directory. Here are the commands:
$ mv .arch-n-opsys .arch-n-opsys.bak
$ cp /usr/lib/smlnj/bin/.arch-n-opsys .

7. Try to run mlj once again

Try to run ./mlj or mlj again and you'll get this view:

MLj 0.2c on x86 under Linux with basis library for JDK 1.1
Copyright (C) 1999 Persimmon IT Inc.

MLj comes with ABSOLUTELY NO WARRANTY. It is free software, and you are
welcome to redistribute it under certain conditions.
See COPYING for details.

Your installation is done!

If you have any question or find my post useful please leave a comment. 

Tuesday, November 11, 2014

Dictionary: First Class Methods/Functions

In Programming language first class methods/functions means the programming language treats the method or function as a first class citizen. 

Based on Structure and Interpretation of Computer Programs 2nd Edition Book, elements with fewest restrictions are said to have first-class status. The rights and privileges  of first-class element are:
  • They may be named by variables.
  • They may be passed as arguments to procedures.
  • They may be returned as the results of procedures.
  • They may be included in data structures.
So if a language support a first-class function, it will let the functions to be passed as a parameters, and returned as a result of procedures. First class function is necessary for the functional programming style.

Example of programming languages that support first-class function are Scheme, ML, Haskel, F#, Perl, Scala, Python, PHP, Lua, JavaScript, C#, C++, and etc.

Sources:
  1. http://en.wikipedia.org/wiki/First-class_function
  2. Abelson, Harold; Sussman, Gerald Jay (1996). Structure and Interpretation of Computer Programs - 2nd Edition. MIT Press.