Awk script that checks duplicate values in an XML

I wanted to test if a key-combination of 3 field values appeared multiple times in an XML file. A colleague of mine gave me head start on text-processing using Awk. OS X ships with the BSD version of awk so this came out quite handy.

The input XML:

<ehbo>
    <order>
        <siebelorderid>10</siebelorderid>
        <siebelordernumber>20</siebelordernumber>
        <cordysordernumber>30</cordysordernumber>
    </order>
    <order>
        <siebelorderid>10</siebelorderid>
        <siebelordernumber>20</siebelordernumber>
        <cordysordernumber>30</cordysordernumber>
    </order>
</ehbo> 

The Awk script:

BEGIN {
print "START";
print "";
}
/SiebelOrderID/ {
       line = $0;
       split(line, a, ">");
       split(a[2], b, "<");
       siebelOrderID = b[1];

  #print "a" siebelOrderID;
}
/SiebelOrderNumber/ {
       line = $0;
       split(line, c, ">");
       split(c[2], d, "<");
       SiebelOrderNumber = d[1];

  #print "b" SiebelOrderNumber;
}
/CordysOrderNumber/ {
       line = $0;
       split(line, e, ">");
       split(e[2], f, "<");
       CordysOrderNumber = f[1];

       #print "c" CordysOrderNumber;
}

/\/Order/ { 

plep = siebelOrderID "_" SiebelOrderNumber "_" CordysOrderNumber;

lijst[plep]++; 

#print "d" plep;

siebelOrderID="";
SiebelOrderNumber="";
CordysOrderNumber="";

}

END {
       for (i in lijst) {
        if (lijst[i]-1) {
print i, lijst[i]
}
       }

print "";
print "FINISHED!";
}

To make life easier, I put the script in a file called "parser.awk" and then used the following command in Terminal:
awk -f parser.awk input.xml

This will print the field combinations that occur multiple times.

References:

• http://www.unix.com/unix-dummies-questions-answers/19844-length-string.html
• http://www.vectorsite.net/tsawk.html
• http://commandlinemac.blogspot.nl/2008/12/learn-to-talk-awk.html

Batch file that loops through files

This week I had to create a batch file that loops through folders and files. Dependent on the path, I needed to create a new.SQL file and append the file names to the text file.

This is the script:

:BEGIN

REM Creating CreateStatements Files...

@echo off

rem This script makes a CreateStatementsFiles.
rem Version: 3
rem Last script update on: 2012-08-30 by Joshua Moesa

:SET_ENVIRONMENT_VARIABLES
rem Setting environment variables
set rootPathBatchFile=C:\DBSCRIPTS\
set scriptRootPath=%rootPathBatchFile%scripts\
set fileOutput=%rootPathBatchFile%output.txt
set fileOutputSub=%rootPathBatchFile%outputSubRoutine.txt

:INIT
rem Cleaning-up

::Cleanup all mk_-files
FOR /R %rootPathBatchFile% %%G IN (mk_*.sql) DO (
DEL %%G
)

:CREATE_BUFFER

rem Printing all SQL filenames to buffer file
FOR /R %rootPathBatchFile% %%G IN (cr_*.sql) DO echo %%G >> %fileOutput%

::DEBUG
::echo %%~pG >> %fileOutput%

:PROCESS_BUFFER_FILE

rem Processing lines in the buffer file

FOR /F "tokens=*" %%A in (%fileOutput%) do (
call :process %%A
)

goto CLEANUP

:process
::echo %1

set fullFileName=%1

call :parse "%fullFileName%"

goto :end

:parse
setlocal
set list=%1
set list=%list:"=%

FOR /f "tokens=4,5,6 delims=\" %%a IN ("%list%") DO (
  if not "%%a" == "" call :sub %%a %%b %%c
  if not "%%b" == "" call :parse "%%b %%b %%c"
)

endlocal

exit /b

:sub

setlocal

::DEBUG
::echo In subroutine
::echo %1 %2 %3 >> %fileOutputSub%
::echo %scriptRootPath%%1\%2\makefile.txt >> %fileOutputSub%

if not exist %scriptRootPath%%1\%2\mk_%2.sql (
 echo prompt create %2 > %scriptRootPath%%1\%2\mk_%2.sql
)

echo @@%3 >> %scriptRootPath%%1\%2\mk_%2.sql

endlocal
exit /b


:CLEANUP

DEL %fileOutput%



:end

Some handy references I used:

• http://www.jamesewelch.com/2008/05/01/how-to-write-a-dos-batch-file-to-loop-through-files/
• http://stackoverflow.com/questions/1100452/get-filename-in-dos-for-loop
• http://weblogs.asp.net/jgalloway/archive/2006/11/20/top-10-dos-batch-tips-yes-dos-batch.aspx
• http://www.nickpisca.com/BLAST/index.php?title=Batch_Txt_Loop_with_Delimiter
• http://stackoverflow.com/questions/155932/how-do-you-loop-through-each-line-in-a-text-file-using-a-windows-batch-file
• http://stackoverflow.com/questions/2524928/dos-batch-iterate-through-a-delimited-string

X2Go client for Mac: connect with Unix

At my assignment, we switched to Unix development machines. Prior to this, we developed on the Microsoft SQL Server 2003 platform so I used the excellent CoRD remote desktop client to create a RDP connections.

For our new situation we're using X2Go.

x2go is an open (GPL/AGPL) source “server based computing” project. Combining the advantages of existing systems it features ease of use, performance and scalability. x2go provides you with access to your desktop as an individual as well as a corporate user - from within your own network and via the internet. x2go is not limited to particular hardware, it supports a variety of devices and architectures. x2go is open source and open minded. Like any open source project we welcome your support

There's a good X2Go client available for Mac which I use to connect to our Unix machines. I had some startup problems due to my recent OS upgrade to Mountain Lion. I noticed that X11 isn't included in Mountain Lion, so I went to XQuartz to download the latest X11 app. After install and re-login, I then went to Preferences and modified the X-Server settings. Don't click Find X11 application because it will reset the setting incorrectly.

After fiddling around with the connection settings, I managed to connect!

References:

• http://www.x2go.org/
• http://cord.sourceforge.net/
• http://www.macrumors.com/2012/02/17/apple-removes-x11-in-os-x-mountain-lion-shifts-support-to-open-source-xquartz/

Time tracking with the free TimeTracker Mac app

Something I'm going to test out this week.
http://code.google.com/p/time-tracker-mac/

This is a screenshot from what's upcoming:

Collaboration within a DJ/producer team

I believe that Flosstradamus nailed it with their set-up; Logic, DropBox and Sharetool

This video shows what they use in their live-sets:

Version control with Git and Bitbucket on a Mac

I was looking into a way to put my code into a private cloud version control system. My choice for now: bitbucket.

• First I installed the git-osx-installer for Snow Leopard (allthough I'm running Mountain Lion now).
• Then I created an account on bitbucket.
• I followed the wonderful bitbucket 101 to get a grasp of the Git commands (coming from a SVN world).
• I installed the beautiful and free SourceTree Git client
• I created a repo on bitbucket and cloned it to my Mac.
• Added files to my working copy and added and committed them into bitbucket.
• Done!

Java Build path and Mac OS X Mountain Lion

After my update from Snow Leopard to Mountain Lion, my Eclipse environment build path got srewed up. My project folder icons showed a red exclamation mark which hinted me.
Have to look into it a bit further (by checking the Java for Developers packages), but I did this as a quick workaround:

1. In Eclipse, go to Preferences
2. At Installed JREs, add a JRE of type MacOS X VM
3. For the JRE Home use location: /System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

Reference:

• http://stackoverflow.com/questions/6614380/jdk-on-osx-10-7-lion
• http://forum.springsource.org/showthread.php?119513-Build-Bath-Issue-after-I-updated-to-MAC-OSX-Lion
• http://maclord.ozarweb.com/mevzu/install-java-on-mac-lion/