Monday, April 30, 2012

UniTrunker Logfile Merging

The program UniTrunker uses XML files to output data pertaining to Systems, Groups, Users, Sites, etc.  It stores call log files in Unicode format with just User numbers and Group numbers, but not text labels associated with them. I wrote a C program to merge the User and Group data with their respective numbers in the call log file.  This is a multiple step process to extract information from the System.xml file into text files for Users, Groups, and Sites; followed by converting those Unicode files to ascii files for handling with the C code; and then running them to created a merged logfile with both User and Group lables appended.  For this I created a batch file to complete the process with 2 inputs from the user during the process.

Prior to running the batch file, all of the needed files need to be placed in the same directory.  These include:

msxsl.exe (which can be downloaded from Microsoft's website)
UTC.exe (the program file which I wrote to perform the merge)
UTMerge.bat (the batch file which calls all the programs and converts files to correct filetypes)
UTxsl.bat(the batch file which calls the msxsl.exe to apply the xsl stylesheets)
UTlog.bat(the batch file which calls UTC.exe program to create a merged log file)
System.xml (this file is exported by going to the systems tab in UniTrunker and exporting the desired system)
users.xsl, groups.xsl, sites.xsl which I have created/modified based on the example from the UT website
UniTrunker-********.LOG file (created daily by UniTrunker and archived to a gzip file)
      -Note this file needs to be unzipped in order for the batch file to work

Once all the files are in the same directory, simply change directories to the location of the files, and then type "utmerge" to call the batch file.

The user will be prompted to enter the name of the .xml file.  Case doesn't matter, but be sure to type it correctly.  After a few seconds of processing the user will be prompted for the date of the logfile to be merged.  After a few more seconds a text file with the merged data will be created in the same directory.

A zip file of the required files can be found here:

UniTrunker Merge Program files

Download and extract the UTMerge.zip file to an accessible location.  Then extract a system.xml file from UniTrunker.  Copy or move a .LOG file from UniTrunker's logfiles to this same directory.  You can go to Run...and type CMD to open a DOS window.  Then change directories to the location of your UTM files and type utm to run the batch file. 

An easier way is to use windows explorer to browse to the directory and double click on UTM.bat. The program will run in a popped-up DOS window.  Carefully type in the information required when prompted, and VOILA, a merged log file will be created!

UPDATE:

I created 2 additional .bat files:UTxsl and UTlog. The UTmerge file can still be used successfully.  However, each time it runs it creates the same users.txt, group.txt, and sites.txt files.  This does no harm, but is redundant and takes time.  I spilt the processes into 2 batch files.  The first to be run should be UTxsl.  This will apply the xsl stylesheet once and create the necessary .txt files.  After these are created, there is no need to run this batch file again unless changes are made in UniTrunker to your system files and subsequently exported.

The second batch file to be run is UTlog.  This simply prompts for the date of the log file you would like to merge with the users and groups files.  It can be run multiple times on logs for different dates without recreating the users.txt and groups.txt files.

UPDATE 2 (February 23, 2013)

After looking for a way to further process the merged log files, I realized the Linux grep command was perfect for looking through logs to isolate individual users or groups or both.  This means I either have to run a grep program on my Windows computer, or move the files to my Linux computer to further process them.  It is a little cumbersome to have to move files to a different platform, but the power to sort the data makes it worth doing.

The power of the the grep command in dissecting the large amount of information in one days' log file is impressive.  For example, after a recent fire department response to a building fire, I sorted the log files based on the fire departments operations channel to see just the calls for the fire separate from the 1000's of other calls on the TRS for the day.

For a better explanation of the grep command see:

http://en.wikipedia.org/wiki/Grep


A sample output from a TG sort using the command:

~$ more UT20130220merged.txt | grep 10848 

20130220210159,2,Call,I,32092,Engine 85,G,10848,Davis Ops 6 - South Davis Metro Fire (Bountiful fire response),488, - 
20130220210242,2,Call,I,32014,Ambulance 83,G,10848,Davis Ops 6 - South Davis Metro Fire (Bountiful fire response),525, - 
20130220210247,2,Call,I,32014,Ambulance 83,G,10848,Davis Ops 6 - South Davis Metro Fire (Bountiful fire response),497, - 
20130220210256,2,Call,I,32031,Engine 85,G,10848,Davis Ops 6 - South Davis Metro Fire (Bountiful fire response),498, - 
20130220210300,2,Call,I,32014,Ambulance 83,G,10848,Davis Ops 6 - South Davis Metro Fire (Bountiful fire response),498, -

Friday, April 20, 2012

Great Utah Shake Out Mock Disaster Audio

I managed to capture some audio from the Davis County EOC during the mock earthquake drill on April 17, 2012 as part of the State of Utah's Great Utah Shakeout Disaster Drill.

Davis County Disaster Talkgroup Audio

Great Utah Shakeout 2012

This Tuesday was the Great Utah Shakeout earthquake drill. I was looking forward to the opportunity to review our family's preparedness at home and then participate in the drills that were to take place in the community. It was fun to have the chance to go to the Davis County EOC and work in the Amateur Radio room. We got to pass mock traffic and interact with the county officials in the mock disaster. Afterwards, the debrief was also informative and reminded me that we always need to look for ways to improve our plans to become prepared for whatever type of disaster that may occur.


The county has a nice radio setup with capabilities on many different modes from CW to packet and band capabilities from HF to UHF.  Area repeater coverage was also excellent with contacts from the North end of the state to the South end.  Seeing how a good station like K7DAV is put together even gave me some ideas for improvements for my own shack.