Any Given Sunday

The Bills pulled out their first win of the season against the Jets, which was the first clue that something was "wrong".

Over half of my Last Man Standing pool (myself included) picked San Diego since they were so highly favored to win, and THEY LOST!

On top of that, my 3-0 Steelers went 3-1, against the freaking CARDINALS!

Must be a bad dream, must be a bad dream, must be a bad dream!

Don't Bother Me

The national "do not call" registry expires every 5 years, so first wave is about up. To re-register.... https://www.donotcall.gov/Register/Reg.aspx

Rhythm's Gonna Get You

Last night was Raina's big 4th birthday, so we went to Chuck E. Cheese. On the way home, our Sirius radio was, of course, tuned to "Kid's Stuff", and annoying little Dora The Explorer was singing some cheesy 80's song, Rhythm's Gonna Get You. The song was bad enough the first time around, did we really need Dora to "sing" it for us?

Anyway, that's not even the point. Half way through the song, Raina asked us if Rhythm is a bad guy? Took us a few seconds to realize what she was getting at. In the Disney movies they've seen, the bad guys "get you", so Rhythm must be a bad guy since it is trying to "get you". It was funny, and also kind of scary. As adults, we see or hear things but don't notice anything odd, but the kids are processing it and picking it apart, not missing a single thing.

Word, Character, Line Counter Ruby Script

I've been on a few projects lately that involve loading data files that I get from other teams. When I get a file, it's nice to see how big it is to get an idea of how long it will take. Opening the file in a text editor and going to end of file to see the row number can be a pain, especially for big files. Also, I'm on a Windows desktop, so tools like "wc" aren't built-in, and launching Cygwin and getting to the right directory can be a pain in itself (cd /cygdrive/c/Documents\ and\ Settings/ran488/Desktop), so I whipped up this quick Ruby script to give me all the file info I want.

Here's the Ruby script (fileinfo.rb) to get the count of lines, characters, and words in a text file. It also spits out the min and max line size as well. The source can be downloaded at http://www.frontiernet.net/~nicholson150/fileinfo.rb.

#!/usr/bin/env ruby if __FILE__ == $0 if ARGV.length < 1 then puts "Usage: #{$0} filename" exit end results = [] words = 0 chars = 0 minline = 0 maxline = 0 filename = ARGV.first File.new(filename, "r").each { |line| results << line } puts "#{filename} has..." puts " -> #{results.size} lines." results.each do |line| chars += line.length words += line.split.length if line.length > maxline then maxline = line.length elsif line.length < minline then minline = line.length end end puts " -> #{words} words." puts " -> #{chars} characters." puts " -> #{minline} character shortest line length." puts " -> #{maxline} characters longest line length." end

Here's an example of the output....

C:\cvs\MqThrottler>fileinfo.rb PA_CSS_Test_Load_0911.txt PA_CSS_Test_Load_0911.txt has... -> 10008 lines. -> 58021 words. -> 4356721 characters. -> 0 character shortest line length. -> 489 characters longest line length.

Lexi

Here's an updated picture of Lexi the Wheaten Terrier. She is about 7 months old here, mostly white and wheat color with just hte black face remaining.

PDFCreator

Apps like OpenOffice.org have built-in PDF export functions, and Mac OS X has a built in PDF export in the print dialog.

But if you're running apps on Windows that don't have this feature, you can download and install PDF Creator. It installs itself as a "printer" on your PC. When you choose to print from any application, choose this printer instead of your normal printer.

Bulk File Renamer in Ruby - file_rename.rb

My updated bulk file renamer script is listed here. (http://www.frontiernet.net/~nicholson150/file_rename.html) along with the Ruby source code.

I first started messing with this concept in this post, but have refined it a bit since.

Along with the Ruby compiler I mentioned yesterday, this could be compiled and dropped in your PATH somewhere (e.g. Windows dir), and then you could run it anywhere as if it were a native Windows/DOS command. I may end up doing that on my home PC to help rename the imported digital photos-- "DCP_05483.JPG" just isn't as descriptive as "Jeep Camping Trip 2007 - 012.jpg"

RubyScript2Exe - The Ruby Compiler

I've been playing with RubyScript2Exe (http://www.erikveenstra.nl/rubyscript2exe/index.html) lately, and it's pretty sweet. It's a Ruby "compiler" to generate executables from your Ruby scripts. Since it embeds the Ruby runtime and any libraries in the executable, even simple scripts can "bloat" up to bigger sizes than you might expect, but if distribution is your thing, so what?

Maybe you want to share some of your cool scripts and development tools you've created, but can't count on your intended audience having Ruby installed (what's wrong with them, anyways?), or maybe it would take too long to explain what Ruby is and how to run the scripts. Or maybe you just don't want people screwing with your perfect source code. Or perhaps, you are ashamed of your trashy, hack-n-whack coding?

Whatever the reason, this thing is the nuts. Just download a single rubyscript2exe.rb script and run it against your app. I've only tried it on single files on Windows so far, but it is supposed to create Linux executables as well, and will allegedly work on more complex, multi-*.rb-file apps.

Crazy Women Drivers

We went to a picnic a few weeks back, and my wife got to "drive" a horse-drawn cart. Here she is with our friend Anita driving the horses.....

Yee-haw!

A Few Notes on my Load Tester Ruby Script

A few days ago I posted a Ruby script I use to simulate multiple users hitting a web application. I have a few notes regarding that script:

1. There is a minor bug in it, that doesn't affect functionality so much as the "logging". I used a global var like a dummy and then reference that in the threads.
   
2. I also added a 3rd command line param to tell the script how many times each thread should run through the list of URL's.
   
3. I also added a sleep() feature so each thread will pause between each run through the list of URL's. Right now that is just a global var instead of a command line param.

I will have to repost the source to get these changes up here. (Actually the *.rb source file is up, I just haven't regenerated the HTML yet.)

On a related note, I wanted to do a better simulation of the types of requests coming through to our prod box. My clever idea, take the actual requests out of Apache's access_log or (in my case) ssl_request_log.

So how to easily grab all those requests to build my file? The format of my ssl_request_log looks a bit like this (some stuff renamed to protect the innocent):

[08/Aug/2007:14:42:23 -0400] 192.237.32.76 SSLv3 RC4-MD5 "GET /appcontext/en/US/CZN2/css/main.css HTTP/1.1" -

Well I just want the URI starting after the GET, so from the Linux/Unix/Cygwin command line do something like this....

$ cat ssl_request_log | cut -d' ' -f 7 2>&1 | tee URIs.txt

Then in vi, I can just put the "http://myserver" portion of the URL at the beginning of each line with a global replace, and I'll have a file with every request, including those for images, stylesheet *.css files, javascript *.js files, etc.

This is much easier than sitting here trying to piece together URL's by hand.

A Great Interview Question

I had to do a few interviews last week, and some of my standard questions were getting a bit stale. So I went out on Google and searched for some new ammo.

I came across this question: "What is the difference between final, finally, and finalize()?"

At first I thought that was too easy, mostly because I knew the answer, and it seemed fairly obvious. Even so, I broke it out in an interview, and the guy nailed the first two, missed the last one.

I was relating that tale to a coworker afterward, and she said, "what was the question he missed?" She nailed the first two, didn't know the last. She's a good programmer; I've worked side by side with her every day for the past few years, and she missed it. That's when I came to the conclusion this is a great question.

Even good programmers don't know everything, and not many Java programmers really have to get into the bowels of memory allocations and garbage collection. This is a great question, not because I want the guy who can answer all 3, but because I want to see how people handle themselves when they don't know the answer. Sometimes that will tell you more about the person than if they just know all the answers.

The first guy I asked this question got the first two 100% correct, then he flat out said he didn't know what finalize was, hadn't run across it, and then he wrote it down in his notebook and said he would have to look it up.

The second guy I asked, he missed it, but didn't seem to know he missed it. For finally, he stammered about a bit and mentioned it was for garbage collection, then switched gears a little and said you usually use it with a try/catch block. Then for finalize, he said he didn't know what it was.

So both missed it, but was one answer better than the other?

In my mind, first guy won. He admitted what he didn't know and showed some level of ambition to look it up. The second guy tried to dazzle me with a little BS for a minute before admitting he didn't know. He didn't write anything down or mention looking it up, he just racked it up as a missed question.

They both went on to answer other technical questions to my satisfaction, but I came away with a slightly difference perspective of each. If they both just nailed every tech question I threw at them, I wouldn't have that added bit of perspective. And how do you know it isn't just "book smarts" -- a lot of those guys that rattle off every new buzzword and argue about architectures and design patterns can't write code to save their lives.


Other Interview Follies:

A lot of funny things do happen when interviewing. Everything I have ever learned about interviewing seemed like common sense, until I met some of these folks:

• One of my manager's standard questions: "How would you structure your ideal work day?" (e.g. 50% development, 20% support, etc.) One guy told us he would sleep in, get to work late, of course he would stay late also, but his brain just doesn't work in the morning unless he has 10 cups of coffee, so he would rather sleep. Honesty is NOT always the best policy.
  
• One lady told us she mostly does front-end work. When I asked her what she was using to build the interfaces (Swing, JSP, JSF, etc.), she told me she mostly does back-end server work. Nice 180, thanks for coming in today!
  
• Another guy just got done telling us about all these web applications he wrote, JSP's, servlets, etc. I asked if he was using STRUTS, and he told me he wasn't sure, he might be using it. Then when I asked what application server he was using, he said UNIX and Windows. I tried to clarify, asking him which J2EE server he was using, he looked confused; so I changed my terminology to "servlet and JSP container", he still looked confused; so I made it multiple choice: "Are you using Websphere, Weblogic, Tomcat, Jboss?" His final answer: "Tomcat sounds familiar."
  

HTTP Load Tester Script in Ruby

Here it is, after running my newly acquired Ruby code to HTML converter script, I am finally able to post decent looking Ruby code. Just not directly on Blogger (yet).

Here's the link to my HTTP "Load Runner" script I have been using to debug certain issues (memory leaks, database connection leaks, etc) with a certain un-named inherited application. It's written in Ruby.

For the more hard-core stuff I use JMeter, but this served two purposes:
1. learning more about Ruby
2. having a quick and simple script for small stuff instead of the heavyweight JMeter.


HTML: http://www.frontiernet.net/~nicholson150/http_load_tester.html
Ruby Source: http://www.frontiernet.net/~nicholson150/http_load_tester.rb

Making Code Pretty For The Web

Any time I post code to this stupid blog, I always run into the same issue. How to make it look like readable code? Seems that just putting code inside <PRE> tags doesn't seem to work -- something about the Google / Blogger engine mangles the crap out of the end if it gets "too long".

For Ruby at least, I just found this blog entry that helps out immensely. It's missing a step, as it assumes you have Rio already, but it was easy enough to figure out.

> gem install rio
> gem install syntax

Then save that Ruby script off locally and run it against your script file. Include the CSS or just add it to the HTML inside <STYLE> tags and you're golden.

Of course, the braindead Blogger engine doesn't seem to be picking up the stylesheet, so the code doesn't appear as pretty as it should -- no syntax coloring or anything. I'll have ot figure that out before I post my HTTP load runner script.

Using JConsole To Connect To Remote JVM

We have been having issues with a certain Java web application we inherited. Actually, we've had a few problems with it, but that's another story.

One issue was that, even though there is nothing running on this Linux box except a single instance of Tomcat (5.5) with this single web application deployed to it, we will get into frequent issues where we get paged for "high CPU utilization". Log in to the machine, and the Java process is taking 100% CPU. It might stay like that for a few hours, or even a few days. Occasionally it clears up on it's own, but usually the server stops responding and we have to restart Tomcat.

We have had another issue where we would get Out of Memory exceptions (Heap space), and the JVM would stop running. The two issues weren't necessarily related, but we couldn't rule out the possibility either.

We don't have much visibility into the application and what's going on since it is a vendor built "black box". We have some customized source code, but most of it is off limits to us. They rolled their own database connection pool, MVC framework, persistence framework, etc.

In comes Jconsole. JConsole is an awesome tool that comes bundled with JDK 1.5 and above. It connects to the JVM and gives you all the info you could want on the various JVM memory pools, garbage collection, threads, classloading, and lets you manage anything exposed via JMX. It also has lots of pretty graphs, such as memory usage and garbage collections over time, for any or all of the memory pools (heap, non-heap, or individual pools, like permgen and eden space). Same goes for threads and loaded classes --current number, peak, total created.

The best thing about JConsole is the ability to connect to remote JVM's so you don't add too much overhead on the box being monitored. I have JConsole hooked up to my test and production servers, and it helped me prove that the two issues above were connected. There is some condition in the application (yet to be found, actually, first step was proof of what's really happening) that causes a substantial memory leak, and once the memory usage gets at it's ceiling, the garbage collection thread basically runs constantly, trying to regain some trivial amount of memory, then filling it up, and running GC again.

With an average of 100 active sessions at a time plus full garbage collection running non-stop, the CPU gets consumed quickly. I monitored the app for about a week and a half, and memory and CPU looked great -- there were a bit over 200 full GC's in that time period. Then this past weekend, we had to restart because of a DNS issue. That was Friday evening, and by Monday morning there were over 2000 full GC's performed, and I was restarting a non-responsive server by lunch time.

Below are two screenshots, one is of several days of "normal" memory usage, notice gradual rise and then sharp decrease at full GC, all the while keeping well below the JVM's allotted memory ceiling. Second is this past weekend's issue, where memory is hovering at ceiling and a full GC doesn't do much. Also notice the old generation memory pool is quite full.




MAKING IT HAPPEN

To set up JConsole to run on a local JVM, you only need ot pass one extra argument to the JVM:

-Dcom.sun.management.jmxremote

To set up remote (with no security), it is a matter of adding a few more parameters to the remote JVM at startup:

-Dcom.sun.management.jmxremote.port=8004 -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false

Then when you start up JConsole, go to the remote tab, enter the server name and the port that you specified (in this case, 8004).

Simple as that. It starts collecting stats immediately and the graphs appear. As you explore the JMX tree, you will notice yo ucan click on some of the stats and the simple integer displays "opens up" into a full graph display. I'm doing this to watch the active sessions patterns through the day and week.

I Feel So Naughty

Most of my development experience has been with structured languages such as C and Java.

Given that, I feel almost naughty working with Ruby's loose typing. It's like I am getting away with something, doing something oh so wrong. For some reason it never seemed odd with shell scripts, but does here. It's both liberating and a little scary at the same time.

No compiler errors, no warnings, no runtime exceptions, no type mismatches, nothing. You can make x = 3 then turn around and make x = "dude", and then set x = FileRenamer.new with out any issues.

And The Tree Came Crashing Down

We went to church this morning, and then off to pick up the kids. When we got home, everything was wet, so we knew it had rained. Then for some reason, I felt the need to go over and check on the chickens. What I saw amazed me. 3 big branches of the maple tree had come crashing down on the coop and the fenced in run. I went back to check it out, and the chicken run was completely full of branches and leaves -- I couldn't see the chickens, but I could hear them clucking and running around through the leaves. The back part of the fence was completely down to the ground.

Luckily, no damage was done to the roof of the coop, no chickens were hurt, and once I cleared out all the branches, I was able to repair the fence with minimal effort. Now it's mostly a cleanup effort.

Another Picture of Lexi

Here's an updated picture of Lexi the Wheaten Terrier. She's growing up and getting a lot lighter now. You can see a lot more of the white hair around her face and her belly is almost completely white now.

Black Raspberry Time

It's that time of year again. The black raspberries are starting to ripen in the back yard and in the woods behind the house. The berries seem to be quite a bit smaller than normal, and a little less abundant than normal, probably because we've had a lack of rain so far this summer. But the berries that are out there are quite yummy. We picked a small bowl on the 4th, and I picked a handful today after work. In the morning, I plan on taking my big bowl out again for round two. Normally I can fill a large mixing bowl in one pass, and this time I just got manybe 1/5 of a bowl the first time through.

Office Art, Part 1

I drew this on the back of one of my project folders and snapped the picture with my ViewSonic v36 Pocket PC. This is what happens when I get bored in meetings.

And 20 Became 19.

Unce upon a time, five Buff Orpington roosters lived among others in a backyard flock, somewhere in upstate New York. A lone Speckled Sussex was even more the outsider, for the flock was mostly made up of Rhode Island Reds.

One sunny day in the summer of 2007, the largest rooster of the flock, a Rhode Island Red who goes by no name, was set aside to become dinner. By "set aside", what I really mean is he was placed into the fenced in backyard, separated from the other chickens and their food, until the time came. But this fenced in yard also happens to be inhabited part-time by a Soft Coated Wheaten Terrier named Lexi.

And so it was, Lexi was let outside and chased the RIR rooster before I could grab either one of them. The rooster ran into some high weeds, the remnants of what used to be a nice flower garden. Lexi ran in after, and came out seconds later, but the rooster did not appear. The chicken must have found escape from the fenced yard, for nobody could find the rooster after much searching.

So the next in line was desperately chosen, as dinner time was drawing nearer with no poultry ready to grill. It was a tough choice, but finally one Buff Orpington rooster stood out as the second largest, and soon succumbed to the meat cleaver. One chop and that's all she wrote.

That evening, dinner plates were stacked high of barbequed pork ribs and a fresh chicken, right from the back yard. Wegman's Memphis Style Barbeque sauce and Frank's Red Hot sauce complimented the smoked meats well.

Soon thereafter, the orginal RIR rooster showed himself. A master of camoflage he proved himself to be, as nobody could find him a few hours earlier, but there he was, right where he had last been spied. He managed to avoid his fate on that June day. But alas, he shall become dinner on another eve, when the 20 will become 18.