Tom Purl’s Web Site

My ’83 Datsun On The Side Of The Information Superhighway

Review – Release It!

leave a comment »

Release It!: Design and Deploy Production-Ready Software (Pragmatic Programmers) Release It!: Design and Deploy Production-Ready Software by Michael Nygard

My rating: 5 of 5 stars
I need to start by saying that this is one of the best technical books I have ever read. To me, it’s easily as enjoyable and useful as Code Complete, The Pragmatic Programmer, or The Mythical Man Month. If you’re a sysadmin, an architect, or a developer that works with medium-to-large-sized systems, then do the following:

  1. Stop reading this post
  2. Order this book from your library or buy it from The Pragmatic Programmer’s web site
  3. Owe me a pint :D

What The Book Is Really About

Actually, there is one thing that I don’t like about this book, but it really has nothing to do with the book. The description of this book on the Pragmatic Programmer’s web site sucks. It’s vague, and it really gives the potential reader a tiny amount of insight into the book’s contents.

What it should have said is that this book contains *tons* of great information on designing, deploying, maintaining and improving medium-to-large-sized IT systems. It’s filled with patterns, anti-patterns, and general best practices that should be part of the shared lexicon of every developer, administrator, and system architect. Also, it does a good job of giving you enough information to be useful without boring you to death. And finally, it’s written very well and is a joy to read.

The Highlights

Thread Dumps & Garbage Collection Tuning

The internals of the Java Virtual Machine (JVM) have been a black box to me for the majority of my career in IT. Thankfully, this book has provided excellent examples of how you can troubleshoot and improve your system using tools that interrogate and manipulate a JVM at runtime.

For me, this was the most interesting and useful part of the book, and I am looking forward to seeing what can be gained by tuning and “poking at” the JVM’s that are in the system that I maintain.

Patterns and Anti-Patterns

It’s great to finally find a book that codifies some patterns that administrators and architects can use.

Transparency

I thought that I new a lot about monitoring and transparency before reading this book, but now I know better. I especially like the concept of a unified “OpsDB”, and I am eager to build something like this myself for the system that I maintain.

Integration Point Risks

I always knew that integration points (e.g. data feeds, databases, LDAP providers, etc.) added risk to you system, but the author does a great job calculating the actual risk. Also, he shows you many ways in which you can avoid brittle integration points.

Caveats

I have one warning about this book, but it’s half-hearted. This book is what I would all Java-centric. All of the case studies involve systems that are written in Java, and some of the sections will only apply directly to you if you are working with Java-based software.

But does that mean that you should avoid this book if you are working with Ruby, PHP, or .Net-based software? Absolutely not. Even though there are a few small sections of the book that won’t directly apply to your line of work, most of them will apply in an indirect way, regardless of your platform. And the other 94% of the book will directly apply to medium-to-large systems of every stripe.

View all my reviews >>

Written by Tom Purl

June 12, 2010 at 1:15 am

New Blog Announcement – Shell Magic

with one comment

I’m excited to say that I am working on a series of screencasts that target Unix command-line newbies. The series is called $hell Magic, and it’s my first attempt at making screencasts.  It was a lot of fun and surprisingly simple to create my first one.

If you would like to learn more about the Unix or Linux command-line, or you know someone else who would, please feel free to check it out.

Written by Tom Purl

June 5, 2010 at 12:39 pm

Review: The Victorian Internet

leave a comment »

The Victorian Internet The Victorian Internet by Tom Standage

My rating: 3 of 5 stars
This book is a fascinating account of the history of the telegraph and how it changed the world. If you are a history and technology buff, you will definitely get a lot out of this book.

Please don’t be fooled by the title, however. This book does not make deep comparisons between the histories of the internet and the telegraph. A small chapter at the end of the book provides the only comparison between the two technologies.

Even with this flaw, this is a solid technical history book, and I really feel like I gained a lot of perspective.

View all my reviews >>

Written by Tom Purl

April 22, 2010 at 2:28 am

Posted in Review

Tagged with , , ,

Running Depot On Tomcat

with one comment

This is my second article about running the Depot application from the Agile Development With Rails book on top of Jruby. The first article, which covers development and testing, can be found here.

This article covers the following topics:

  • Packaging our RoR project into a WAR file
  • Deploying that WAR file on Tomcat

Please note that this article only covers deploying Jruby-On-Rails applications in a non-Production (i.e. non-public-facing) environment. This example is designed to allow a developer to deploy a Jruby-On-Rails application on her workstation. If you want to install Tomcat in a public-facing environment, then you need a different tutorial :)

Tomcat Installation

Tomcat was my choice for the following reasons:

  1. It’s very easy to install. Just unpack it and go (if you already have the proper version of Java installed).
  2. It’s very easy to administer on my netbook (which is where I’m learning about RoR and Jruby).
  3. It’s relatively lightweight compared to JBoss or WebLogic.
  4. It’s very easy to deploy war files. Just drop them in $TOMCAT_HOME/webapps, restart Tomcat, and you’re up-and-running.

So let’s install Tomcat:

$ cd /tmp
$ wget http://apache.opensourceresources.org/tomcat/tomcat-6/v6.0.24/bin/apache-tomcat-6.0.24.tar.gz
$ tar xvfz apache-tomcat-6.0.24.tar.gz
$ sudo cp -r apache-tomcat-6.0.24 /opt
# please replace tom:tom with your user and group
$ sudo chown -R tom:tom /opt/apache-tomcat-6.0.24

Assuming that you have the minimum version of Java (I’m using 1.6.0_15 from the sun-java6-jdk Ubuntu package), you can start Tomcat like this:

$ cd /opt/apache-tomcat-6.0.24/bin
$ ./startup.sh
Using CATALINA_BASE:   /opt/apache-tomcat-6.0.24
Using CATALINA_HOME:   /opt/apache-tomcat-6.0.24
Using CATALINA_TMPDIR: /opt/apache-tomcat-6.0.24/temp
Using JRE_HOME:        /usr/lib/jvm/java-6-sun
Using CLASSPATH:       /opt/apache-tomcat-6.0.24/bin/bootstrap.jar

Great, now you can test your installation by pointing your browser at the following url:

  • http://localhost:8080

You can also shut the server down using the shutdown.sh script from the same directory.

Packaging Depot In A WAR File

Now that Tomcat is up-and-running, let’s build our WAR file. If you haven’t already installed warbler, then you can do so with the following command:

$ jruby -S gem install warbler

Warbler is a gem that makes it very easy to package RoR applications as WAR files. However, before we can use it, we need to tell it that we’re using the activerecord-jdbcsqlite3-adapter driver. Here’s how you do that:

  1. Run the warble config command from the root directory of your project. This command will create the config/warble.rb file.
  2. Add the following line to the newly-created config/warble.rb file:
    1. config.gems << "activerecord-jdbcsqlite3-adapter"

That’s it! Now you can build your war file:

$ warble

In a minute or so, you should see a file named depot-jruby.war in the root of your project directory. If you need to rebuild you war file, then simply execute the following command:

$ warble war:clean &amp;&amp; warble

Deploying On Tomcat

Now we can finally deploy the WAR file to our Tomcat instance. Here’s how I do it on my machine:

$ cd /opt/apache-tomcat-6.0.24
$ cp /home/tom/Dev/ruby/depot-jruby/depot-jruby.war ./webapps
$ ./bin/shutdown.sh &amp;&amp; ./bin/startup.sh

Then you can test your application by visiting the following URL:

  • http://localhost:8080/depot-jruby/store

Conclusion

I know that I said this in my previous tutorial, but that was a lot easier than I thought it would be. It only took 3 simple steps, and the Tomcat installation step only needs to be done once.

Written by Tom Purl

March 25, 2010 at 1:49 am

Week 1 Hosting SSHD – Gee Whiz Security Stats

with 2 comments

I installed the 8.0 version of FreeBSD this week in a VirtualBox image, and I thought that it would be fun to let some of my friends use it as a web server test bed.  I therefore setup an SSH server and hardened it using some of the recommendations from this web page:

Specifically, I set the following properties:

  • I explicitly turned off root logins. This is the default on FreeBSD, and it should be the default on every Linux and Unix server (even though it isn’t).
  • I deny access to anyone using the SSH 1 protocol.
  • I turned off password authentication.  Instead, anyone who logs in has to use keypair authentication.

With all of these settings, I feel like my SSH server is safe enough.  However, I was curious to see if anyone had actually tried to crack it.  I mean, it’s only been up for 3 days.  The bad robots of the Internet could not have possibly found my little SSH garden in the shade, right?

A quick look at /var/log/auth.log brings me back to Earth.  To my surprise, their have already been hundreds of attempted logins from multiple sources.  Here’s the statistics related to people who are trying to crack my SSH server.  Remember, this server is 3-days old.

  • 865 failed login attempts
  • 281 unique login names
    • Some of the names are fairly comical (12345?), and many of them are very uncommon (Agatha?).
  • 0 login attempts using the root user name.
    • This was a big surprise for me.
  • The attacks originated from 6 IP addresses.
    • It’s anyone’s guess if all of these computers are part of the same botnet or if there are multiple groups trying to crack my server.

I can’t help but compare any publicly-available, networked service to a car that is parked in the middle of a city containing a billion people.  No matter how small and obscure your car is, you can’t afford to park it without locking your doors.

Written by Tom Purl

March 20, 2010 at 7:17 pm

Posted in Uncategorized

Tagged with , ,

Stupid Unix Tricks – Creating A Self-Updating Dashboard

with one comment

Have you ever wanted to automatically run the same command every 5, 10, or 60 seconds on a Unix/Linux machine until the output changed?

For example, let’s say that you’re deploying a new application on top of a Tomcat app server. After starting Tomcat and deploying your application, it can take a couple of minutes before the application is really available.  However, you want to know exactly when the new version of the application is usable (give or take 10 seconds).

A good way of testing whether the application is really “up” is to ping a URL.  You can do this using a web browser, but this approach has a few disadvantages.  First, it’s a manual,  tedious process.  Second, web browsers can be a little flaky sometimes. For example, they may cache your last results, giving you incorrect results.

For me, I get much better results testing this sort of thing with curl.  Here’s an example of how you could test the availability of the Hello World application that comes with Tomcat 6.x:

$ curl http://localhost:8080/examples/servlets/servlet/HelloWorldExample
<html>
<head>
<title>Hello World!</title>
</head>
<body bgcolor="white">
<a href="../helloworld.html">
<img src="../images/code.gif" height=24 width=24 align=right border=0 alt="view code"></a>
<a href="../index.html">
<img src="../images/return.gif" height=24 width=24 align=right border=0 alt="return"></a>
<h1>Hello World!</h1>
</body>
</html>

This is fine, but it can also very labor-intensive and tedious to execute the same command over and over.  Wouldn’t it be nice if we could just create an ad-hoc “dashboard” for this command?

Here’s how you can do it.  All we have to do is a little shell magic:

$ while true; do curl http://localhost:8080/examples/servlets/servlet/HelloWorldExample; sleep 10; done

Here’s what I’m doing:

  • First, I’m creating an infinite while loop.  You’ll see why this isn’t a bad idea in a second.
  • Next, I’m running our curl command.
  • After that, I’m sleeping for 10 seconds.  This is very important, because you don’t want to command to execute multiple times a second in an infinite loop.

This loop will repeat itself forever until you kill it with a Ctrl-C.

And here’s my results (assuming that I just restarted the server and it took that servlet 20 seconds to respond):

curl: (7) couldn't connect to host # First execution
curl: (7) couldn't connect to host # second
<html>                             # Success!
<head>
<title>Hello World!</title>
</head>
<body bgcolor="white">
<a href="../helloworld.html">
<img src="../images/code.gif" height=24 width=24 align=right border=0 alt="view code"></a>
<a href="../index.html">
<img src="../images/return.gif" height=24 width=24 align=right border=0 alt="return"></a>
<h1>Hello World!</h1>
</body>
</html>

Ta-da!  Easy, automatic command testing.  Of course, you can use this trick with commands other than curl.  It’s very handy in any situation where you want to check a command for a change in output.

Written by Tom Purl

March 17, 2010 at 10:27 pm

Running The Depot Application With Jruby

with one comment

I completed the Depot application from the Agile Development With Rails book, and I feel that I have a good understanding of the basics of developing with Ruby on Rails. It’s a great framework, and creating the Depot application was a fun and enlightening process.

I thought that it might be equally enlightening to “port” the Depot application to Jruby using the Jruby On Rails libraries. I had two basic milestones in mind:

  1. Development & Testing – I wanted to be able to develop and test my application just like I did with CRuby, using tools such as mongrel and rake.
  2. J2EE Deployment – The most popular process for deploying a Java-based application is to build a WAR file and then deploy it to a Java application server such as Tomcat (which was my choice). I wanted to be able to complete these steps with my RoR application.

The entire process was surprisingly easy. The first milestone, Development and Testing, is covered in this article. The J2EE Deployment milestone will be covered in a different blog article.

Development & Testing

Here are some simple steps that will have your application running with mongrel very quickly.

Please note that I am executing the steps below on a netbook running Ubuntu 9.10. These commands should work on most Linux distributions, however. Also, they should work with a few small adjustments on a Windows machine.

Prepping Your Project Directory

First, you’re not going to want to make these changes to your original version of the Depot code. You will probably want to create a copy of it that is designed to work with Jruby. Here’s how I did it on my machine:

$ cd /home/tom/Dev/ruby
$ cp -r depot depot-jruby

The location of your depot folder will probably be different on your machine, but I’m sure that you can make the necessary adjustments :)

Also, please note that it is a good idea to “freeze” your rails version now if you have not done so already. You can do this by executing the following command:

$ cd /home/tom/Dev/ruby/depot-jruby
$ jruby -S rake rails:freeze:edge RELEASE=2.2.2

Installing Jruby

Next, you’ll want to install Jruby. There’s a lot of ways of doing this, but this method worked pretty well on my development machine:

$ cd /tmp
$ wget http://jruby.kenai.com/downloads/1.4.0/jruby-bin-1.4.0.tar.gz
$ tar xvfz jruby-bin-1.4.0.tar.gz
$ sudo cp -r jruby-1.4.0 /opt
# please replace tom:tom with your user and group
$ sudo chown -R tom:tom /opt/jruby-1.4.0
$ sudo ln -s /opt/jruby-1.4.0 /opt/jruby

Then simply add /opt/jruby to your PATH, and see if you can execute jirb.

Installing Rails And Other Libraries

This process is just as easy using Jruby as it is using CRuby.

Install the gems using these commands:

$ jruby -S gem install rails --version 2.2.2
$ jruby -S gem install mongrel jruby-openssl jdbc-sqlite3

A few notes on those commands:

  • Prepend the gem command with jruby -S to ensure that you are using the right version of the gem command. This also works with any other Jruby command, such as rake.
  • Since the example uses the 2.2.2 version of Rails, we’re going to need it too.
  • jdbc-sqlite3 is the Java version of the sqlite3 DB driver.
  • The jruby-openssl package is nice to have regardless of what you’re installing with rubygems.

Making Depot Safe For JDBC

You can’t use the C-based database drivers with Jruby, but luckily, the Jruby developers have made it very easy to use the JDBC-based equivalents.

First, make sure that you are using version 0.9.2 of the activerecord-jdbc-adapter driver or higher. You can check this by executing the following command:

$ jruby -S gem list | grep activerecord-jdbc-adapter
activerecord-jdbc-adapter (0.9.2)

Next, execute the following command from the root directory of your project:

$ jruby script/generate jdbc
exists  config/initializers
create  config/initializers/jdbc.rb
create  lib/tasks
create  lib/tasks/jdbc.rake

That’s it! Your Depot application is now ready to use Jruby-on-Rails.

Please note that I did not modify my config/database.yml file or hack any part of the Rails libraries to make JDBC work. These steps used to be necessary, but the command listed above gives you a much easier and cleaner way of using JDBC. For more information, please check out this article:

Test

To test my work, I did the following:

  • I ran the mongrel app server and executed some manual tests of the Depot web site. That worked well.
  • I execute all of my tests using the rake test command. This also worked as well as it had with the CRuby version of my application.

Success! Now it would be nice to review all of the files that changed or were added. Since I checked my project into my git repository before making any changes, I can get the information that I need pretty easily:

$ git status
# branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       new file:   config/initializers/jdbc.rb
#       new file:   lib/tasks/jdbc.rake

That’s it! No code or config changes were necessary.

Conclusion

As you can see, it’s fairly easy to “port” the Depot application to use Java-based versions of mongrel, rails, and the other libraries from the Agile Development With Rails book. In my next article, I will take this application and deploy it on top of a Tomcat server.

Written by Tom Purl

March 2, 2010 at 2:49 am

Tab Sweep – Week Of 2/14/10

leave a comment »

I spent most of my “hobby-time” this week “converting” my demo Ruby On Rails (RoR) application to Jruby.  I hope to have a post on all of my steps posted this week.

Written by Tom Purl

February 22, 2010 at 5:55 pm

Tab Sweep – Week Of 2/7/10

leave a comment »

I had lots of great opportunities to work with unfamiliar technologies last week, so I have an unusually large list of links in my tab sweep.

Solaris Zones

I learned late last week that Solaris Zones use different monitoring tools than any other Unix or Linux distribution, so I spent a good portion of this week learning about them.

RoR Deployment With Capistrano

I get the impression that a lot of people have a love/hate relationship with Capistrano, and I believe that I’m joining those ranks. When your recipe finally works, it works very well, but getting to that point can be very frustrating. Here are some links that helped me figure things out:

Heroku + Toto

I discovered the Toto blogging “engine” this week, which led me to Heroku. It’s a very interesting deployment and hosting model for Ruby web applications, and I look forward to learning more about it.

Written by Tom Purl

February 15, 2010 at 12:00 am

Review: Rails Deployment: Production Configuration and Advanced Rails Tactics

leave a comment »

Rails Deployment: Production Configuration and Advanced Rails Tactics Rails Deployment: Production Configuration and Advanced Rails Tactics by Ezra Zygmuntowicz

My rating: 4 of 5 stars
Overall, this book had a lot of very good information, and it was very helpful to me as I deployed my first Ruby-On-Rails application into a “Production” environment.

Here’s the high points:

  • Lots of good information on tools such as Capistrano, MySQL, Mongrel, Apache, and nginx.
  • The authors clearly know what they are talking about.
  • It’s helpful if you’re deploying a “toy” application (like I am) and if you’re deploying a large, clustered application.
  • It doesn’t assume that you’re already an expert on either Web app deployment or Ruby-On-Rails.

Here are some of the things that could be better:

  • This book was published over a year ago, and it already feels out-of-date. For example, there isn’t one mention of Phusion Passenger, even though this tool seems to be the new standard app server for Ruby-On-Rails in Production environments.
  • This is very subjective, but I feel like the information could have been organized a little better. I felt as if the author jumped around a bit sometimes.
  • Also, some of the passages were a little difficult to read due to their incorrect sentence structure. My writing isn’t perfect either, but I believe that the editor should have fixed more of these mistakes.

If you’re deploying a Ruby app in any setting, then this is a good book to get. I just don’t know if I would actually buy it.

View all my reviews >>

Written by Tom Purl

February 13, 2010 at 2:47 am