Tom Purl’s Web Site

I’m excited to say that I am working on a series of screencasts that target Unix command-line newbies. The series is called $hell Magic, and it’s my first attempt at making screencasts.  It was a lot of fun and surprisingly simple to create my first one.

If you would like to learn more about the Unix or Linux command-line, or you know someone else who would, please feel free to check it out.

This is my second article about running the Depot application from the Agile Development With Rails book on top of Jruby. The first article, which covers development and testing, can be found here.

This article covers the following topics:

• Packaging our RoR project into a WAR file
• Deploying that WAR file on Tomcat

Please note that this article only covers deploying Jruby-On-Rails applications in a non-Production (i.e. non-public-facing) environment. This example is designed to allow a developer to deploy a Jruby-On-Rails application on her workstation. If you want to install Tomcat in a public-facing environment, then you need a different tutorial

Tomcat Installation

Tomcat was my choice for the following reasons:

1. It’s very easy to install. Just unpack it and go (if you already have the proper version of Java installed).
2. It’s very easy to administer on my netbook (which is where I’m learning about RoR and Jruby).
3. It’s relatively lightweight compared to JBoss or WebLogic.
4. It’s very easy to deploy war files. Just drop them in $TOMCAT_HOME/webapps, restart Tomcat, and you’re up-and-running.

So let’s install Tomcat:

$ cd /tmp
$ wget http://apache.opensourceresources.org/tomcat/tomcat-6/v6.0.24/bin/apache-tomcat-6.0.24.tar.gz
$ tar xvfz apache-tomcat-6.0.24.tar.gz
$ sudo cp -r apache-tomcat-6.0.24 /opt
# please replace tom:tom with your user and group
$ sudo chown -R tom:tom /opt/apache-tomcat-6.0.24

Assuming that you have the minimum version of Java (I’m using 1.6.0_15 from the sun-java6-jdk Ubuntu package), you can start Tomcat like this:

$ cd /opt/apache-tomcat-6.0.24/bin
$ ./startup.sh
Using CATALINA_BASE:   /opt/apache-tomcat-6.0.24
Using CATALINA_HOME:   /opt/apache-tomcat-6.0.24
Using CATALINA_TMPDIR: /opt/apache-tomcat-6.0.24/temp
Using JRE_HOME:        /usr/lib/jvm/java-6-sun
Using CLASSPATH:       /opt/apache-tomcat-6.0.24/bin/bootstrap.jar

Great, now you can test your installation by pointing your browser at the following url:

• http://localhost:8080

You can also shut the server down using the shutdown.sh script from the same directory.

Packaging Depot In A WAR File

Now that Tomcat is up-and-running, let’s build our WAR file. If you haven’t already installed warbler, then you can do so with the following command:

$ jruby -S gem install warbler

Warbler is a gem that makes it very easy to package RoR applications as WAR files. However, before we can use it, we need to tell it that we’re using the activerecord-jdbcsqlite3-adapter driver. Here’s how you do that:

1. Run the warble config command from the root directory of your project. This command will create the config/warble.rb file.
2. Add the following line to the newly-created config/warble.rb file:
   1. config.gems << "activerecord-jdbcsqlite3-adapter"

That’s it! Now you can build your war file:

$ warble

In a minute or so, you should see a file named depot-jruby.war in the root of your project directory. If you need to rebuild you war file, then simply execute the following command:

$ warble war:clean && warble

Deploying On Tomcat

Now we can finally deploy the WAR file to our Tomcat instance. Here’s how I do it on my machine:

$ cd /opt/apache-tomcat-6.0.24
$ cp /home/tom/Dev/ruby/depot-jruby/depot-jruby.war ./webapps
$ ./bin/shutdown.sh && ./bin/startup.sh

Then you can test your application by visiting the following URL:

• http://localhost:8080/depot-jruby/store

Conclusion

I know that I said this in my previous tutorial, but that was a lot easier than I thought it would be. It only took 3 simple steps, and the Tomcat installation step only needs to be done once.

Have you ever wanted to automatically run the same command every 5, 10, or 60 seconds on a Unix/Linux machine until the output changed?

For example, let’s say that you’re deploying a new application on top of a Tomcat app server. After starting Tomcat and deploying your application, it can take a couple of minutes before the application is really available.  However, you want to know exactly when the new version of the application is usable (give or take 10 seconds).

A good way of testing whether the application is really “up” is to ping a URL.  You can do this using a web browser, but this approach has a few disadvantages.  First, it’s a manual,  tedious process.  Second, web browsers can be a little flaky sometimes. For example, they may cache your last results, giving you incorrect results.

For me, I get much better results testing this sort of thing with curl.  Here’s an example of how you could test the availability of the Hello World application that comes with Tomcat 6.x:

$ curl http://localhost:8080/examples/servlets/servlet/HelloWorldExample
<html>
<head>
<title>Hello World!</title>
</head>
<body bgcolor="white">
<a href="../helloworld.html">
<img src="../images/code.gif" height=24 width=24 align=right border=0 alt="view code"></a>
<a href="../index.html">
<img src="../images/return.gif" height=24 width=24 align=right border=0 alt="return"></a>
<h1>Hello World!</h1>
</body>
</html>

This is fine, but it can also very labor-intensive and tedious to execute the same command over and over.  Wouldn’t it be nice if we could just create an ad-hoc “dashboard” for this command?

Here’s how you can do it.  All we have to do is a little shell magic:

$ while true; do curl http://localhost:8080/examples/servlets/servlet/HelloWorldExample; sleep 10; done

Here’s what I’m doing:

• First, I’m creating an infinite while loop.  You’ll see why this isn’t a bad idea in a second.
• Next, I’m running our curl command.
• After that, I’m sleeping for 10 seconds.  This is very important, because you don’t want to command to execute multiple times a second in an infinite loop.

This loop will repeat itself forever until you kill it with a Ctrl-C.

And here’s my results (assuming that I just restarted the server and it took that servlet 20 seconds to respond):

curl: (7) couldn't connect to host # First execution
curl: (7) couldn't connect to host # second
<html>                             # Success!
<head>
<title>Hello World!</title>
</head>
<body bgcolor="white">
<a href="../helloworld.html">
<img src="../images/code.gif" height=24 width=24 align=right border=0 alt="view code"></a>
<a href="../index.html">
<img src="../images/return.gif" height=24 width=24 align=right border=0 alt="return"></a>
<h1>Hello World!</h1>
</body>
</html>

Ta-da!  Easy, automatic command testing.  Of course, you can use this trick with commands other than curl.  It’s very handy in any situation where you want to check a command for a change in output.

I completed the Depot application from the Agile Development With Rails book, and I feel that I have a good understanding of the basics of developing with Ruby on Rails. It’s a great framework, and creating the Depot application was a fun and enlightening process.

I thought that it might be equally enlightening to “port” the Depot application to Jruby using the Jruby On Rails libraries. I had two basic milestones in mind:

1. Development & Testing – I wanted to be able to develop and test my application just like I did with CRuby, using tools such as mongrel and rake.
2. J2EE Deployment – The most popular process for deploying a Java-based application is to build a WAR file and then deploy it to a Java application server such as Tomcat (which was my choice). I wanted to be able to complete these steps with my RoR application.

The entire process was surprisingly easy. The first milestone, Development and Testing, is covered in this article. The J2EE Deployment milestone will be covered in a different blog article.

Development & Testing

Here are some simple steps that will have your application running with mongrel very quickly.

Please note that I am executing the steps below on a netbook running Ubuntu 9.10. These commands should work on most Linux distributions, however. Also, they should work with a few small adjustments on a Windows machine.

Prepping Your Project Directory

First, you’re not going to want to make these changes to your original version of the Depot code. You will probably want to create a copy of it that is designed to work with Jruby. Here’s how I did it on my machine:

$ cd /home/tom/Dev/ruby
$ cp -r depot depot-jruby

The location of your depot folder will probably be different on your machine, but I’m sure that you can make the necessary adjustments

Also, please note that it is a good idea to “freeze” your rails version now if you have not done so already. You can do this by executing the following command:

$ cd /home/tom/Dev/ruby/depot-jruby
$ jruby -S rake rails:freeze:edge RELEASE=2.2.2

Installing Jruby

Next, you’ll want to install Jruby. There’s a lot of ways of doing this, but this method worked pretty well on my development machine:

$ cd /tmp
$ wget http://jruby.kenai.com/downloads/1.4.0/jruby-bin-1.4.0.tar.gz
$ tar xvfz jruby-bin-1.4.0.tar.gz
$ sudo cp -r jruby-1.4.0 /opt
# please replace tom:tom with your user and group
$ sudo chown -R tom:tom /opt/jruby-1.4.0
$ sudo ln -s /opt/jruby-1.4.0 /opt/jruby

Then simply add /opt/jruby to your PATH, and see if you can execute jirb.

Installing Rails And Other Libraries

This process is just as easy using Jruby as it is using CRuby.

Install the gems using these commands:

$ jruby -S gem install rails --version 2.2.2
$ jruby -S gem install mongrel jruby-openssl jdbc-sqlite3

A few notes on those commands:

• Prepend the gem command with jruby -S to ensure that you are using the right version of the gem command. This also works with any other Jruby command, such as rake.
• Since the example uses the 2.2.2 version of Rails, we’re going to need it too.
• jdbc-sqlite3 is the Java version of the sqlite3 DB driver.
• The jruby-openssl package is nice to have regardless of what you’re installing with rubygems.

Making Depot Safe For JDBC

You can’t use the C-based database drivers with Jruby, but luckily, the Jruby developers have made it very easy to use the JDBC-based equivalents.

First, make sure that you are using version 0.9.2 of the activerecord-jdbc-adapter driver or higher. You can check this by executing the following command:

$ jruby -S gem list | grep activerecord-jdbc-adapter
activerecord-jdbc-adapter (0.9.2)

Next, execute the following command from the root directory of your project:

$ jruby script/generate jdbc
exists  config/initializers
create  config/initializers/jdbc.rb
create  lib/tasks
create  lib/tasks/jdbc.rake

That’s it! Your Depot application is now ready to use Jruby-on-Rails.

Please note that I did not modify my config/database.yml file or hack any part of the Rails libraries to make JDBC work. These steps used to be necessary, but the command listed above gives you a much easier and cleaner way of using JDBC. For more information, please check out this article:

• Fresh 0.9.2 activerecord-jdbc-adapter Release

Test

To test my work, I did the following:

• I ran the mongrel app server and executed some manual tests of the Depot web site. That worked well.
• I execute all of my tests using the rake test command. This also worked as well as it had with the CRuby version of my application.

Success! Now it would be nice to review all of the files that changed or were added. Since I checked my project into my git repository before making any changes, I can get the information that I need pretty easily:

$ git status
# branch master
# Changes to be committed:
#   (use "git reset HEAD <file>..." to unstage)
#
#       new file:   config/initializers/jdbc.rb
#       new file:   lib/tasks/jdbc.rake

That’s it! No code or config changes were necessary.

Conclusion

As you can see, it’s fairly easy to “port” the Depot application to use Java-based versions of mongrel, rails, and the other libraries from the Agile Development With Rails book. In my next article, I will take this application and deploy it on top of a Tomcat server.

Overview

One of my goals for 2010 is to gain a decent understanding of the Ruby On Rails framework.  I don’t think that this knowledge will directly help me with my current job (as a FileNet P8 administrator), but I do think that it will provide some good benefits.

First, learning about something that’s a little bit outside of your comfort zone can often help you with your day-to-day tasks.  For example, if you teach yourself the basics of Java development, then you will know how to read a stack trace.  Having this knowledge is very important if you want to administer software written in Java.

Also,  it’s just fun.  Creating something useful out of a few lines of text never ceases to amaze me.  Also, I find that learning new technical skills is good for your brain.

Finally, who knows what the future may bring?  It never hurts to know as much as you can about popular technologies in general when you work in IT.

So to get started, I purchased a copy of Agile Web Development with Rails, Third Edition from the Pragmatic Programmers’ web site.  This book seems to be one of the best for beginners, and I really love that you can buy DRM-free ebooks from the Pragmatic Programmers’ web site.

Installing Ubuntu Packages

So enough background information.  Here’s how I set up my system to get started with Rails development.  First, I installed all of the necessary software on my Ubuntu 9.10 machine that was  available as an Ubuntu package:

sudo apt-get install rails ruby-full rubygems rake mongrel git-core sqlite3

Here’s why I installed more than just the rails and ruby pacakges.  When you install Ruby from source on a Linux machine, you get a lot of software.  In addition to the Ruby runtime, you get fun tools like irb, rubygems, rdoc, ri, and rake.  However, when you install the ruby package on Ubuntu and Debian, you just get the Ruby runtime along with a a couple of libraries.  Since I’m lazy efficient guy, I decided to install the ruby-full package instead to save a few keystrokes.  Unfortunately, the ruby-full package still doesn’t include rake or rubygems, so I installed those manually.

Mongrel is a developer’s web server that is designed to work very well with Ruby on Rails development.  Since I am also impatient, I wanted a web server that would work a little faster than WEBrick.

Finally, I installed git because I wanted to share code between two different computers.  Git is a version control system that is very popular with Ruby developers, so I figured that I would use the most popular Ruby-related tools while I’m learning about Ruby.

Installing Software Without apt-get

Natrually, you can’t install everything that you need with apt-get.  For starters, the version of rails that is used in Agile Web Development with Rails, Third Edition is 2.2.2, so let’s install that using rubygems:

sudo gem install rails --version 2.2.2

This operation took a little while (~ 7 minutes) on my machine for some reason.

Note: For some reason, this bog

Next, I’m a Vim user, so I would like to see if the rails.vim plugin would help me at all.  There is an official Ubuntu package for this plugin called vim-rails, but when I try to install it using apt-get, I get the following error:

Some packages could not be installed. This may mean that you have
requested an impossible situation or if you are using the unstable
distribution that some required packages have not yet been created
or been moved out of Incoming.
The following information may help to resolve the situation:

The following packages have unmet dependencies:
vim-rails: Depends: vim-full but it is not installable
E: Broken packages

When I tried to install the vim-full package on my machine, I got this error:

Package vim-full is not available, but is referred to by another package.
This may mean that the package is missing, has been obsoleted, or
is only available from another source
E: Package vim-full has no installation candidate

Ok, so after all of that, let’s just install it the easy way:

(Download the latest version of rails.zip from the rails.vim download page)

$ cp rails.zip ~/.vim

$ cd ~/.vim

$ unzip rails.zip

Finally, open vim or gvim and execute the following command:

:helptags ~/.vim/doc

In case your interested, here are the other Vim-related packages that I have installed on my machine:

• vim-common
• vim-gnome
• vim-gui-commom
• vim-runtime
• vim-tiny

Conclusion

And that’s it!  I now appear to have everything that I need to complete the beginning examples in Agile Web Development with Rails using Ubuntu 9.10 and Vim.

Update (1/20/10) – I added the sqlite3 package to the apt-get line above.  This package is necessary if you want to query your database manually.

I’ve been using an 8 GB Creative Zen player now for about a year, and in general, I’m very happy. It was about 25% cheaper than a comparable iPod, it can do more stuff, and it has ended up being very durable and easy-to-use.

Another thing that I initially liked about the Zen was how well it worked with Ubuntu Linux 8.04. From Amarok, I could add and manage files as easily as I could with my old iPod.

Then I updated my OS to use Ubuntu 9.04. The Zen would mount automatically when you plugged it in, but I couldn’t access it using any music player such as Amarok, RhythmBox, or Banshee. And if I could access the player using one of those apps, the connection was always very flaky. I might be able to move one or two songs, but eventually, something would crash and I would have to start over again.

This thread on the Ubuntu forums shows that many other Zen users were having the same problem:

• http://ubuntuforums.org/showthread.php?t=1146528

The basic advice was to unmount the Zen after it was automatically mounted, and then access it using your favorite music player. This only worked for me sporadically at first. And if that wasn’t weird enough, the Amarok maintainers decided to do the following:

• Completely change the Amarok GUI interface
• Completely change the way you copy music to an MP3 player.

I figured almost everything out eventually, so I thought I would share it here.

1. First, plug your Zen into your computer. You should now see a Zen icon on your desktop.
   • 
2. Next, right-click on that icon and choose the Unmount Volume option. This may seem counter-intuitive, but it’s what you need to do.
3. Next, open Amarok. If you don’t see the Amarok window after a minute, then look for the howling wolf icon in your task tray. You will need to right-click on it and choose Restore.
   • Note: This process may work with other music players, but I haven’t tested it.
4. After Amarok opens, you should see the following items in Amarok:
   1. An MP3 player icon with a red X and green + next to it.
      • 
   2. Your Zen listed on the left-hand side of the screen.
      • 
   3. If you don’t see it at first, then make sure that you have clicked on the Collection button on the left-hand side of the screen.
   4. Also, If you see the MP3 player icon but your player isn’t listed on the left-hand side of the screen, then click on the green + button.

Now you can copy individual songs from your Local Collection by right-clicking on a song or album and choosing Copy To Collection -> Creative Zen. That should do it. I’ve had pretty good luck with this system as long as I didn’t try to move over 6 albums’ worth of MP3′s at once.

And here are some of the differences that you need to get used to when using the 2.x version of Amarok:

• You can no longer copy playlists over to your Zen. This is a big disappointment to me, and I can’t figure out why the Amarok maintainers took this functionality away.
• You can no longer “queue” up MP3′s that you want to copy to your Zen and then copy them all over at once. Again, another great feature that was taken away for some mysterious reason.
• Album covers are no longer copied to your Zen. This isn’t a huge disappointment, but it was always a nice feature of the pre-2.x version of Amarok.

In the end, I’m very appreciative that Canonical, the Gnome group, and the Amarok maintainers provide such excellent software that makes my life so much more enjoyable. And I’m glad that I finally figured out a somewhat-reliable process for copying MP3′s from my hard drive to my Zen.

I was recently tasked with writing a command-line script in Jruby that performed simple CRUD operations against a simple database table. I’ve written tons of scripts in the past that manipulated tables, but I had never done so using Jruby. I therefore decided to first write a “bare-minimum” test script that would help me dip my toes in this unfamiliar water.

I wanted to write my test script against a database manager that ran on my laptop, but I didn’t want to go through the hassle of installing and configuring Oracle or MySql. Also, I soon learned that a lot of Ruby database drivers don’t work with Jruby, so I needed a small, simple database that had a good JDBC driver. I ended up installing and configuring Apache Derby, and it worked out very well for me.

So anyways, here’s the script (jdbctest.rb):

require 'java'
require './derbyclient.jar'
require './derbytools.jar'

# This example assumes that you have the free DerbyDB software installed on
# your machine.  It also assumes that you are running it in "server" mode by
# executing the startNetworkServer.bat script.

# This is what I'm using instead of Class.forName("org.apache.derby.jdbc.ClientDriver").newInstance()
import 'org.apache.derby.jdbc.ClientDriver'

connString = "jdbc:derby://localhost:1527/MyDbTest"
conn = Java::JavaSql::DriverManager.getConnection(connString)

stmt = conn.createStatement

rs = stmt.executeQuery("select * from derbyDb")

while (rs.next) do
    puts "#{rs.getString("NUM")} -- #{rs.getString("ADDR")}"
end

rs.close
stmt.close
conn.close()

I then placed the jdbctest.rb script in a folder with the derbyclient.jar and derbytools.jar file, and then executed the script like this:

c:\Dev\jruby\somefolder>jruby .\jdbctest.rb
180 -- Grand Ave.
1910 -- Union St.

To make this script work, you need to do the following:

1. Install Derby using this tutorial: Step 1 – Install Software
   • This is a very simple task that took me all of 6 minutes after downloading the software.
2. Create the MyDbTest database and the derbyDB table using this tutorial: Step 2 – ij Basics
   • You should also populate this table using the SQL located under the Execute SQL Statements section.
3. Once you’ve created and queried your table using ij, start your Derby server using this tutorial: Step 4 – Derby Network Server
   • You should validate the server using ij. Please see the Test network server connection with ij section in the Step 4 tutorial for help.

Please note that this might seem like a lot of work, but it’s really very simple, especially when you compare it installing and configuring most relational database managers. It took me about 20 minutes from A to Z, and I had no prior experience with Derby.

That’s it! I hope that this tutorial helps a few other people who want to get started with Jruby and JDBC.

At my job as a middleware sysadmin, I’m a big fan of automating complex and repetitive tasks. It cut down on defects, saves time, and makes my job a heckuva lot easier.

In the past, I’ve automated many tasks using Apache Ant, and I’ve had pretty good experiences. It’s light-weight, easy-to-learn, and it integrates very well with Java-based software (which I very often use). The problem is that it’s not terribly flexible. If you want to use it to compile and distribute Java programs, then it’s a great tool. However, take it a few degrees out of its comfort zone, and things get significantly more complicated.

The problem seems to be that Ant-based build files are written in XML. XML’s a great language for declaring things in a device-independent manner, but it’s not a terribly good medium for writing programs. There are Ant extensions that make it easier to write build files that need to perform complex operations, but they haven’t really helped me much in the past.

Like I said, I’ve had a lot of success with Ant in the past, but every so often, I like to reevaluate the tools that I use. That opportunity appeared the other day when I was tasked with automating the deployment process for a new web application. I have been teaching myself Ruby for a little while, and have heard great things about Rake, so I decided to give it a try.

My First Rakefile

The automation script that I needed to write needed to be able to do the following:

1. Copy some configuration files from one place to another. For example, if I’m deploying my application in the UAT environment, I would need to copy “config/db.conf.uat” to “config/db.conf”.
2. I need to be able to specify my target environment (e.g dev, uat, prod, etc.) at runtime.
3. Delete any temporary or backup files
4. Create a WAR file
5. Use that WAR file to create an EAR file.

The sixth and most important requirement was that this deployment script needed to be usable by people who had never used Rake or Ruby before. I therefore needed to give it a command-line help interface that was very simple and intuitive. I did not want to force other admins to learn Ruby just so they could use my Rakefile. Of course, I would like them to learn the basics of the language eventually, but nobody likes to be forced to learn new technology before they can see the benefits.

So here it is:

 1  require 'rake/clean'
 2
 3  CLEAN.exclude("**/*.bak")
 4  CLOBBER.include('build', 'dist')
 5
 6  task :default => [:usage]
 7
 8  ### Begin directory and file declarations
 9  directory "build"
 10 directory "dist"
 11
 12 file 'dist/myapp.ear' => 'build/myapp.war' do |task|
 13     puts "Building ear file..."
 14     sh "jar cf #{task.name} -C build myapp.war META-INF"
 15 end
 16 ### End directory and file declarations
 17
 18 desc "Usage statement for people who are unfamiliar with Rake or this particular Rakefile"
 19 task :usage do
 20     puts """
 21 Please execute the following command to build a deployable ear file:
 22
 23    jruby -S rake ear env=[dev|test|uat|prod]
 24
 25 For more task options, please execute the following command:
 26
 27    jruby -S rake -T"""
 28 end
 29
 30 desc "Make sure that an env var was passed to this build script"
 31 task :validate do
 32    if ENV['env'].nil?
 33        $stderr.puts "\nERROR: Empty env value"
 34        Rake::Task['usage'].invoke
 35        exit 1
 36    end
 37 end
 38
 39 desc "Update the environment-specific configuration files"
 40 task :setenv => [:validate] do
 41
 42    env = ENV['env']
 43
 44    webinf_dir = "src/myapp/WEB-INF"
 45    config_files = ['db.properties', 'soap.properties']
 46
 47    config_files.each do |config_file|
 48        cp "#{webinf_dir}/#{config_file}.#{env}", "#{webinf_dir}/#{config_file}"
 49    end
 50
 51 end
 52
 53 desc "Create a war file that can eventually be packaged as an ear file"
 54 task :war => [:setenv, :build, :dist, "clean"] do
 55    puts "Building war file..."
 56    sh "jar cf build/myapp.war -C src/myapp ."
 57 end
 58
 59 desc "Create a deployable ear file."
 60 task :ear => [:war, 'dist/myapp.ear'] do
 61     puts "The ear file has been created in dist directory"
 62 end

Ok, there’s a lot going on here, and it’s especially confusing if you’re new to Ruby. Here’s basically how I see things from a very high level:

Tasks And Prerequisites

Tasks are a lot like Ant targets, which are a lot like functions (although there are some big differences). They perform some unit of work, and are typically entry points into your script. For example, I could type in rake war, and I would immediately start running the instructions within the task starting at line 54, after the pre-requisites have run.

What’s a prerequisite? For the :war task, the pre-requisites are everything following the => notation. So when I attempt to run this task, the first thing that is actually executed is the :setenv task, followed by the :build task, and so on.

Directory and File Tasks

You may also notice that the :build task (at line 9) isn’t actually a task at all, but a “directory”. What I’m doing is specifying the directory at the beginning of the file, and then creating it later as a prerequisite for creating my war file. This seems a little strange to me, but apparently it’s a common way of doing things in Rakefiles.

Also, please note that if you do things this way, you save yourself a bit of coding. When you specify directories this way, rake takes care of recreating the directory after it’s been deleted and leaving it alone if it already exists. You can do this yourself with Ruby code, but why waste your time and energy?

Rake also has the concept of a file task. You can see an example of this on line 12, and it has the following basic syntax:

• On the left side of the => operator, you have the name of the file that you want to create.
• On the right side, you have the name of the file or files that will be used to create this file. Since my ear file is basically a war file with a little bit of added metadata, I only specified the war file for this task.
• This is a little weird, but the task allows you to refer to itself. That’s what I’m doing with the task variable. And in this case, #{task.name} would be equal to dist/myapp.ear.

You might have noticed that I don’t really gain much by putting the code at line 14 into its own file task. I could have simply placed it within the :ear task, and saved a few lines of code and a little confusion. I’m doing it because this method of defining files seems to be more “idiomatically Rake-ish” to me, and that may make this file a little easier to change in the future.

Cleaning Up

What may be really confusing is that the clean prerequisite task isn’t defined anywhere. Well, it kinda-sorta is actually. At line 1, I require the rake/clean module. This tells rake that I want to use it’s built-in clean and clobber tasks. Traditionally, you use the clean task when you want to delete temporary files from your project, and clobber when you want to delete those files plus any files that can be recreated.

Next, I specify what I would and wouldn’t like rake to delete for me when I execute these tasks. At line 3, I’m telling rake that I don’t want to delete .bak files when I clean house. At line 4, I’m saying that when I really want to clean things up in my project (without affecting anything that I can’t replace), I also want to delete the dist and build directories, which contain my ear and war files respectively.

Shell Interaction

Once my :war task is finally done with its prerequisites, it can finally execute the code within its block. Line 55 simply prints a string to STDOUT, and line 56 simply executes the jar command to create a war file. Please note that rake has built-in versions of common shell commands such as mkdir, cp and sh that work across platforms. This is a very nice feature to have, and saves me a bit of coding time.

Command-Line Arguments And Help

Since I wanted to be able to specify build properties from the command line, I added some common command-line tools to my script. This is first implemented at line 42, within the :setenv task. There, I’m reading the value of the env parameter that was passed to the script at runtime.

Of course, before I assume that a parameter exists, I need to validate it. Tht’s why the :settenv task has the :validate task as a prerequsite. It checks to see if some env value exists. If it does, the task exits normally, but if it doesn’t, two things happen:

1. I print an error message to STDERR
2. The :usage task is invoked
3. The script exits with a non-zero return code

The second step should be pretty easy-to-understand by now. I simply invoke the :usage task, which does nothing but print usage information. Also, please note that the :usage task is defined as my default task at line 6. This means that if someone just types rake in my project’s root directory, they will see a usage message.

The third step surprised me. A lot of build tools (including Ant) don’t really make it easy to just exit a build script, especially if you want to return an error code. Since Rakefiles are just Ruby scripts, I can use a very common scripting idiom to exit my script in a somewhat-graceful way.

Questions

Here are some of the questions that went through my mind while I was writing this:

Is Rake better than Ant?

This is a big judgement call, but Rake definitely has its advantages. I like the fact that my build file is an actual script, not an specification of a process written in XML. If I was building a big Java program, I would probably stick with Ant since it has better Java integration. However, for my relatively simple projects, Rake could be a very useful tool.

Why not just write a regular Ruby script instead?

• Command-Line Interface Consistency
  • Anyone running a Rakefile always executes the same command: rake [task] [paramN=foo]. This interface consistency makes it easier to properly run a Rakefile without knowing a lot about what it does.
• Code Reduction
  • Rake is program and a framework (DSL?). This framework gives you some nice built-in stuff that cuts down on your total lines-of-code while improving script quality
• Dependency Based Programming
  • Martin Fowler explains (in his rake tutorial) why it is so much easier to use Rake to write these types of scripts.

As I’ve mentioned in some of my earlier articles, you really don’t need a very powerful media server if you using the MediaMVP media extender with MVPMC firmware. For example, I’m able to use my MediaMVP to watch MythTv movies, Xvid AVI’s, and home movie’s stored as MPEG’s without putting much of a load on my media server, even though it has slow disks, a 1.6 Ghz processor, and 512 MB of RAM. Once I had everything working in my “sandbox environment” (that is, every component was two feet away from every other component), I really thought that the “hard part” was over. All I had to was (queue ominous music) set up the network.

The short version of this story is that I haven’t been able to set up a network that is fast enough for my needs yet. Continue reading for the gory details

A Little Overview

You might be wondering why networking is such a big deal with the MediaMVP. You probably have a “Wireless G” network running in your home, and you probably never have any speed or bandwidth issues.

The problem is that, while a Wireless G network is great for surfing the web, it’s typically not good enough to stream MPEG movies. I’ll explain why later in this article. The total amount that you do need is debatable (and somewhat configurable in MVPMC), but according to the Network page on the MythTv wiki, you should shoot for a 16 Mb/s connection for viewing standard definition (SD) videos. I’ve learned so far that, depending on what you’re viewing, you can get by on less bandwidth, but this appears to be a good number if you want to watch a variety of different SD video formats.

Basically, I need a way to “wire” the televisions in my living room and kitchen into my media server in my basement. The goal is to implement “networked television” so I can watch recorded television shows and home movies in a quick-and-easy fashion.

Why Isn’t Wireless Networking An Option?

Everyone with whom I’ve spoken on this topic has told me to avoid streaming MPEG’s over a wireless network. I know this is only anecdotal evidence, but my friends who have tried this know a lot more about networking and hardware than I do. I’m happy to benefit for their wasted time and effort

Also, the network in my home simply isn’t fast enough. I ran a few tests of my network using Iperf, and found that I was only getting about 15Mb/s of bandwidth on my wireless network. This is actually pretty good, but probably less than what I need.

Another problem with wireless networks is that they simply aren’t reliable. One minute, you’re copying a large file at blinding speeds and, poof, your laptop loses it’s connection to the router. Why this happens is a mystery since I can’t look “under the hood” of my router, so I just live with the occasional flakiness. I could buy a better router, but my friends who have tried the same told me that it only improves the situation slightly. No consumer-grade, wireless router seems to be immune from this problem.

Finally, another big reason why I don’t want to stream big, honkin’ MPEG’s over my wireless network is that almost every other computer in my network also uses it. I really don’t want my wife’s laptop and my MediaMVP to be fighting for bandwidth. I want them both to work quickly and easily.

Wired Networking – The Ideal Solution?

The obvious choice for me at first was wired networking. It’s relatively cheap (this blog is about technology and frugality), it’s very fast, and it seemed like it would be pretty easy to install. I needed to wire two rooms on my main floor into my router in the basement. All I had to do (my favorite example of “famous last words”) was drop some CAT-5e (plain-old networking) cable from my main floor into the basement from two separate rooms. A real no-brainer, right?

Well, I guess it depends. I thought that installation would be easy, but three and a half hours later, after making no progress whatsoever, I took a step back to re-evaluate my options. Here’s what I came up with:

1. I really hate it when I spend a lot of time on a project and get nothing back in return. I’m not the busiest guy in the world, but I’m far too busy to be playing with a hobby for that long for no good reason. I really felt like I wasn’t valuing my free time, which is my most valuable asset.
2. One of the rooms simply couldn’t be wired for ethernet. There was no non-evasive way for me to wire the room for ethernet.
3. The other room could be wired for ethernet, but it would involve more time and effort than I had in the short term, and it would require me to drill another hole in the wall. I prefer using existing holes for this sort of thing, not drilling new ones.

Since frugality is about your use of time and money, I decided to exchange more of the latter for the former. It was time to go back to the drawing board.

Powerline Networking – The Best Of Both Worlds?

One of my friends who also happened to own a MediaMVP, also had trouble running ethernet cable in his house. He therefore decided to try out a powerline networking kit. The kit includes two devices: one that plugs into your MediaMVP using CAT-5e, and one that plugs into your router or switch. The two devices “talk” to each other (i.e. exchange networking packets) over the power network in your house.

This method of networking seemed really strange to me at first, but my friend said that it worked really well for him. I therefore hopped on over to Newegg to get a second opinion, and, lo and behold, there seemed to be universal praise for these devices. I found multiple people on Newegg who used the devices to stream movies, and they all said that this form of networking was more than fast enough for their needs. I thought I had found the perfect solution.

I therefore ordered a ZyXEL PL-100KIT powerline networking kit from Newegg. It’s maximum stated bandwidth is 85 Mb/s, which is over 5 times more than what I required. Unfortunately, once I set up the networking kit and plugged everything in, I was only able to get between 4 and 6 Mb/s of bandwidth according to Iperf and MVPMC. Not only was this around 20% of my bandwidth goal, it was around 7% of the stated maximum.

Now, believe it or not, I was actually able to view some content through my MediaMVP, even with the lackluster bandwidth. I had the best results with MythTv videos that had been recorded with my Hauppauge PVR-150. Those rarely fell out of sync, and when they did, you just needed to fast-forward a couple of seconds. Medium-quality Xvid AVI’s were watchable, but only after setting VLC’s video transcoding setting to “VCD”, which is barely watchable in my opinion. My home videos that I converted to MPEG’s are simply unwatchable.

I figured that something had to be wrong with my particular setup, since so many other people had stated that this model worked well with streaming video. I therefore tried tweaking my setup in the following ways:

• Powerline Adapter Tweaks

  1. Of course, I made sure that none of my powerline networking adapters were plugged into a powerstrip or surge protector.
  2. I made sure that nothing else was plugged into the same outlet as my powerline adaptors.
  3. I tried to plug the adapters into sockets that were in close proximity, even though they had to be on different floors.

  • Some people recommend plugging the adapters into outlets on the same circuit, but I don’t believe that I have any circuits that span floors.

• Router Tweaks

  1. Changed my LAN network type option from “auto-negotiate” to “100 Mb/s”.
  2. Updated my router’s firmware to the most current version

Unfortunately, none of these changes seemed to make a difference. As of today, my MediaMVP works wonderfully with MVPMC, but I can barely use it due to my inadequate network.

Conclusion

When I started this project, I viewed it as a frugal way (again, both in time and money) to enable myself to watch recorded movies and television shows that are stored on my media server. In the end, I ended up doubling my financial budget and tripling my time budget for this project.

Based on all of my experiences so far, I can make the following recommendations:

• This Project Requires A Decent Amount Of Computer Knowledge
  • I’m a systems administrator for a living, and I have a good number of friends who have done stuff like this before. These resources really save me a lot of time on these types of projects. If you don’t have these same resources, then this type of project will probably take you much more time and cause you much more aggravation.
• This Project Could Take You A Lot Of Time
  • There are plenty of examples of people on the internet who have had no problem setting up an MediaMVP system with powerline networking. I’m living proof that the home network part of the puzzle doesn’t always work as well as advertised. Be prepared to spend a decent amount of time troubleshooting and configuring this type of system.

I still see the MediaMVP as being a great device, and look forward to the day where it “just works”. For example, I’m already growing accustomed to watching my favorite daytime TV shows while I work in the kitchen at night, and the rest of my family really loves it too. Therefore, I’m going to continue trying to make things work with a powerline network, but I’m going to try a different brand. I’ve heard good things about the Netgear XE103G and Linksys PLK200 kits. I’ll probably end up trying the Linksys kit for the following reasons:

• It has a higher theoretical maximum bandwidth (200 Mb/s compared to 85 Mb/s)
• It’s basically the same price as the Netgear XE103G
• It received great customer reviews on both Amazon and Newegg
• It received a good review from CNET

I hope to get this part of my system working soon, and will share the results when I’m done.

A friend of mine recently volunteered to update and maintain a web site for a local non-profit. He basically wants to clean up the formatting on the site (using CSS whenever possible), add more rich media such as flash-based slide shows, and make it more collaborative by adding features like a forum.

Of course, since he has a day job and is doing this in his spare time, he would like to accomplish all of this in a way that is inexpensive, relatively simple, and intuitive for a technical professional who isn’t a web designer.

Since I’ve been in his shoes before, he asked if I knew of any tools that could help him build a professional-looking site easily and inexpensively. Here’s what I came up with, and hopefully it will help other technical professionals/hobbyists get started with web development.

Software

Firefox + Firebug

A vanilla instance of Firefox itself isn’t an inherently superior browser for web development over IE or Opera. Once you add Firebug, however, it turns into the best free web application development environment that I’ve ever seen. Here are some of the things you can do with Firebug:

• Find which HTML snippet maps to which element on a web site by hovering over the snippet
  • For me, this is the best feature
• View the document’s HTML or CSS as a tree
• Debug Javascript
• Edit HTML and CSS in memory so you can test-drive the addition or removal of features.
• et cetera

Honestly, this tool easily saved me half of my web development time. It’s worth checking out, even if you’re a dedicated IE or Opera user.

Note: Apparently, there is also a scaled-down version of Firebug that works with Internet Explorer. More information can be found at the Firebug home page.

XAMPP

XAMPP is a “distribution” of web server software that is configured to work very well together, out-of-the-box. It includes Apache2, PHP, MySql, Perl, phpMyAdmin, and a bunch of other things that I’ve never even had any time to use. It’s simply an excellent way to set up a server-side, LAMP development environment on Windows in a hurry.

A Decent Text Editor

There are a lot of integrated development environments (IDE’s) that do a pretty decent job of helping you create robust web sites. Eclipse, for example, has a highly-regarded PHP plugin, and Netbeans has a robust set of “Ruby On Rails” plugins. If you’re working on a very complex web site, or you don’t use a decent text editor on a regular basis, then you’ll probably save time with a decent, free IDE.

However, if you are already using a decent text editor (e.g. Jedit, Vim, Emacs, Textpad, Textmate, etc.) for other tasks, then you should look into using them for HTML editing too. Your favorite editor probably has some excellent plugins to help you write code using HTML, JavaScript, PHP, etc.

The main benefit of using a familiar text editor over an unfamiliar, web-centric IDE is the size of your learning curve. Most IDE’s, while powerful, can be difficult to master. Also, IDE’s usually force you to manage your project in “the Eclipse” way or “the ASP.Net” way, which may be very different from your normal process. A good text editor, however, should already be very familiar to you and flexible enough to allow you to manage your project in a way that works well for you.

This isn’t to say that IDE’s are a bad idea. Just keep the text editor option in mind if you find your IDE to be too bloated, expensive, or difficult to use.

HTML Tidy

HTML Tidy is a great command-line utility for validating web pages. You can use it to automatically format your HTML, but I mostly use it to validate my web pages and point out any mistakes I may have made (such as forgetting to close a tag).

Online Resources

Open Source Web Design

Here are the two hardest tasks for most technologists (sysadmins, programmers, etc.) who aren’t dedicated web developers:

1. Designing the look of the web site

2. Implementing the design

This should be evident by what they wear, but it’s worth stating: most technologists are not good at tasks that require visual creativity. If you need someone to write an efficient sorting algorithm or trace some packets, then they’re the people to call. Designing a clean, good-looking UI, however, is something that most technologists simply can’t do.

And that’s just half the story. Even if the technologist can create a good design on paper, it’s even harder to actually implement the design. CSS is great, don’t get me wrong, but it has a huge learning curve, and just enough browser-based idiosyncrasies to drive you nuts.

In comes the Open Source Web Design project to the rescue. This site has a repository of page designs and the CSS files that implement them, all for free. Just browse the designs, download the ones that work best for you, and your 95% along the path of creating a solid, attractive web site.

Alertbox

Ok, now you site looks pretty good, but it’s just a shell of its future self. It needs content and customization, which brings up a lot of questions. Does my web site need an RSS feed, or should I just use a mailing list? Will a bunch of dancing, AJAX-y crap bring in more traffic or will it just frustrate my users?

Jakob Nielsen’s Alertbox answers questions like this using two guiding principles: usability and quality content. Your site should be usable by your target audience, and it should contain the content that they need. Anything that is contrary to these principles should be avoided at all costs.

Of course, that sort of thing is much more easily said than implemented, so Jakob does tons of research on the subject and then publishes a lot of his results on his web site. Check these out if you want to do a better job of connecting with your “customers”, whomever they may be.

Dead-Tree Resources

Now that I think about it, I’ve only cracked open one actual book on fundamental web design. Not surprisingly, there’s a wealth of information available on HTML, CSS, Javascript, and other web technologies online. If you do need some help with CSS, however, I highly recommend Eric Meyer On CSS. It’s easy-to-read, succinct, and packed with loads of useful examples.