Making Latkes

I spend my days making great software with the team at OpenSky.com. At the end of the day the value of what we do is not measured by how many tasks we complete, how many lines of code we write, or even how many features we deliver, but by happy customers and successful merchants. On the last day of Hannukah and I found myself making latkes and thinking about the parallels between my experience as a cook and as a software developer.

Latkes are little potato pancakes, fried in oil and served with applesauce and sour cream. They are really tasty hot from the pan, but you can also make latkes ahead of time and reheat them so they can be served all at once. My plan was to start cooking at noon and to be done by 2pm so that I would have time to go for a swim and still arrive on time at the party with my latkes at 4pm.

When I set to work at noon I decided to make the latkes in three batches because my mixing bowl wasn’t big enough to hold all the potatoes at once. This worked out serendipitously because when 2pm rolled around I had only just finished the second batch. I could have soldiered on to make the third batch and arrived at the party tired and grumpy at having missed my swim, but I realized that I had made 24 latkes and that was enough. Since I hadn’t grated the last batch of potatoes I skipped making the third batch completely. Aha! I thought, this is the value of delivering work in smaller batches: it makes it possible to meet my schedule by reducing my scope.

My pan is big enough to fit four latkes at a time and in an effort to maximize throughput I started out frying the latkes four at a time. I quickly learned that when I did that quality suffered for two reasons: 1) the temperature of the oil dropped – the latkes did not cook as quickly and soaked up more oil – the recipes warned of this, but apparently I have to learn by doing, and 2) having only 3 latkes in the pan left me some room to maneuver – I could slide the pancakes around to brown them more evenly. Maybe this is why I only finished two batches instead of three, but I focused on the value of the delivered product: two dozen crispy delicious potato pancakes – they were a hit :)

Lots of people contributed to the party – each did their bit, and it all came together on schedule for a great evening.

Happy making!

Using scala implicits to make my code more readable.

Scala implicits can be very frustrating to new users of the language, they were for me.

Their operation seems magical because the compiler chooses when and where to apply them.

A simple use case for implicits is providing type conversions that would otherwise cause a lot of boilerplate code clutter.

Here’s an example adapted from some tests I wrote to check payment calculations for an e-commerce web site.

Payments are represented by a case class like this:

case class Payment(date: DateTime, amount: BigDecimal, note: Option[String])

Here’s how I created a testPayment to compare with my calculations:

val date = ISODateTimeFormat.date.parseDateTime("2014-04-01")
val testPayment = Payment(date, BigDecimal(100.00), Some("comment"))

There’s a lot of boilerplate to get the right types.

I cleaned things up a bit by introducing some implicits in the body of my test:

  
implicit def String2DateTime(value: String): DateTime = ISODateTimeFormat.date.parseDateTime(value)
implicit def String2OptionString(value: String): Option[String] = Some(value)
implicit def Double2BigDecimal(value: Double): BigDecimal = BigDecimal(value)

Once these implicits were in scope my testPayment creation became much easier to read:

val testPayment = Payment("2014-04-01", 100.00, "comment")

Posted in Uncategorized | Comments Off on Using scala implicits to make my code more readable.

Riak 1.3.0 dev cluster on Ubuntu 12.x

Riak 1.3.0 is out and I decided that I would rather have my dev cluster on a stable linux box rather than an OS/x laptop.

I took https://github.com/xing/riak-dev-cluster as a starting point. This repo provides a way to install a riak dev cluster on OS/x.

My modifications to build and run a dev cluster on Ubuntu 12.x are at https://github.com/nopolabs/riak-dev-cluster

I added a couple rake tasks:
rake bind_all
to bind to all IPs (0.0.0.0) and
rake bind_local
to bind to localhost (127.0.0.1), convenient for me since I still want to do client development on my laptop.

I needed to edit the app.config files for the riak servers to change the IP bindings. These files contain Erlang terms, and I found a script that could be used to update them (http://adamschepis.com/blog/2011/08/25/a-script-to-update-riak-config-files/). But I ended up using sed and regular expressions to get the job done. Learning Erlang is still somewhere in the future for me 😉

You might find https://github.com/hectcastro/vagrant-riak-cluster an interesting alternative. It uses Vagrant to a riak cluster on virtualboxes.

You will need to have an erlang development installed to build riak:

sudo apt-get install erlang erlang-base erlang-tools erlang-appmon erlang-pman erlang-observer erlang-ic erlang-parsetools erlang-toolbar erlang-et erlang-mnesia erlang-debugger erlang-percept

Posted in Uncategorized | Comments Off on Riak 1.3.0 dev cluster on Ubuntu 12.x

Installing Riak 1.2.1 on Mac OS/X 10.7

Update: For Riak 1.3 this may be a better solution than what I describe below: https://github.com/simonvc/riak-dev-cluster

I’ve been hearing a lot about Riak recently. When the Portland Basho team scheduled some co-working office hours I took the opportunity to get Riak installed and to talk to some experts.

Riak’s web site and documentation are well organized and very helpful.

I already had the proper version of Erlang (R15B01) installed, so I skipped straight to installing from a binary tarball which looked like the easiest and least intrusive option.

But when I tried to run Riak after unpacking the tarball I got an error:

~/riak-1.2.1 $ bin/riak start
Error reading /Users/drevel/Code/riak-1.2.1/tarball/riak-1.2.1/etc/app.config

After some googling and poking around I decided to try installing from source instead. That went better:

~/riak-1.2.1/rel/riak $ bin/riak start
~/riak-1.2.1/rel/riak $ bin/riak ping
pong
~/riak-1.2.1/rel/riak $ bin/riak stop
ok

But, what’s the point of running a single Riak node? Not much, fortunately the source build provides a devrel target to help set up a cluster on localhost.

Following the instructions got me to this:

~/riak-1.2.1/dev $ dev1/bin/riak start
Riak failed to start within 15 seconds,
see the output of 'riak console' for more information.
If you want to wait longer, set the environment variable
WAIT_FOR_ERLANG to the number of seconds to wait.

Hmm… Trying again using console instead of start I got a lot of output, but the important piece was:

16:12:56.257 [error] Supervisor riak_api_sup had child riak_api_pb_listener started with riak_api_pb_listener:start_link("127.0.0.1", 8081) at undefined exit with reason bad return value: {error,eaddrinuse} in context start_error

Turns out McAfee antivirus is using port 8081 on my laptop.

So, I wrote a script to change the ports used by the dev release:

change_riak_ports.sh

After changing the port numbers I was able to start the four riak instances, but I couldn’t join them as a cluster:

~/riak-1.2.1/dev $ dev2/bin/riak-admin cluster join dev1@127.0.0.1
Node is not running!
~/riak-1.2.1/dev $ dev2/bin/riak-admin cluster join plan
Node is not running!
~/riak-1.2.1/dev $ dev2/bin/riak chkconfig
config is OK
~/riak-1.2.1/dev $ dev2/bin/riak ping

=ERROR REPORT==== 18-Feb-2013::10:28:01 ===
Error when reading /Users/drevel/.erlang.cookie: eaccesescript: exception error: no match of right hand side value 
                 {error,
                     {shutdown,
                         {child,undefined,net_sup_dynamic,
                             {erl_distribution,start_link,
                                 [['dev2_maint_39472@127.0.0.1',longnames]]},
                             permanent,1000,supervisor,
                             [erl_distribution]}}}

Aha! I needed to delete ~/erlang.cookie and restart my instances.

Now I have my local riak cluster up and running on my laptop :)

Posted in Uncategorized | Comments Off on Installing Riak 1.2.1 on Mac OS/X 10.7

Installing Redis 2.4.5 on Ubuntu 10.04 LTS

Redis is an open-source key value store. I visited the interactive tutorial and was intrigued by the features it offers.

Here’s how I installed a local copy for further investigation:

# download and unpack the sources (see http://redis.io/download for the latest stable version)
wget http://redis.googlecode.com/files/redis-2.4.5.tar.gz
tar -zxvf redis-2.4.5.tar.gz

# build
cd redis-2.4.5/
make

# test
# I needed to install tcl8.5 to run the tests: sudo apt-get install tcl8.5
make test

#install
sudo make install
cd utils
sudo ./install_server.sh

# run the redis cli (/usr/local/bin/redis-cli)
redis-cli
Posted in Uncategorized | Comments Off on Installing Redis 2.4.5 on Ubuntu 10.04 LTS

Enabling SSL for Couchdb v1.1

Having installed Couchdb v1.1 I wanted to take advantage of its native support for SSL. I found instructions at couchbase.org and on wiki.apache.org which got me through editing etc/couchdb/local.ini. Uncomment and edit lines in the [daemons] and [ssl] sections to look something like this:

[daemons]
; enable SSL support by uncommenting the following line and supply the PEM's below.
; the default ssl port CouchDB listens on is 6984
httpsd = {couch_httpd, start_link, [https]}

[ssl]
cert_file = /var/lib/couchdb/build-couchdb/build/etc/ssl/couch_cert.pem
key_file = /var/lib/couchdb/build-couchdb/build/etc/ssl/couch_key.pem

The most difficult part proved to be generating the required private key and self-signed certificate. couchbase.org has instructions, but they didn’t work for me. I followed openssl’s keys.txt to generate a non-password-protected private key, and certificates.txt to create a test certificate self-signed using the private key.

openssl genrsa -out couch_key.pem 4096
openssl req -new -x509 -key couch_key.pem -out couch_cert.pem -days 1095

Restart couchdb and test with curl:

sudo /etc/init.d/couchdb-1.1.0 restart
curl -k https://atom:6984/
# expect: {"couchdb":"Welcome","version":"1.1.0"}

The troubleshooting tips on wiki.apache.org were helpful, in particular the suggestion to test using a known good private key and certificate from MochiWeb.

Posted in Uncategorized | Comments Off on Enabling SSL for Couchdb v1.1

Installing Couchdb v1.1 on Ubuntu 10.04 LTS

The repositories for Ubuntu 10.04 LTS only contain packages for Couchdb v1.0. The Couchdb Wiki has instructions for manual installation on Ubuntu, but it also provides a pointer to the build-couchdb project on GitHub.

Using build-couchdb, here’s how I built, installed and ran Couchdb on my Ubuntu 10.04 LTS system:

# remove the old Couchdb (could have just disabled...)
sudo apt-get uninstall couchdb

# get the dependencies required by build-couchdb
sudo apt-get install make gcc zlib1g-dev libssl-dev libreadline5-dev rake

# download and build as couchdb user
sudo su couchdb
cd ~

# clone and prepare build-couchdb
git clone git://github.com/iriscouch/build-couchdb
cd build-couchdb/
git submodule init
git submodule update

# build couchdb
rake/etc/init.d/couchdb-0.11.0

# switch to root to install init script
exit
sudo su

# install init script
cd /etc/init.d
ln -s ~couchdb/build-couchdb/build/etc/init.d/couchdb couchdb-1.1.0
update-rc.d couchdb-1.1.0 defaults

# start couchdb
/etc/init.d/couchdb-1.1.0 start

# back to user mode
exit

# test
curl http://127.0.0.1:5984
# expected output: {"couchdb":"Welcome","version":"1.1.0"}

Drop dead simple – thanks to build-couchdb :)

Posted in Uncategorized | Comments Off on Installing Couchdb v1.1 on Ubuntu 10.04 LTS

Installing RabbitMQ on Ubuntu 10.04 LTS

Following these instructions, here’s how I installed RabbitMQ on my Ubuntu 10.04 LTS system:

# add the rabbitmq repository to /etc/apt/sources.list 
sudo bash -c 'echo "deb http://www.rabbitmq.com/debian/ testing main" >> /etc/apt/sources.list'

# add rabbitmq's public key so that aptitude will trust the repository
wget http://www.rabbitmq.com/rabbitmq-signing-key-public.asc
sudo apt-key add rabbitmq-signing-key-public.asc

# Update aptitude's list of available packages
sudo apt-get update

# Install rabbitmq :)
sudo apt-get install rabbitmq-server

# enable the rabbit management plugin
sudo rabbitmq-plugins enable rabbitmq_management

# restart rabbit
sudo /etc/init.d/rabbitmq-server restart
Posted in Uncategorized | Comments Off on Installing RabbitMQ on Ubuntu 10.04 LTS

Installing MongoDB 2.0 on Ubuntu 10.04 LTS

10Gen publishes apt-gettable packages for MongoDB that are more up-to-date than those in Debian or Ubuntu.

The details about getting and installing 10gen’s packages are here.

Following those instructions, here’s how I installed MongoDB on my Ubuntu 10.04 LTS system:

# add the mongodb repository to /etc/apt/sources.list
sudo bash -c "echo 'deb http://downloads-distro.mongodb.org/repo/ubuntu-upstart dist 10gen' >> /etc/apt/sources.list"

# add 10gen's GPG key so that aptitude will trust the repository
sudo apt-key adv --keyserver keyserver.ubuntu.com --recv 7F0CEB10

# Update aptitude's list of available packages
sudo apt-get update

# Install mongodb
sudo apt-get install mongodb-10gen
Posted in Uncategorized | Comments Off on Installing MongoDB 2.0 on Ubuntu 10.04 LTS