22 Feb 2009

So many projects...

… and so little time. I really should make a point of updating this blog more frequently, as I’ve been doing a lot of new and interesting (at least to me) technical stuff in the *nix space. Up until the end of last year, my day job has primarily focused on application capability. What I mean is I spent most of my time working on application specific configuration and code changes to add features or fix bugs to better support the business’s needs. Most of these applications are large, complex enterprise tools that fall into the “one size fits all because you can customize through configuration” category. Granted, the line between code and configuration is unclear at times. On occasion, I would get down to the application framework or operating system level. It was always a treat.

Late last year, a key person for the company website’s site operations team moved on, and I was pulled in to fill a fraction of the void he left behind. I’m only working there at half capacity because I still need to support my previous areas, but it has been a refreshing deviation from the work I’ve been doing for the past 7 or so years. And when I get interested and excited about something, I begin to explore.

Virtualization

One area I’ve spent some time exploring is system virtualization software, and this is for several reasons. I only have one machine at home to do all my exploratory work. This poses a problem when playing around with load balancing configurations or testing network software. Virtual servers allow me to simulate many machines on a single box, so while it isn’t perfect for emulating a production environment, it makes it possible with a non-existent budget. Trying out different flavors of applications can also have a negative effect (sometimes damaging) on a box. Installing, configuring, testing and then uninstalling can lead to garbage being left behind. If the install is invasive to the basic services of a box and you don’t fully understand all aspects of these changes, it can interfere with the health of the system in the long term. Using a virtual environment for this kind of playing around means you can simply discard the environment if you decide not to go with that solution. One added benefit of installing critical services into a virtual environment is you can easily back-up and transport that environment into another machine if the host box goes belly-up. Maybe I should get another machine…

Website - the whole package

I’ve had a few websites outside of work that I’ve developed (with someone else’s code as a starting pointing) and maintain, this site being one of them. I’ve never put a proper backup/restore process into place for full, rapid restores. In the event of a failure, I would be able to restore most, but it would take a long time and some stuff may fall through the cracks.

Django 1.0 came out a while ago. I’ve successfully upgraded this site, but have some more upgrade work to do yet. This will likely take about 3 days to complete, if I could work on it full time. Unless I can use some dedicated time over a long weekend, it will probably grow into 3 weeks.

This site has only held my blog this far. I think I want to expand it to serve as my project site as well (of which I currently have none), as I’ve got bandwidth and storage space to spare. Source code will be hosted via mercurial, possibly through hgweb. I haven’t decided if I want to use Trac. This partly depends on its RAM consumption because I need to keep several apps running on this hosting service, and memory is my limiting factor. I’ve already done some promising prototyping of a move from mod_python to mod_wsgi, so I appear to be making myself some wiggle room.

My Scala + db4o + Swing personal project

I’ve blogged about this project previously. I’ve succeeded in putting together a lot of the domain specific code this far, but I’m now toiling with the application framework (API for managing the main window, opening/closing views, starting and stopping services). I haven’t decided on the user experience yet: single tabbed window, multi window, etc. The options are plentiful, but my goal is to keep it light weight with the potential for using it in a smart phone. I’ve started going down the path of OSGi (another learning experience). That may or may not continue. This is one project I would like to host out of this web site.

sbaz - Scala’s package manager

I am presently the named maintainer for sbaz. Sbaz will start to become a bigger part of this blog. Presently, I am still struggling with defining a path forward for sbaz. I’m considering hosting some sbaz universes out of this site (e.g. my personal project above, a managed developer release universe, etc.). My biggest obstacle with this is my limited memory resources, so deploying the existing Java servlet won’t be possible. Perhaps using PHP, Python or flat files to host a universe should be my first enhancement.

General system administration notes

I need a reliable location to document many of my system administration learnings, and this website seems to be a logical location. To date, I’ve peppered README files throughout my machine’s filesystem, but the point behind the README files is to repeat the steps I need to perform on a new (e.g. replacement) machine. I’ve also failed to document some good stuff because I hope to, at some point, have a central repository. So, I delay documenting until I forget to do it all together, a pattern I’ve recently recognized and am changing… leading to the before mentioned README files. Some of the things I intend to document include:

  • MySQL, rsync, etc. commands (scripts) needed to back-up and restore websites
  • Configuration for monitoring mdadm, SMART, capacity and other hard drive details
  • Virtualization via OpenVZ, KVM and VMware
  • Remote monitoring tool (mon or nagios) for virtual machines, website, hdhomerun and services
  • Local mail server configuration (including security concerns)
  • Apache configuration details (including security concerns)
  • DNS configuration for third party hosting, local network, dynamic IP, etc.
  • I may even consider documenting SAP transaction codes and technology details (BADI, BAPI, BDOC, BSP, complex ABAP selects, OO vs. Procedural ABAP details, etc.) as these are starting to fade a bit.
  • Just about anything I’ve had to research when solving a problem, and don’t address often enough to know off hand.